Unhandled LoadStoreAlignment exception

carlcarl US
edited April 18 in Mongoose OS

I've started getting this error, just serving files from fs/:

mgos_http_ev         0x3ffb4334 HTTP connection from 192.168.1.151:60510
mgos_http_ev         0x3ffb4334 GET /index.html
set_spiffs_errno     .: res = 0, e = -10002
drop_dir             './index.html' -> 'index.html'
set_spiffs_errno     ./index.html: res = 0, e = -10002
mg_uri_to_local_path '/index.html' -> './index.html' + ''
drop_dir             './index.html' -> 'index.html'
set_spiffs_errno     ./index.html: res = 0, e = -10002
mg_send_http_file    0x3ffb4334 GET [./index.html] exists=1 is_dir=0 is_dav=0 is_cgi=0 index=
mg_http_is_authorize ./index.html  1 1
mg_http_is_authorize ./index.html .htpasswd 0 1
mg_http_serve_file   0x3ffb4334 [./index.html] text/html
drop_dir             './index.html' -> 'index.html'
set_spiffs_errno     ./index.html: res = 0, e = -10002
drop_dir             './index.html' -> 'index.html'
set_spiffs_errno     index.html: res = 1, e = -10002
set_spiffs_errno     fstat: res = 0, e = -10002
set_spiffs_errno     read: res = 128, e = -10002
set_spiffs_errno     read: res = 128, e = -10002
set_spiffs_errno     read: res = 128, e = -10002
set_spiffs_errno     read: res = 128, e = -10002
set_spiffs_errno     read: res = 128, e = -10002
set_spiffs_errno     read: res = 128, e = -10Guru Meditation Error of type LoadStoreAlignment occurred on core  0. Exception was unhandled.
Register dump:
PC      : 0x400dd0cc  PS      : 0x00060b30  A0      : 0x800dd528  A1      : 0x3ffcc950
A2      : 0x3ffd2460  A3      : 0x3ffc2588  A4      : 0x3ffb44a4  A5      : 0x3ffb3da4
A6      : 0x3ffc2588  A7      : 0x00001016  A8      : 0x40082b2a  A9      : 0x3ffcc920
A10     : 0x3ffb4810  A11     : 0x3ffcc946  A12     : 0x00000004  A13     : 0x3ffb481c
A14     : 0x0160c9a9  A15     : 0x0160c9a9  SAR     : 0x00000010  EXCCAUSE: 0x00000009
EXCVADDR: 0x40082b2e  LBEG    : 0x4000c349  LEND    : 0x4000c36b  LCOUNT  : 0xffffffff

Backtrace: 0x400dd0cc:0x3ffcc950 0x400dd528:0x3ffcc970 0x400dde40:0x3ffcca40 0x40092618:0x3ffcca60 0x40093f7a:0x3ffccab0

Any idea what this is and how I can track it down?

Comments

  • rojerrojer Dublin, Ireland

    we need that backtrace symbolized. do you build your own firmware?

  • rojerrojer Dublin, Ireland
    edited April 18

    ok, then you'll need to do it yourself. run xtensa-elf32-gdb on the .elf file (find it in the build directory) and call disas on the 0x400... addresses from the backtrace (the second address is the stack frame pointer). or you can post the .elf file somewhere and i can do it.

  • ok, long story:

    (gdb) disas 0x400dd0cc
    Dump of assembler code for function sta_rx_eapol:
       0x400dd0c0 <+0>: entry   a1, 32
       0x400dd0c3 <+3>: l32r    a8, 0x400d2790
       0x400dd0c6 <+6>: l32i.n  a10, a4, 4
       0x400dd0c8 <+8>: l32i.n  a8, a8, 0
       0x400dd0ca <+10>:    l32i.n  a10, a10, 4
       0x400dd0cc <+12>:    l32i.n  a9, a8, 4
       0x400dd0ce <+14>:    beqz.n  a9, 0x400dd0f8 <sta_rx_eapol+56>
       0x400dd0d0 <+16>:    l32i.n  a8, a8, 8
       0x400dd0d2 <+18>:    bnei    a8, 2, 0x400dd0f8 <sta_rx_eapol+56>
       0x400dd0d5 <+21>:    l32r    a2, 0x400d1a90
       0x400dd0d8 <+24>:    l32i    a2, a2, 0x17c
       0x400dd0db <+27>:    beqz    a2, 0x400dd180 <sta_rx_eapol+192>
       0x400dd0de <+30>:    l32i.n  a2, a2, 8
       0x400dd0e0 <+32>:    beqz    a2, 0x400dd180 <sta_rx_eapol+192>
    ---snip---
    
    (gdb) disas 0x400dd528
    Dump of assembler code for function sta_input:
       0x400dd1dc <+0>: entry   a1, 208
       0x400dd1df <+3>: l32r    a8, 0x400d2594
       0x400dd1e2 <+6>: l32i.n  a7, a3, 16
       0x400dd1e4 <+8>: l32i    a6, a8, 144
       0x400dd1e7 <+11>:    l32i.n  a7, a7, 24
       0x400dd1e9 <+13>:    addi.n  a6, a6, 1
       0x400dd1eb <+15>:    l32i    a14, a2, 144
       0x400dd1ee <+18>:    s32i    a4, a1, 132
       0x400dd1f1 <+21>:    extui   a4, a7, 0, 12
       0x400dd1f4 <+24>:    s32i    a6, a8, 144
       0x400dd1f7 <+27>:    s32i    a5, a1, 116
       0x400dd1fa <+30>:    s32i    a4, a1, 112
    ---snip---
    
       0x400dd516 <+826>:   j   0x400dddec <sta_input+3088>
       0x400dd519 <+829>:   or  a12, a10, a10
       0x400dd51c <+832>:   or  a11, a14, a14
       0x400dd51f <+835>:   or  a10, a2, a2
       0x400dd522 <+838>:   s32i    a8, a1, 164
       0x400dd525 <+841>:   call8   0x400dd0c0 <sta_rx_eapol>
       0x400dd528 <+844>:   l32i    a8, a1, 164
       0x400dd52b <+847>:   bnei    a10, 1, 0x400dd531 <sta_input+853>
       0x400dd52e <+850>:   j   0x400ddde1 <sta_input+3077>
       0x400dd531 <+853>:   beqz.n  a7, 0x400dd536 <sta_input+858>
       0x400dd533 <+855>:   j   0x400ddd60 <sta_input+2948>
       0x400dd536 <+858>:   l32r    a4, 0x400d1a90
       0x400dd539 <+861>:   addmi   a4, a4, 0x100
       0x400dd53c <+864>:   l8ui    a4, a4, 186
    ---snip---
    
    (gdb) disas 0x400dde40
    Dump of assembler code for function sta_rx_cb:
       0x400dde2c <+0>: entry   a1, 32
       0x400dde2f <+3>: l32r    a8, 0x400d1a90
       0x400dde32 <+6>: or  a11, a2, a2
       0x400dde35 <+9>: l32i    a10, a8, 16
       0x400dde38 <+12>:    or  a13, a4, a4
       0x400dde3b <+15>:    mov.n   a12, a3
       0x400dde3d <+17>:    call8   0x400dd1dc <sta_input>
       0x400dde40 <+20>:    mov.n   a2, a10
       0x400dde42 <+22>:    retw.n
    End of assembler dump.
    
    (gdb) disas 0x40092618
    Dump of assembler code for function ppRxPkt:
       0x40091f04 <+0>: entry   a1, 80
       0x40091f07 <+3>: movi.n  a7, 16
       0x40091f09 <+5>: j   0x40092767 <ppRxPkt+2147>
       0x40091f0c <+8>: or  a10, a2, a2
       0x40091f0f <+11>:    call8   0x40098a24 <wDevCheckBlockError>
       0x40091f12 <+14>:    l32i    a5, a2, 16
       0x40091f15 <+17>:    l32i.n  a3, a2, 4
       0x40091f17 <+19>:    addi    a4, a5, 28
       0x40091f1a <+22>:    s32i.n  a4, a3, 4
       0x40091f1c <+24>:    l32i.n  a3, a2, 36
       0x40091f1e <+26>:    mov.n   a11, a5
       0x40091f20 <+28>:    l32i.n  a3, a3, 0
       0x40091f22 <+30>:    mov.n   a10, a2
       0x40091f24 <+32>:    and a3, a3, a7
    ---snip---
    
       0x4009260d <+1801>:  jx  a0
       0x40092610 <+1804>:  addi.n  a2, a3, 11
       0x40092612 <+1806>:  or  a10, a2, a2
       0x40092615 <+1809>:  callx8  a4
       0x40092618 <+1812>:  j   0x40092767 <ppRxPkt+2147>
       0x4009261b <+1815>:  lsi f0, a1, 0x35c
       0x4009261e <+1818>:  bnez.n  a2, 0x4009265c <ppRxPkt+1880>
       0x40092620 <+1820>:  l8ui    a2, a1, 50
       0x40092623 <+1823>:  lsi f2, a1, 0x148
    ---snip---
    
       0x40092756 <+2130>:  addi    a2, a2, 18
       0x40092759 <+2133>:  j   0x400acdf1
       0x4009275c <+2136>:  call0   0x4001d820
       0x4009275f <+2139>:  mov.n   a10, a2
       0x40092761 <+2141>:  l32r    a8, 0x400917bc
       0x40092764 <+2144>:  callx8  a8
       0x40092767 <+2147>:  call8   0x40091ea0 <ppDequeueRxq_Locked>
       0x4009276a <+2150>:  mov.n   a2, a10
       0x4009276c <+2152>:  beqz.n  a10, 0x40092771 <ppRxPkt+2157>
       0x4009276e <+2154>:  j   0x40091f0c <ppRxPkt+8>
       0x40092771 <+2157>:  retw.n
    ---snip---
    
    (gdb) disas 0x40093f7a
    Dump of assembler code for function ppTask:
       0x40093e90 <+0>: entry   a1, 64
       0x40093e93 <+3>: l32r    a2, 0x40092a14
       0x40093e96 <+6>: l32r    a4, 0x40091978
       0x40093e99 <+9>: l32r    a3, 0x40092f7c
       0x40093e9c <+12>:    movi.n  a5, 24
       0x40093e9e <+14>:    l32i    a10, a2, 0
       0x40093ea1 <+17>:    movi    a13, 0
       0x40093ea4 <+20>:    movi    a12, -1
       0x40093ea7 <+23>:    addi    a11, a1, 16
       0x40093eaa <+26>:    call8   0x40083240 <xQueueGenericReceive>
       0x40093ead <+29>:    bnei    a10, 1, 0x40093e9e <ppTask+14>
       0x40093eb0 <+32>:    l32i.n  a10, a1, 16
    ---snip---
    
       0x40093f62 <+210>:   l32r    a8, 0x40080684
       0x40093f65 <+213>:   callx8  a8
       0x40093f68 <+216>:   j   0x40093e9e <ppTask+14>
       0x40093f6b <+219>:   sub.s   f0, f12, f0
       0x40093f6e <+222>:   call8   0x40091d54 <ppProcTxDone>
       0x40093f71 <+225>:   j   0x40093e9e <ppTask+14>
       0x40093f74 <+228>:   ill
       0x40093f77 <+231>:   call8   0x40091f04 <ppRxPkt>
       0x40093f7a <+234>:   j   0x40093e9e <ppTask+14>
       0x40093f7d <+237>:   ill
       0x40093f80 <+240>:   l8ui    a10, a1, 20
       0x40093f83 <+243>:   call8   0x40093a40 <ppResortTxAMPDU>
       0x40093f86 <+246>:   j   0x40093e9e <ppTask+14>
    ---snip---
    

    Hope that helps.

  • rojerrojer Dublin, Ireland

    hm. this does look like a legit unaligned 32-bit load somewhere in the guts of wifi code. 0x40082b2a is an address in IRAM and is not 4-byte aligned, so l32i.n a9, a8, 4 throws an exception.

  • rojerrojer Dublin, Ireland

    is it reproducible? i've never seen it happen, and we do test serving files from fs on our CI.

  • Yeah, I basically can't serve files at all at this point. Note that an RPC call to get files works fine.

  • rojerrojer Dublin, Ireland
    edited April 19

    i was able to reproduce it and filed https://github.com/espressif/esp-idf/issues/530
    i don't see anything in my code that could be triggering it, http uses lwip's socket interface (unlike 8266, where we use low-level tcp api), so wifi rx path is at least two steps away from anything we ever touch... let's see what they have to say.
    somewhat bizarrely, it is smaller files that tend to cause this, larger files can be fetched just fine. and... you might've guessed it: our test fetches a larger file.

    was there time when this wasn't happening?

  • Yeah, it only started recently, when I started building out an app. Maybe the number of files? Odd.

  • rojerrojer Dublin, Ireland

    ok. i'll do some more digging and bisecting tomorrow.

  • Hmm. Still getting a crash, but in different place now. Let me decode some symbols...

  • mgos_http_ev         0x3ffd47cc HTTP connection from 192.168.1.151:59780
    mgos_http_ev         0x3ffd47cc GET /example.txt
    set_spiffs_errno     .: res = 0, e = -10002
    drop_dir             './example.txt' -> 'example.txt'
    set_spiffs_errno     ./example.txt: res = 0, e = -10002
    mg_uri_to_local_path '/example.txt' -> './example.txtGuru Meditation Error of type LoadProhibited occurred on core  0. Exception was unhandled.
    Register dump:
    PC      : 0x400014fd  PS      : 0x00060730  A0      : 0x800f6560  A1      : 0x3ffc96a0
    A2      : 0x00060730  A3      : 0x0006072c  A4      : 0x000000ff  A5      : 0x0000ff00
    A6      : 0x00ff0000  A7      : 0xff000000  A8      : 0x00000000  A9      : 0x3ffc9970
    A10     : 0x00000003  A11     : 0x00060723  A12     : 0x00060720  A13     : 0x0a2a9364
    A14     : 0x00000001  A15     : 0x00000000  SAR     : 0x00000000  EXCCAUSE: 0x0000001c
    EXCVADDR: 0x00060730  LBEG    : 0x400014fd  LEND    : 0x4000150d  LCOUNT  : 0xffffffff
    
    Backtrace: 0x400014fd:0x3ffc96a0 0x400f6560:0x3ffc96b0 0x400f4615:0x3ffc99c0 0x40106f5e:0x3ffc9a80 0x40110444:0x3ffc9b00 0x40103e54:0x3ffc9b20 0x4000bd86:0x3ffc9b40 0x40001180:0x3ffc9b60 0x40059301:0x3ffc9b80 0x4005937d:0x3ffc9ba0 0x400593b2:0x3ffc9bc0 0x4011265e:0x3ffc9be0 0x4011654d:0x3ffc9c30 0x40118936:0x3ffcac70 0x4010fa84:0x3ffcb7b0 0x401169b1:0x3ffcb830 0x40117d6e:0x3ffcb870 0x401182ac:0x3ffcb8a0 0x401182d8:0x3ffcb990 0x401169b1:0x3ffcbb30 0x401172a1:0x3ffcbb70 0x401176a1:0x3ffcbba0 0x40117a0e:0x3ffcbc10 0x40114caa:0x3ffcbc70 0x4010f88f:0x3ffcbc90 0x40111119:0x3ffcbcb0
    
    (gdb) disas 0x400014fd
    No function contains specified address.
    (gdb) disas 0x400f6560
    Dump of assembler code for function _svfprintf_r:
       0x400f4b54 <+0>: entry   a1, 0x310
       0x400f4b57 <+3>: s32i    a2, a1, 0x28c
       0x400f4b5a <+6>: l32i    a10, a1, 0x28c
       0x400f4b5d <+9>: s32i    a3, a1, 0x290
       0x400f4b60 <+12>:    s32i    a4, a1, 0x284
       0x400f4b63 <+15>:    s32i    a5, a1, 0x270
       0x400f4b66 <+18>:    s32i    a6, a1, 0x274
       0x400f4b69 <+21>:    s32i    a7, a1, 0x278
       0x400f4b6c <+24>:    l32r    a8, 0x400d054c <_stext+1332>
       0x400f4b6f <+27>:    callx8  a8
       0x400f4b72 <+30>:    l32i.n  a10, a10, 0
    ---snip---
    

    Call stack looks like:
    0x400014fd ??
    0x400f6560 _svfprintf_r
    0x400f4615 snprintf
    0x40106f5e mgos_debug_write
    0x40110444 debug_write
    0x40103e54 esp_vfs_write
    0x4000bd86 ??
    0x40001180 ??
    ...

  • Ah, nevermind. Did a clean build and all looks good now.

  • rojerrojer Dublin, Ireland

    yeah, i was going to say - it looked like the same thing. ok, great.

  • Thanks for the fix!

Sign In or Register to comment.