Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Illegal instruction when calling Math.exp() #1376

Closed
tberthe opened this issue Apr 8, 2015 · 32 comments
Closed

Illegal instruction when calling Math.exp() #1376

tberthe opened this issue Apr 8, 2015 · 32 comments
Labels
arm Issues and PRs related to the ARM platform. confirmed-bug Issues with confirmed bugs. v8 engine Issues and PRs related to the V8 dependency.

Comments

@tberthe
Copy link

tberthe commented Apr 8, 2015

I'm using io.js on a Raspberry Pi (Raspbian) and calling Math.exp() generates an illegal instruction.
I have tryed many packages: iojs-v1.2.0-linux-armv6l.tar.gz, iojs-v1.4.1-linux-armv6l.tar.gz and iojs-v1.6.4-linux-armv6l.tar.gz
For example, a simple file with:

var v = Math.exp(-0.5);

produces this error.

@bnoordhuis
Copy link
Member

I'd like to know what instruction it's tripping on. Can you try this?

$ gdb --args iojs -e 'Math.exp(-0.5)'
> run
# wait for the illegal instruction trap
> disassemble $pc,$pc+32

@tberthe
Copy link
Author

tberthe commented Apr 8, 2015

$ gdb --args ~/interpreters/iojs-v1.6.4-linux-armv6l/bin/iojs -e 'Math.exp(-0.5)'
GNU gdb (GDB) 7.4.1-debian
Copyright (C) 2012 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "arm-linux-gnueabihf".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>...
Reading symbols from /home/pi/interpreters/iojs-v1.6.4-linux-armv6l/bin/iojs...done.
(gdb) r
Starting program: /home/pi/interpreters/iojs-v1.6.4-linux-armv6l/bin/iojs -e Math.exp\(-0.5\)
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/arm-linux-gnueabihf/libthread_db.so.1".

Program received signal SIGILL, Illegal instruction.
0x003abda0 in _armv7_neon_probe ()
(gdb) disassemble $pc,$pc+32
Dump of assembler code from 0x3abda0 to 0x3abdc0:
=> 0x003abda0 <_armv7_neon_probe+0>:    vorr    q15, q15, q15
   0x003abda4 <_armv7_neon_probe+4>:    bx      lr
   0x003abda8 <_armv7_tick+0>:  mrc     15, 0, r0, cr9, cr13, {0}
   0x003abdac <_armv7_tick+4>:  bx      lr
   0x003abdb0 <OPENSSL_atomic_add+0>:   ldrex   r2, [r0]
   0x003abdb4 <OPENSSL_atomic_add+4>:   add     r3, r2, r1
   0x003abdb8 <OPENSSL_atomic_add+8>:   strex   r2, r3, [r0]
   0x003abdbc <OPENSSL_atomic_add+12>:  cmp     r2, #0
End of assembler dump.

But io.js has a strange behavior since calling only gdb --args ~/interpreters/iojs-v1.6.4-linux-armv6l/bin/iojs and then run also produces the illegal instruction whereas calling ~/interpreters/iojs-v1.6.4-linux-armv6l/bin/iojs produces the error only when entering the call to 'Math.exp()' ...

@bnoordhuis
Copy link
Member

Oh, I forgot that openssl does feature detection. Can you repeat the above (enter continue after disassembling) until you hit the final SIGILL?

@tberthe
Copy link
Author

tberthe commented Apr 8, 2015

$ gdb --args ~/interpreters/iojs-v1.6.4-linux-armv6l/bin/iojs -e 'Math.exp(-0.5)'
GNU gdb (GDB) 7.4.1-debian
Copyright (C) 2012 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "arm-linux-gnueabihf".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>...
Reading symbols from /home/pi/interpreters/iojs-v1.6.4-linux-armv6l/bin/iojs...done.
(gdb) r
Starting program: /home/pi/interpreters/iojs-v1.6.4-linux-armv6l/bin/iojs -e Math.exp\(-0.5\)
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/arm-linux-gnueabihf/libthread_db.so.1".

Program received signal SIGILL, Illegal instruction.
0x003abda0 in _armv7_neon_probe ()
(gdb) disassemble $pc,$pc+32
Dump of assembler code from 0x3abda0 to 0x3abdc0:
=> 0x003abda0 <_armv7_neon_probe+0>:    vorr    q15, q15, q15
   0x003abda4 <_armv7_neon_probe+4>:    bx      lr
   0x003abda8 <_armv7_tick+0>:  mrc     15, 0, r0, cr9, cr13, {0}
   0x003abdac <_armv7_tick+4>:  bx      lr
   0x003abdb0 <OPENSSL_atomic_add+0>:   ldrex   r2, [r0]
   0x003abdb4 <OPENSSL_atomic_add+4>:   add     r3, r2, r1
   0x003abdb8 <OPENSSL_atomic_add+8>:   strex   r2, r3, [r0]
   0x003abdbc <OPENSSL_atomic_add+12>:  cmp     r2, #0
End of assembler dump.
(gdb) continue
Continuing.

Program received signal SIGILL, Illegal instruction.
0x003abda8 in _armv7_tick ()
(gdb) disassemble $pc,$pc+32
Dump of assembler code from 0x3abda8 to 0x3abdc8:
=> 0x003abda8 <_armv7_tick+0>:  mrc     15, 0, r0, cr9, cr13, {0}
   0x003abdac <_armv7_tick+4>:  bx      lr
   0x003abdb0 <OPENSSL_atomic_add+0>:   ldrex   r2, [r0]
   0x003abdb4 <OPENSSL_atomic_add+4>:   add     r3, r2, r1
   0x003abdb8 <OPENSSL_atomic_add+8>:   strex   r2, r3, [r0]
   0x003abdbc <OPENSSL_atomic_add+12>:  cmp     r2, #0
   0x003abdc0 <OPENSSL_atomic_add+16>:  bne     0x3abdb0 <OPENSSL_atomic_add>
   0x003abdc4 <OPENSSL_atomic_add+20>:  mov     r0, r3
End of assembler dump.
(gdb) continue
Continuing.
[New Thread 0xb6ca7450 (LWP 29165)]
[New Thread 0xb64a7450 (LWP 29166)]
[New Thread 0xb5ca7450 (LWP 29167)]
[New Thread 0xb54a7450 (LWP 29168)]

Program received signal SIGILL, Illegal instruction.
0x42783070 in ?? ()
(gdb) disassemble $pc,$pc+32
Dump of assembler code from 0x42783070 to 0x42783090:
=> 0x42783070:  movt    r12, #16368     ; 0x3ff0
   0x42783074:  vmov.32 d3[1], r12
   0x42783078:  vadd.f64        d1, d1, d3
   0x4278307c:  lsr     r4, r5, #11
   0x42783080:  ldr     r12, [pc, #68]  ; 0x427830cc
   0x42783084:  and     r5, r5, r12
   0x42783088:  ldr     r12, [pc, #64]  ; 0x427830d0
   0x4278308c:  add     r4, r4, r12
End of assembler dump.```

@bnoordhuis
Copy link
Member

Ah, movt again? I'm not sure how that can happen, that should have been fixed as of 297cadb. Can you post the contents of /proc/cpuinfo? What happens when you pass --noenable_movw_movt on the command line?

@mscdex mscdex added v8 engine Issues and PRs related to the V8 dependency. confirmed-bug Issues with confirmed bugs. labels Apr 8, 2015
@tberthe
Copy link
Author

tberthe commented Apr 8, 2015

Of course:

$ cat /proc/cpuinfo
processor       : 0
model name      : ARMv6-compatible processor rev 7 (v6l)
Features        : swp half thumb fastmult vfp edsp java tls 
CPU implementer : 0x41
CPU architecture: 7
CPU variant     : 0x0
CPU part        : 0xb76
CPU revision    : 7

Hardware        : BCM2708
Revision        : 000e
Serial          : 00000000afbc086e

Using --noenable_movw_movt does not change anything: illegal instruction and the gdb output is the same.

@bnoordhuis
Copy link
Member

Thanks. I can't think of a plausible reason why it's failing.

@rvagg Can I get (or do I still have) access to the rpi1 buildbot?

@silverwind
Copy link
Contributor

Confirmed on my RPi1, RPi2 is fine.

@silverwind
Copy link
Contributor

Might have a partial stack there:

(gdb) bt
#0  0x2b481070 in ?? ()
#1  0x00422bb0 in v8::internal::fast_exp(double) ()
#2  0x00722df8 in v8::internal::Runtime_MathExpRT(int, v8::internal::Object**, v8::internal::Isolate*) ()
#3  0x2eb08090 in ?? ()
Backtrace stopped: previous frame identical to this frame (corrupt stack?)

@bnoordhuis
Copy link
Member

Ah, interesting. What happens when you run with --nofast_math?

@silverwind
Copy link
Contributor

Works:

$ iojs --nofast_math -p "Math.exp(-1)"
0.36787944117144233

@bnoordhuis
Copy link
Member

Okay, good to know. Just to be sure, is the offending instruction in the top stack frame a movt for you as well or is it e.g. a vmov?

@silverwind
Copy link
Contributor

Looks pretty much the same as @tberthe's:

(gdb) disassemble $pc,$pc+32
Dump of assembler code from 0x2b481070 to 0x2b481090:
=> 0x2b481070:  movt    r12, #16368 ; 0x3ff0
   0x2b481074:  vmov.32 d3[1], r12
   0x2b481078:  vadd.f64    d1, d1, d3
   0x2b48107c:  lsr r4, r5, #11
   0x2b481080:  ldr r12, [pc, #68]  ; 0x2b4810cc
   0x2b481084:  and r5, r5, r12
   0x2b481088:  ldr r12, [pc, #64]  ; 0x2b4810d0
   0x2b48108c:  add r4, r4, r12

This is from a core dump. For some reason, I can't get it to crash while live debugging, I blame it on my shell.

@silverwind
Copy link
Contributor

Tried all Math function, also crashes on

  • Math.tanh(.5)
  • Math.cosh(.5)

Math.exp() crashes without an argument, above two need a number argument to crash.

@silverwind
Copy link
Contributor

The other two functions show the exact same crash signature, so they're just calling into fast_exp.

@silverwind
Copy link
Contributor

Also, current v8 options:

$ iojs --v8-options | head -2
target arm v6 vfp2 hard
ARMv7=0 VFP3=0 VFP32DREGS=0 NEON=0 SUDIV=0 UNALIGNED_ACCESSES=0
MOVW_MOVT_IMMEDIATE_LOADS=0 COHERENT_CACHE=0 USE_EABI_HARDFLOAT=1

@bnoordhuis
Copy link
Member

The code in upstream V8 seems to be identical so it's probably best to file an issue in the V8 bug tracker.

@silverwind
Copy link
Contributor

This is how far I got. Still interested in finding the cause myself, but v8 guys might be faster to find this ;)

@bnoordhuis
Copy link
Member

V8 issue: https://code.google.com/p/v8/issues/detail?id=4019

/cc @Fishrock123 - this and the --nofast_math workaround should probably be called out in the release notes.

@Fishrock123
Copy link
Contributor

@bnoordhuis You mean in future release notes? Sure.

@rvagg
Copy link
Member

rvagg commented Apr 9, 2015

@bnoordhuis still want to play with this?

ssh [email protected] -p 2224

The network is slow on these but there is a reference clone of io.js so you can use --reference git/io.js.reference/ when you're cloning to speed it up significantly. ccache is also used and should be well primed for builds so you can PATH=/usr/lib/ccache/:$PATH to use that.

@rvagg
Copy link
Member

rvagg commented Apr 9, 2015

(updated comment to be ssh [email protected] -p 2224, I have iojs-www as an ssh alias sorry)

@silverwind
Copy link
Contributor

@bnoordhuis let me know if above ssh works for you 😉

@bnoordhuis
Copy link
Member

Someone from ARM promised to look at it today so I'll wait for his findings.

@silverwind Go ahead and delete my account on your rpi. And thanks again. :-)

@silverwind
Copy link
Contributor

@bnoordhuis thanks 😉

I think it's worth to float a patch for this until the fix lands in v8.

bnoordhuis added a commit to bnoordhuis/io.js that referenced this issue Apr 11, 2015
The "fast" implementation of Math.exp() and Math.tanh() emits ARMv7
movt/movw instructions on ARMv6 (notably, the original Raspberry Pi.)

Disable fast math for now.  The adventurous can enable it again with
the --fast_math switch.

Refs: nodejs#1376
V8-Bug: https://code.google.com/p/v8/issues/detail?id=4019
bnoordhuis added a commit to bnoordhuis/io.js that referenced this issue Apr 11, 2015
The "fast" implementation of Math.exp() and Math.tanh() emits ARMv7
movt/movw instructions on ARMv6 (notably, the original Raspberry Pi.)

Disable fast math for now.  The adventurous can enable it again with
the --fast_math switch.

PR-URL: nodejs#1398
Refs: nodejs#1376
Reviewed-By: Roman Reiss <[email protected]>
V8-Bug: https://code.google.com/p/v8/issues/detail?id=4019
rvagg added a commit that referenced this issue Apr 14, 2015
Notable changes:

* C++ API: Fedor Indutny contributed a feature to V8 which has been
  backported to the V8 bundled in io.js. SealHandleScope allows a C++
  add-on author to seal a HandleScope to prevent further, unintended
  allocations within it. Currently only enabled for debug builds of
  io.js. This feature helped detect the leak in #1075 and is now
  activated on the root HandleScope in io.js. (Fedor Indutny) #1395.
* ARM: This release includes significant work to improve the state of
  ARM support for builds and tests. The io.js CI cluster's ARMv6,
  ARMv7 and ARMv8 build servers are now all (mostly) reporting passing
  builds and tests.
  - ARMv8 64-bit (AARCH64) is now properly supported, including a
    backported fix in libuv that was mistakenly detecting the
    existence of `epoll_wait()`. (Ben Noordhuis) #1365.
  - ARMv6: #1376 reported a problem with Math.exp() on ARMv6 (incl
    Raspberry Pi). The culprit is erroneous codegen for ARMv6 when
    using the "fast math" feature of V8. --nofast_math has been turned
    on for all ARMv6 variants by default to avoid this, fast math can
    be turned back on with --fast_math. (Ben Noordhuis) #1398.
  - Tests: timeouts have been tuned specifically for slower platforms,
    detected as ARMv6 and ARMv7. (Roman Reiss) #1366.
* npm: Upgrade npm to 2.7.6. See the release notes
  (https://github.com/npm/npm/releases/tag/v2.7.6) for details.
@brendanashworth
Copy link
Contributor

Is this fixed (and can be closed) or are we waiting for the v8 update on this?

@bnoordhuis
Copy link
Member

We're working around it now by disabling fast math. The core issue (v8:4019) is still unfixed.

@indutny
Copy link
Member

indutny commented Jun 2, 2015

Is it still happening?

@silverwind
Copy link
Contributor

Yes, of course:

$ iojs --fast-math -p "Math.exp(-1)"
[1]    2773 illegal hardware instruction (core dumped)  iojs --fast-math -p "Math.exp(-1)"

Patch has landed it seems:

https://chromium.googlesource.com/v8/v8/+/81703350bbb9923d211fe5b79e90bd458b0916e2%5E!/#F0

@trevnorris
Copy link
Contributor

For reference, the first V8 version that patch is available is 4.5.24.

@brendanashworth brendanashworth added the arm Issues and PRs related to the ARM platform. label Jul 13, 2015
targos added a commit to targos/node that referenced this issue Aug 31, 2015
targos added a commit that referenced this issue Sep 3, 2015
@ronkorving
Copy link
Contributor

So this has been resolved now?

rvagg pushed a commit to rvagg/io.js that referenced this issue Sep 4, 2015
@targos
Copy link
Member

targos commented Sep 4, 2015

Yes, it has been resolved by the V8 4.5 upgrade. Fast math is back with 074315f

@targos targos closed this as completed Sep 4, 2015
targos added a commit that referenced this issue Sep 6, 2015
targos added a commit that referenced this issue Sep 6, 2015
targos added a commit that referenced this issue Sep 6, 2015
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
arm Issues and PRs related to the ARM platform. confirmed-bug Issues with confirmed bugs. v8 engine Issues and PRs related to the V8 dependency.
Projects
None yet
Development

No branches or pull requests