Cargo 0.14.0-nightly issue

[rivaldealer]

Recently installing Rust nightly on Mac OS X 10.11.6, I came across an issue with cargo. Unfortunately this error comes up every time I run cargo

Illegal instruction: 4

However, I've installed cargo 0.13.0-nightly (eca9e15 2016-11-01) from the stable Rust pkg installer and cargo seems to be running fine. I'm not sure what might be causing this, but I'm willing to help in any way possible to narrow it down.

[alexcrichton]

Thanks for the report! I think this will be fixed by #3332, but to be 100% sure, can you get a backtrace of the faulting instruction in the executable?

[rivaldealer]

Since rust's backtrace was unable to send out the backtrace, I've done it in lldb, hopefully this is somewhat helpful.

(lldb) thread backtrace all
* thread #1: tid = 0x11583, 0x00000001003c93ee cargo`lh_new + 196, queue = 'com.apple.main-thread', stop reason = EXC_BAD_INSTRUCTION (code=EXC_I386_INVOP, subcode=0x0)
  * frame #0: 0x00000001003c93ee cargo`lh_new + 196
    frame #1: 0x0000000100334177 cargo`OBJ_NAME_init + 48
    frame #2: 0x000000010033431e cargo`OBJ_NAME_add + 35
    frame #3: 0x00000001003d1b1e cargo`EVP_add_cipher + 42
    frame #4: 0x00000001003d32dc cargo`OpenSSL_add_all_ciphers + 19
    frame #5: 0x00000001003d32c3 cargo`OPENSSL_add_all_algorithms_noconf + 14
    frame #6: 0x00000001003132bc cargo`libssh2_init + 28
    frame #7: 0x000000010030532b cargo`git_transport_ssh_global_init + 11
    frame #8: 0x00000001002c5d7f cargo`init_once + 79
    frame #9: 0x00007fff8557abf6 libsystem_pthread.dylib`__pthread_once_handler + 65
    frame #10: 0x00007fff9014dfc4 libsystem_platform.dylib`_os_once + 41
    frame #11: 0x00007fff8557ab95 libsystem_pthread.dylib`pthread_once + 57
    frame #12: 0x00000001002c5d09 cargo`git_libgit2_init + 57
    frame #13: 0x00000001002985a8 cargo`std::sync::once::Once::call_once::_$u7b$$u7b$closure$u7d$$u7d$::heb5e79581d08779c + 24
    frame #14: 0x000000010045e936 cargo`std::sync::once::Once::call_inner::h9e2528dcbe65c87c + 438
    frame #15: 0x000000010029b62c cargo`git2::config::Config::open_default::h167c66de9b2bb1bc + 428
    frame #16: 0x0000000100190f73 cargo`cargo::ops::registry::http_proxy::he6e5b7da7a9c692c + 131
    frame #17: 0x00000001001910bd cargo`cargo::ops::registry::http_proxy_exists::h0318015e29232b61 + 29
    frame #18: 0x000000010001bb16 cargo`cargo::execute::h5572ff483e8f5999 + 182
    frame #19: 0x0000000100015a88 cargo`cargo::call_main_without_stdin::hf248bc49460e4199 + 3384
    frame #20: 0x000000010001b76f cargo`cargo::main::hba646a3e23cce277 + 1231
    frame #21: 0x000000010046991b cargo`__rust_maybe_catch_panic + 27
    frame #22: 0x0000000100468e17 cargo`std::rt::lang_start::h5d71a3afaaa4b2ff + 391
    frame #23: 0x0000000100001734 cargo`start + 52

[alexcrichton]

Awesome, thanks! Looks like it is indeed OpenSSL which I suspected. Could you also run disas to get a disassembly of the function running and peek at the actual faulting instruction?

[rivaldealer]

Oh, right I forgot the disassembly...
The original log:

* thread #1: tid = 0x11583, 0x00000001003c93ee cargo`lh_new + 196, queue = 'com.apple.main-thread', stop reason = EXC_BAD_INSTRUCTION (code=EXC_I386_INVOP, subcode=0x0)
    frame #0: 0x00000001003c93ee cargo`lh_new + 196
cargo`lh_new:
->  0x1003c93ee <+196>: vxorps %xmm0, %xmm0, %xmm0
    0x1003c93f2 <+200>: vmovups %xmm0, 0x48(%rbx)
    0x1003c93f7 <+205>: vmovups %xmm0, 0x38(%rbx)
    0x1003c93fc <+210>: vmovups %xmm0, 0x68(%rbx)

And yes sir, I'd be glad to; here's the full disas as requested:

cargo`lh_new:
    0x1003c932a <+0>:   pushq  %rbp
    0x1003c932b <+1>:   movq   %rsp, %rbp
    0x1003c932e <+4>:   pushq  %r15
    0x1003c9330 <+6>:   pushq  %r14
    0x1003c9332 <+8>:   pushq  %rbx
    0x1003c9333 <+9>:   pushq  %rax
    0x1003c9334 <+10>:  movq   %rsi, %r15
    0x1003c9337 <+13>:  movq   %rdi, %r14
    0x1003c933a <+16>:  leaq   0x1b78bc(%rip), %rsi      ; "lhash.c"
    0x1003c9341 <+23>:  movl   $0xb0, %edi
    0x1003c9346 <+28>:  movl   $0x78, %edx
    0x1003c934b <+33>:  callq  0x100332d57               ; CRYPTO_malloc
    0x1003c9350 <+38>:  movq   %rax, %rbx
    0x1003c9353 <+41>:  xorl   %eax, %eax
    0x1003c9355 <+43>:  testq  %rbx, %rbx
    0x1003c9358 <+46>:  je     0x1003c9426               ; <+252>
    0x1003c935e <+52>:  leaq   0x1b7898(%rip), %rsi      ; "lhash.c"
    0x1003c9365 <+59>:  movl   $0x80, %edi
    0x1003c936a <+64>:  movl   $0x7a, %edx
    0x1003c936f <+69>:  callq  0x100332d57               ; CRYPTO_malloc
    0x1003c9374 <+74>:  movq   %rax, (%rbx)
    0x1003c9377 <+77>:  xorl   %ecx, %ecx
    0x1003c9379 <+79>:  testq  %rax, %rax
    0x1003c937c <+82>:  jne    0x1003c9393               ; <+105>
    0x1003c937e <+84>:  movq   %rbx, %rdi
    0x1003c9381 <+87>:  callq  0x100332f81               ; CRYPTO_free
    0x1003c9386 <+92>:  xorl   %eax, %eax
    0x1003c9388 <+94>:  jmp    0x1003c9426               ; <+252>
    0x1003c938d <+99>:  incq   %rcx
    0x1003c9390 <+102>: movq   (%rbx), %rax
    0x1003c9393 <+105>: movq   $0x0, (%rax,%rcx,8)
    0x1003c939b <+113>: cmpq   $0xf, %rcx
    0x1003c939f <+117>: jne    0x1003c938d               ; <+99>
    0x1003c93a1 <+119>: testq  %r15, %r15
    0x1003c93a4 <+122>: cmoveq 0x216d7c(%rip), %r15      ; (void *)0x00007fff9014e800: _platform_strcmp
    0x1003c93ac <+130>: movq   %r15, 0x8(%rbx)
    0x1003c93b0 <+134>: testq  %r14, %r14
    0x1003c93b3 <+137>: leaq   0x77(%rip), %rax          ; lh_strhash
    0x1003c93ba <+144>: cmovneq %r14, %rax
    0x1003c93be <+148>: movq   %rax, 0x10(%rbx)
    0x1003c93c2 <+152>: movl   $0x8, 0x18(%rbx)
    0x1003c93c9 <+159>: movl   $0x10, 0x1c(%rbx)
    0x1003c93d0 <+166>: movl   $0x0, 0x20(%rbx)
    0x1003c93d7 <+173>: movl   $0x8, 0x24(%rbx)
    0x1003c93de <+180>: movq   $0x200, 0x28(%rbx)        ; imm = 0x200 
    0x1003c93e6 <+188>: movq   $0x100, 0x30(%rbx)        ; imm = 0x100 
->  0x1003c93ee <+196>: vxorps %xmm0, %xmm0, %xmm0
    0x1003c93f2 <+200>: vmovups %xmm0, 0x48(%rbx)
    0x1003c93f7 <+205>: vmovups %xmm0, 0x38(%rbx)
    0x1003c93fc <+210>: vmovups %xmm0, 0x68(%rbx)
    0x1003c9401 <+215>: vmovups %xmm0, 0x58(%rbx)
    0x1003c9406 <+220>: vmovups %xmm0, 0x88(%rbx)
    0x1003c940e <+228>: vmovups %xmm0, 0x78(%rbx)
    0x1003c9413 <+233>: vmovups %xmm0, 0x9c(%rbx)
    0x1003c941b <+241>: vmovups %xmm0, 0x8c(%rbx)
    0x1003c9423 <+249>: movq   %rbx, %rax
    0x1003c9426 <+252>: addq   $0x8, %rsp
    0x1003c942a <+256>: popq   %rbx
    0x1003c942b <+257>: popq   %r14
    0x1003c942d <+259>: popq   %r15
    0x1003c942f <+261>: popq   %rbp
    0x1003c9430 <+262>: retq   

[alexcrichton]

Ok perfect, thanks! Indeed looks like some new SIMD instruction is being used when it shouldn't. I believe this happened as the OpenSSL we used was accidentally compiled with -march=native, meaning it was tuned for the CPU of the building machine. Presumably the building machine is a newer CPU than the one you're running which then causes this fault.

In that case I believe this is closed by #3332, so I'm going to close in favor of that.