darwin: scdaemon built with gnupg21 crashes on startup

[jwiegley]

If you build gnupg21 and then run:

~/.nix-profile/libexec/scdaemon --server --verbose --no-detach

It will immediately die with Segmentation fault: 11. Here is the stack trace:

(lldb) run
scdaemon[67198]: handler for fd -1 started
Process 67198 stopped
* thread #2: tid = 0x3a25f, 0x00007fff86bda4e9 libobjc.A.dylib`objc_msgSend + 41, stop reason = EXC_BAD_ACCESS (code=1, address=0x0)
    frame #0: 0x00007fff86bda4e9 libobjc.A.dylib`objc_msgSend + 41
libobjc.A.dylib`objc_msgSend:
->  0x7fff86bda4e9 <+41>: cmpq   (%r10), %rsi
    0x7fff86bda4ec <+44>: jne    0x7fff86bda4f2            ; <+50>
    0x7fff86bda4ee <+46>: jmpq   *0x8(%r10)
    0x7fff86bda4f2 <+50>: cmpq   $0x1, (%r10)

Process 67198 launched: '/Users/johnw/.nix-profile/libexec/scdaemon' (x86_64)
(lldb) bt
* thread #2: tid = 0x3a25f, 0x00007fff86bda4e9 libobjc.A.dylib`objc_msgSend + 41, stop reason = EXC_BAD_ACCESS (code=1, address=0x0)
  * frame #0: 0x00007fff86bda4e9 libobjc.A.dylib`objc_msgSend + 41
    frame #1: 0x00007fff998e6adc IOKit`IORegistryEntrySearchCFProperty + 46
    frame #2: 0x00000001001a7bde libusb-1.0.0.dylib`process_new_device + 94
    frame #3: 0x00000001001a268a libusb-1.0.0.dylib`darwin_init + 234
    frame #4: 0x000000010019d22f libusb-1.0.0.dylib`libusb_init + 559
    frame #5: 0x0000000100010643 scdaemon`ccid_open_reader + 99
    frame #6: 0x0000000100007e85 scdaemon`apdu_open_reader + 133
    frame #7: 0x0000000100003ee7 scdaemon`scd_command_handler + 471
    frame #8: 0x0000000100003544 scdaemon`start_connection_thread + 148
    frame #9: 0x0000000100195f8e libnpth.0.dylib`thread_start + 30
    frame #10: 0x00007fff8fb9399d libsystem_pthread.dylib`_pthread_body + 131
    frame #11: 0x00007fff8fb9391a libsystem_pthread.dylib`_pthread_start + 168
    frame #12: 0x00007fff8fb91351 libsystem_pthread.dylib`thread_start + 13

This prevents smart cards from being used with GnuPG 2.1. I'm able to use scdaemon from GPGTools just fine with Nix's gnupg21. Here are the binaries linked against Nix's scdaemon:

15:00:53 Vulcan:~/bae/xhtml-deliverable $ otool -L ~/.nix-profile/libexec/scdaemon
/Users/johnw/.nix-profile/libexec/scdaemon:
	/nix/store/bz5pf8mmfknv85w64xdvx0qbzvdsjqcb-libgcrypt-1.7.3/lib/libgcrypt.20.dylib (compatibility version 22.0.0, current version 22.3.0)
	/nix/store/i8a9vwkf7ijzkj8q5h051bal2wmqpisv-libgpg-error-1.24/lib/libgpg-error.0.dylib (compatibility version 20.0.0, current version 20.1.0)
	/nix/store/qmgn7wq1psm8mkvab8bn60pk2fs10x2k-libksba-1.3.5/lib/libksba.8.dylib (compatibility version 20.0.0, current version 20.6.0)
	/nix/store/dgh61053wlc50zil8hswafzg29b04xac-libassuan-2.4.3/lib/libassuan.0.dylib (compatibility version 8.0.0, current version 8.3.0)
	/nix/store/kmjlaad8ddlsd6mbrqmximgdrwk2rdj4-npth-1.2/lib/libnpth.0.dylib (compatibility version 1.0.0, current version 1.5.0)
	/nix/store/p6r0hq4drx20nic9b8hdnpxdghjm3xjb-libusb-1.0.20/lib/libusb-1.0.0.dylib (compatibility version 2.0.0, current version 2.0.0)
	/nix/store/wg557dq764j8x4q49c01vczs8fl4z5sn-CF-osx-10.9.5/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation (compatibility version 150.0.0, current version 855.15.0)
	/System/Library/Frameworks/IOKit.framework/Versions/A/IOKit (compatibility version 1.0.0, current version 275.0.0)
	/nix/store/vii41bjljhdsa655yww4jcymnibhv039-gettext-0.19.8/lib/libintl.9.dylib (compatibility version 11.0.0, current version 11.4.0)
	/nix/store/wr4xak6fw0prhdx8qbaf8pijcp1iyajm-libiconv-osx-10.11.6/lib/libiconv.dylib (compatibility version 7.0.0, current version 7.0.0)
	/nix/store/a131zkygxrk04qs2kl3vq83z1wnhh03c-Libsystem-osx-10.11.6/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1226.10.1)

[copumpkin]

Try this magic:

DYLD_FRAMEWORK_PATH=/System/Library/Frameworks result/libexec/scdaemon --server --verbose --no-detach

There's a not-entirely-obscene way to automate this but it takes some work.

[jwiegley]

@copumpkin That works! So what does this mean?

Last login: Thu Nov 17 13:34:40 on ttys012
13:41:51 Vulcan:~ $ ~/.nix-profile/libexec/scdaemon --server --verbose --no-detach
scdaemon[73507]: handler for fd -1 started
Segmentation fault: 11
13:42:00 Vulcan:~ $ DYLD_FRAMEWORK_PATH=/System/Library/Frameworks ~/.nix-profile/libexec/scdaemon --server --verbose --no-detach
scdaemon[73533]: handler for fd -1 started
scdaemon[73533]: detected reader 'Yubico Yubikey 4 OTP+U2F+CCID'
scdaemon[73533]: reader slot 0: not connected
OK GNU Privacy Guard's Smartcard server readyscdaemon[73533]: updating reader 0 (0) status: 0x0000->0x0007 (0->1)

[jwiegley]

Here's the output from scdaemon 2.0 (from GPGTools):

13:47:16 Vulcan:~ $ /usr/local/MacGPG2/libexec/scdaemon --server --verbose --no-detach
scdaemon[75260]: handler for fd -1 started
scdaemon[75260]: PC/SC OPEN failed: sharing violation (0x8010000b)
OK GNU Privacy Guard's Smartcard server ready

[LnL7]

@copumpkin any idea what the issue is here or how to solve it properly?

[copumpkin]

The issue is that we have two simultaneous versions of CoreFoundation.framework, one built purely, and the other one brought in transitively because we depend on an impure framework from the outside world (which in turn depends on the system CoreFoundation). I can't remember if that's an automatic recipe for failure (they do stuff at startup to global things, and might conflict there) or if things only go south if you pass a CF value allocated with one of the libraries to the other, but it generally gets unpleasant.

I think the best solution we have for now is roughly as follows:

1. Fix wrapProgram to not screw things up if you apply more than one wrapper to the same program (see here)
2. Write a hook for all our macOS SDK frameworks that automatically adds a "fixup" step to anyone depending on them, where the step basically calls wrapProgram on everything in bin to pass in DYLD_FRAMEWORK_PATH as we did above
3. Allow people to disable A small set of package updates #2 if it's annoying or to invoke it by hand if there are programs outside of bin

[matthewbauer]

@copumpkin
There's got to be a better way to do this. Is there no way to set DYLD_FRAMEWORK_PATH at the linker level? That would make things more automatic.

I'm pretty weary of the "wrappers upon wrappers" stuff. Even if it works, I like having actual binaries in /bin/ as opposed to scripts.

[copumpkin]

@matthewbauer I don't know how to do it, if it's possible. If Apple allowed a custom dyld (it used to up until 10.11) we'd have more options, but I don't think there's much else.

[pikajude]

I'd rather us do what we do for other system frameworks and turn CF into a skeleton derivation of headers with a symlink to the real dylib. I don't see the purpose of a CF library that will reliably exhibit this kind of behavior, purity-wise, especially if the accepted solution is to "just use the system one" which conflicts with the purity goal to begin with.

[copumpkin]

@pikajude well, the difference is which programs you use the system one on. With the pure one, we get to have a pure(r) ruby, vim, etc., and only the things that are impure for other reasons (i.e., need other impure frameworks) turn impure with CF as well.

It might still not be worth the pain, but I wouldn't cast them as quite the same thing.

[copumpkin]

A core part of "my view" for Nix for darwin is that I can pretty much use most "unixy" tools (i.e., things that aren't overly Mac-specific or have deep integration with the OS) with minimal impurities. As far as I know, we currently just have the kernel, libSystem, dyld, /bin/sh, and /usr/bin/env for the majority of the sorts of tools you'd find in homebrew. Homebrew cask is a whole other story, and is where this sort of CF impurity starts mattering. Of course there are a few awkward overlaps like this GPG tool that are materially worse under this world than an impurer one, and while I think we do have general solutions to it, they're not very appealing for now.

[LnL7]

I'm working on a solution for this that fixes the dylibs, it looks like only a few of the libraries like libusb1 will need the setup hook.

[matthewbauer]

Is this confirmed fixed?

[LnL7]

I only tested it on a machine without xcode so that was without pinentry, but I'm pretty sure libusb1 was the problem.

[periklis]

@matthewbauer & @LnL7 i am using gnupg with scdaemon 2.1.19 and pinentry-mac-0.9.4 from nixpkgs/master. It works like a charm without DYLD_FRAMEWORK_PATH.

[LnL7]

Sweet!