Install reported successful despite failed daemon startup #742
Comments
|
This is super interesting. We test regularly on a fleet of macs, but they're in a rack and don't use external storage. Thanks for such a good log / record of what you're seeing, we'll hunt this down. |
|
Notes from a mac on AWS, running macOS Sonoma 14.1 on an mac2-m2.metal, which I'd expect to exhibit similar symptoms. |
|
Not that I've used other macs on AWS withtout this problem, so if you have trouble reproducing, make sure to use the exact same instance type and OS version. |
|
I could replicate this with the host above. Here's the relevant part of the plan: {
"version": "0.15.1",
"actions": [
{
"action": {
"action": "create_apfs_volume",
"disk": "disk5",
"name": "Nix Store",
"case_sensitive": false,
"encrypt": false,
"create_or_append_synthetic_conf": {
"action": {
"path": "/etc/synthetic.conf",
"user": null,
"group": null,
"mode": null,
"buf": "nix\n",
"position": "End"
},
"state": "Uncompleted"
},
"create_synthetic_objects": {
"action": null,
"state": "Uncompleted"
},
"unmount_volume": {
"action": {
"disk": "disk5",
"name": "Nix Store"
},
"state": "Uncompleted"
},
"create_volume": {
"action": {
"disk": "disk5",
"name": "Nix Store",
"case_sensitive": false
},
"state": "Uncompleted"
},
"create_fstab_entry": {
"action": {
"apfs_volume_label": "Nix Store",
"existing_entry": "None"
},
"state": "Uncompleted"
},
"encrypt_volume": null,
"setup_volume_daemon": {
"action": {
"path": "/Library/LaunchDaemons/org.nixos.darwin-store.plist",
"apfs_volume_label": "Nix Store",
"mount_service_label": "org.nixos.darwin-store",
"mount_point": "/nix",
"encrypt": false,
"needs_bootout": false
},
"state": "Uncompleted"
},
...and I got very similar output: |
|
I have solved this, but it'll take time to get it integrated into the installation flow. I've created a signed program that shims access to the Nix daemon. This gets us half way there. It has a caveat, though: The machine must grant Full Disk Access to that program. This can be done in two ways:
This is basically the only way, by design by Apple. Their security is good! :). Next steps:
Big thanks to @angerman whose comments on that linked issue were right on point, and got me a lot of the way through. |
|
@grahamc Triggered by a comment in the linked issue I tried to run the daemon manually, not as a service, in a separate terminal. This workaround appeared to be successful. Does this change your assessment in any way, as to whether the shim is necessary? |
|
If you run What you can do, though that's really a bit, uh, hacky:
😒 I don't want to go there again. |
|
@angerman This was done with no graphical session. I just did ssh, sudo, nix-daemon. If that gives FDA then I don't understand why a special shim is necessary, and if that does not give FDA then I don't understand what prevents nix-daemon started as a service from working and why a shim is necessary. I'm probably wrong, just wanted to point out the manual workaround if that could give @grahamc any ideas. |
|
Unfortunately the shim is still likely to be necessary in almost all cases. The behavior of the sandbox is a bit mysterious, and different behaviors can make it be a non-issue... sometimes. The shim isn't very complicated, though, which is good news. |
|
@svenssonaxel you may be able to verify three things: Iirc if ssh-keygen-wrapper (or whatever that thing is called) has FDA, so will your ssh session as well. By default on a blank macOS install nothing has FDA permissions. |
@angerman Sorry, I don't have access to such a machine currently. |
|
I'm trying to install nix for the first time, so please excuse my ignorance. I hit the same error tonight on an Intel iMac Pro booted from an external drive, MacOS 14.6.1 Sonoma. Same errors in /var/log/nix-daemon.log. I used the command-line installer. The install script reported success, but the self-test subcommand (and "nix run nixpkgs#hello") reported I eventually traced it down to the same errors in /var/log/nix-daemon: Is this problem fixed in the graphical installer, and is that why it's a signed app? Thanks. |
|
Hey folks, this problem (which is specific to macOS on EC2) has been solved. See: https://determinate.systems/posts/unattended-nix-install-macos-aws-ec2/ |
If the daemon startup fails, the installation is still reported as successful. While this might be good for statistics, it might be better to check that the daemon has started and report failure otherwise, so that the issues can be addressed.
In this particular case, the failure looks similar to NixOS/nix#6291 after running nix-installer on an AWS mac2-m2pro.metal.
The text was updated successfully, but these errors were encountered: