Nix install failed on macOS 11.2.3 with FileVault and a pre-existing unencrypted Nix Store volume

[rdparker]

After installing nix using the old method and realizing the created volume was unencrypted I attempted to reinstall using the most recent version of abathur's installer from NixOS/nix/#4289. It went through a number of steps. to remove the pre-existing install and failed mounting the disk.

A second run of the same produced a working install.

The full nix install log follows:

% sh <(curl https://abathur-nix-install-tests.cachix.org/serve/7sqcns3hn4brpm53j1afkmmx8im6zjb2/install) --tarball-url-prefix https://abathur-nix-install-tests.cachix.org/serve
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  3656    0  3656    0     0   1915      0 --:--:--  0:00:01 --:--:--  1914
downloading Nix 2.4pre19700101_97269f6 binary tarball for x86_64-darwin from 'https://abathur-nix-install-tests.cachix.org/serve/nd0ijj1zp6chap43nnc5k9qwlyylidcb/nix-2.4pre19700101_97269f6-x86_64-darwin.tar.xz' to '/var/folders/nt/807f8y4j179bjf58wq36pymh0000gp/T/nix-binary-tarball-unpack.XXXXXXXXXX.usLEvzFX'...
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 25.4M    0 25.4M    0     0  1486k      0 --:--:--  0:00:17 --:--:-- 2716k
Switching to the Multi-user Installer
Welcome to the Multi-User Nix Installation

This installation tool will set up your computer with the Nix package
manager. This will happen in a few stages:

1. Make sure your computer doesn't already have Nix. If it does, I
   will show you instructions on how to clean up your old install.

2. Show you what we are going to install and where. Then we will ask
   if you are ready to continue.

3. Create the system users and groups that the Nix daemon uses to run
   builds.

4. Perform the basic installation of the Nix files daemon.

5. Configure your shell to import special Nix Profile files, so you
   can use Nix.

6. Start the Nix daemon.

Would you like to see a more detailed list of what we will do?
[y/n] n


---- let's talk about sudo -----------------------------------------------------
This script is going to call sudo a lot. Every time we do, it'll
output exactly what it'll do, and why.

Just like this:

---- sudo execution ------------------------------------------------------------
I am executing:

    $ sudo echo

to demonstrate how our sudo prompts look


This might look scary, but everything can be undone by running just a
few commands. We used to ask you to confirm each time sudo ran, but it
was too many times. Instead, I'll just ask you this one time:

Can we use sudo?
[y/n] y

Yay! Thanks! Let's get going!

~~> Fixing any leftover Nix volume state
Before I try to install, I'll check for any existing Nix volume config
and ask for your permission to remove it (so that the installer can
start fresh). I'll also ask for permission to fix any issues I spot.

---- Found existing Nix volume -------------------------------------------------
  special:	disk1s7
     uuid:	023367B4-9F80-4D37-94C9-32A97AE8FEF2
encrypted:	no

---- warning! ------------------------------------------------------------------
FileVault is on, but your Nix Store volume isn't encrypted.

Should I encrypt it and add the decryption key to your keychain?
[y/n] y

Volume Nix Store on Nix Store mounted

---- sudo execution ------------------------------------------------------------
I am executing:

    $ sudo /usr/bin/security -i

to add your Nix volume's password to Keychain

Password:

---- sudo execution ------------------------------------------------------------
I am executing:

    $ sudo /usr/sbin/diskutil apfs encryptVolume Nix Store -user disk -stdinpassphrase

to encrypt your Nix volume

Encrypting with the new "Disk" crypto user on disk1s7
The new "Disk" user will be the only one who has initial access to disk1s7
The new APFS crypto user UUID will be 023367B4-9F80-4D37-94C9-32A97AE8FEF2
Encryption has likely completed due to AES hardware; see "diskutil apfs list"
Volume Nix Store on disk1s7 force-unmounted

During install, I add 'nix' to /etc/synthetic.conf, which instructs
macOS to create an empty root directory for mounting the Nix volume.
Can I remove /etc/synthetic.conf?
[y/n] y


---- sudo execution ------------------------------------------------------------
I am executing:

    $ sudo rm /etc/synthetic.conf

to remove /etc/synthetic.conf

During install, I add '/nix' to /etc/fstab so that macOS knows what
mount options to use for the Nix volume.
Can I remove /etc/fstab?
[y/n] y


---- sudo execution ------------------------------------------------------------
I am executing:

    $ sudo rm /etc/fstab

to remove /etc/fstab


---- Nix config report ---------------------------------------------------------
        Temp Dir:	/var/folders/nt/807f8y4j179bjf58wq36pymh0000gp/T/tmp.4i3p5bRs2r
        Nix Root:	/nix
     Build Users:	32
  Build Group ID:	30000
Build Group Name:	nixbld

build users:
    Username:	UID
     _nixbld1:	301
     _nixbld2:	302
     _nixbld3:	303
     _nixbld4:	304
     _nixbld5:	305
     _nixbld6:	306
     _nixbld7:	307
     _nixbld8:	308
     _nixbld9:	309
     _nixbld10:	310
     _nixbld11:	311
     _nixbld12:	312
     _nixbld13:	313
     _nixbld14:	314
     _nixbld15:	315
     _nixbld16:	316
     _nixbld17:	317
     _nixbld18:	318
     _nixbld19:	319
     _nixbld20:	320
     _nixbld21:	321
     _nixbld22:	322
     _nixbld23:	323
     _nixbld24:	324
     _nixbld25:	325
     _nixbld26:	326
     _nixbld27:	327
     _nixbld28:	328
     _nixbld29:	329
     _nixbld30:	330
     _nixbld31:	331
     _nixbld32:	332

Ready to continue?
[y/n] y


---- Preparing a Nix volume ----------------------------------------------------
    Nix traditionally stores its data in the root directory /nix, but
    macOS now (starting in 10.15 Catalina) has a read-only root directory.
    To support Nix, I will create a volume and configure macOS to mount it
    at /nix.

~~> Configuring /etc/synthetic.conf to make a mount-point at /nix

---- sudo execution ------------------------------------------------------------
I am executing:

    $ sudo /usr/bin/ex /etc/synthetic.conf

to add Nix to /etc/synthetic.conf


~~> Creating a Nix volume

~~> Configuring /etc/fstab to specify volume mount options

---- sudo execution ------------------------------------------------------------
I am executing:

    $ sudo /usr/sbin/vifs

to add nix to fstab

Volume on disk1s7 failed to mount
This is an encrypted and locked APFS Volume; use "diskutil apfs unlockVolume"

---- oh no! --------------------------------------------------------------------
Jeeze, something went wrong. If you can take all the output and open
an issue, we'd love to fix the problem so nobody else has this issue.

:(

We'd love to help if you need it.

You can open an issue at https://github.com/nixos/nix/issues

Or feel free to contact the team:
 - IRC: in #nixos on irc.freenode.net
 - twitter: @nixos_org
 - forum: https://discourse.nixos.org

[abathur]

I was able to reproduce this on a spare system. I've pushed a commit to #4289 which should fix this issue.

I've also updated that PR with the latest installer.

cc @domenkozar @toonn