Add --inputs-only to nix copy

[iFreilicht]

Is your feature request related to a problem? Please describe.

In this neverending saga of a thread, there is a seemingly simple objective: copy the closure of a NixOS system configuration to another machine, not via ssh, but via a file, and make that file as small as possible.

The best Nix can currently do is to copy the entire closure, including all outputs. You would think that either nix-copy-closure or nix copy can do better than that, but no:

• nix-copy-closure only copies inputs by default, but it can only copy via ssh, not to a local binary cache.
• nix copy does support a file:// URL for the --to and --from arguments, but copies the entire closure with outputs by default, and --substitute-on-destination is obviously only possible on ssh stores.
• nix-store --export and nix-store --import are ~enough to kind of do this, but you would basically have to build two big scripts around them that resolve dependency graphs themselves. --import apparently also requires inputs to be signed, which is annoying and not documented in its manpage.
• nix-store -q -R + nix derivation + cp + nix store add-path are in a similar boat. You can probably build a big script that does this, but it's cumbersome.

This is very dissatisfying. Nix should be very good at solving this problem efficiently and quickly, but currently the frontend offers no solution.

Describe the solution you'd like

Add an --inputs-only argument to nix copy that will only copy the closure of all derivations and inputs required to build the installables argument, no matter what type the --to argument is.

This would enable one to nix copy --to file:///media/drive/cache --inputs-only, zip that directory, bring it to another machine in any conceivable way, nix copy --from file:///media/drive/cache, and then nix build the toplevel derivation.

There are some cases where this isn't possible. For example, the current implementation of profiles does not create a derivation in the store for every profile generation, so only copying the inputs is not possible.

Describe alternatives you've considered

It might be possible to add support for local binary caches to nix-copy-closure instead, which would solve the same problem.

A separate tool (potentially called nix copy-closure) could also be written for this. nix copy is currently a very thin wrapper around the store API, so it might not be desirable to extend its functionality too much. The advantage here might be that it can be dedicated to copying a closure to a directory, instead of requiring the somewhat surprising file:// prefix.

Additional context
See the thread linked above.

I'm willing to attempt an implementation of this myself.

Priorities

Add 👍 to issues you find important.

[nixos-discourse]

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/compile-nixos-derivations-into-a-single-file-for-a-single-host/30920/58

[mark-c-a]

Thank you @iFreilicht for working on this. I'd like to document how this new feature may work.

CLI example how it might work

• on a "builder" machine, you can build (but not activate) NixOS configuration for the "builder" or for another host. In this case for foobar host.

nix build --no-link --print-out-paths .#nixosConfigurations.foobar.config.system.build.toplevel

• this will return for example: /nix/store/hbwb7jyvd91iqz6f8g7bfddq3xhjiss6-nixos-system-foobar-23.05.20230805.9607b91
• then it will be possible to capture the NixOS configuration without copying source *.nix files that were used to build it

nix copy --inputs-only --to file:///tmp/file /nix/store/hbwb7jyvd91iqz6f8g7bfddq3xhjiss6-nixos-system-foobar-23.05.20230805.9607b91
# or
nix copy --inputs-only --to - /nix/store/hbwb7jyvd91iqz6f8g7bfddq3xhjiss6-nixos-system-foobar-23.05.20230805.9607b91 | gzip > /tmp/file.gz

• now the file can be backed, archived or copied to a target host foobar
• after copying the file to /tmp:

nix copy --from file:///tmp/file
# or
gunzip --stdout /tmp/file.gz | nix copy --from -

• now this can be build the same way as nixos-rebuild would do

nix build /nix/store/hbwb7jyvd91iqz6f8g7bfddq3xhjiss6-nixos-system-foobar-23.05.20230805.9607b91

• and activated

/nix/store/hbwb7jyvd91iqz6f8g7bfddq3xhjiss6-nixos-system-foobar-23.05.20230805.9607b91/bin/switch-to-configuration boot
reboot

Why to add such functionality

• You want to configure untrusted VPS in Cloud but you don't want to copy all your source code with a flake that has configuration for many other hosts. Such source code can contain a lot of sensitive data like for example initial user passwords, user password hashes, Wi-Fi passwords, IP addresses, firewall settings, API access keys and a lot of comments documenting your whole infrastructure. And nothing from that might be relevant to that single VPS.
• Your might not be able to have SSH access so nix copy --substitute-on-destination can't be used. But someone / something else has access but again you don't want to give the individual/system the source code with a lot of sensitive data.
• You might want to capture (into a file) all state / configuration that can be then easily restored (build) on the same host or other host with the same hardware. The configuration might use flakes but it might also have flake inputs from other flake repositories (in other words your main flake has inputs from other local filesystem flakes). It is easy to capture a configuration from a single flake since you can just store commit ID (and commnit has flake.lock) but this gets difficult when you have other repos in your main flake inputs.

[nixos-discourse]

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/compile-nixos-derivations-into-a-single-file-for-a-single-host/30920/59

[Ericson2314]

You can nix copy a derivation and its closure. What's wrong with that? I am lost.

[iFreilicht]

Ohhh it seems I was confused about system configurations not having a deriver because this is the case for profiles. I'll continue the discussion in the linked thread and close this if that is the solution.

[iFreilicht]

We finally found that the issue was caused by an outdated version of Nix. Like @Ericson2314 said, nix copy with a derivation works perfectly fine, and we found it to work properly in Nix 2.15.1 and 2.17.0, but not in 2.13.3. We didn't test any other versions.

This means the functionality requested by this ticket is offered by the current version of nix.