How to link nix statically

[nh2]

I did a quick perf profile on my nix-instantiate that takes 2 seconds, and it suggests that most time (in fact, suspiciously too much time) is spent do_lookup_x, which seems to be shared library symbol lookup.

Is it possible to link nix-instantiate (or even all nix programs) statically?

[edolstra]

~2 seconds for dynamic linking sounds way too high. For example, on my laptop:

$ time nix-instantiate --version
nix-instantiate (Nix) 1.12pre5350_7689181e                                                                                                                                                                                                                   
                                                                                                                                                                                                                                                             
real    0m0.018s                                                                                                                                                                                                                                             
user    0m0.015s                                                                                                                                                                                                                                             
sys     0m0.003s                                                                                                                                                                                                                                             

[edolstra]

Can you provide the perf report output?

[nh2]

~2 seconds for dynamic linking sounds way too high. For example, on my laptop:

Yes, when using --version I also get immediate output. I wonder if we're paying only if the functions are used. But in any case the perf output also looks suspicious to me, so I'm wondering if I'm being tricked by something. Here's the file:

perf.data.gz

I got this from git tag 1.11.8, built with ./dev-shell --run 'make clean && configurePhase && make -j4 && make install' and then perf record --call-graph dwarf -- ~/src/nix/src/nix-instantiate/nix-instantiate ...

[knedlsepp]

Now that nix 2.0 is out with the amazing mount-namespace option in nix run, statically linked nix binaries sound really tempting. Wouldn't this imply that an unprivileged user could simply download nix and then nix run the entire nixpkgs repository? Are there any reasons why nix is not already distributed in that manner? Licensing issues for linked libraries?

[edolstra]

Nix once had a flag to build it statically but it was removed (5414b3b). I think we only used it for Cygwin.

[takeda]

I'm trying to generate rpm that installs nix so I can use it on CentOS machines. The problem I'm seeing is that nix has many dependencies that are not even available on the standard system (for example libsodium available in epel, nix uses -std=c++14 which is available in g++ 5.0 but the compiled binary will then depend on corresponding libstdc++.

It is quickly becoming a mess. Since nix is meant to be a package manager having another package manager to resolve its dependencies will degrade user experience.

[thomasjm]

It seems a static build of nix is in Nixpkgs now due to NixOS/nixpkgs@c86af90, does that solve this issue?

[Ericson2314]

#3765 needs to be fixed up. I think @matthewbauer has some tricks up his sleave for this. (do note that PR while I opened it is his work.)

[thomasjm]

Thanks @Ericson2314. I was just writing up an issue actually to describe some issues I've seen with the static Nix build as it stands today. Basically it seems to still depend on paths in /nix/store/*, so (for example) copying it into a fresh Docker container and trying to run it fails with ENOENT errors. Does that sound familiar under #3765 or should I file my issue to track? I have a sort of nice Docker repro :)

[kloenk]

Thanks @Ericson2314. I was just writing up an issue actually to describe some issues I've seen with the static Nix build as it stands today. Basically it seems to still depend on paths in /nix/store/*, so (for example) copying it into a fresh Docker container and trying to run it fails with ENOENT errors. Does that sound familiar under #3765 or should I file my issue to track? I have a sort of nice Docker repro :)

As I worked on the worked protocol I noticed that it queries for pathes over the protocol, and fails if there aren't valid (pathes like bohemgc).

[matthewbauer]

Thanks @Ericson2314. I was just writing up an issue actually to describe some issues I've seen with the static Nix build as it stands today. Basically it seems to still depend on paths in /nix/store/*, so (for example) copying it into a fresh Docker container and trying to run it fails with ENOENT errors. Does that sound familiar under #3765 or should I file my issue to track? I have a sort of nice Docker repro :)

There are still references to .nix files in /corepkgs, which are needed if you want to build things. That can be overridden by setting NIX_DATA_DIR=$PWD and doing (mkdir -p nix/corepkgs && cd nix/corepkgs && curl -O https://raw.githubusercontent.com/NixOS/nix/master/corepkgs/derivation.nix). I think @edolstra is planning to inline the rest of these files like was done in 9950cde.

You may or may not also need a "sandbox shell" which can be overridden with --option sandbox-paths /bin/sh=/bin/sh

[thomasjm]

@matthewbauer I tried both of those things and I'm still getting the same result. Pretty much all nix commands fail with

error: getting status of '/nix/store/3242w1nl4l7pkhbm9v7qr0fwc899dagw-nix-2.3.7-x86_64-unknown-linux-musl': No such file or directory

The path it's trying to find is that of the statically built nix itself. Running strings on the binary shows a number of /nix/store paths baked in:

> strings nix | grep nix/store
/nix/store/3242w1nl4l7pkhbm9v7qr0fwc899dagw-nix-2.3.7-x86_64-unknown-linux-musl
/nix/store/3242w1nl4l7pkhbm9v7qr0fwc899dagw-nix-2.3.7-x86_64-unknown-linux-musl/share
/nix/store/3242w1nl4l7pkhbm9v7qr0fwc899dagw-nix-2.3.7-x86_64-unknown-linux-musl/libexec
/nix/store/3242w1nl4l7pkhbm9v7qr0fwc899dagw-nix-2.3.7-x86_64-unknown-linux-musl/bin
/nix/store/m2ln0j0inycvxhvnh53hz2hiql05nxz0-nix-2.3.7-x86_64-unknown-linux-musl-man/share/man
/bin/sh=/nix/store/w2ydlnvl0x3mcbclcfmrlqrmbv1w2j35-busybox-1.31.1-x86_64-unknown-linux-musl/bin/busybox
/nix/store
/nix/store/zzc9z2lzlvnw1vksfqkf4j3x0b9ipfza-openssl-1.1.1g-x86_64-unknown-linux-musl/etc/ssl/ct_log_list.cnf
OPENSSLDIR: "/nix/store/zzc9z2lzlvnw1vksfqkf4j3x0b9ipfza-openssl-1.1.1g-x86_64-unknown-linux-musl/etc/ssl"
ENGINESDIR: "/nix/store/zzc9z2lzlvnw1vksfqkf4j3x0b9ipfza-openssl-1.1.1g-x86_64-unknown-linux-musl/lib/engines-1.1"
/nix/store/zzc9z2lzlvnw1vksfqkf4j3x0b9ipfza-openssl-1.1.1g-x86_64-unknown-linux-musl/lib/engines-1.1
/nix/store/zzc9z2lzlvnw1vksfqkf4j3x0b9ipfza-openssl-1.1.1g-x86_64-unknown-linux-musl/etc/ssl/private
/nix/store/zzc9z2lzlvnw1vksfqkf4j3x0b9ipfza-openssl-1.1.1g-x86_64-unknown-linux-musl/etc/ssl
/nix/store/zzc9z2lzlvnw1vksfqkf4j3x0b9ipfza-openssl-1.1.1g-x86_64-unknown-linux-musl/etc/ssl/certs

Like I said, I'm happy to provide a repro if it would be helpful.

[matthewbauer]

@matthewbauer I tried both of those things and I'm still getting the same result. Pretty much all nix commands fail with

error: getting status of '/nix/store/3242w1nl4l7pkhbm9v7qr0fwc899dagw-nix-2.3.7-x86_64-unknown-linux-musl': No such file or directory

The path it's trying to find is that of the statically built nix itself. Running strings on the binary shows a number of /nix/store paths baked in:

> strings nix | grep nix/store
/nix/store/3242w1nl4l7pkhbm9v7qr0fwc899dagw-nix-2.3.7-x86_64-unknown-linux-musl
/nix/store/3242w1nl4l7pkhbm9v7qr0fwc899dagw-nix-2.3.7-x86_64-unknown-linux-musl/share
/nix/store/3242w1nl4l7pkhbm9v7qr0fwc899dagw-nix-2.3.7-x86_64-unknown-linux-musl/libexec
/nix/store/3242w1nl4l7pkhbm9v7qr0fwc899dagw-nix-2.3.7-x86_64-unknown-linux-musl/bin
/nix/store/m2ln0j0inycvxhvnh53hz2hiql05nxz0-nix-2.3.7-x86_64-unknown-linux-musl-man/share/man
/bin/sh=/nix/store/w2ydlnvl0x3mcbclcfmrlqrmbv1w2j35-busybox-1.31.1-x86_64-unknown-linux-musl/bin/busybox
/nix/store
/nix/store/zzc9z2lzlvnw1vksfqkf4j3x0b9ipfza-openssl-1.1.1g-x86_64-unknown-linux-musl/etc/ssl/ct_log_list.cnf
OPENSSLDIR: "/nix/store/zzc9z2lzlvnw1vksfqkf4j3x0b9ipfza-openssl-1.1.1g-x86_64-unknown-linux-musl/etc/ssl"
ENGINESDIR: "/nix/store/zzc9z2lzlvnw1vksfqkf4j3x0b9ipfza-openssl-1.1.1g-x86_64-unknown-linux-musl/lib/engines-1.1"
/nix/store/zzc9z2lzlvnw1vksfqkf4j3x0b9ipfza-openssl-1.1.1g-x86_64-unknown-linux-musl/lib/engines-1.1
/nix/store/zzc9z2lzlvnw1vksfqkf4j3x0b9ipfza-openssl-1.1.1g-x86_64-unknown-linux-musl/etc/ssl/private
/nix/store/zzc9z2lzlvnw1vksfqkf4j3x0b9ipfza-openssl-1.1.1g-x86_64-unknown-linux-musl/etc/ssl
/nix/store/zzc9z2lzlvnw1vksfqkf4j3x0b9ipfza-openssl-1.1.1g-x86_64-unknown-linux-musl/etc/ssl/certs

Like I said, I'm happy to provide a repro if it would be helpful.

You may need to use this PR: #3765. I think the "getting status" issue in nix-build was fixed at some point in master.

[stale]

I marked this as stale due to inactivity. → More info