[WIP] [preview] Single static binary for Nix+deps (bash, coreutils, ...) via ALLVM #1973
Comments
|
cc #1418 |
Try it!Either add the ALLVM cache's public key as described in the musl PR or alternatively instruct Nix to use a local store and ignore signatures, super WIP instructions can be found here, which I'm planning on cleaning up soon (next few days). |
|
Can we get a "why" section here? 😀 |
|
Haha sure: And most importantly "for science!" 😁 |
|
I really like the idea of a static drop-in nix binary. No more impure install script! |
|
👍 To this! This would be helpful for a "portable Nix". Right now I have been running Nix inside a Chroot on computers that I don't have root access to. Being able to download a "nix" binary and put it in ~/.local/bin, combined with the new @dtzWill How hard would this approach be to port to MacOS or other non-Linux OS? It seems like it should be portable but there could be some Linux-specific hack going on somewhere. |
|
While having a single-binary Nix executable would be great, it seems unnecessarily complicated to include stuff like coreutils, bash, xz, ... in that executable. Most of those dependencies could be eliminated pretty easily. For example, IIRC, the only reason why we depend on the Random points:
|
|
FWIW, thought I'd share latest version:
New store path:
GLHF, will respond/follow-up more properly ASAP :). |
|
So, I've figured out you can get "single file" executables through "nix-bundle" (with the help of Arx). You can try this out right now, by running this bundle: This is from the Musl branch of nix-bundle: You still need unshare unfortunately- but it's fairly small with Musl! Current build is ~10MB. |
|
FWIW, some commits to help Nix run "from anywhere" ("relocatable") instead of requiring baked-in paths:
|
|
Ok will look into those. Probably not easy to upstream to Nix? Anyway here is where the sizes come from: Probably still room for improvement but 30MB is pretty good. |
No, I don't think they're well-suited for upstream, at least not as-is :). Perhaps at a high-level they're reasonable, but stylistically they're a bit of a mismatch too. If folks think either of these make sense for upstreaming please LMK and I'd be happy to put together some PR's and do at least a little reworking. Those sizes are pretty great! Not quite as small--and at a glance seems to miss 'unzip' which Nix's If only the user namespaces were more widely supported out-of-the-box, hopefully distributions/kernel/etc get this sorted soon. Beyond use for awesomeness like nix-bundle, they're required for Nix's sandbox support--which is essential for Nix use on non-NixOS, the likely target environment for both of our efforts. Just to be clear, this isn't just a statically-built Nix, it's a statically linked "multicall" binary (like busybox) with Nix and most required runtime utilities (only known exception is 'git', which is supported in ALLVM of course but seems an especially poor fit for this sort of inclusion). |
Not currently but I'm looking into it in nix-community/nix-bundle#34 |
|
Few updates on this: Will be presenting paper on Software Multiplexing at upcoming OOPSLA'18! Paper PDF: https://wdtz.org/files/oopsla18-allmux-dietz.pdf It even mentions NixOS/nixpkgs (in passing)! :) Please contact me with any feedback/comments/questions! Recently Nix grew support for using a single binary for all This gets most, nearly all, of the benefits ignoring the inclusion of utilities and And I agree. Although of course I'll add that in my biased opinion Also there's value in the ability to easily determine how effective such Updated multiplexed Nix can be found in my nur-packages repo: https://github.com/dtzWill/nur-packages For example, you can install it with nix-env with a command such as: $ nix-env -I dtz-nur=https://github.com/dtzWill/nur-packages/archive/master.tar.gz -f '<dtz-nur>' -iA pkgs.nix-muxFor reasons outlined earlier I wouldn't recommend using this for the I'd be happy to help as best I can but even so it's probably I have a Dockerfile for unpacking this into a tiny container base image, The plan is still to make all of this public (and much more), Bug me if you're interested as that helps motivate prioritizing these This issue can probably be closed now, Thanks for the support and feedback, ~Will |
|
@dtzWill Congrats on the OOPSLA paper :-) Will have a read. |
|
@dtzWill this is awesome work IMO and very important work, for the reasons outlined in #1418. In fact fixing #1418 in some way is in the critical path for making Bazel play nice with Nix to the point where Bazel can transparently unpack a local Nix, use it to manage system dependencies for build targets, then proceed with the build. See tweag/rules_nixpkgs#54. I noticed that there is no PR yet. Are you ready at this point to submit a PR upstream? |
|
What is needed for this? |
|
@dtzWill , I've been trying to build a static binary for a haskell package(postgrest), but I'm stuck at the following error when booting-strap: I've tried to specify the cipher with
Is this a problem of your site? Or a bug of |
|
@hughjfchen That works with curl 7.68.0 (20.03, OpenSSL 1.1.1d), and 7.65.3 (19.09, OpenSSL 1.1.1d), but not curl 7.64 (19.03, OpenSSL 1.0.2u). Try with a more recent curl and openssl, possibly from a newer nixpkgs. |
|
@tilpner , Thank you so much! I upgraded my nix to 2.3.3 which uses libcurl 7.65.3 and now it works. |
|
Sorry for the trouble! And thanks for the investigation and solution, @tilpner! FWIW because these bits are critical for building a number of configurations I also made them available elsewhere so if I (or my server) disappeared folks might have a chance to proceed anyway. Copies of bootstrap files I host (from anything that made it into nixpkgs tree) are additionally available here: https://github.com/dtzWill/nixos-musl-bootstrap-bits . Hopefully this helps anyone experiencing issues that can't update, or if there are problems in the future. |
|
@dtzWill , I'd upgraded nix to the newest version and now can download from |
|
FWIW, my paper on "Software Multiplexing" was published in OOPSLA'18! DOI: https://doi.org/10.1145/3276524 I'll update the original post accordingly, forgot I mention it here :). |
|
Update2: I investigated this a little bit, and AFAICT this should now be resolved! Test case was using a copy of Sorry for the trouble, let me know if anything still isn't working for you1 |
|
@dtzWill , thanks for your update. I rerun the build and unfortunately, it still failed at the test-init.sh of the coreutils package with the same error: Segmentation fault. Do you think any other thing we can do with it? Collect some more data to diagnose? Skip the test? |
|
I marked this as stale due to inactivity. → More info |
|
I am still interested. Is there a way ahead to turn this into a |
|
I marked this as stale due to inactivity. → More info |
|
still interested |
|
I marked this as stale due to inactivity. → More info |
|
still interested |
I'm still working on writing up the details, but as a bit of a preview I present to you:
What
Single static binary that contains:
Organization
Here's the result of running 'tree' on this: https://gist.github.com/dtzWill/cc721df847f4464e4dc57a0ee29f0cb5
Basically every executable entry point is symlink'd to the same binary, which for this I figured might as well be called "nix" :).
Everything not usually included as part of "nix" is in
libexec/nixwhich is also prepended toPATHwhen invoking utilities. Commit used can be looked up on my nix "fork" (everything generally useful has been submitted upstream, just saying that the commit identifier can be resolved here :)).All of these tools are included to ensure a quality experience when using Nix, without any dependency on having the right versions of the right tools installed just to evaluate common Nix expressions (such as those in nixpkgs).
How
Two key components (first available now, second I'm working to make available):
allmuxtool from the allvm-tools package.Paper will hopefully be published soon[1][2] :). (EDIT: published, see [1] for info and link!)
".nix" file
This is incomplete but hopefully gives a bit of transparency and detail about how this was constructed:
https://gist.github.com/321ab77bb5bcde6af74c277c8ad4cb4c
Interested/determined individuals can also inspect the derivation chain with nix commands pointed at the ALLVM cache. GLHF.
Path Info
$ nix path-info --json --store https://cache.allvm.org /nix/store/826wlhflzcwjcb1cn53gn7rqzg6h7i0s-nix-mux-2.0dtz6036_0f4a51e/bin/nix|jq[ { "path": "/nix/store/826wlhflzcwjcb1cn53gn7rqzg6h7i0s-nix-mux-2.0dtz6036_0f4a51e", "narHash": "sha256:09xsri3nhcabf7wjkz0mjr0p9v0vd2325q1lqziz91ik7knl5rsj", "narSize": 24456160, "references": [ "/nix/store/826wlhflzcwjcb1cn53gn7rqzg6h7i0s-nix-mux-2.0dtz6036_0f4a51e" ], "deriver": "/nix/store/l8c7m7w4v81wv5x3iwhgslw9isdxv18r-nix-mux-2.0dtz6036_0f4a51e.drv", "signatures": [ "gravity.cs.illinois.edu-1:ViMgRoj5BGQ0OdqWIGoUEWJ7H8ZJlhuULivJzMS+ZBWuNGjG8tMhgUlQivfE/xQZvfPSpDzvUb7EkxxJ01l8Ag==" ], "url": "nar/0sp3isv52j9a2qhk9f7siimdm7d54rvjrx9020gfr8g5wipg735a.nar.xz", "downloadHash": "sha256:0sp3isv52j9a2qhk9f7siimdm7d54rvjrx9020gfr8g5wipg735a", "downloadSize": 5945000 } ]That's right! All of this-- Nix and many important utilities-- as a single statically linked binary (even works on WSL (Windows)!!!) compressed (
nar.xz) to just under 6MB! Try comparing that to disk usage or memory usage of traditional Nix deployments!(Improvements are not "Nix's fault"--Nix is great! 👍 All improvements are from an improved/alternative executable file format, semantics) that are not in any way specific to Nix. I just put this together because ❤️ Nix and it's a good demonstration of
allmuxIMO.Caveats
(Known) Bugs and Limitations
/nix/store/...) it is installed to.fork[3] in ways that make musl sad. I'm still investigating/debugging, will file issue/submit PR once I sort this out :). Anyway this means that very rarelynix-daemoncrashes, basically due to heap corruption resulting from doing things post-fork that aren't "allowed".[1] Paper available to all, published in OOPSLA'18, DOI link: https://doi.org/10.1145/3276524
[2] Paper itself is built using entirely ALLVM-driven "allexe"s (ALLVM executables), many of which are JIT-compiled because we can ^_^
[3] generally speaking threads +
forkis a "bad idea", see this discussion from 2014. Fixing this is a bit difficult- - and if I understand correctly an upcoming POSIX standard may change language making this valid? Not sure.The text was updated successfully, but these errors were encountered: