Bindmount files instead of hardlinking or copying to chroot #8965

Artturin · 2023-09-10T21:29:28Z

Hide whitespace changes when reviewing

16591eb#diff-19f999107b609d37cfb22c58e7f0bc1cf76edf1180e238dd6389e03cc279b604 (2013) added support for files to doBind

This is work towards allowing users to change the location of chrootRootDir, to, for example, a tmpfs.

inspired by trofi on matrix

It looks like build sandbox created by nix-daemon runs on the same filesystem, as /nix/store including things like /tmp which makes all small temporary files hit the disk. Is it intentional? If it is is there an easy way to redirect chroot's root to be tmpfs?

CC @ShamrockLee who made #8614

Motivation

Context

Priorities

Add 👍 to pull requests you find important.

src/libstore/build/local-derivation-goal.cc

Ericson2314 · 2023-09-13T05:08:30Z

Do we even need a separate map? or can we put everything in one map?

Big +1 on the idea itself (bind > hardlink/copy) and the broader goal of being able to do builds on a different file system.

CC @benradf @ryantm Doing this will remove one small thing we had to work around with layered store also.

Artturin · 2023-09-14T03:25:15Z

Unsure if this diff of

sudo inotifywait -m /nix -r --include '.chroot'

nix-build '<nixpkgs>' -A "hello" --check -vvvvvvvvvvvvvvvvvvvvvvvv

is correct

diff <(sort inotifylog-before) <(sort inotifylog-after)

4a5
> /nix/store/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.d
43a45,46
> /nix/store/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/nix/store/ ACCESS,ISDIR 5ccllrgswk61a44yl7jcxs20jql1y32b-gnused-4.9
> /nix/store/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/nix/store/ ACCESS,ISDIR 5wspmqhgsfs7dza9gcyrj5yxbyd00q7y-coreutils-9.3
46a50,53
> /nix/store/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/nix/store/ CLOSE_NOWRITE,CLOSE,ISDIR 5ccllrgswk61a44yl7jcxs20jql1y32b-gnused-4.9
> /nix/store/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/nix/store/ CLOSE_NOWRITE,CLOSE,ISDIR 5wspmqhgsfs7dza9gcyrj5yxbyd00q7y-coreutils-9.3
> /nix/store/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/nix/store/ CLOSE_WRITE,CLOSE 5yzw0vhkyszf2d179m0qfkgxmp5wjjx4-move-docs.sh
> /nix/store/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/nix/store/ CLOSE_WRITE,CLOSE 6xg259477c90a229xwmb53pdfkn6ig3g-default-builder.sh
49,53d55
< /nix/store/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/nix/store/ CREATE 9n7n6vy7aqncbsdzr3w6bw3bq1a6xm7k-patch-shebangs.sh
< /nix/store/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/nix/store/ CREATE bxsly8a56yb8kyrq03s82a3vyc8fqrb3-multiple-outputs.sh
< /nix/store/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/nix/store/ CREATE cickvswrvann041nqxb0rxilc46svw1n-prune-libtool-files.sh
< /nix/store/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/nix/store/ CREATE fyaryjvghbkpfnsyw97hb3lyb37s1pd6-move-lib64.sh
< /nix/store/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/nix/store/ CREATE h9lc1dpi14z7is86ffhl3ld569138595-audit-tmpdir.sh
57,65c59,63
< /nix/store/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/nix/store/ CREATE,ISDIR 5ccllr
< /nix/store/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/nix/store/ CREATE kd4xwxjpjxi71jkm6ka0np72if9rm3y0-move-sbin.sh
< /nix/store/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/nix/store/ CREATE m54bmrhj6fqz8nds5zcj97w9s9bckc9v-compress-man-pages.sh
< /nix/store/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/nix/store/ CREATE ngg1cv31c8c7bcm2n8ww4g06nq7s4zhm-set-source-date-epoch-to-latest.sh
< /nix/store/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/nix/store/ CREATE pa10z4ngm0g83kx9mssrqzz30s84vq7k-hello-2.12.1.tar.gz
< /nix/store/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/nix/store/ CREATE pag6l61paj1dc9sv15l7bm5c17xn5kyk-move-systemd-user-units.sh
< /nix/store/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/nix/store/ CREATE sgdq3c3lp6qabfwnrq26qzvrxx101hgi-strip.sh
< /nix/store/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/nix/store/ CREATE wgrbkkaldkrlrni33ccvm3b6vbxzb656-make-symlinks-relative.sh
< /nix/store/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/nix/store/ CREATE xyff06pkhki3qy1ls77w10s0v79c9il0-reproducible-builds.sh
---
> /nix/store/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/nix/store/ CREATE,ISDIR 5ccllrgswk61a44yl7jcxs20jql1y32b-gnused-4.9
> /nix/store/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/nix/store/ CREATE,ISDIR 5wspmqhgsfs7dza9gcyrj5yxbyd00q7y-coreutils-9.3
> /nix/store/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/nix/store/ CREATE,ISDIR 7b943a2k4amjmam6dnwnxnj8qbba9lbq-busybox-static-x86_64-unknown-linux-musl-1.35.0
> /nix/store/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/nix/store/ OPEN 5yzw0vhkyszf2d179m0qfkgxmp5wjjx4-move-docs.sh
> /nix/store/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/nix/store/ OPEN 6xg259477c90a229xwmb53pdfkn6ig3g-default-builder.sh
68a67,68
> /nix/store/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/nix/store/ OPEN,ISDIR 5ccllrgswk61a44yl7jcxs20jql1y32b-gnused-4.9
> /nix/store/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/nix/store/ OPEN,ISDIR 5wspmqhgsfs7dza9gcyrj5yxbyd00q7y-coreutils-9.3

the nix log shows binding of files
sandbox setup: bind mounting '/nix/store/5yzw0vhkyszf2d179m0qfkgxmp5wjjx4-move-docs.sh' to '/nix/store/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/nix/store/5yzw0vhkyszf2d179m0qfkgxmp5wjjx4-move-docs.sh'

full inotifylog-after

/nix/store/ CREATE,ISDIR dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot
/nix/store/ ATTRIB,ISDIR dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot
/nix/store/ OPEN,ISDIR dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot
/nix/store/ ACCESS,ISDIR dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot
/nix/store/ CLOSE_NOWRITE,CLOSE,ISDIR dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot
/nix/store/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/ CREATE,ISDIR nix
/nix/store/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/ OPEN,ISDIR nix
/nix/store/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/ ACCESS,ISDIR nix
/nix/store/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/ CLOSE_NOWRITE,CLOSE,ISDIR nix
/nix/store/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/ CREATE,ISDIR dev
/nix/store/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/ OPEN,ISDIR dev
/nix/store/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/ ACCESS,ISDIR dev
/nix/store/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/ CLOSE_NOWRITE,CLOSE,ISDIR dev
/nix/store/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/dev/ CREATE stdin
/nix/store/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/dev/ CREATE stdout
/nix/store/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/dev/ CREATE stderr
/nix/store/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/ CREATE,ISDIR bin
/nix/store/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/ OPEN,ISDIR bin
/nix/store/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/ ACCESS,ISDIR bin
/nix/store/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/ CLOSE_NOWRITE,CLOSE,ISDIR bin
/nix/store/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/ CREATE,ISDIR build
/nix/store/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/ OPEN,ISDIR build
/nix/store/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/ ACCESS,ISDIR build
/nix/store/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/ CLOSE_NOWRITE,CLOSE,ISDIR build
/nix/store/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/dev/ CREATE full
/nix/store/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/dev/ OPEN full
/nix/store/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/dev/ CLOSE_WRITE,CLOSE full
/nix/store/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/dev/ CREATE kvm
/nix/store/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/dev/ OPEN kvm
/nix/store/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/dev/ CLOSE_WRITE,CLOSE kvm
/nix/store/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/dev/ CREATE null
/nix/store/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/dev/ OPEN null
/nix/store/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/dev/ CLOSE_WRITE,CLOSE null
/nix/store/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/dev/ CREATE random
/nix/store/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/dev/ OPEN random
/nix/store/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/dev/ CLOSE_WRITE,CLOSE random
/nix/store/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/dev/ CREATE tty
/nix/store/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/dev/ OPEN tty
/nix/store/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/dev/ CLOSE_WRITE,CLOSE tty
/nix/store/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/dev/ CREATE urandom
/nix/store/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/dev/ OPEN urandom
/nix/store/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/dev/ CLOSE_WRITE,CLOSE urandom
/nix/store/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/dev/ CREATE zero
/nix/store/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/dev/ OPEN zero
/nix/store/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/dev/ CLOSE_WRITE,CLOSE zero
/nix/store/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/nix/store/ CREATE,ISDIR 039nii0j16pkzabxi9nm5dkh0z8awaww-findutils-4.9.0
/nix/store/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/nix/store/ OPEN,ISDIR 039nii0j16pkzabxi9nm5dkh0z8awaww-findutils-4.9.0
/nix/store/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/nix/store/ ACCESS,ISDIR 039nii0j16pkzabxi9nm5dkh0z8awaww-findutils-4.9.0
/nix/store/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/nix/store/ CLOSE_NOWRITE,CLOSE,ISDIR 039nii0j16pkzabxi9nm5dkh0z8awaww-findutils-4.9.0
/nix/store/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/nix/store/ CREATE,ISDIR 0gjbkfmy505imr4q5iil7pfk2i2km3z7-libmpc-1.3.1
/nix/store/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/nix/store/ OPEN,ISDIR 0gjbkfmy505imr4q5iil7pfk2i2km3z7-libmpc-1.3.1
/nix/store/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/nix/store/ ACCESS,ISDIR 0gjbkfmy505imr4q5iil7pfk2i2km3z7-libmpc-1.3.1
/nix/store/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/nix/store/ CLOSE_NOWRITE,CLOSE,ISDIR 0gjbkfmy505imr4q5iil7pfk2i2km3z7-libmpc-1.3.1
/nix/store/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/nix/store/ CREATE,ISDIR 1qxkj91n172qnypdrqdybh4p203nax7g-stdenv-linux
/nix/store/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/nix/store/ OPEN,ISDIR 1qxkj91n172qnypdrqdybh4p203nax7g-stdenv-linux
/nix/store/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/nix/store/ ACCESS,ISDIR 1qxkj91n172qnypdrqdybh4p203nax7g-stdenv-linux
/nix/store/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/nix/store/ CLOSE_NOWRITE,CLOSE,ISDIR 1qxkj91n172qnypdrqdybh4p203nax7g-stdenv-linux
/nix/store/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/nix/store/ CREATE,ISDIR 5ccllrgswk61a44yl7jcxs20jql1y32b-gnused-4.9
/nix/store/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/nix/store/ OPEN,ISDIR 5ccllrgswk61a44yl7jcxs20jql1y32b-gnused-4.9
/nix/store/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/nix/store/ ACCESS,ISDIR 5ccllrgswk61a44yl7jcxs20jql1y32b-gnused-4.9
/nix/store/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/nix/store/ CLOSE_NOWRITE,CLOSE,ISDIR 5ccllrgswk61a44yl7jcxs20jql1y32b-gnused-4.9
/nix/store/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/nix/store/ CREATE,ISDIR 5wspmqhgsfs7dza9gcyrj5yxbyd00q7y-coreutils-9.3
/nix/store/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/nix/store/ OPEN,ISDIR 5wspmqhgsfs7dza9gcyrj5yxbyd00q7y-coreutils-9.3
/nix/store/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/nix/store/ ACCESS,ISDIR 5wspmqhgsfs7dza9gcyrj5yxbyd00q7y-coreutils-9.3
/nix/store/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/nix/store/ CLOSE_NOWRITE,CLOSE,ISDIR 5wspmqhgsfs7dza9gcyrj5yxbyd00q7y-coreutils-9.3
/nix/store/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/nix/store/ CREATE 5yzw0vhkyszf2d179m0qfkgxmp5wjjx4-move-docs.sh
/nix/store/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/nix/store/ OPEN 5yzw0vhkyszf2d179m0qfkgxmp5wjjx4-move-docs.sh
/nix/store/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/nix/store/ CLOSE_WRITE,CLOSE 5yzw0vhkyszf2d179m0qfkgxmp5wjjx4-move-docs.sh
/nix/store/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/nix/store/ CREATE 6xg259477c90a229xwmb53pdfkn6ig3g-default-builder.sh
/nix/store/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/nix/store/ OPEN 6xg259477c90a229xwmb53pdfkn6ig3g-default-builder.sh
/nix/store/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/nix/store/ CLOSE_WRITE,CLOSE 6xg259477c90a229xwmb53pdfkn6ig3g-default-builder.sh
/nix/store/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/nix/store/ CREATE,ISDIR 7b943a2k4amjmam6dnwnxnj8qbba9lbq-busybox-static-x86_64-unknown-linux-musl-1.35.0
/nix/store/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.d

Ericson2314 · 2023-09-14T04:20:17Z

Odd the files mentioned in the diff seem unrelated, but the code looks very good!

Artturin · 2023-09-14T04:40:22Z

diff of having the chroot on a different filesystem

diff --git a/src/libstore/build/local-derivation-goal.cc b/src/libstore/build/local-derivation-goal.cc
index 16a0b40b7..b1b519288 100644
--- a/src/libstore/build/local-derivation-goal.cc
+++ b/src/libstore/build/local-derivation-goal.cc
@@ -651,7 +651,7 @@ void LocalDerivationGoal::startBuilder()
            environment using bind-mounts.  We put it in the Nix store
            to ensure that we can create hard-links to non-directory
            inputs in the fake Nix store in the chroot (see below). */
-        chrootRootDir = worker.store.Store::toRealPath(drvPath) + ".chroot";
+        chrootRootDir = canonPath("/data-4TB/test-chroot") + "/" + drvPath.to_string() + ".chroot";
         deletePath(chrootRootDir);

         /* Clean up the chroot directory automatically. */

$ wc -l different-fs-before
12136 different-fs-before
$ wc -l different-fs-after
12028 different-fs-after

diff <(sort different-fs-before) <(sort different-fs-after)

diff with st5kFvz3 stey2ptR noise removed

4d3
< /data-4TB/test-chroot/ ACCESS,ISDIR dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot
48d46
< /data-4TB/test-chroot/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/ ACCESS,ISDIR etc
217,239d214
< /data-4TB/test-chroot/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/etc/ ACCESS,ISDIR
< /data-4TB/test-chroot/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/etc/ ACCESS,ISDIR
< /data-4TB/test-chroot/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/etc/ ACCESS passwd
< /data-4TB/test-chroot/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/etc/ ATTRIB,ISDIR
< /data-4TB/test-chroot/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/etc/ ATTRIB,ISDIR
< /data-4TB/test-chroot/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/etc/ CLOSE_NOWRITE,CLOSE,ISDIR
< /data-4TB/test-chroot/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/etc/ CLOSE_NOWRITE,CLOSE passwd
< /data-4TB/test-chroot/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/etc/ CLOSE_WRITE,CLOSE group
< /data-4TB/test-chroot/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/etc/ CLOSE_WRITE,CLOSE hosts
< /data-4TB/test-chroot/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/etc/ CLOSE_WRITE,CLOSE passwd
< /data-4TB/test-chroot/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/etc/ CREATE hosts
< /data-4TB/test-chroot/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/etc/ CREATE passwd
< /data-4TB/test-chroot/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/etc/ DELETE group
< /data-4TB/test-chroot/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/etc/ DELETE hosts
< /data-4TB/test-chroot/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/etc/ DELETE passwd
< /data-4TB/test-chroot/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/etc/ DELETE_SELF
< /data-4TB/test-chroot/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/etc/ MODIFY group
< /data-4TB/test-chroot/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/etc/ MODIFY hosts
< /data-4TB/test-chroot/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/etc/ MODIFY passwd
< /data-4TB/test-chroot/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/etc/ OPEN hosts
< /data-4TB/test-chroot/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/etc/ OPEN,ISDIR
< /data-4TB/test-chroot/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/etc/ OPEN passwd
< /data-4TB/test-chroot/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/etc/ OPEN passwd
305,312d279
< /data-4TB/test-chroot/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/nix/store/ ACCESS 5yzw0vhkyszf2d179m0qfkgxmp5wjjx4-move-docs.sh
< /data-4TB/test-chroot/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/nix/store/ ACCESS 6xg259477c90a229xwmb53pdfkn6ig3g-default-builder.sh
< /data-4TB/test-chroot/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/nix/store/ ACCESS 6xg259477c90a229xwmb53pdfkn6ig3g-default-builder.sh
< /data-4TB/test-chroot/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/nix/store/ ACCESS 9n7n6vy7aqncbsdzr3w6bw3bq1a6xm7k-patch-shebangs.sh
< /data-4TB/test-chroot/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/nix/store/ ACCESS bxsly8a56yb8kyrq03s82a3vyc8fqrb3-multiple-outputs.sh
< /data-4TB/test-chroot/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/nix/store/ ACCESS cickvswrvann041nqxb0rxilc46svw1n-prune-libtool-files.sh
< /data-4TB/test-chroot/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/nix/store/ ACCESS fyaryjvghbkpfnsyw97hb3lyb37s1pd6-move-lib64.sh
< /data-4TB/test-chroot/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/nix/store/ ACCESS h9lc1dpi14z7is86ffhl3ld569138595-audit-tmpdir.sh
476,487d442
< /data-4TB/test-chroot/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/nix/store/ ACCESS kd4xwxjpjxi71jkm6ka0np72if9rm3y0-move-sbin.sh
< /data-4TB/test-chroot/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/nix/store/ ACCESS m54bmrhj6fqz8nds5zcj97w9s9bckc9v-compress-man-pages.sh
< /data-4TB/test-chroot/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/nix/store/ ACCESS ngg1cv31c8c7bcm2n8ww4g06nq7s4zhm-set-source-date-epoch-to-latest.sh
< /data-4TB/test-chroot/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/nix/store/ ACCESS pa10z4ngm0g83kx9mssrqzz30s84vq7k-hello-2.12.1.tar.gz
< /data-4TB/test-chroot/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/nix/store/ ACCESS pa10z4ngm0g83kx9mssrqzz30s84vq7k-hello-2.12.1.tar.gz
< /data-4TB/test-chroot/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/nix/store/ ACCESS pa10z4ngm0g83kx9mssrqzz30s84vq7k-hello-2.12.1.tar.gz
< /data-4TB/test-chroot/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/nix/store/ ACCESS pa10z4ngm0g83kx9mssrqzz30s84vq7k-hello-2.12.1.tar.gz
< /data-4TB/test-chroot/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/nix/store/ ACCESS pa10z4ngm0g83kx9mssrqzz30s84vq7k-hello-2.12.1.tar.gz
< /data-4TB/test-chroot/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/nix/store/ ACCESS pag6l61paj1dc9sv15l7bm5c17xn5kyk-move-systemd-user-units.sh
< /data-4TB/test-chroot/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/nix/store/ ACCESS sgdq3c3lp6qabfwnrq26qzvrxx101hgi-strip.sh
< /data-4TB/test-chroot/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/nix/store/ ACCESS wgrbkkaldkrlrni33ccvm3b6vbxzb656-make-symlinks-relative.sh
< /data-4TB/test-chroot/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/nix/store/ ACCESS xyff06pkhki3qy1ls77w10s0v79c9il0-reproducible-builds.sh
498d452
< /data-4TB/test-chroot/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/nix/store/ ATTRIB pag6l61paj1dc9sv15l7bm5c17xn5kyk-move-systemd-user-units.sh
509,515d462
< /data-4TB/test-chroot/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/nix/store/ CLOSE_NOWRITE,CLOSE 5yzw0vhkyszf2d179m0qfkgxmp5wjjx4-move-docs.sh
< /data-4TB/test-chroot/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/nix/store/ CLOSE_NOWRITE,CLOSE 6xg259477c90a229xwmb53pdfkn6ig3g-default-builder.sh
< /data-4TB/test-chroot/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/nix/store/ CLOSE_NOWRITE,CLOSE 9n7n6vy7aqncbsdzr3w6bw3bq1a6xm7k-patch-shebangs.sh
< /data-4TB/test-chroot/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/nix/store/ CLOSE_NOWRITE,CLOSE bxsly8a56yb8kyrq03s82a3vyc8fqrb3-multiple-outputs.sh
< /data-4TB/test-chroot/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/nix/store/ CLOSE_NOWRITE,CLOSE cickvswrvann041nqxb0rxilc46svw1n-prune-libtool-files.sh
< /data-4TB/test-chroot/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/nix/store/ CLOSE_NOWRITE,CLOSE fyaryjvghbkpfnsyw97hb3lyb37s1pd6-move-lib64.sh
< /data-4TB/test-chroot/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/nix/store/ CLOSE_NOWRITE,CLOSE h9lc1dpi14z7is86ffhl3ld569138595-audit-tmpdir.sh
621,629d567
< /data-4TB/test-chroot/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/nix/store/ CLOSE_NOWRITE,CLOSE kd4xwxjpjxi71jkm6ka0np72if9rm3y0-move-sbin.sh
< /data-4TB/test-chroot/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/nix/store/ CLOSE_NOWRITE,CLOSE m54bmrhj6fqz8nds5zcj97w9s9bckc9v-compress-man-pages.sh
< /data-4TB/test-chroot/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/nix/store/ CLOSE_NOWRITE,CLOSE ngg1cv31c8c7bcm2n8ww4g06nq7s4zhm-set-source-date-epoch-to-latest.sh
< /data-4TB/test-chroot/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/nix/store/ CLOSE_NOWRITE,CLOSE pa10z4ngm0g83kx9mssrqzz30s84vq7k-hello-2.12.1.tar.gz
< /data-4TB/test-chroot/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/nix/store/ CLOSE_NOWRITE,CLOSE pa10z4ngm0g83kx9mssrqzz30s84vq7k-hello-2.12.1.tar.gz
< /data-4TB/test-chroot/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/nix/store/ CLOSE_NOWRITE,CLOSE pag6l61paj1dc9sv15l7bm5c17xn5kyk-move-systemd-user-units.sh
< /data-4TB/test-chroot/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/nix/store/ CLOSE_NOWRITE,CLOSE sgdq3c3lp6qabfwnrq26qzvrxx101hgi-strip.sh
< /data-4TB/test-chroot/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/nix/store/ CLOSE_NOWRITE,CLOSE wgrbkkaldkrlrni33ccvm3b6vbxzb656-make-symlinks-relative.sh
< /data-4TB/test-chroot/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/nix/store/ CLOSE_NOWRITE,CLOSE xyff06pkhki3qy1ls77w10s0v79c9il0-reproducible-builds.sh
860,889d797
< /data-4TB/test-chroot/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/nix/store/ MODIFY 5yzw0vhkyszf2d179m0qfkgxmp5wjjx4-move-docs.sh
< /data-4TB/test-chroot/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/nix/store/ MODIFY 6xg259477c90a229xwmb53pdfkn6ig3g-default-builder.sh
< /data-4TB/test-chroot/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/nix/store/ MODIFY 9n7n6vy7aqncbsdzr3w6bw3bq1a6xm7k-patch-shebangs.sh
< /data-4TB/test-chroot/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/nix/store/ MODIFY bxsly8a56yb8kyrq03s82a3vyc8fqrb3-multiple-outputs.sh
< /data-4TB/test-chroot/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/nix/store/ MODIFY cickvswrvann041nqxb0rxilc46svw1n-prune-libtool-files.sh
< /data-4TB/test-chroot/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/nix/store/ MODIFY fyaryjvghbkpfnsyw97hb3lyb37s1pd6-move-lib64.sh
< /data-4TB/test-chroot/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/nix/store/ MODIFY h9lc1dpi14z7is86ffhl3ld569138595-audit-tmpdir.sh
< /data-4TB/test-chroot/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/nix/store/ MODIFY kd4xwxjpjxi71jkm6ka0np72if9rm3y0-move-sbin.sh
< /data-4TB/test-chroot/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/nix/store/ MODIFY m54bmrhj6fqz8nds5zcj97w9s9bckc9v-compress-man-pages.sh
< /data-4TB/test-chroot/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/nix/store/ MODIFY ngg1cv31c8c7bcm2n8ww4g06nq7s4zhm-set-source-date-epoch-to-latest.sh
< /data-4TB/test-chroot/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/nix/store/ MODIFY pa10z4ngm0g83kx9mssrqzz30s84vq7k-hello-2.12.1.tar.gz
< /data-4TB/test-chroot/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/nix/store/ MODIFY pa10z4ngm0g83kx9mssrqzz30s84vq7k-hello-2.12.1.tar.gz
< /data-4TB/test-chroot/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/nix/store/ MODIFY pa10z4ngm0g83kx9mssrqzz30s84vq7k-hello-2.12.1.tar.gz
< /data-4TB/test-chroot/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/nix/store/ MODIFY pa10z4ngm0g83kx9mssrqzz30s84vq7k-hello-2.12.1.tar.gz
< /data-4TB/test-chroot/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/nix/store/ MODIFY pa10z4ngm0g83kx9mssrqzz30s84vq7k-hello-2.12.1.tar.gz
< /data-4TB/test-chroot/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/nix/store/ MODIFY pa10z4ngm0g83kx9mssrqzz30s84vq7k-hello-2.12.1.tar.gz
< /data-4TB/test-chroot/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/nix/store/ MODIFY pa10z4ngm0g83kx9mssrqzz30s84vq7k-hello-2.12.1.tar.gz
< /data-4TB/test-chroot/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/nix/store/ MODIFY pa10z4ngm0g83kx9mssrqzz30s84vq7k-hello-2.12.1.tar.gz
< /data-4TB/test-chroot/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/nix/store/ MODIFY pa10z4ngm0g83kx9mssrqzz30s84vq7k-hello-2.12.1.tar.gz
< /data-4TB/test-chroot/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/nix/store/ MODIFY pa10z4ngm0g83kx9mssrqzz30s84vq7k-hello-2.12.1.tar.gz
< /data-4TB/test-chroot/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/nix/store/ MODIFY pa10z4ngm0g83kx9mssrqzz30s84vq7k-hello-2.12.1.tar.gz
< /data-4TB/test-chroot/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/nix/store/ MODIFY pa10z4ngm0g83kx9mssrqzz30s84vq7k-hello-2.12.1.tar.gz
< /data-4TB/test-chroot/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/nix/store/ MODIFY pa10z4ngm0g83kx9mssrqzz30s84vq7k-hello-2.12.1.tar.gz
< /data-4TB/test-chroot/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/nix/store/ MODIFY pa10z4ngm0g83kx9mssrqzz30s84vq7k-hello-2.12.1.tar.gz
< /data-4TB/test-chroot/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/nix/store/ MODIFY pa10z4ngm0g83kx9mssrqzz30s84vq7k-hello-2.12.1.tar.gz
< /data-4TB/test-chroot/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/nix/store/ MODIFY pa10z4ngm0g83kx9mssrqzz30s84vq7k-hello-2.12.1.tar.gz
< /data-4TB/test-chroot/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/nix/store/ MODIFY pag6l61paj1dc9sv15l7bm5c17xn5kyk-move-systemd-user-units.sh
< /data-4TB/test-chroot/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/nix/store/ MODIFY sgdq3c3lp6qabfwnrq26qzvrxx101hgi-strip.sh
< /data-4TB/test-chroot/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/nix/store/ MODIFY wgrbkkaldkrlrni33ccvm3b6vbxzb656-make-symlinks-relative.sh
< /data-4TB/test-chroot/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/nix/store/ MODIFY xyff06pkhki3qy1ls77w10s0v79c9il0-reproducible-builds.sh
896d803
< /data-4TB/test-chroot/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/nix/store/ OPEN 5yzw0vhkyszf2d179m0qfkgxmp5wjjx4-move-docs.sh
898,899d804
< /data-4TB/test-chroot/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/nix/store/ OPEN 6xg259477c90a229xwmb53pdfkn6ig3g-default-builder.sh
< /data-4TB/test-chroot/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/nix/store/ OPEN 9n7n6vy7aqncbsdzr3w6bw3bq1a6xm7k-patch-shebangs.sh
902,903d806
< /data-4TB/test-chroot/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/nix/store/ OPEN bxsly8a56yb8kyrq03s82a3vyc8fqrb3-multiple-outputs.sh
< /data-4TB/test-chroot/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/nix/store/ OPEN cickvswrvann041nqxb0rxilc46svw1n-prune-libtool-files.sh
906,907d808
< /data-4TB/test-chroot/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/nix/store/ OPEN fyaryjvghbkpfnsyw97hb3lyb37s1pd6-move-lib64.sh
< /data-4TB/test-chroot/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/nix/store/ OPEN h9lc1dpi14z7is86ffhl3ld569138595-audit-tmpdir.sh
1015,1016d915
< /data-4TB/test-chroot/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/nix/store/ OPEN kd4xwxjpjxi71jkm6ka0np72if9rm3y0-move-sbin.sh
< /data-4TB/test-chroot/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/nix/store/ OPEN m54bmrhj6fqz8nds5zcj97w9s9bckc9v-compress-man-pages.sh
1019,1020d917
< /data-4TB/test-chroot/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/nix/store/ OPEN ngg1cv31c8c7bcm2n8ww4g06nq7s4zhm-set-source-date-epoch-to-latest.sh
< /data-4TB/test-chroot/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/nix/store/ OPEN pa10z4ngm0g83kx9mssrqzz30s84vq7k-hello-2.12.1.tar.gz
1022,1023d918
< /data-4TB/test-chroot/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/nix/store/ OPEN pa10z4ngm0g83kx9mssrqzz30s84vq7k-hello-2.12.1.tar.gz
< /data-4TB/test-chroot/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/nix/store/ OPEN pag6l61paj1dc9sv15l7bm5c17xn5kyk-move-systemd-user-units.sh
1026,1027d920
< /data-4TB/test-chroot/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/nix/store/ OPEN sgdq3c3lp6qabfwnrq26qzvrxx101hgi-strip.sh
< /data-4TB/test-chroot/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/nix/store/ OPEN wgrbkkaldkrlrni33ccvm3b6vbxzb656-make-symlinks-relative.sh
1029d921
< /data-4TB/test-chroot/dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/nix/store/ OPEN xyff06pkhki3qy1ls77w10s0v79c9il0-reproducible-builds.sh

in before

OPEN 5yzw0vhkyszf2d179m0qfkgxmp5wjjx4-move-docs.sh
ACCESS 5yzw0vhkyszf2d179m0qfkgxmp5wjjx4-move-docs.sh
CLOSE_NOWRITE,CLOSE 5yzw0vhkyszf2d179m0qfkgxmp5wjjx4-move-docs.sh

in after

CREATE 5yzw0vhkyszf2d179m0qfkgxmp5wjjx4-move-docs.sh
OPEN 5yzw0vhkyszf2d179m0qfkgxmp5wjjx4-move-docs.sh
CLOSE_WRITE,CLOSE 5yzw0vhkyszf2d179m0qfkgxmp5wjjx4-move-docs.sh

Artturin · 2023-09-14T05:12:05Z

I removed the automatic cleaning of the chroot dir by removing

/* Clean up the chroot directory automatically. */
autoDelChroot = std::make_shared<AutoDelete>(chrootRootDir);

these may not be correct as the fs is zfs with compression

# before
$ sudo du -shc --apparent-size before-dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/nix/store
1.3M	before-dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/nix/store
1.3M	total

# after
$ sudo du -shc --apparent-size dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/nix/store
197K	dyzlaidm3j18dqsfdfpixkpiljnclgx4-hello-2.12.1.drv.chroot/nix/store
197K	total

files
before
701 B Thu Sep 14 08:06:06 2023 5yzw0vhkyszf2d179m0qfkgxmp5wjjx4-move-docs.sh
after
0 B Thu Sep 14 08:18:04 2023 5yzw0vhkyszf2d179m0qfkgxmp5wjjx4-move-docs.sh

in both
the dirs are 2 B Thu Sep 14 08:18:04 2023 5wspmqhgsfs7dza9gcyrj5yxbyd00q7y-coreutils-9.3/

so seems to be working

Ericson2314 · 2023-09-15T15:42:43Z

Why do we not want cleanup? Also, what exactly were the inotify logs from outside the sandbox supposed to demonstrate? I've been a bit lost.

Artturin · 2023-09-15T16:22:15Z

Why do we not want cleanup? Also, what exactly were the inotify logs from outside the sandbox supposed to demonstrate? I've been a bit lost.

I removed it for testing locally.

I thought the inotify logs would have shown something useful like less writes.

pwaller · 2023-09-15T21:18:08Z

I thought the inotify logs would have shown something useful like less writes.

I guess the same amount of file access will happen via the FS. But given that the FS is not backed by a block device, there will be no corresponding writes to a block device. To observe those in a similar manner you'd need something like biosnoop, but it's fairly obvious that if you have a tmpfs that there will be no such writes. Another issue is that writes are delayed until dirty page writeback happens, which can be some time after the filesystem writes happen.

ShamrockLee · 2023-09-16T01:39:32Z

I'll test it on AFS and CERN EOS today.

ShamrockLee · 2023-09-18T06:39:37Z

I'll test it on AFS and CERN EOS today.

Build and run hello and pkgsStatic.hello successfully on AFS using CERN LXPLUS.

CERN EOS on LXPULS still cannot run Nix with local chroot store, as cern-eos/eos@e0da166 hasn't been released by cern-eos/eos, and therefore hasn't been deployed onto CERN LXPLUS.

Artturin · 2023-09-20T00:03:52Z

hydraJobs.tests.{remoteBuilds,nix-copy,nix-copy-closure,containers} pass

Artturin · 2023-09-25T00:06:12Z

Been using this for a week on x86_64-linux applied on top of the lazy-trees branch and no issues

https://github.com/Artturin/nix/tree/lazy-trees-mine

there were no conflicts when applying

Artturin · 2023-10-05T00:37:27Z

Been using this for ~2 weeks and no issues

Ericson2314 · 2023-10-12T18:51:40Z

OK cool so we're not worrying about the inotify logs (which I never was, seems fine) this looks good to me!

virusdave · 2023-10-12T19:41:17Z

OK cool so we're not worrying about the inotify logs (which I never was, seems fine) this looks good to me!

+100

This looks good, and is better than my own version. It fixes a real problem in the wild, so hopefully we can land it soon. 👍

edolstra · 2023-10-16T11:45:26Z

src/libstore/build/local-derivation-goal.hh

-    typedef map<Path, ChrootPath> DirsInChroot; // maps target path to source path
-    DirsInChroot dirsInChroot;
+    typedef map<Path, ChrootPath> FilesAndDirsInChroot; // maps target path to source path
+    FilesAndDirsInChroot filesAndDirsInChroot;


Maybe just "PathsInChroot" to be less verbose?

edolstra · 2023-10-16T11:50:23Z

It looks like build sandbox created by nix-daemon runs on the same filesystem, as /nix/store including things like /tmp which makes all small temporary files hit the disk. Is it intentional?

Yes, the chroot is required to be on the same filesystem so that the build outputs can be moved efficiently from the chroot to their final location.

NixOS@16591eb#diff-19f999107b609d37cfb22c58e7f0bc1cf76edf1180e238dd6389e03cc279b604 (2013) added support for files to doBind This is work towards allowing users to change the location of chrootRootDir, to, for example, a tmpfs. inspired by trofi on matrix > It looks like build sandbox created by nix-daemon runs on the same filesystem, as /nix/store including things like /tmp which makes all small temporary files hit the disk. Is it intentional? If it is is there an easy way to redirect chroot's root to be tmpfs? dirsInChroot -> pathsInChroot

Ericson2314 · 2023-10-19T14:36:10Z

Yes, the chroot is required to be on the same filesystem so that the build outputs can be moved efficiently from the chroot to their final location.

I don't think we should force that. E.g. if one is using NFS it is probably much better to do one big move at the end. Synchronizing the state during the build is very counter productive. Ultimately, all the final bytes need to get copied over the network from the builder to the data server, so there is "mv avoids O(n) anyways".

Ericson2314 · 2023-10-19T14:42:22Z

Anyways, this PR doesn't directly change any logic saying what build dirs are or not allowed, so this stuff shouldn't be a blocker merging this. Is anything else needed or can we merge this?

virusdave · 2023-10-26T03:19:30Z

Yay!

infinisil · 2023-12-31T23:15:20Z

This causes #9579

Bindmount files instead of hardlinking or copying to chroot (cherry picked from commit 622191c) Change-Id: I278ec1baacdfa9044992b58fdec8f14d6d7d09ce

Artturin mentioned this pull request Sep 10, 2023

WIP Allow users to change chrootRootDirBase location #8966

Draft

Artturin force-pushed the bindfilesinchroot branch from 267f471 to 26546f2 Compare September 10, 2023 22:43

Ericson2314 reviewed Sep 13, 2023

View reviewed changes

src/libstore/build/local-derivation-goal.cc Show resolved Hide resolved

Artturin force-pushed the bindfilesinchroot branch from 26546f2 to 4e63ed3 Compare September 14, 2023 01:01

Artturin changed the title ~~WIP Bindmount files instead of hardlinking~~ Bindmount files instead of hardlinking Sep 14, 2023

Artturin marked this pull request as ready for review September 14, 2023 18:36

Artturin requested a review from thufschmitt as a code owner September 14, 2023 18:36

Artturin force-pushed the bindfilesinchroot branch 2 times, most recently from 2ab5985 to 8dbafb4 Compare September 14, 2023 19:02

Artturin changed the title ~~Bindmount files instead of hardlinking~~ Bindmount files instead of hardlinking or copying to chroot Sep 14, 2023

Artturin requested review from Ericson2314 and roberth September 25, 2023 00:06

Artturin mentioned this pull request Oct 12, 2023

Stop hardlinking inputs inside a builder chroot sandbox #9148

Closed

Ericson2314 approved these changes Oct 12, 2023

View reviewed changes

edolstra reviewed Oct 16, 2023

View reviewed changes

Artturin force-pushed the bindfilesinchroot branch from 8dbafb4 to d393725 Compare October 16, 2023 21:59

Artturin added 4 commits October 17, 2023 01:26

remove linkOrCopy and use bindmounts for files in addDependency

630c254

factor out doBind from runChild

11e47e7

use doBind in addDependency

b8dfa3d

Artturin force-pushed the bindfilesinchroot branch from d393725 to b8dfa3d Compare October 16, 2023 22:26

Artturin requested a review from edolstra October 25, 2023 01:51

edolstra merged commit 622191c into NixOS:master Oct 25, 2023
8 checks passed

Trundle mentioned this pull request Nov 13, 2023

Update comment to reflect bind mounts are now used for store in chroot #9351

Merged

cole-h mentioned this pull request Nov 26, 2023

bug: permission denied when trying to mkOutOfStoreSymlink nix-community/home-manager#4692

Open

2 tasks

infinisil mentioned this pull request Dec 31, 2023

Permission denied error when building symlink derivation #9579

Closed

fricklerhandwerk added the store Issues and pull requests concerning the Nix store label Jan 8, 2024

roberth mentioned this pull request Jan 10, 2024

Fix permission denied when building symlink derivation which points t… #9723

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Bindmount files instead of hardlinking or copying to chroot #8965

Bindmount files instead of hardlinking or copying to chroot #8965

Artturin commented Sep 10, 2023 •

edited

Loading

Ericson2314 commented Sep 13, 2023

Artturin commented Sep 14, 2023 •

edited

Loading

Ericson2314 commented Sep 14, 2023

Artturin commented Sep 14, 2023 •

edited

Loading

Artturin commented Sep 14, 2023 •

edited

Loading

Ericson2314 commented Sep 15, 2023

Artturin commented Sep 15, 2023

pwaller commented Sep 15, 2023 •

edited

Loading

ShamrockLee commented Sep 16, 2023 •

edited

Loading

ShamrockLee commented Sep 18, 2023 •

edited

Loading

Artturin commented Sep 20, 2023 •

edited

Loading

Artturin commented Sep 25, 2023 •

edited

Loading

Artturin commented Oct 5, 2023

Ericson2314 commented Oct 12, 2023 •

edited

Loading

virusdave commented Oct 12, 2023

edolstra Oct 16, 2023

Artturin Oct 16, 2023

edolstra commented Oct 16, 2023

Ericson2314 commented Oct 19, 2023

Ericson2314 commented Oct 19, 2023

virusdave commented Oct 26, 2023

infinisil commented Dec 31, 2023

Bindmount files instead of hardlinking or copying to chroot #8965

Bindmount files instead of hardlinking or copying to chroot #8965

Conversation

Artturin commented Sep 10, 2023 • edited Loading

Motivation

Context

Priorities

Ericson2314 commented Sep 13, 2023

Artturin commented Sep 14, 2023 • edited Loading

Ericson2314 commented Sep 14, 2023

Artturin commented Sep 14, 2023 • edited Loading

Artturin commented Sep 14, 2023 • edited Loading

Ericson2314 commented Sep 15, 2023

Artturin commented Sep 15, 2023

pwaller commented Sep 15, 2023 • edited Loading

ShamrockLee commented Sep 16, 2023 • edited Loading

ShamrockLee commented Sep 18, 2023 • edited Loading

Artturin commented Sep 20, 2023 • edited Loading

Artturin commented Sep 25, 2023 • edited Loading

Artturin commented Oct 5, 2023

Ericson2314 commented Oct 12, 2023 • edited Loading

virusdave commented Oct 12, 2023

edolstra Oct 16, 2023

Choose a reason for hiding this comment

Artturin Oct 16, 2023

Choose a reason for hiding this comment

edolstra commented Oct 16, 2023

Ericson2314 commented Oct 19, 2023

Ericson2314 commented Oct 19, 2023

virusdave commented Oct 26, 2023

infinisil commented Dec 31, 2023

Artturin commented Sep 10, 2023 •

edited

Loading

Artturin commented Sep 14, 2023 •

edited

Loading

Artturin commented Sep 14, 2023 •

edited

Loading

Artturin commented Sep 14, 2023 •

edited

Loading

pwaller commented Sep 15, 2023 •

edited

Loading

ShamrockLee commented Sep 16, 2023 •

edited

Loading

ShamrockLee commented Sep 18, 2023 •

edited

Loading

Artturin commented Sep 20, 2023 •

edited

Loading

Artturin commented Sep 25, 2023 •

edited

Loading

Ericson2314 commented Oct 12, 2023 •

edited

Loading