Build: use nix path-info to test for build output availability
#1352
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
When using the Perl bindings, this usually takes the wrong store, i.e. `auto` even if `store_uri` is a flat-file binary cache. See also NixOS/nix#9859 Using fork/exec here on purpose to make sure that the output of `nix path-info` isn't spammed into stderr. This could probably done with `system("foo &>/dev/null")`, but I didn't want to worry about shell injection issues then. And finally, I can't seem to silence STDERR for `system` with a list as argument. Not a Perl dev though, so not sure if there's a better way.
|
I don't quite understand the things you are saying in that issue, but I soon as well just add the missing functionality. We have a |
|
Does this even work when using the nixos module? |
|
It definitely works since I just realized I forgot to silence stdout and now I get paths printed to hydra-server's journal whenever I open a build view. Nix is part of hydra-server's env:
Fair. Should probably use a nix-* equivalent then 🤷 |
|
I am looking at the perl binding stuff. |
…well `nix-store -q` (and `--outputs`) doesn't seem to complain if a path doesn't exist in a store anymore, so let's do it that way. This approach works fine on Nix 2.3 as well, it only warns about `experimental-features` being an unknown option, but that's silenced along with the rest of stderr.
|
I made NixOS/nix#9863 to generalize the Perl bindings so we can fix this the right way. It would be nice to replace this with PR against I think we can basically do a big |
Ma27
added a commit
to Ma27/hydra
that referenced
this pull request
Feb 7, 2024
Implements support for Nix's new Perl bindings[1]. The current state basically does `openStore()`, but always uses `auto` and doesn't support stores at other URIs. Even though the stores are cached inside the Perl implementation, I decided to instantiate those once in the Nix helper module. That way store openings aren't cluttered across the entire codebase. Also, there are two stores used later on - MACHINE_LOCAL_STORE for `auto`, BINARY_CACHE_STORE for the one from `store_uri` in `hydra.conf` - and using consistent names should make the intent clearer then. This doesn't contain any behavioral changes, i.e. the build product availability issue from NixOS#1352 isn't fixed. This patch only contains the migration to the new API. [1] NixOS/nix#9863
Ma27
added a commit
to Ma27/hydra
that referenced
this pull request
Feb 7, 2024
Replaces / Closes NixOS#1352 Consider the following setup: * `store_uri` points to a "special" store, e.g. `s3://` or `file://`. * Hydra runs inside e.g. an nspawn container and exclusively builds on remote systems. Then, build products are never in the nspawn's `/nix/store`. This in turn means that `$MACHINE_LOCAL_STORE->isValidPath` is always `false` (or `0` 🤷) for output paths and thus Hydra wrongly claims that the output got garbage collected. Instead, use the BINARY_CACHE_STORE to look for the availability in the correct store. Everything touching the `drv` rather than the output paths still uses MACHINE_LOCAL_STORE: this is because `hydra-eval-jobs` does `openStore()` on `auto`, so the derivations end up there rather than on the BINARY_CACHE_STORE.
Ma27
added a commit
to Ma27/hydra
that referenced
this pull request
Feb 12, 2024
Implements support for Nix's new Perl bindings[1]. The current state basically does `openStore()`, but always uses `auto` and doesn't support stores at other URIs. Even though the stores are cached inside the Perl implementation, I decided to instantiate those once in the Nix helper module. That way store openings aren't cluttered across the entire codebase. Also, there are two stores used later on - MACHINE_LOCAL_STORE for `auto`, BINARY_CACHE_STORE for the one from `store_uri` in `hydra.conf` - and using consistent names should make the intent clearer then. This doesn't contain any behavioral changes, i.e. the build product availability issue from NixOS#1352 isn't fixed. This patch only contains the migration to the new API. [1] NixOS/nix#9863
Ma27
added a commit
to Ma27/hydra
that referenced
this pull request
Feb 12, 2024
Replaces / Closes NixOS#1352 Consider the following setup: * `store_uri` points to a "special" store, e.g. `s3://` or `file://`. * Hydra runs inside e.g. an nspawn container and exclusively builds on remote systems. Then, build products are never in the nspawn's `/nix/store`. This in turn means that `$MACHINE_LOCAL_STORE->isValidPath` is always `false` (or `0` 🤷) for output paths and thus Hydra wrongly claims that the output got garbage collected. Instead, use the BINARY_CACHE_STORE to look for the availability in the correct store. Everything touching the `drv` rather than the output paths still uses MACHINE_LOCAL_STORE: this is because `hydra-eval-jobs` does `openStore()` on `auto`, so the derivations end up there rather than on the BINARY_CACHE_STORE.
Ma27
added a commit
to Ma27/hydra
that referenced
this pull request
Feb 12, 2024
Implements support for Nix's new Perl bindings[1]. The current state basically does `openStore()`, but always uses `auto` and doesn't support stores at other URIs. Even though the stores are cached inside the Perl implementation, I decided to instantiate those once in the Nix helper module. That way store openings aren't cluttered across the entire codebase. Also, there are two stores used later on - MACHINE_LOCAL_STORE for `auto`, BINARY_CACHE_STORE for the one from `store_uri` in `hydra.conf` - and using consistent names should make the intent clearer then. This doesn't contain any behavioral changes, i.e. the build product availability issue from NixOS#1352 isn't fixed. This patch only contains the migration to the new API. [1] NixOS/nix#9863
Ma27
added a commit
to Ma27/hydra
that referenced
this pull request
Feb 12, 2024
Replaces / Closes NixOS#1352 Consider the following setup: * `store_uri` points to a "special" store, e.g. `s3://` or `file://`. * Hydra runs inside e.g. an nspawn container and exclusively builds on remote systems. Then, build products are never in the nspawn's `/nix/store`. This in turn means that `$MACHINE_LOCAL_STORE->isValidPath` is always `false` (or `0` 🤷) for output paths and thus Hydra wrongly claims that the output got garbage collected. Instead, use the BINARY_CACHE_STORE to look for the availability in the correct store. Everything touching the `drv` rather than the output paths still uses MACHINE_LOCAL_STORE: this is because `hydra-eval-jobs` does `openStore()` on `auto`, so the derivations end up there rather than on the BINARY_CACHE_STORE.
Mic92
pushed a commit
that referenced
this pull request
Aug 21, 2024
Implements support for Nix's new Perl bindings[1]. The current state basically does `openStore()`, but always uses `auto` and doesn't support stores at other URIs. Even though the stores are cached inside the Perl implementation, I decided to instantiate those once in the Nix helper module. That way store openings aren't cluttered across the entire codebase. Also, there are two stores used later on - MACHINE_LOCAL_STORE for `auto`, BINARY_CACHE_STORE for the one from `store_uri` in `hydra.conf` - and using consistent names should make the intent clearer then. This doesn't contain any behavioral changes, i.e. the build product availability issue from #1352 isn't fixed. This patch only contains the migration to the new API. [1] NixOS/nix#9863
rhelmot
pushed a commit
to rhelmot/hydra
that referenced
this pull request
Dec 6, 2024
Replaces / Closes NixOS#1352 Consider the following setup: * `store_uri` points to a "special" store, e.g. `s3://` or `file://`. * Hydra runs inside e.g. an nspawn container and exclusively builds on remote systems. Then, build products are never in the nspawn's `/nix/store`. This in turn means that `$MACHINE_LOCAL_STORE->isValidPath` is always `false` (or `0` 🤷) for output paths and thus Hydra wrongly claims that the output got garbage collected. Instead, use the BINARY_CACHE_STORE to look for the availability in the correct store. Everything touching the `drv` rather than the output paths still uses MACHINE_LOCAL_STORE: this is because `hydra-eval-jobs` does `openStore()` on `auto`, so the derivations end up there rather than on the BINARY_CACHE_STORE.
rhelmot
pushed a commit
to rhelmot/hydra
that referenced
this pull request
Dec 6, 2024
Replaces / Closes NixOS#1352 Consider the following setup: * `store_uri` points to a "special" store, e.g. `s3://` or `file://`. * Hydra runs inside e.g. an nspawn container and exclusively builds on remote systems. Then, build products are never in the nspawn's `/nix/store`. This in turn means that `$MACHINE_LOCAL_STORE->isValidPath` is always `false` (or `0` 🤷) for output paths and thus Hydra wrongly claims that the output got garbage collected. Instead, use the BINARY_CACHE_STORE to look for the availability in the correct store. Everything touching the `drv` rather than the output paths still uses MACHINE_LOCAL_STORE: this is because `hydra-eval-jobs` does `openStore()` on `auto`, so the derivations end up there rather than on the BINARY_CACHE_STORE.
rhelmot
pushed a commit
to rhelmot/hydra
that referenced
this pull request
Dec 7, 2024
Replaces / Closes NixOS#1352 Consider the following setup: * `store_uri` points to a "special" store, e.g. `s3://` or `file://`. * Hydra runs inside e.g. an nspawn container and exclusively builds on remote systems. Then, build products are never in the nspawn's `/nix/store`. This in turn means that `$MACHINE_LOCAL_STORE->isValidPath` is always `false` (or `0` 🤷) for output paths and thus Hydra wrongly claims that the output got garbage collected. Instead, use the BINARY_CACHE_STORE to look for the availability in the correct store. Everything touching the `drv` rather than the output paths still uses MACHINE_LOCAL_STORE: this is because `hydra-eval-jobs` does `openStore()` on `auto`, so the derivations end up there rather than on the BINARY_CACHE_STORE.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
cc @dasJ @Ericson2314
Hacky workaround for NixOS/nix#9859, but after I managed to fix the issue on my end, I decided I could still submit it against upstream.
When using the Perl bindings, this usually takes the wrong store, i.e.
autoeven ifstore_uriis a flat-file binary cache.See also NixOS/nix#9859
Using fork/exec here on purpose to make sure that the output of
nix path-infoisn't spammed into stderr. This could probably done withsystem("foo &>/dev/null"), but I didn't want to worry about shell injection issues then. And finally, I can't seem to silence STDERR forsystemwith a list as argument. Not a Perl dev though, so not sure if there's a better way.