Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow Perl bindings to perform operations on arbitrary stores #9859

Closed
Ma27 opened this issue Jan 26, 2024 · 0 comments · Fixed by #9863
Closed

Allow Perl bindings to perform operations on arbitrary stores #9859

Ma27 opened this issue Jan 26, 2024 · 0 comments · Fixed by #9863
Labels

Comments

@Ma27
Copy link
Member

Ma27 commented Jan 26, 2024

Describe the bug

Store operations done via Nix's Perl bindings are always made on the "default" store (i.e. auto) because open() doesn't accept a store uri and caches the store internally:

https://github.com/NixOS/nix/blob/2.19.3/perl/lib/Nix/Store.xs#L23-L35

Steps To Reproduce

n/a

Expected behavior

Being able to perform operations on arbitrary stores.

nix-env --version output

nix-env (Nix) 2.19.3

Additional context

I discovered this by debugging Hydra: I wondered why a lot of builds were claiming to be garbage collected even though this wasn't the case:

2024-01-26_18h49m13s_screenshot

The cause for that was that my Hydra doesn't run builds on localhost, but exclusively on remote machines it connects to via ssh.
Then, the store paths are downloaded into the store specified via store_uri in hydra.conf.
Because I use a flat-file binary cache (file:///var/lib/hydra/store), sub isLocalStore in Hydra returns true and thus the output paths of a Hydra build are checked with isValidPath from the Perl bindings:

https://github.com/NixOS/hydra/blob/7f5889559e3118c66763428c410958dcec6a6fa8/src/lib/Hydra/Controller/Build.pm#L80-L86

This however doesn't use the binary cache for that, but the machine's /nix/store and thus wrongly claims that the build product was garbage collected.

Priorities

Add 👍 to issues you find important.

@Ma27 Ma27 added the bug label Jan 26, 2024
Ma27 added a commit to Ma27/hydra that referenced this issue Jan 26, 2024
When using the Perl bindings, this usually takes the wrong store, i.e.
`auto` even if `store_uri` is a flat-file binary cache.

See also NixOS/nix#9859

Using fork/exec here on purpose to make sure that the output of `nix path-info`
isn't spammed into stderr. This could probably done with
`system("foo &>/dev/null")`, but I didn't want to worry about shell
injection issues then. And finally, I can't seem to silence STDERR for
`system` with a list as argument. Not a Perl dev though, so not sure if
there's a better way.
Ma27 added a commit to Ma27/hydra that referenced this issue Jan 26, 2024
When using the Perl bindings, this usually takes the wrong store, i.e.
`auto` even if `store_uri` is a flat-file binary cache.

See also NixOS/nix#9859

Using fork/exec here on purpose to make sure that the output of `nix path-info`
isn't spammed into stderr. This could probably done with
`system("foo &>/dev/null")`, but I didn't want to worry about shell
injection issues then. And finally, I can't seem to silence STDERR for
`system` with a list as argument. Not a Perl dev though, so not sure if
there's a better way.
Ericson2314 added a commit to obsidiansystems/nix that referenced this issue Jan 26, 2024
Fix NixOS#9859

It's a breaking change but that's fine; I can just update Hydra to use
the new bindings.
Ericson2314 added a commit to obsidiansystems/nix that referenced this issue Jan 26, 2024
Fix NixOS#9859

It's a breaking change but that's fine; we can just update Hydra to use
the new bindings.
Ericson2314 added a commit to obsidiansystems/nix that referenced this issue Jan 26, 2024
Fix NixOS#9859

It's a breaking change but that's fine; we can just update Hydra to use
the new bindings.
Ericson2314 added a commit to obsidiansystems/nix that referenced this issue Jan 26, 2024
Fix NixOS#9859

It's a breaking change but that's fine; we can just update Hydra to use
the new bindings.
Ericson2314 added a commit to obsidiansystems/nix that referenced this issue Feb 7, 2024
Fix NixOS#9859

It's a breaking change but that's fine; we can just update Hydra to use
the new bindings.
Ericson2314 added a commit to obsidiansystems/nix that referenced this issue Feb 8, 2024
Fix NixOS#9859

It's a breaking change but that's fine; we can just update Hydra to use
the new bindings.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant