Make fetch* source derivation names (optionally) more descriptive and homogenize them across fetchers
#49862
Conversation
oxij
force-pushed
the
tree/repo-to-name
branch
from
234705b to
97dd9ed
Compare
GrahamcOfBorg
added
10.rebuild-darwin-stdenv
|
No, we expressly switched to |
|
Failure on aarch64-linux (full log) Attempted: gnuradio-rds Partial log (click to expand)
|
|
Timed out, unknown build status on x86_64-linux (full log) Attempted: gnuradio-rds Partial log (click to expand)
|
|
In practice switching from `fetchgit` to anything else triggers a rebuild anyway because, e.g. tarballs are usually produced by `git-archive` (which `fetchgit` does not do) followed by `autogen.sh`/`bootstrap.sh`. And similarly for most other fetchers.
The only exceptions to this rule are different versions of the same thing, like `fetchgit` and `fetchgitLocal`, and fetchers based of `fetchzip`.
This patchset doesn't touch the first and keeping the hashes for the `fetchzip`-based is a trivial addition to this, if anybody actually cares.
Also note that on master _most_ fetchers don't use `-source` suffix, only the most common ones do.
I'd say closing this is premature.
|
|
In defense of the current naming scheme, I would like to add that I often switch between github and a other git mirrors, and then I also switch between |
|
As I said above, to get the equivalent of the current naming scheme in that regard all you need to do is to make `repoToName` of this patchset ignore its first argument. One mass rebuild later things will stabilize again.
as long as the hash produced by `nix-prefetch-git` is the same in both cases, which they always are in my experience - maybe I was lucky...
`fetchGit` and `fetchgit` do the same thing, pretty much, so that's not surprising. But `fetchgit` and `fetchFromGitHub` do different things, their results don't match on a lot of repositories.
Nevertheless, could it be possible to pass something like `name ? "source"` as an attribute of an argument set to allow explicit overrides, if necessary, and maybe add the corresponding option to the `nix-prefetch-git` function (now there is an `--out` option but I didn't quite understand what it did)?
I don't understand, please elaborate. AFAIK you can pass `name` to most `fetch*` already, the point here is that the default naming of those `fetch*` outputs is very inconvenient for when you want to glance at the source real quick.
|
|
@oxij I meant exactly this thing you mentioned above:
In my (limited) experience switching from a github repo and fetchFromGitHub to its On the second point, I thought that e.g. |
|
For instance, repositories with `.gitattributes` file usually produce different hashes when switching between `fetchgit` and `fetchFromGitHub`. Some random examples: firefox/tor-browser/palemoon, qTox, linux, nixpkgs itself...
`fetchTarball` and `nix-prefetch-url` commands *did not* have the optional `name` argument, which would then be added to the store paths to make them more *informative*
`nix-prefetch-url` has `--name` option, see the man page.
`fetchTarball`, judging from a glance at the sources, does not.
If I am wrong then I (personally, at least) do see this issue as closed. Otherwise, I wonder if such an optional argument could be added (and then it wouldn't have to cause any massive rebuilds, would it?).
This patchset causes a single massive rebuild. Prefetch scripts, as noted above, can either be fixed by reimplementing some of `repoToName` in bash or, IMO, preferably, just dropped in favor of building derivations without requiring those hashes via nix itself.
|
Thank you for the information - good to know about these cases. BTW, I checked that both |
|
Reopening... |
github-actions
bot
added
6.topic: kernel
github-actions
bot
added
6.topic: python
and removed
6.topic: kernel
|
There's an RFC for this now 😅, see the above links. I'm afraid I'm a bit low on time to review this, but I'm hoping that this idea gets others helping out with the RFC |
ofborg
bot
added
10.rebuild-darwin: 0
|
Alright, so I rebased this, switched the base to Since the first two commits are a noop, the last commit here now neatly shows the actual state So, in short, the reality is that those names are a mess ATM. |
ofborg
bot
added
10.rebuild-darwin-stdenv
…ctions This patch adds `lib.repoRevToName` function that generalizes away most of the code used for derivation name generation by `fetch*` functions (`fetchzip`, `fetchFromGitHub`, etc, except those which are delayed until latter commits for mass-rebuild reasons). It's first argument controls how the resulting name will look (see below). Since `lib` has no equivalent of Nixpkgs' `config`, this patch adds `config.nameSourcesPrettily` option to Nixpkgs and then re-exposes `lib.repoRevToName config.nameSourcesPrettily` expression as `pkgs.repoRevToNameMaybe` which is then used in `fetch*` derivations. The result is that different values of `config.nameSourcesPrettily` now control how the `src` derivations produced by `fetch*` functions are to be named, e.g.: - `nameSourcesPrettily = false` (the default): ``` $ nix-instantiate -A fuse.src /nix/store/<hash>-source.drv ``` - `nameSourcesPrettily = true`: ``` $ nix-instantiate -A fuse.src /nix/store/<hash>-libfuse-2.9.9-source.drv ``` - `nameSourcesPrettily = "full"`: ``` $ nix-instantiate -A fuse.src /nix/store/<hash>-libfuse-2.9.9-github-source.drv ``` See documentation of `config.nameSourcesPrettily` for more info.
This is slightly more efficient.
…enize `name`s in `fetchsvn` In effect, this homogenizes derivation names produced by all `fetch*` functions so that switching from e.g. `fetchFromGitHub` to `fetchgit` would be a noop (assuming the content hash does not change, which is not always the case for `fetchFromGitHub` since GitHub uses `git archive` internally and `fetchgit` does not) with both the default `config.nameSourcesPrettily` setting and with `config.nameSourcesPrettily = true`.
wegank
added
the
2.status: merge conflict
Description of changes
This patch adds
lib.repoRevToNamefunction that generalizes away most of the code used for derivation name generation byfetch*(fetchzip,fetchFromGitHub, etc) functions.It's first argument controls how the resulting name will look (see below).
Since
libhas no equivalent of Nixpkgs'config, this patch addsconfig.nameSourcesPrettilyoption to Nixpkgs and then re-exposeslib.repoRevToName config.nameSourcesPrettilyexpression aspkgs.repoRevToNameMaybewhich is then used infetch*derivations.The result is that different values of
config.nameSourcesPrettilynow control how thesrcderivations produced byfetch*functions are to be named, e.g.:nameSourcesPrettily = false(the default):nameSourcesPrettily = true:nameSourcesPrettily = "full":See documentation of
config.nameSourcesPrettilyfor more info.In effect, this patch also homogenizes derivation names produced by
fetch*functions so that switching from e.g.fetchFromGitHubtofetchgitwould be a noop (assuming the content hash does not change, which is not always the case forfetchFromGitHubsince GitHub usesgit archiveinternally andfetchgitdoes not).