pkgs/top-level: make package sets composable

[wolfgangwalther]

Description of changes

The various pkgsXYZ top-level package sets did not pass localSystem / crossSystem options to lower levels, so far. This change propagates the original arguments, i.e. localSystem0 / crossSystem0 to lower levels. Those include the overrides made by an upper package set.

Example: pkgsStatic adds crossSystem = { isStatic = true }. pkgsStatic.pkgsMusl previously lost this setting, because it didn't propagate crossSystem0, yet.

The first commit removes / rewrites some outdated comments, which referred to the cross and i686 package sets. Meanwhile, we have a lot more, so I rewrote those comments to not say anything about specific sets anymore.

The second commit adds some tests to confirm composability of package sets. This helped me coming up with the fixes, I'm not sure whether it should be kept. It takes about 8-9 seconds for me to run. It also helps showing what the remaining commits fix by uncommenting the previously broken tests.

The remaining commits fix those composability issues. This:

• will fix Build failure: pkgsCross.mingwW64.pkgsStatic.stdenv #281596 (@yshui, @wegank)
• could possibly fix pkgsStatic: Pass hostPlatform.gcc attribute #283460 with a more principal approach (@rodarima, @alyssais, @K900)
• supersedes the fix in pkgsStatic: make pkgsMusl a no-op #136549 (@r-burns, @rnhmjoj)

One open question for me is how to deal with pkgsXYZ.pkgsCross. This has practical challenges (#114510), but is also conceptually unsound: In a way pkgsCross expects to reset crossSystem from scratch, because the whole point about this layer is to start with "well known" example configurations. IMHO, it's fine to make modifications on top of that. e.g. pkgsCross.XYZ.pkgsLLVM or pkgsCross.XYZ.pkgsStatic, but it makes little sense to have pkgsLLVM.pkgsCross... or pkgsStatic.pkgsCross.... Probably completely unusable is pkgsCross.XYZ.pkgsCross.ABC.

One idea would be to just restrict pkgsCross to be used at the top-level, for example by adding something like this in each package set, including pkgsCross itself:

overlays = [ (self': super': {
  pkgsCross = throw "only use pkgsCross at the top-level";
})]

That would also disallow pkgsMusl.pkgsCross.XYZ, which I think is about the only remotely reasonable use-case here: "cross compiling from a native musl environment". I'm not sure whether anyone uses that, though, and whether it's worth supporting at all.

Things done

• Built on platform(s)
  • x86_64-linux
  • aarch64-linux
  • x86_64-darwin
  • aarch64-darwin
• For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)
  • sandbox = relaxed
  • sandbox = true
• Tested, as applicable:
  • NixOS test(s) (look inside nixos/tests)
  • and/or package tests
  • or, for functions and "core" functionality, tests in lib/tests or pkgs/test
  • made sure NixOS tests are linked to the relevant packages
• Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
• Tested basic functionality of all binary files (usually in ./result/bin/)
• 24.05 Release Notes (or backporting 23.05 and 23.11 Release notes)
  • (Package updates) Added a release notes entry if the change is major or breaking
  • (Module updates) Added a release notes entry if the change is significant
  • (Module addition) Added a release notes entry if adding a new NixOS module
• Fits CONTRIBUTING.md.

---

Add a 👍 reaction to pull requests you find important.

[K900]

Eval is broken :(

[K900]

That said, this is generally good stuff and we should land this (though maybe after 24.05?)

[lf-]

I don't have time to review this right now but I just want to say, thank you so much for writing this, and your commit messages are great!

[wolfgangwalther]

Eval is broken :(

I haven't been able to find out why, yet :/

[wolfgangwalther]

Eval is broken :(

I haven't been able to find out why, yet :/

Will change to "draft" until I have the time to figure that out.

[wolfgangwalther]

I rebased on latest master and should have fixed eval along the way.

[wolfgangwalther]

Improved the commit messages, fixed another eval failure and added another commit to do the following, as mentioned above:

One idea would be to just restrict pkgsCross to be used at the top-level

Let's see where ofborg fails this time.

[wolfgangwalther]

Rebased and cherry-picked #314845. This makes ./outpaths.nix evaluate locally for me.

[wolfgangwalther]

eval passes now. Ready for review. Please ignore the first commit (see other comment).

[Ericson2314]

Thank for doing a very thorough job documenting your work, @wolfgangwalther, but I am a little worried about passing this information. it was originally very intentional that I transformed the ugly/bad localSystem and crossSystem to other better arguments things in the process of going from pkgs/top-level/default.nix -> pkgs/top-level/stage.nix.

I think I see why you are doing this, but another way to look at this is that having the other package sets mashed into stage.nix like this is --- while convenient --- and architectural abomination. The package sets are are supposed to be intended, and the proper way to get them is just to call nixpkgs with different arguments. Having some convience example calls is fine, but it was a mistake to ever make this part of the package set. pkgsCross being bad makes it unsurprising to me that supporting it forces further architectural imperfections. It's "virally" the wrong thing.

[wolfgangwalther]

@Ericson2314 I'm not sure what the takeaway from this is. Are you saying that package sets should not be composed? I.e. we should only have (example) pkgsStatic at the top-level, but not pkgsCross.XYZ.pkgsStatic?

Clearly the status-quo, where it's possible to call them, but they only compose in parts, but not in others, is a problem.

[wolfgangwalther]

Overall, I can find 5 different approaches how to do deal with those package sets:

• Get rid of all of them and force users to pass arguments to nixpkgs.
• Get rid of them when nested. pkgsStatic exists, but pkgsCross.XYZ.pkgsStatic is forbidden (either undefined or throwing an error).
• Status quo: Have them, but let them compose badly.
• Change them to always override all previous settings. So pkgsCross.XYZ.pkgsStatic would not be cross anymore, but just the same as pkgsStatic directly.
• Make them compose well. pkgsCross.XYZ.pkgsStatic would do exactly what you'd expect it to do.

Obviously, I'm in favor of the last option.

Once we agree on where this should go, we can look into how to make this nicer architecturally.

[wolfgangwalther]

but I am a little worried about passing this information. it was originally very intentional that I transformed the ugly/bad localSystem and crossSystem to other better arguments things in the process of going from pkgs/top-level/default.nix -> pkgs/top-level/stage.nix.

I have changed the approach from "pass localSystem and crossSystem to stage.nix" to "make nixpkgsFun itself composable". So "dealing with the bad/ugly localSystem/crossSystem" only happens in default.nix now - stage.nix is a lot cleaner than before.

@Ericson2314 Do you like this approach better?

[wolfgangwalther]

After rebasing again, eval passes. Ready for review, again.

[aforemny]

Reading through the comments, I feel settling on one of @wolfgangwalther's proposed solutions would be the way forward.

I very much agree that the current interface suggests arbitrary composability, and if the implementation is not supposed to live up to that suggestion, the interface should be changed to be more clear, for instance by removing nested pkgsStatic, pkgsCross, or have them throw with an error message detailing how to achieve the intended outcome.

I am personally leaning towards making the current interface work, ie. have all of pkgsStatic, pkgsCross compose well, as it is very easy to use and would be a backwards-compatible solution.

However, I do not feel qualified making a decision here.

@Ericson2314 could you provide some guidance on progressing here, possibly in light of the latest changes? I think first we should settle on the solution that we want (change the interface or change the implementation), and then see if this PR does address that.

I am interested in this, is there a way I could help out?

edit: Ah, I missed some of the conversation afterwards. It seems this is progressing towards making the current interface compose well. What is left to do here?

[wolfgangwalther]

Ah, I missed some of the conversation afterwards. It seems this is progressing towards making the current interface compose well. What is left to do here?

I'm not sure about that, I don't draw the same conclusions about the discussion, so far. I haven't received much feedback, but the message I understood so far is along the lines of "those pkgsX attributes never were a good idea, so I'm not keen on improving them". Which I can kind of understand where it's coming from and might make sense on a green field, but not in the case we have: Those attributes exist and don't work well, so leaving them in a broken state is not helpful.

[wolfgangwalther]

I am interested in this, is there a way I could help out?

@aforemny You could start with #351608, helping to push that forward. This is a single commit that I was able to extract from this PR here. Maybe taking smaller steps could help.

[r-burns]

Those attributes exist and don't work well, so leaving them in a broken state is not helpful.

Agree 100%. In an ideal world maybe we'd have a better mechanism for doing this and we wouldn't be using pkgsCross internally at all. But as it is we rely on these and they're broken, so why not improve them a bit? It seems your changes are quite well-contained, so I'm not as worried about the "virality" of this PR.

It seems that a lot of good changes to the cross machinery stall out because it's hard to gain confidence in them. Maybe that's what's happening here? I'd be happy to help out with testing this on a variety of cross-compilation scenarios if that will improve confidence in the implications of this PR.

[wolfgangwalther]

I rebased this and changed the division of labor between top-level/default.nix and top-level/stage.nix a bit compared to my previous approach. The code is much cleaner now and also less likely to break when new package sets are added. Yes, we still use localSystem / crossSystem in stage.nix, we have to. But it's actually reduced after the refactors at the end, a step in the right direction I guess.

All the new tests still pass, so all the related issues about composition of those package sets are still solved.

Since @Ericson2314 doesn't seem to have the bandwidth to look at this, maybe somebody else from @NixOS/stdenv can help out reviewing this?

[wolfgangwalther]

Rebased to latest master to resolve a merge conflict.

Eval worked locally before, but now has an infinite recursion. Note that this comes up with the test-suite introduced in the 4th commit, so before any functional changes made by this PR. This happened more than once in the last couple of months now, so the tests here also seem to uncover other problems that we're not even aware of, yet.

Will try to find out what happened there later.

Edit: It's this line that is triggering the infinite recursion: assert isIdempotent "pkgsLLVM";. So related to pkgsLLVM.