Staging next (with glibc 2.30)

[FRidh]

Motivation for this change

This is most likely the last staging cycle before branch-off.
Notably contains changes for #66528.

Things done

• Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
• Built on platform(s)
  • NixOS
  • macOS
  • other Linux distributions
• Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
• Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
• Tested execution of all binary files (usually in ./result/bin/)
• Determined the impact on package closure size (by running nix path-info -S before and after)
• Ensured that relevant documentation is up to date
• Fits CONTRIBUTING.md.

[worldofpeace]

I believe @disassembler and I agreed we for sure wanted to get this into 20.03.

[vcunat]

I should say here that the glibc PR broke cross evaluation (I found after merging), but hopefully we can fix that soon.

[FRidh]

Indeed, that needs to be fixed before it can go into master.

[bhipple]

If possible, it'd be nice to get #78501 in, which adds an improved fetchCargo implementation for Rust applications. This defaults to false and has no functional impact, but it causes a bunch of rebuilds due to attribute shuffling, so I have it going to staging.

Ideally I'd like to get it in 20.03, because even though it's a no-op, once merged I'd like to start porting Rust applications to use the newer fetchCargo, but that would complicate backports if the newer fetch implementation isn't in the release branch.

I believe it's ready to go, and it's intended just as a performance optimization around disk usage in the stores and binary caches, plus allowing for hashed mirror usage, but if on review it looks like it still needs work feel free to omit it (I don't want to hold up the release).

[FRidh]

@bhipple it needs some approved reviews before it would go into staging.

[Ma27]

I should say here that the glibc PR broke cross evaluation (I found after merging), but hopefully we can fix that soon.

@vcunat @FRidh did you already pushed a commit to fix this issue? And if not, can you please point me to some failures that were caused by this to take a closer look? :)

[FRidh]

It's not fixed yet.
@Ma27

$ nix-build -A pkgsCross.raspberryPi.openssl
error: infinite recursion encountered, at /home/freddy/Code/libraries/nixpkgs/pkgs/top-level/all-packages.nix:13581:3
(use '--show-trace' to show detailed location information)

Note pkgsStatic does evaluate fine.

[FRidh]

@vcunat later today the building of staging-next is finished. If we would merge this into master, we may do another staging cycle before branch-off. It would of course mean cross is broken on master. I'm assuming that fixing that would be a rather non-invasive change and that it is thus worthwhile to merge this anyway. What do you think?

[vcunat]

ATM I don't know what's broken, but I'm confident I can put some time into it during this weekend.

[vcunat]

BTW, the unstable channel is already blocked on an UEFI test: https://hydra.nixos.org/job/nixos/trunk-combined/tested#tabs-status I'm not sure if that was caused by the previous staging iteration or something else.

[FRidh]

@vcunat that's tracked in #79304

[Ma27]

The error @FRidh showed me can be fixed with the following patch:

index 60c0730dce1..193381b04b4 100644
--- a/pkgs/stdenv/linux/default.nix
+++ b/pkgs/stdenv/linux/default.nix
@@ -225,7 +225,7 @@ in
       inherit (prevStage)
         ccWrapperStdenv
         gcc-unwrapped coreutils gnugrep
-        perl gnum4 bison;
+        perl gnum4 bison python3;
       dejagnu = super.dejagnu.overrideAttrs (a: { doCheck = false; } );

       # We need libidn2 and its dependency libunistring as glibc dependency.
@@ -268,7 +268,7 @@ in
       inherit (prevStage)
         ccWrapperStdenv
         binutils coreutils gnugrep
-        perl patchelf linuxHeaders gnum4 bison libidn2 libunistring;
+        perl patchelf linuxHeaders gnum4 bison libidn2 libunistring python3;
       ${localSystem.libc} = getLibc prevStage;
       # Link GCC statically against GMP etc.  This makes sense because
       # these builds of the libraries are only used by GCC, so it
@@ -298,7 +298,7 @@ in
       # because gcc (since JAR support) already depends on zlib, and
       # then if we already have a zlib we want to use that for the
       # other purposes (binutils and top-level pkgs) too.
-      inherit (prevStage) gettext gnum4 bison gmp perl texinfo zlib linuxHeaders libidn2 libunistring;
+      inherit (prevStage) gettext gnum4 bison gmp perl texinfo zlib linuxHeaders libidn2 libunistring python3;
       ${localSystem.libc} = getLibc prevStage;
       binutils = super.binutils.override {
         # Don't use stdenv's shell but our own
@@ -399,7 +399,7 @@ in
         inherit (prevStage)
           gzip bzip2 xz bash coreutils diffutils findutils gawk
           gnumake gnused gnutar gnugrep gnupatch patchelf
-          attr acl zlib pcre libunistring libidn2;
+          attr acl zlib pcre libunistring libidn2 python3;
         ${localSystem.libc} = getLibc prevStage;
       } // lib.optionalAttrs (super.stdenv.targetPlatform == localSystem) {
         # Need to get rid of these when cross-compiling.

Unfortunately I got another "infinite recursion" error because of libiconv. As libiconv seems to be a part of glibc, but also a dependency for libunistring (which is needed by glibc 2.30) this appears to be a dependency cycle that somehow breaks the evaluation of cross-builds (tbh I have no idea why this doesn't break the evaluation of non-cross builds). Unfortunately I'm not sufficiently familiar with the stdenv, which is why I failed at trying to fix this as well :/

[FRidh]

Cross will now be broken on master, but I don't want to build up merge conflicts by delaying merging and this way we can do one more staging cycle (the main reason).

[vcunat]

After 811448e the -small channel should succeed, at least.