pkgsMusl.pkgsStatic.stdenv is broken

[r-burns]

Describe the bug
The gcc used for pkgsMusl.pkgsStatic.stdenv fails to build on x86_64-linux, due to a faulty bintools setup, as pointed out by Sandro here: #112096 (comment).

I think the reason for this is that the binutils hostPlatform is different from the targetPlatform (host is dynamic musl, target is static) but the configs are both x86_64-unknown-linux-musl. So our package definitions expect the resulting bintools to have a x86_64-unknown-linux-musl targetPrefix, but they do not because the autotools configure script only sees matching host/target configs, so it produces unprefixed ld, as, etc binaries.

To Reproduce
nix-build '<nixpkgs>' -A pkgsMusl.pkgsStatic.stdenv

Notify maintainers
Nixpkgs cross maintainers: @matthewbauer @Ericson2314

Metadata

• system: "x86_64-linux"
• host os: Linux 5.4.87, NixOS, 21.03pre268206.536fe36e23a (Okapi)
• multi-user?: yes
• sandbox: yes
• version: nix-env (Nix) 2.3.10
• channels(root): "nixos-21.03pre268206.536fe36e23a"
• nixpkgs: /nix/var/nix/profiles/per-user/root/channels/nixos

Maintainer information:

# a list of nixpkgs attributes affected by the problem
attribute:
  - binutils
  - gcc
# a list of nixos modules affected by the problem
module:

[samueldr]

Isn't pkgsStatic already musl? I ask, but I know it is.

nix-repl> pkgsStatic.targetPlatform.isMusl
true

I'm not sure what the difference should be when comparing pkgsMusl.pkgsStatic.* vs. pkgsStatic.* considering pkgsStatic is using musl.

To me, it feels like pkgsMusl.pkgsStatic should be a no-op, but obviously (as I'm observing) it is not.

[r-burns]

Musl is not necessarily static, so pkgsMusl can be dynamic. I think pkgsStatic implies musl but only because glibc is bad at static linking. So I think pkgsStatic.pkgsMusl would be redundant but pkgsMusl.pkgsStatic would not be.

[samueldr]

Yes, sorry, I should have made it clear that they shouldn't be expected to be equivalent, but at the current time, I would have assumed them to be equivalent. Since pkgsSatic is using musl, pkgsMusl.pkgsStatic to me should get to the same end result.

I'm wondering what are the differences in semantics that makes (the current implementation details heavy) pkgsStatic not be equivalent to pkgsMusl.pkgsStatic. Why is pkgsMusl.pkgStatic more broken?

[r-burns]

Hmm yeah that's really weird. I'm not sure of the underlying cause, but here's one observable difference:

$ nix eval -f . pkgsStatic.pkgsMusl.stdenv.hostPlatform.isStatic
false
$ nix eval -f . pkgsMusl.pkgsStatic.stdenv.hostPlatform.isStatic
true

[r-burns]

Wait a minute, that makes no sense. Shouldn't pkgsStatic.pkgsMusl mean musl and static? Does the pkgsMusl set "start over" and forget that it's still supposed to be static?

[samueldr]

Definitions:

• nixpkgs/pkgs/top-level/stage.nix

  Lines 161 to 179 in b1e00e4

  	# All packages built with the Musl libc. This will override the
  	# default GNU libc on Linux systems. Non-Linux systems are not
  	# supported.
  	pkgsMusl = if stdenv.hostPlatform.isLinux then nixpkgsFun {
  	overlays = [ (self': super': {
  	pkgsMusl = super';
  	})] ++ overlays;
  	${if stdenv.hostPlatform == stdenv.buildPlatform
  	then "localSystem" else "crossSystem"} = {
  	parsed = stdenv.hostPlatform.parsed // {
  	abi = {
  	gnu = lib.systems.parse.abis.musl;
  	gnueabi = lib.systems.parse.abis.musleabi;
  	gnueabihf = lib.systems.parse.abis.musleabihf;
  	}.${stdenv.hostPlatform.parsed.abi.name}
  	or lib.systems.parse.abis.musl;
  	};
  	};
  	} else throw "Musl libc only supports Linux systems.";

• nixpkgs/pkgs/top-level/stage.nix

  Lines 212 to 231 in b1e00e4

  	# Fully static packages.
  	# Currently uses Musl on Linux (couldn’t get static glibc to work).
  	pkgsStatic = nixpkgsFun ({
  	overlays = [ (self': super': {
  	pkgsStatic = super';
  	})] ++ overlays;
  	crossOverlays = [ (import ./static.nix) ];
  	} // lib.optionalAttrs stdenv.hostPlatform.isLinux {
  	crossSystem = {
  	isStatic = true;
  	parsed = stdenv.hostPlatform.parsed // {
  	abi = {
  	gnu = lib.systems.parse.abis.musl;
  	gnueabi = lib.systems.parse.abis.musleabi;
  	gnueabihf = lib.systems.parse.abis.musleabihf;
  	}.${stdenv.hostPlatform.parsed.abi.name}
  	or lib.systems.parse.abis.musl;
  	};
  	};
  	});

I guess one difference is the possible use of localSystem in pkgsMusl rather than crossSystem

Though I don't really grok how this "resets" pkgsMusl... I guess nixpkgsFun does not compose recursively on all attributes of the platform?

Should pkgsStatic also do if stdenv.hostPlatform == stdenv.buildPlatform then "localSystem" else "crossSystem"? Or is it that pkgsMusl should always cross? (probably not)

[r-burns]

Thanks for posting that, I had assumed the pkgsStatic/pkgsMusl logic would be too complicated to skim but that's actually very well-contained and self-documenting. Is the "resetting" just a matter of pkgsMusl setting crossSystem.parsed but not crossSystem.isStatic? I think it would be fairly simple to have pkgsMusl propagate stdenv.hostPlatform.isStatic if that behavior would be more correct.

Also, I think I understand the non-commutativity now. When pkgsMusl is native, it switches over the build system to use musl as well. I think this is because the ordinary linux stdenv is able to bootstrap with musl or glibc, so we can take the native compilation pathway as it's more reliable.

$ nix eval nixpkgs.stdenv.buildPlatform.isMusl                                
false
$ nix eval nixpkgs.pkgsMusl.stdenv.buildPlatform.isMusl
true

However, there is no native static stdenv. The static stdenv still uses a dynamic buildPlatform, and goes the cross-compilation route to produce a final static package. So pkgsMusl.pkgsStatic is cross-compiling from dynamic to static musl, while pkgsStatic.pkgsMusl triggers the stdenv.hostPlatform != stdenv.buildPlatform case in pkgsMusl and ends up cross-compiling from dynamic glibc to static musl. Well, actually dynamic glibc to dynamic musl, since hostPlatform.isStatic isn't propagated.

Primitive demo:

for set in pkgsMusl.pkgsStatic pkgsStatic.pkgsMusl; do
    for platform in buildPlatform hostPlatform; do
        x=nixpkgs.$set.stdenv.$platform
        echo $set.$platform: $(nix eval $x.config) $(nix eval $x.isStatic)
    done
done

pkgsMusl.pkgsStatic.buildPlatform: "x86_64-unknown-linux-musl" false
pkgsMusl.pkgsStatic.hostPlatform: "x86_64-unknown-linux-musl" true
pkgsStatic.pkgsMusl.buildPlatform: "x86_64-unknown-linux-gnu" false
pkgsStatic.pkgsMusl.hostPlatform: "x86_64-unknown-linux-musl" false

Ultimately this is all pretty tangential to the original issue, lol. I guess my conclusion is that pkgsMusl.pkgsStatic is broken due to the issue in the OP but otherwise sound and useful. Meanwhile pkgsStatic.pkgsMusl is completely busted.

[stale]

I marked this as stale due to inactivity. → More info

[r-burns]

pkgsMusl.pkgsStatic.stdenv was fixed by #119625.

pkgsStatic.pkgsMusl.stdenv is still broken.

[stale]

I marked this as stale due to inactivity. → More info

[uri-canva]

Opened a new issue for pkgsStatic using musl, so it's easier to find and people can refer to it: #196329.