Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pkgsBuildBuild.pkgsi686Linux in pkgsCross is build for cross platform #212494

Open
Cynerd opened this issue Jan 24, 2023 · 9 comments
Open

pkgsBuildBuild.pkgsi686Linux in pkgsCross is build for cross platform #212494

Cynerd opened this issue Jan 24, 2023 · 9 comments
Labels
0.kind: build failure A package fails to build 6.topic: cross-compilation Building packages on a different platform than they will be used on

Comments

@Cynerd
Copy link
Contributor

Cynerd commented Jan 24, 2023

Steps To Reproduce

Steps to reproduce the behavior:

  1. build nix build nixpkgs#legacyPackages.x86_64-linux.pkgsCross.armv7l-hf-multiplatform.pkgsBuildBuild.gcc_multi

Build log

From config.log for 32bit gcc.

/nix/store/49zl26cmymymzzh1xqahd6cp1g10l141-glibc-2.35-224-dev/include/gnu/stubs.h:7:11: fatal error: gnu/stubs-32.h: No such file or directory
    7 | # include <gnu/stubs-32.h>
      |           ^~~~~~~~~~~~~~~~

Additional context

The error is caused by missing file in glibc-multi. That is created by merging glibc and pkgsi686Linux.glibc. It turns out that issue is in pkgsi686Linux.glibc which for some reason uses some options from host platform. This is output of nix show-derivation nixpkgs#legacyPackages.x86_64-linux.pkgsCross.armv7l-hf-multiplatform.pkgsBuildBuild.pkgsi686Linux.glibc

{
  "/nix/store/27skn0aayk7xivdrsqz7sh4zr5v9c61i-glibc-armv7l-unknown-linux-gnueabihf-2.35-224.drv": {
    "args": [
      "-e",
      "/nix/store/6xg259477c90a229xwmb53pdfkn6ig3g-default-builder.sh"
    ],
    "builder": "/nix/store/pn5sy7dwndzpxbih7xjl7g7inkdphxzq-bash-5.2-p15/bin/bash",
    "env": {
      "BASH_SHELL": "/bin/sh",
      "NIX_CFLAGS_COMPILE": "-Wno-error=missing-attributes",
      "NIX_HARDENING_ENABLE": "pic strictoverflow format relro bindnow",
      "NIX_NO_SELF_RPATH": "1",
      "__structuredAttrs": "",
      "bin": "/nix/store/6xbpkixalf0s429qd7d5syv0gghwxxlh-glibc-armv7l-unknown-linux-gnueabihf-2.35-224-bin",
      "buildInputs": "/nix/store/kmw32416a7ya2pk5kbrfyyk9b6szpsxk-linux-headers-6.1",
      "builder": "/nix/store/pn5sy7dwndzpxbih7xjl7g7inkdphxzq-bash-5.2-p15/bin/bash",
      "cmakeFlags": "-DCMAKE_SYSTEM_NAME=Linux -DCMAKE_SYSTEM_PROCESSOR=armv7l -DCMAKE_HOST_SYSTEM_NAME=Linux -DCMAKE_HOST_SYSTEM_PROCESSOR=i686",
      "configureFlags": "-C --enable-add-ons --sysconfdir=/etc --enable-stack-protector=strong --enable-bind-now --with-headers=/nix/store/kmw32416a7ya2pk5kbrfyyk9b6szpsxk-linux-headers-6.1/include --disable-profile --enable-kernel=3.10.0 --without-fp --with-__thread --disable-crypt --build=i686-unknown-linux-gnu --host=armv7l-unknown-linux-gnueabihf",
      "debug": "/nix/store/9mwwaw0bi4pd6zyvr8ms73c677x1a159-glibc-armv7l-unknown-linux-gnueabihf-2.35-224-debug",
      "depsBuildBuild": "/nix/store/2n1gi30hscji7s0j7lk76c2fh0zm657w-gcc-wrapper-11.3.0",
      "depsBuildBuildPropagated": "",
      "depsBuildTarget": "",
      "depsBuildTargetPropagated": "",
      "depsHostHost": "",
      "depsHostHostPropagated": "",
      "depsTargetTarget": "",
      "depsTargetTargetPropagated": "",
      "dev": "/nix/store/ypnmzg2awsqnqd8bq60cjx42742f7inw-glibc-armv7l-unknown-linux-gnueabihf-2.35-224-dev",
      "doCheck": "",
      "doInstallCheck": "",
      "enableParallelBuilding": "1",
      "enableParallelChecking": "1",
      "hardeningDisable": "fortify pie stackprotector",
      "installFlags": "sysconfdir=$(out)/etc",
      "is64bit": "",
      "linuxHeaders": "/nix/store/kmw32416a7ya2pk5kbrfyyk9b6szpsxk-linux-headers-6.1",
      "makeFlags": "OBJCOPY=armv7l-unknown-linux-gnueabihf-objcopy",
      "mesonFlags": "--cross-file=/nix/store/liv5rkz44z1cy77w4a92fg2ibnf852ng-cross-file.conf",
      "name": "glibc-armv7l-unknown-linux-gnueabihf-2.35-224",
      "nativeBuildInputs": "/nix/store/r10al970d8z0mkrwxww4lw0jkq9rhhpd-bison-3.8.2 /nix/store/0k4afl42b6hd0m8mjpl5mna5q5siraij-python3-minimal-3.10.9 /nix/store/g1669dakr0yxrx2w6c7czqih8cq3351r-separate-debug-info.sh",
      "out": "/nix/store/93b480qcg6kd29s1j6slfh03krqiq84a-glibc-armv7l-unknown-linux-gnueabihf-2.35-224",
      "outputs": "out bin dev static debug",
      "patches": "/nix/store/sxv7vwsanif38wri00hr2cg91ffagcjf-2.35-master.patch.gz /nix/store/8haph3ng4mgsqr6p4024vj8k6kg3mqc4-nix-locale-archive.patch /nix/store/b1w7zbvm39ff1i52iyjggyvw2rdxz104-dont-use-system-ld-so-cache.patch /nix/store/mnglr8rr7nl444h7p50ysyq8qd0fm1lm-dont-use-system-ld-so-preload.patch /nix/store/za0pg7fmysrcwrqcal26fnmzw6vycgdn-fix_path_attribute_in_getconf.patch /nix/store/7kw224hdyxd7115lrqh9a4dv2x8msq2s-fix-x64-abi.patch /nix/store/001gp43bjqzx60cg345n2slzg7131za8-nix-nss-open-files.patch /nix/store/l9yiy982yxkijaazijr741w36a97llcs-0001-Revert-Remove-all-usage-of-BASH-or-BASH-in-installed.patch",
      "pname": "glibc",
      "postConfigure": "# Hack: get rid of the `-static' flag set by the bootstrap stdenv.\n# This has to be done *after* `configure' because it builds some\n# test binaries.\nexport NIX_CFLAGS_LINK=\nexport NIX_LDFLAGS_BEFORE=\n\nexport NIX_DONT_SET_RPATH=1\nunset CFLAGS\n\n# Apparently --bindir is not respected.\nmakeFlagsArray+=(\"bindir=$bin/bin\" \"sbindir=$bin/sbin\" \"rootsbindir=$bin/sbin\")\n",
      "postInstall": "# This is based on http://www.linuxfromscratch.org/lfs/view/development/chapter06/glibc.html\n# Instead of using their patch to build a build-native localedef,\n# we simply use the one from buildPackages\npushd ../glibc-2*/localedata\nexport I18NPATH=$PWD GCONV_PATH=$PWD/../iconvdata\nmkdir -p $NIX_BUILD_TOP//nix/store/hnnmq5vp5y32imk3c4i2pdvsqxvw4chz-glibc-2.35-224/lib/locale\n/nix/store/sl93gr469wkjrngs46490qm9y2ma0din-glibc-2.35-224-bin/bin/localedef \\\n  --alias-file=../intl/locale.alias \\\n  -i locales/C \\\n  -f charmaps/UTF-8 \\\n  --prefix $NIX_BUILD_TOP \\\n  --little-endian \\\n  C.UTF-8\ncp -r $NIX_BUILD_TOP//nix/store/hnnmq5vp5y32imk3c4i2pdvsqxvw4chz-glibc-2.35-224/lib/locale $out/lib\npopd\n\ntest -f $out/etc/ld.so.cache && rm $out/etc/ld.so.cache\n\nif test -n \"$linuxHeaders\"; then\n    # Include the Linux kernel headers in Glibc, except the `scsi'\n    # subdirectory, which Glibc provides itself.\n    (cd $dev/include && \\\n     ln -sv $(ls -d $linuxHeaders/include/* | grep -v scsi\\$) .)\nfi\n\n# Fix for NIXOS-54 (ldd not working on x86_64).  Make a symlink\n# \"lib64\" to \"lib\".\nif test -n \"$is64bit\"; then\n    ln -s lib $out/lib64\nfi\n\n# Get rid of more unnecessary stuff.\nrm -rf $out/var $bin/bin/sln\n\n# Backwards-compatibility to fix e.g.\n# \"configure: error: Pthreads are required to build libgomp\" during `gcc`-build\n# because it's not actually needed anymore to link against `pthreads` since\n# it's now part of `libc.so.6` itself, but the gcc build breaks if\n# this doesn't work.\nln -sf $out/lib/libpthread.so.0 $out/lib/libpthread.so\nln -sf $out/lib/librt.so.1 $out/lib/librt.so\nln -sf $out/lib/libdl.so.2 $out/lib/libdl.so\nln -sf $out/lib/libutil.so.1 $out/lib/libutil.so\ntouch $out/lib/libpthread.a\n\n# Put libraries for static linking in a separate output.  Note\n# that libc_nonshared.a and libpthread_nonshared.a are required\n# for dynamically-linked applications.\nmkdir -p $static/lib\nmv $out/lib/*.a $static/lib\nmv $static/lib/lib*_nonshared.a $out/lib\n# Some of *.a files are linker scripts where moving broke the paths.\nsed \"/^GROUP/s|$out/lib/lib|$static/lib/lib|g\" \\\n  -i \"$static\"/lib/*.a\n\n# Work around a Nix bug: hard links across outputs cause a build failure.\ncp $bin/bin/getconf $bin/bin/getconf_\nmv $bin/bin/getconf_ $bin/bin/getconf\n",
      "postPatch": "# Needed for glibc to build with the gnumake 3.82\n# http://comments.gmane.org/gmane.linux.lfs.support/31227\nsed -i 's/ot \\$/ot:\\n\\ttouch $@\\n$/' manual/Makefile\n\n# nscd needs libgcc, and we don't want it dynamically linked\n# because we don't want it to depend on bootstrap-tools libs.\necho \"LDFLAGS-nscd += -static-libgcc\" >> nscd/Makefile\n\n# Ensure that `__nss_files_fopen` can still be wrapped by `libredirect`.\nsed -i -e '/libc_hidden_def (__nss_files_fopen)/d' nss/nss_files_fopen.c\nsed -i -e '/libc_hidden_proto (__nss_files_fopen)/d' include/nss_files.h\n",
      "preBuild": "",
      "preConfigure": "export PWD_P=$(type -tP pwd)\nfor i in configure io/ftwtest-sh; do\n    # Can't use substituteInPlace here because replace hasn't been\n    # built yet in the bootstrap.\n    sed -i \"$i\" -e \"s^/bin/pwd^$PWD_P^g\"\ndone\n\nmkdir ../build\ncd ../build\n\nconfigureScript=\"`pwd`/../$sourceRoot/configure\"\n\nmakeFlags=\"$makeFlags BUILD_LDFLAGS=-Wl,-rpath,/nix/store/1jb8f8yw7yvchsw3gpsi5bwkfa37anbv-glibc-armv7l-unknown-linux-gnueabihf-2.35-224/lib OBJDUMP=/nix/store/1cq6nm161b4jw0kqr0lm77qiqprb9znw-armv7l-unknown-linux-gnueabihf-binutils-2.39/bin/objdump\"\n\n\nsed -i s/-lgcc_eh//g \"../$sourceRoot/Makeconfig\"\n\ncat > config.cache << \"EOF\"\nlibc_cv_forced_unwind=yes\nlibc_cv_c_cleanup=yes\nlibc_cv_gnu89_inline=yes\nEOF\n\n# ./configure has logic like\n#\n#     AR=`$CC -print-prog-name=ar`\n#\n# This searches various directories in the gcc and its wrapper. In nixpkgs,\n# this returns the bare string \"ar\", which is build ar. This can result as\n# a build failure with the following message:\n#\n#     libc_pic.a: error adding symbols: archive has no index; run ranlib to add one\n#\n# (Observed cross compiling from aarch64-linux -> armv7l-linux).\n#\n# Nixpkgs passes a correct value for AR and friends, so to use the correct\n# set of tools, we only need to delete this special handling.\nsed -i \\\n  -e '/^AR=/d' \\\n  -e '/^AS=/d' \\\n  -e '/^LD=/d' \\\n  -e '/^OBJCOPY=/d' \\\n  -e '/^OBJDUMP=/d' \\\n  $configureScript\n",
      "preInstall": "",
      "profilingLibraries": "",
      "propagatedBuildInputs": "",
      "propagatedNativeBuildInputs": "",
      "separateDebugInfo": "1",
      "src": "/nix/store/hsji8dkz2azgn3ylyzx9898qyrf46axv-glibc-2.35.tar.xz",
      "static": "/nix/store/wqmz7xq2pw08rhhayyvg271wqpfwcna3-glibc-armv7l-unknown-linux-gnueabihf-2.35-224-static",
      "stdenv": "/nix/store/9772prci5wkkqkjm4kj8n4m915ygkc0m-stdenv-linux",
      "strictDeps": "1",
      "system": "i686-linux",
      "version": "2.35-224",
      "withLibcrypt": ""
    },
    "inputDrvs": {
      "/nix/store/2gfgl2n7wxzr3w3z6qshgd8qyx45a8lj-glibc-2.35-224.drv": [
        "bin",
        "out"
      ],
      "/nix/store/5kj6nifh4080ym1kimcvb8ax1mzvkk5r-armv7l-unknown-linux-gnueabihf-binutils-2.39.drv": [
        "out"
      ],
      "/nix/store/70sabiv6cnaj943jszsib3ki9x3q6sg7-bison-3.8.2.drv": [
        "out"
      ],
      "/nix/store/9k3zwhzkzj7hbfs5a62imrpg58jhmdhv-bash-5.2-p15.drv": [
        "out"
      ],
      "/nix/store/bd67qkpfzdx6isqnrb46zvr4f8akwbly-stdenv-linux.drv": [
        "out"
      ],
      "/nix/store/pwb6lncqmal5nqwzwi73dv9grsn1hg8c-gcc-wrapper-11.3.0.drv": [
        "out"
      ],
      "/nix/store/qm4vj29kcp188mbbabk6ndpjb15711sz-linux-headers-6.1.drv": [
        "out"
      ],
      "/nix/store/varvf7907q8xm1p1ccil359l2zbcrnq3-glibc-armv7l-unknown-linux-gnueabihf-2.35-224.drv": [
        "out"
      ],
      "/nix/store/wpsjbgw43jyji44vralbnmylfqrrq4x7-python3-minimal-3.10.9.drv": [
        "out"
      ],
      "/nix/store/zxa99fi6bvk41r17xkqa4d6payipyxhx-glibc-2.35.tar.xz.drv": [
        "out"
      ]
    },
    "inputSrcs": [
      "/nix/store/001gp43bjqzx60cg345n2slzg7131za8-nix-nss-open-files.patch",
      "/nix/store/6xg259477c90a229xwmb53pdfkn6ig3g-default-builder.sh",
      "/nix/store/7kw224hdyxd7115lrqh9a4dv2x8msq2s-fix-x64-abi.patch",
      "/nix/store/8haph3ng4mgsqr6p4024vj8k6kg3mqc4-nix-locale-archive.patch",
      "/nix/store/b1w7zbvm39ff1i52iyjggyvw2rdxz104-dont-use-system-ld-so-cache.patch",
      "/nix/store/g1669dakr0yxrx2w6c7czqih8cq3351r-separate-debug-info.sh",
      "/nix/store/l9yiy982yxkijaazijr741w36a97llcs-0001-Revert-Remove-all-usage-of-BASH-or-BASH-in-installed.patch",
      "/nix/store/liv5rkz44z1cy77w4a92fg2ibnf852ng-cross-file.conf",
      "/nix/store/mnglr8rr7nl444h7p50ysyq8qd0fm1lm-dont-use-system-ld-so-preload.patch",
      "/nix/store/sxv7vwsanif38wri00hr2cg91ffagcjf-2.35-master.patch.gz",
      "/nix/store/za0pg7fmysrcwrqcal26fnmzw6vycgdn-fix_path_attribute_in_getconf.patch"
    ],
    "outputs": {
      "bin": {
        "path": "/nix/store/6xbpkixalf0s429qd7d5syv0gghwxxlh-glibc-armv7l-unknown-linux-gnueabihf-2.35-224-bin"
      },
      "debug": {
        "path": "/nix/store/9mwwaw0bi4pd6zyvr8ms73c677x1a159-glibc-armv7l-unknown-linux-gnueabihf-2.35-224-debug"
      },
      "dev": {
        "path": "/nix/store/ypnmzg2awsqnqd8bq60cjx42742f7inw-glibc-armv7l-unknown-linux-gnueabihf-2.35-224-dev"
      },
      "out": {
        "path": "/nix/store/93b480qcg6kd29s1j6slfh03krqiq84a-glibc-armv7l-unknown-linux-gnueabihf-2.35-224"
      },
      "static": {
        "path": "/nix/store/wqmz7xq2pw08rhhayyvg271wqpfwcna3-glibc-armv7l-unknown-linux-gnueabihf-2.35-224-static"
      }
    },
    "system": "i686-linux"
  }
}

The system is according to it i686-linux but it doesn't look like it. I was unable to find the real reason for this. It might be something in stage.nix. I also tried other packages in pkgsi686Linux; they all seem to contain paths to armv7l instead of i686.

Notify maintainers

@johnae @matthewbauer @danbst

Metadata

[user@system:~]$ nix-shell -p nix-info --run "nix-info -m"
 - system: `"x86_64-linux"`
 - host os: `Linux 6.1.3, NixOS, 23.05 (Stoat), 23.05.20230122.6c582bd`
 - multi-user?: `yes`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.12.0`
 - channels(root): `"nixos"`
 - nixpkgs: `/nix/var/nix/profiles/per-user/root/channels/nixos`
@Cynerd Cynerd added the 0.kind: build failure A package fails to build label Jan 24, 2023
@Cynerd
Copy link
Contributor Author

Cynerd commented Jan 25, 2023

Eh...

$ nix eval nixpkgs#legacyPackages.x86_64-linux.pkgsCross.armv7l-hf-multiplatform.pkgsBuildBuild.stdenv.hostPlatform.system
"x86_64-linux"
$ nix eval nixpkgs#legacyPackages.x86_64-linux.pkgsCross.armv7l-hf-multiplatform.pkgsBuildBuild.pkgsi686Linux.stdenv.hostPlatform.system
"armv7l-linux"
$ nix eval nixpkgs#legacyPackages.x86_64-linux.pkgsCross.armv7l-hf-multiplatform.pkgsBuildBuild.pkgsMusl.stdenv.hostPlatform.system
"armv7l-linux"

@rrbutani rrbutani added the 6.topic: cross-compilation Building packages on a different platform than they will be used on label Jan 25, 2023
@Cynerd
Copy link
Contributor Author

Cynerd commented Jan 26, 2023

It turns out that having armv7l there is not actually something completely invalid, but rather just strange and non-intuitive. I thought that the following chain is true but no (I am ignoring target there for simplicity)..:

legacyPackages.x86_64-linux
-> build: x86_64-linux, host: x86_64-linux
legacyPackages.x86_64-linux.pkgsi686Linux
-> build: i686-linux, host: i686-linux
legacyPackages.x86_64-linux.pkgsCross.armv7l-hf-multiplatform
-> build: x86_64-linux, host: i686-linux
legacyPackages.x86_64-linux.pkgsCross.armv7l-hf-multiplatform.pkgsBuildBuild
-> build: x86_64-linux, host: x86_64-linux
legacyPackages.x86_64-linux.pkgsCross.armv7l-hf-multiplatform.pkgsBuildBuild.pkgsi686Linux
-> build: i686-linux, host: i686-linux

This would make sense to me but no... It turns out it is instead:

legacyPackages.x86_64-linux.pkgsCross.armv7l-hf-multiplatform.pkgsBuildBuild.pkgsi686Linux
-> build: i686-linux, host: armv7l-linux
legacyPackages.x86_64-linux.pkgsCross.armv7l-hf-multiplatform.pkgsBuildBuild.pkgsi686Linux.buildPackages
-> build: i686-linux, host: i686-linux

What makes it confusing is that this is the same package set, but it doesn't feel like it should be:

legacyPackages.x86_64-linux.pkgsCross.armv7l-hf-multiplatform.pkgsBuildBuild.pkgsi686Linux
legacyPackages.x86_64-linux.pkgsCross.armv7l-hf-multiplatform.pkgsBuildHost.pkgsi686Linux

The core issue is then probably that glibc_multi should not be possible to be built when the host is not i686 as it results in gcc_multi build failure.

It might also be desirable to resolve the confusion I fell into here.

@Cynerd
Copy link
Contributor Author

Cynerd commented Jan 30, 2023

For completeness, my original entry point to this was stdenv_32bit which uses gcc_multi. The typical approach in the packaging when software needs a 32-bit host compiler, I would expect, is to use stdenv_32bit. The same applies when I need it to compile some build tool during cross-compilation. I would simply use buildPackages.stdenv_32bit but that doesn't work, and neither does buildPackages.buildPackages.stdenv_32bit. Those environments consider a local system to be i686 but a host system to be the cross-compilation target.

@Cynerd
Copy link
Contributor Author

Cynerd commented Mar 8, 2023

I have more discoveries because nobody else seems to care about this... It turns out that behaviour, where pkgsi686Linux has host and target set to the target of the original package set, is a limitation of package sets instantiation. There is only possible to pass localSystem and crossSystem. The pkgsi686Linux set is generated by reevaluating the package set with these variables set to something other than the original set. The issue is that when the original set has the same host and build platform but a different target, it can only set the build and host platform. There is just no possibility of getting set where the target is different from the host directly. The only option is to use pkgsi686Linux.buildPackages to get the correct combination of platforms.

@wucke13
Copy link
Contributor

wucke13 commented May 7, 2023

@Cynerd I ran into this on multiple cross compilation attempts. What do you think would be a way forward?

@Cynerd
Copy link
Contributor Author

Cynerd commented May 17, 2023

@wucke13 I think the core issue is that we can't set a target platform. We can only set localSystem and crossSystem. The neighbor package set like the ones discussed here sometimes requires a custom target platform that can't be directly archived through the standard nixpkgs set evaluation. It might be possible to evaluate it with the correct crossPlatform but then take buildPackages of that set instead. Honestly, I am not experienced enough to fix this right now, that is also why I am writing this here and not fixing it.

@wolfgangwalther
Copy link
Contributor

I have more discoveries because nobody else seems to care about this... It turns out that behaviour, where pkgsi686Linux has host and target set to the target of the original package set, is a limitation of package sets instantiation. There is only possible to pass localSystem and crossSystem. The pkgsi686Linux set is generated by reevaluating the package set with these variables set to something other than the original set. The issue is that when the original set has the same host and build platform but a different target, it can only set the build and host platform. There is just no possibility of getting set where the target is different from the host directly. The only option is to use pkgsi686Linux.buildPackages to get the correct combination of platforms.

I didn't fully understand this whole issue - but I fixed the fact that pkgsi686Linux changes the localSystem instead of the crossSystem when "you are already cross" in #303849, specifically in 1028e73e8428f659f2c462c950355cd221b82107.

Can you try whatever you did here ;) on my branch again and see whether that helps?

@wolfgangwalther
Copy link
Contributor

On my branch mentioned above:

nix-repl> pkgs.pkgsCross.armv7l-hf-multiplatform.pkgsBuildBuild.stdenv.hostPlatform.system
"x86_64-linux"

nix-repl> pkgs.pkgsCross.armv7l-hf-multiplatform.pkgsBuildBuild.pkgsi686Linux.stdenv.hostPlatform.system
"i686-linux"

nix-repl> pkgs.pkgsCross.armv7l-hf-multiplatform.pkgsBuildBuild.pkgsMusl.stdenv.hostPlatform.system
"x86_64-linux"

nix-repl> pkgs.pkgsCross.armv7l-hf-multiplatform.pkgsBuildBuild.pkgsi686Linux.stdenv.buildPlatform.system
"x86_64-linux"

nix-repl> pkgs.pkgsCross.armv7l-hf-multiplatform.pkgsBuildBuild.pkgsi686Linux.buildPackages.stdenv.buildPlatform.system
"x86_64-linux"

nix-repl> pkgs.pkgsCross.armv7l-hf-multiplatform.pkgsBuildBuild.pkgsi686Linux.buildPackages.stdenv.hostPlatform.system
"x86_64-linux"

This looks a lot more what you'd expect, I guess?

wolfgangwalther added a commit to wolfgangwalther/nixpkgs that referenced this issue Sep 25, 2024
The previous logic to decide whether to "go musl" or "go i686" or not
was based on comparing stdenv.buildPlatform and stdenv.hostPlatform.
Those are the same for something like pkgsCross.gnu64 on linux x64. This
then leads to the following:

> pkgsCross.gnu64.pkgsMusl.stdenv.buildPlatform.config
"x86_64-unknown-linux-musl"

> pkgsCross.gnu64.pkgsMusl.stdenv.hostPlatform.config
"x86_65-unknown-linux-gnu"

This means the addition of pkgsMusl changed the *local system* and not
the *cross system* - which is surely not what is intended here. Instead
the cross system should be changed to musl / i686.

A better way to detect this is to check whether a crossSystem had
already explicitly been passed to nixpkgsFun.

This also makes the following work as expected:
pkgsCross.aarch64-darwin.pkgsx86_64Darwin
pkgsCross.mingwW64.pkgsLinux

Both now change the cross system instead of the local system.

Resolves NixOS#212494
@kuruczgy
Copy link
Contributor

kuruczgy commented Dec 3, 2024

@wolfgangwalther yep those look better, but the derivation @Cynerd wants is still not compiling (with their workaround reverted). Not sure why though, all 3 platforms seem fine, yet arm is still sneaking in somewhere

$ git checkout eb5a92b773db407046e7481c594d2fcba5aa31a3
$ git revert adfac122cd6a669c65dcf5a7d51544d9c122b977
$ nix eval .#legacyPackages.x86_64-linux.pkgsCross.raspberryPi.buildPackages.pkgsi686Linux.buildPlatform.system
"x86_64-linux"
$ nix eval .#legacyPackages.x86_64-linux.pkgsCross.raspberryPi.buildPackages.pkgsi686Linux.hostPlatform.system
"i686-linux"
$ nix eval .#legacyPackages.x86_64-linux.pkgsCross.raspberryPi.buildPackages.pkgsi686Linux.targetPlatform.system
"i686-linux"
$ nix build .#legacyPackages.x86_64-linux.pkgsCross.raspberryPi.buildPackages.glibc_multi.out
error: builder for '/nix/store/dyaqx9i9c9zbggl7icf23xsakk4ww9h8-i686-unknown-linux-gnu-nolibc-gcc-13.3.0.drv' failed with exit code 2;
       last 25 log lines:
       > checking whether fwrite_unlocked is declared... yes
       > checking whether getchar_unlocked is declared... yes
       > checking whether getc_unlocked is declared... yes
       > checking whether putchar_unlocked is declared... yes
       > checking whether putc_unlocked is declared... yes
       > checking whether getrlimit is declared... yes
       > checking whether setrlimit is declared... yes
       > checking whether getrusage is declared... yes
       > checking whether mallinfo is declared... yes
       > checking whether mallinfo2 is declared... yes
       > checking whether ldgetname is declared... no
       > checking whether times is declared... yes
       > checking whether sigaltstack is declared... yes
       > checking for struct tms... yes
       > checking for clock_t... yes
       > checking for F_SETLKW... yes
       > checking for O_CLOEXEC... yes
       > checking for fcntl.h... (cached) yes
       > checking whether O_NONBLOCK is declared... yes
       > checking for AF_UNIX... yes
       > checking for AF_INET6... yes
       > checking for _LK_LOCK... no
       > checking if mkdir takes one argument... no
       > Unknown CPU given in --with-arch=armv6kz.
       > make: *** [Makefile:4544: configure-gcc] Error 1
       For full logs, run 'nix log /nix/store/dyaqx9i9c9zbggl7icf23xsakk4ww9h8-i686-unknown-linux-gnu-nolibc-gcc-13.3.0.drv'.
error: 1 dependencies of derivation '/nix/store/dy7fwhipjcf4rkzpp5ia96855b3qwzkk-i686-unknown-linux-gnu-nolibc-gcc-wrapper-13.3.0.drv' failed to build
error: 1 dependencies of derivation '/nix/store/4cxd0w2vd164b4r3m45fbddb07sar5hv-libgcc-i686-unknown-linux-gnu-13.3.0.drv' failed to build
error: 1 dependencies of derivation '/nix/store/72p2bvj3jalgk2ffjlwwjh8mhmciabxb-stdenv-linux.drv' failed to build
error: 1 dependencies of derivation '/nix/store/gfv8cfiwpld8i7qrmw1ijxxnajzd6207-glibc-i686-unknown-linux-gnu-2.39-52.drv' failed to build
error: 1 dependencies of derivation '/nix/store/lnkg5jjz0dyshjrgk0hraj5pmkl3pdqg-glibc-multi-2.39-52.drv' failed to build

wolfgangwalther added a commit to wolfgangwalther/nixpkgs that referenced this issue Dec 15, 2024
The various pkgsXYZ top-level package sets did not pass localSystem /
crossSystem to lower levels, so far. This change propagates original
arguments to lower levels, which include the overrides made by an upper
package sets.

There is an extensive test-suite to test various combinations of package
sets in pkgs/test/top-level. There are a few basic promises made:

- Package sets must be idempotent. pkgsMusl.pkgsMusl === pkgsMusl.

- Once pkgsCross is used any subsequent package sets should affect the
  **host platform** and not the build platform. Examples:
  - pkgsMusl.pkgsCross.gnu64 is a cross compilation from musl to glibc
  - pkgsCross.gnu64.pkgsMusl is a cross compilation to musl

- Modifications from an earlier layer should not be lost, unless
  explicitly overwritten. Examples:
  - pkgsStatic.pkgsMusl should still be static.
  - pkgsStatic.pkgsCross.gnu64 should be static, but with glibc instead of
    musl.

Exceptions / TODOs:
- pkgsExtraHardening is currently not idempotent, because it applies the
  same flags over and over again.

Supersedes NixOS#136549
Resolves NixOS#114510
Resolves NixOS#212494
Resolves NixOS#281596
wolfgangwalther added a commit to wolfgangwalther/nixpkgs that referenced this issue Dec 16, 2024
The various pkgsXYZ top-level package sets did not pass localSystem /
crossSystem to lower levels, so far. This change propagates original
arguments to lower levels, which include the overrides made by an upper
package sets.

There is an extensive test-suite to test various combinations of package
sets in pkgs/test/top-level. There are a few basic promises made:

- Package sets must be idempotent. pkgsMusl.pkgsMusl === pkgsMusl.

- Once pkgsCross is used any subsequent package sets should affect the
  **host platform** and not the build platform. Examples:
  - pkgsMusl.pkgsCross.gnu64 is a cross compilation from musl to glibc
  - pkgsCross.gnu64.pkgsMusl is a cross compilation to musl

- Modifications from an earlier layer should not be lost, unless
  explicitly overwritten. Examples:
  - pkgsStatic.pkgsMusl should still be static.
  - pkgsStatic.pkgsCross.gnu64 should be static, but with glibc instead of
    musl.

Exceptions / TODOs:
- pkgsExtraHardening is currently not idempotent, because it applies the
  same flags over and over again.

Supersedes NixOS#136549
Resolves NixOS#114510
Resolves NixOS#212494
Resolves NixOS#281596
wolfgangwalther added a commit to wolfgangwalther/nixpkgs that referenced this issue Dec 21, 2024
The various pkgsXYZ top-level package sets did not pass localSystem /
crossSystem to lower levels, so far. This change propagates original
arguments to lower levels, which include the overrides made by an upper
package sets.

There is an extensive test-suite to test various combinations of package
sets in pkgs/test/top-level. There are a few basic promises made:

- Package sets must be idempotent. pkgsMusl.pkgsMusl === pkgsMusl.

- Once pkgsCross is used any subsequent package sets should affect the
  **host platform** and not the build platform. Examples:
  - pkgsMusl.pkgsCross.gnu64 is a cross compilation from musl to glibc
  - pkgsCross.gnu64.pkgsMusl is a cross compilation to musl

- Modifications from an earlier layer should not be lost, unless
  explicitly overwritten. Examples:
  - pkgsStatic.pkgsMusl should still be static.
  - pkgsStatic.pkgsCross.gnu64 should be static, but with glibc instead of
    musl.

Exceptions / TODOs:
- pkgsExtraHardening is currently not idempotent, because it applies the
  same flags over and over again.

Supersedes NixOS#136549
Resolves NixOS#114510
Resolves NixOS#212494
Resolves NixOS#281596
wolfgangwalther added a commit to wolfgangwalther/nixpkgs that referenced this issue Dec 21, 2024
The various pkgsXYZ top-level package sets did not pass localSystem /
crossSystem to lower levels, so far. This change propagates original
arguments to lower levels, which include the overrides made by an upper
package sets.

There is an extensive test-suite to test various combinations of package
sets in pkgs/test/top-level. There are a few basic promises made:

- Package sets must be idempotent. pkgsMusl.pkgsMusl === pkgsMusl.

- Once pkgsCross is used any subsequent package sets should affect the
  **host platform** and not the build platform. Examples:
  - pkgsMusl.pkgsCross.aarch64-multiplatform is a cross compilation from
musl to glibc/aarch64
  - pkgsCross.aarch64-multiplatform.pkgsMusl is a cross compilation to
musl/aarch64

- Modifications from an earlier layer should not be lost, unless
  explicitly overwritten. Examples:
  - pkgsStatic.pkgsMusl should still be static.
  - pkgsStatic.pkgsCross.gnu64 should be static, but with glibc instead
of
    musl.

Exceptions / TODOs:
- pkgsExtraHardening is currently not idempotent, because it applies the
  same flags over and over again.

Supersedes NixOS#136549
Resolves NixOS#114510
Resolves NixOS#212494
Resolves NixOS#281596
wolfgangwalther added a commit to wolfgangwalther/nixpkgs that referenced this issue Dec 21, 2024
The various pkgsXYZ top-level package sets did not pass localSystem /
crossSystem to lower levels, so far. This change propagates original
arguments to lower levels, which include the overrides made by an upper
package sets.

There is an extensive test-suite to test various combinations of package
sets in pkgs/test/top-level. There are a few basic promises made:

- Package sets must be idempotent. pkgsMusl.pkgsMusl === pkgsMusl.

- Once pkgsCross is used any subsequent package sets should affect the
  **host platform** and not the build platform. Examples:
  - pkgsMusl.pkgsCross.aarch64-multiplatform is a cross compilation from
musl to glibc/aarch64
  - pkgsCross.aarch64-multiplatform.pkgsMusl is a cross compilation to
musl/aarch64

- Modifications from an earlier layer should not be lost, unless
  explicitly overwritten. Examples:
  - pkgsStatic.pkgsMusl should still be static.
  - pkgsStatic.pkgsCross.gnu64 should be static, but with glibc instead
of musl.

Exceptions / TODOs:
- pkgsExtraHardening is currently not idempotent, because it applies the
  same flags over and over again.

Supersedes NixOS#136549
Resolves NixOS#114510
Resolves NixOS#212494
Resolves NixOS#281596
wolfgangwalther added a commit to wolfgangwalther/nixpkgs that referenced this issue Dec 21, 2024
The various pkgsXYZ top-level package sets did not pass localSystem /
crossSystem to lower levels, so far. This change propagates original
arguments to lower levels, which include the overrides made by an upper
package sets.

There is an extensive test-suite to test various combinations of package
sets in pkgs/test/top-level. There are a few basic promises made:

- Package sets must be idempotent. pkgsMusl.pkgsMusl === pkgsMusl.

- Once pkgsCross is used any subsequent package sets should affect the
  **host platform** and not the build platform. Examples:
  - pkgsMusl.pkgsCross.aarch64-multiplatform is a cross compilation from
musl to glibc/aarch64
  - pkgsCross.aarch64-multiplatform.pkgsMusl is a cross compilation to
musl/aarch64

- Modifications from an earlier layer should not be lost, unless
  explicitly overwritten. Examples:
  - pkgsStatic.pkgsMusl should still be static.
  - pkgsStatic.pkgsCross.gnu64 should be static, but with glibc instead
of musl.

Exceptions / TODOs:
- pkgsExtraHardening is currently not idempotent, because it applies the
  same flags over and over again.

Supersedes NixOS#136549
Resolves NixOS#114510
Resolves NixOS#212494
Resolves NixOS#281596
wolfgangwalther added a commit to wolfgangwalther/nixpkgs that referenced this issue Jan 11, 2025
The various pkgsXYZ top-level package sets did not pass localSystem /
crossSystem to lower levels, so far. This change propagates original
arguments to lower levels, which include the overrides made by an upper
package sets.

There is an extensive test-suite to test various combinations of package
sets in pkgs/test/top-level. There are a few basic promises made:

- Package sets must be idempotent. pkgsMusl.pkgsMusl === pkgsMusl.

- Once pkgsCross is used any subsequent package sets should affect the
  **host platform** and not the build platform. Examples:
  - pkgsMusl.pkgsCross.aarch64-multiplatform is a cross compilation from
musl to glibc/aarch64
  - pkgsCross.aarch64-multiplatform.pkgsMusl is a cross compilation to
musl/aarch64

- Modifications from an earlier layer should not be lost, unless
  explicitly overwritten. Examples:
  - pkgsStatic.pkgsMusl should still be static.
  - pkgsStatic.pkgsCross.gnu64 should be static, but with glibc instead
of musl.

Exceptions / TODOs:
- pkgsExtraHardening is currently not idempotent, because it applies the
  same flags over and over again.

Supersedes NixOS#136549
Resolves NixOS#114510
Resolves NixOS#212494
Resolves NixOS#281596
wolfgangwalther added a commit to wolfgangwalther/nixpkgs that referenced this issue Jan 11, 2025
The various pkgsXYZ top-level package sets did not pass localSystem /
crossSystem to lower levels, so far. This change propagates original
arguments to lower levels, which include the overrides made by an upper
package sets.

There is an extensive test-suite to test various combinations of package
sets in pkgs/test/top-level. There are a few basic promises made:

- Package sets must be idempotent. pkgsMusl.pkgsMusl === pkgsMusl.

- Once pkgsCross is used any subsequent package sets should affect the
  **host platform** and not the build platform. Examples:
  - pkgsMusl.pkgsCross.aarch64-multiplatform is a cross compilation from
musl to glibc/aarch64
  - pkgsCross.aarch64-multiplatform.pkgsMusl is a cross compilation to
musl/aarch64

- Modifications from an earlier layer should not be lost, unless
  explicitly overwritten. Examples:
  - pkgsStatic.pkgsMusl should still be static.
  - pkgsStatic.pkgsCross.gnu64 should be static, but with glibc instead
of musl.

Exceptions / TODOs:
- pkgsExtraHardening is currently not idempotent, because it applies the
  same flags over and over again.

Supersedes NixOS#136549
Resolves NixOS#114510
Resolves NixOS#212494
Resolves NixOS#281596
wolfgangwalther added a commit to wolfgangwalther/nixpkgs that referenced this issue Jan 11, 2025
The various pkgsXYZ top-level package sets did not pass localSystem /
crossSystem to lower levels, so far. This change propagates original
arguments to lower levels, which include the overrides made by an upper
package sets.

There is an extensive test-suite to test various combinations of package
sets in pkgs/test/top-level. There are a few basic promises made:

- Package sets must be idempotent. pkgsMusl.pkgsMusl === pkgsMusl.

- Once pkgsCross is used any subsequent package sets should affect the
  **host platform** and not the build platform. Examples:
  - pkgsMusl.pkgsCross.aarch64-multiplatform is a cross compilation from
musl to glibc/aarch64
  - pkgsCross.aarch64-multiplatform.pkgsMusl is a cross compilation to
musl/aarch64

- Modifications from an earlier layer should not be lost, unless
  explicitly overwritten. Examples:
  - pkgsStatic.pkgsMusl should still be static.
  - pkgsStatic.pkgsCross.gnu64 should be static, but with glibc instead
of musl.

Exceptions / TODOs:
- pkgsExtraHardening is currently not idempotent, because it applies the
  same flags over and over again.

Supersedes NixOS#136549
Resolves NixOS#114510
Resolves NixOS#212494
Resolves NixOS#281596
wolfgangwalther added a commit to wolfgangwalther/nixpkgs that referenced this issue Jan 11, 2025
The various pkgsXYZ top-level package sets did not pass localSystem /
crossSystem to lower levels, so far. This change propagates original
arguments to lower levels, which include the overrides made by an upper
package sets.

There is an extensive test-suite to test various combinations of package
sets in pkgs/test/top-level. There are a few basic promises made:

- Package sets must be idempotent. pkgsMusl.pkgsMusl === pkgsMusl.

- Once pkgsCross is used any subsequent package sets should affect the
  **host platform** and not the build platform. Examples:
  - pkgsMusl.pkgsCross.aarch64-multiplatform is a cross compilation from
musl to glibc/aarch64
  - pkgsCross.aarch64-multiplatform.pkgsMusl is a cross compilation to
musl/aarch64

- Modifications from an earlier layer should not be lost, unless
  explicitly overwritten. Examples:
  - pkgsStatic.pkgsMusl should still be static.
  - pkgsStatic.pkgsCross.gnu64 should be static, but with glibc instead
of musl.

Exceptions / TODOs:
- pkgsExtraHardening is currently not idempotent, because it applies the
  same flags over and over again.

Supersedes NixOS#136549
Resolves NixOS#114510
Resolves NixOS#212494
Resolves NixOS#281596
wolfgangwalther added a commit to wolfgangwalther/nixpkgs that referenced this issue Jan 19, 2025
The various pkgsXYZ top-level package sets did not pass localSystem /
crossSystem to lower levels, so far. This change propagates original
arguments to lower levels, which include the overrides made by an upper
package sets.

There is an extensive test-suite to test various combinations of package
sets in pkgs/test/top-level. There are a few basic promises made:

- Package sets must be idempotent. pkgsMusl.pkgsMusl === pkgsMusl.

- Once pkgsCross is used any subsequent package sets should affect the
  **host platform** and not the build platform. Examples:
  - pkgsMusl.pkgsCross.aarch64-multiplatform is a cross compilation from
musl to glibc/aarch64
  - pkgsCross.aarch64-multiplatform.pkgsMusl is a cross compilation to
musl/aarch64

- Modifications from an earlier layer should not be lost, unless
  explicitly overwritten. Examples:
  - pkgsStatic.pkgsMusl should still be static.
  - pkgsStatic.pkgsCross.gnu64 should be static, but with glibc instead
of musl.

Exceptions / TODOs:
- pkgsExtraHardening is currently not idempotent, because it applies the
  same flags over and over again.

Supersedes NixOS#136549
Resolves NixOS#114510
Resolves NixOS#212494
Resolves NixOS#281596
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
0.kind: build failure A package fails to build 6.topic: cross-compilation Building packages on a different platform than they will be used on
Projects
None yet
Development

No branches or pull requests

5 participants