Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

The NixOS SD image cannot be cross-compiled #106759

Closed
7 of 9 tasks
Atemu opened this issue Dec 12, 2020 · 28 comments
Closed
7 of 9 tasks

The NixOS SD image cannot be cross-compiled #106759

Atemu opened this issue Dec 12, 2020 · 28 comments
Labels
0.kind: bug Something is broken 6.topic: cross-compilation Building packages on a different platform than they will be used on

Comments

@Atemu
Copy link
Member

Atemu commented Dec 12, 2020 •
edited
Loading

Describe the bug
A clear and concise description of what the bug is.

Neither unstable, nor 20.09 can be cross compiled to armv7. I haven't tested aarch64 recently but, a few weeks back, it was just as broken.

Packages that don't cross compile anymore:

Additional context
Add any other context about the problem here.

Simplest configuration.nix you should be able to build:

{ lib, ... }:

{
  imports = [
    <nixpkgs/nixos/modules/installer/cd-dvd/sd-image-armv7l-multiplatform.nix>
  ];

  nixpkgs = {
    crossSystem = lib.systems.examples.armv7l-hf-multiplatform;
    localSystem = { system = "x86_64-linux"; };
  };
}

(Could this become a hydra job?)

Notify maintainers

@Ericson2314 @matthewbauer

Metadata
Please run nix-shell -p nix-info --run "nix-info -m" and paste the result.

Maintainer information:

# a list of nixpkgs attributes affected by the problem
attribute:
# a list of nixos modules affected by the problem
module:
@Atemu Atemu added the 0.kind: bug Something is broken label Dec 12, 2020
This was referenced Dec 12, 2020
Merged
Merged
Closed
Closed
@Ericson2314
Copy link
Member

For Mesa, you will need something to do something like de3f4fe

@Ericson2314 Ericson2314 mentioned this issue Dec 12, 2020
Open
@Ericson2314
Copy link
Member

See mesonbuild/meson#7453 for the lack of upstream functionality which is why we keep on hitting this issue with packages using Meson.

@matthewbauer
Copy link
Member

We do have a job for mesa: https://hydra.nixos.org/job/nixpkgs/cross-trunk/rpi.mesa.x86_64-linux

@veprbl veprbl added the 6.topic: cross-compilation Building packages on a different platform than they will be used on label Dec 13, 2020
This was referenced Dec 13, 2020
Merged
Closed
@Atemu
Copy link
Member Author

Atemu commented Dec 13, 2020

^ Mesa fix and spidermonkey bug report

Spidermonkey isn't too critical since that's only needed for the manual and should hopefully not stay a dependency for too long.

@Atemu Atemu mentioned this issue Dec 15, 2020
Closed
@Atemu
Copy link
Member Author

Atemu commented Dec 15, 2020

^ New spidermonkey bug report.

Turns out polkit (and therefore also udisks) depends on in, so we can't skip it either.

However, with:

  security.polkit.enable = false;
  services.udisks2.enable = false;

and merging open fixes locally, I was at least able to build an sdcard image!

This was referenced Dec 15, 2020
Closed
Closed
@Gaelan
Copy link
Contributor

Gaelan commented Dec 19, 2020

Also worth noting, although it doesn't prevent cross-compilation: b3640e0 results in a separate libbfd, and therefore llvm, being built when target = aarch32, despite both of those tools being target-agnostic.

@Ericson2314
Copy link
Member

@Gaelan we can make that patch unconditional on staging, I suppose.

@flokli flokli mentioned this issue Dec 25, 2020
Closed
@flokli flokli mentioned this issue Jan 1, 2021
Merged
10 tasks
@flokli
Copy link
Contributor

flokli commented Jan 1, 2021

At least the aarch64 sd image did build in #108173.

@misuzu
Copy link
Contributor

misuzu commented Jan 20, 2021

nix is also broken:

$ nix-build channel:nixos-unstable -A pkgsCross.armv7l-hf-multiplatform.nix
these 2 derivations will be built:
  /nix/store/9a2fjmigyqab2qfzi2m84dixi0inpfbn-aws-sdk-cpp-1.8.113-armv7l-unknown-linux-gnueabihf.drv
  /nix/store/1ssspmj2bvf6l7kavpcihlimd47wyx97-nix-2.3.10-armv7l-unknown-linux-gnueabihf.drv
building '/nix/store/9a2fjmigyqab2qfzi2m84dixi0inpfbn-aws-sdk-cpp-1.8.113-armv7l-unknown-linux-gnueabihf.drv'...
unpacking sources
unpacking source archive /nix/store/pwgmmhrfzq8ymbyck3jbd7jpqkk9j6xp-source
source root is source
patching sources
applying patch /nix/store/frvfi397b88f6c91p371z5r2940x005p-cmake-dirs.patch
patching file cmake/AWSSDKConfig.cmake
Hunk #1 succeeded at 86 (offset -11 lines).
patching file cmake/utilities.cmake
Hunk #1 succeeded at 39 (offset -4 lines).
Hunk #2 succeeded at 54 (offset -4 lines).
patching file toolchains/pkg-config.pc.in
applying patch /nix/store/yf9794whcciambbhx6vms7vhyj4mpsga-aws-sdk-cpp-TransferManager-ContentEncoding.patch
patching file aws-cpp-sdk-transfer/include/aws/transfer/TransferHandle.h
patching file aws-cpp-sdk-transfer/include/aws/transfer/TransferManager.h
patching file aws-cpp-sdk-transfer/source/transfer/TransferManager.cpp
updateAutotoolsGnuConfigScriptsPhase
configuring
fixing cmake files...
cmake flags: -DCMAKE_FIND_USE_SYSTEM_PACKAGE_REGISTRY=OFF -DCMAKE_FIND_USE_PACKAGE_REGISTRY=OFF -DCMAKE_EXPORT_NO_PACKAGE_REGISTRY=ON -DCMAKE_BUILD_TYPE=Release -DCMAKE_SKIP_BUILD_RPATH=ON -DBUILD_TESTING=OFF -D
CMAKE_INSTALL_LOCALEDIR=/nix/store/nlgw7vh3nrzcjcswfzkb5lrbpckmddw1-aws-sdk-cpp-1.8.113-armv7l-unknown-linux-gnueabihf/share/locale -DCMAKE_INSTALL_LIBEXECDIR=/nix/store/nlgw7vh3nrzcjcswfzkb5lrbpckmddw1-aws-sdk-
cpp-1.8.113-armv7l-unknown-linux-gnueabihf/libexec -DCMAKE_INSTALL_LIBDIR=/nix/store/nlgw7vh3nrzcjcswfzkb5lrbpckmddw1-aws-sdk-cpp-1.8.113-armv7l-unknown-linux-gnueabihf/lib -DCMAKE_INSTALL_DOCDIR=/nix/store/nlgw
7vh3nrzcjcswfzkb5lrbpckmddw1-aws-sdk-cpp-1.8.113-armv7l-unknown-linux-gnueabihf/share/doc/aws-cpp-sdk-all -DCMAKE_INSTALL_INFODIR=/nix/store/nlgw7vh3nrzcjcswfzkb5lrbpckmddw1-aws-sdk-cpp-1.8.113-armv7l-unknown-li
nux-gnueabihf/share/info -DCMAKE_INSTALL_MANDIR=/nix/store/nlgw7vh3nrzcjcswfzkb5lrbpckmddw1-aws-sdk-cpp-1.8.113-armv7l-unknown-linux-gnueabihf/share/man -DCMAKE_INSTALL_OLDINCLUDEDIR=/nix/store/knf9jzldjcly49jck
8y8pic54sygn13n-aws-sdk-cpp-1.8.113-armv7l-unknown-linux-gnueabihf-dev/include -DCMAKE_INSTALL_INCLUDEDIR=/nix/store/knf9jzldjcly49jck8y8pic54sygn13n-aws-sdk-cpp-1.8.113-armv7l-unknown-linux-gnueabihf-dev/includ
e -DCMAKE_INSTALL_SBINDIR=/nix/store/nlgw7vh3nrzcjcswfzkb5lrbpckmddw1-aws-sdk-cpp-1.8.113-armv7l-unknown-linux-gnueabihf/sbin -DCMAKE_INSTALL_BINDIR=/nix/store/nlgw7vh3nrzcjcswfzkb5lrbpckmddw1-aws-sdk-cpp-1.8.11
3-armv7l-unknown-linux-gnueabihf/bin -DCMAKE_INSTALL_NAME_DIR=/nix/store/nlgw7vh3nrzcjcswfzkb5lrbpckmddw1-aws-sdk-cpp-1.8.113-armv7l-unknown-linux-gnueabihf/lib -DCMAKE_POLICY_DEFAULT_CMP0025=NEW -DCMAKE_OSX_SYS
ROOT= -DCMAKE_OSX_ARCHITECTURES=x86_64 -DCMAKE_FIND_FRAMEWORK=LAST -DCMAKE_STRIP=/nix/store/abj35mb9qx0494jbik029r9h1ck7siqr-armv7l-unknown-linux-gnueabihf-binutils-2.34/bin/armv7l-unknown-linux-gnueabihf-strip
-DCMAKE_RANLIB=/nix/store/abj35mb9qx0494jbik029r9h1ck7siqr-armv7l-unknown-linux-gnueabihf-binutils-2.34/bin/armv7l-unknown-linux-gnueabihf-ranlib -DCMAKE_AR=/nix/store/abj35mb9qx0494jbik029r9h1ck7siqr-armv7l-unk
nown-linux-gnueabihf-binutils-2.34/bin/armv7l-unknown-linux-gnueabihf-ar -DCMAKE_C_COMPILER=armv7l-unknown-linux-gnueabihf-gcc -DCMAKE_CXX_COMPILER=armv7l-unknown-linux-gnueabihf-g++ -DCMAKE_INSTALL_PREFIX=/nix/
store/nlgw7vh3nrzcjcswfzkb5lrbpckmddw1-aws-sdk-cpp-1.8.113-armv7l-unknown-linux-gnueabihf -DBUILD_DEPS=OFF -DCMAKE_SKIP_BUILD_RPATH=OFF -DCUSTOM_MEMORY_MANAGEMENT=0 -DENABLE_TESTING=OFF -DCURL_HAS_H2=0 -DBUILD_O
NLY=s3;transfer -DCMAKE_SYSTEM_NAME=Linux -DCMAKE_SYSTEM_PROCESSOR=armv7l -DCMAKE_HOST_SYSTEM_NAME=Linux -DCMAKE_HOST_SYSTEM_PROCESSOR=x86_64
-- Could NOT find Git (missing: GIT_EXECUTABLE)
-- TARGET_ARCH not specified; inferring host OS to be platform compilation target
-- Building AWS libraries as shared objects
-- Generating linux build config
-- Building project version: 1.8.113
-- The CXX compiler identification is GNU 10.2.0
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /nix/store/8md3fz3fsa7i4n32v34wiwxpr8nkx3cv-armv7l-unknown-linux-gnueabihf-stage-final-gcc-debug-wrapper-10.2.0/bin/armv7l-unknown-linux-gnueabihf-g++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found ZLIB: /nix/store/j5j26f5vb79f1k1dm2yyf33z0pmfmsm3-zlib-1.2.11-armv7l-unknown-linux-gnueabihf/lib/libz.so (found version "1.2.11")
--   Zlib library: /nix/store/j5j26f5vb79f1k1dm2yyf33z0pmfmsm3-zlib-1.2.11-armv7l-unknown-linux-gnueabihf/lib/libz.so
-- Encryption: Openssl
-- Found OpenSSL: /nix/store/xfhz0rvqfyja4pxnlv29c649cdyx4vdq-openssl-1.1.1i-armv7l-unknown-linux-gnueabihf/lib/libcrypto.so (found version "1.1.1i")
--   Openssl include directory: /nix/store/2k6z1k39pfz6yggcnl0xamxnj96qi3m6-openssl-1.1.1i-armv7l-unknown-linux-gnueabihf-dev/include
--   Openssl library: /nix/store/xfhz0rvqfyja4pxnlv29c649cdyx4vdq-openssl-1.1.1i-armv7l-unknown-linux-gnueabihf/lib/libssl.so;/nix/store/xfhz0rvqfyja4pxnlv29c649cdyx4vdq-openssl-1.1.1i-armv7l-unknown-linux-gnuea
bihf/lib/libcrypto.so
-- Http client: Curl
-- Found CURL: /nix/store/379nw3zfwah0jcmzih1d9z8wyqis7z9f-curl-7.74.0-armv7l-unknown-linux-gnueabihf/lib/libcurl.so (found version "7.74.0")
--   Curl include directory: /nix/store/5498i8ywdkk7phy804lx8q7dxdcp6b6i-curl-7.74.0-armv7l-unknown-linux-gnueabihf-dev/include
--   Curl library: /nix/store/379nw3zfwah0jcmzih1d9z8wyqis7z9f-curl-7.74.0-armv7l-unknown-linux-gnueabihf/lib/libcurl.so
-- Performing Test HAVE_ATOMICS_WITHOUT_LIBATOMIC
-- Performing Test HAVE_ATOMICS_WITHOUT_LIBATOMIC - Success
-- Considering s3
-- Considering transfer
-- Looking for pathconf
-- Looking for pathconf - found
-- Looking for umask
-- Looking for umask - found
-- The C compiler identification is GNU 10.2.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /nix/store/8md3fz3fsa7i4n32v34wiwxpr8nkx3cv-armv7l-unknown-linux-gnueabihf-stage-final-gcc-debug-wrapper-10.2.0/bin/armv7l-unknown-linux-gnueabihf-gcc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Updating version info to 1.8.113
-- Custom memory management disabled
-- Performing Test CURL_HAS_TLS_PROXY
CMake Error: TRY_RUN() invoked in cross-compiling mode, please set the following cache variables appropriately:
   CURL_HAS_TLS_PROXY_EXITCODE (advanced)
   CURL_HAS_TLS_PROXY_EXITCODE__TRYRUN_OUTPUT (advanced)
For details see /build/source/build/TryRunResults.cmake
-- Performing Test CURL_HAS_TLS_PROXY - Failed
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed
-- Check if compiler accepts -pthread
-- Check if compiler accepts -pthread - yes
-- Found Threads: TRUE
-- Found LibCrypto: /nix/store/xfhz0rvqfyja4pxnlv29c649cdyx4vdq-openssl-1.1.1i-armv7l-unknown-linux-gnueabihf/lib/libcrypto.so
-- LibCrypto Include Dir: /nix/store/2k6z1k39pfz6yggcnl0xamxnj96qi3m6-openssl-1.1.1i-armv7l-unknown-linux-gnueabihf-dev/include
-- LibCrypto Shared Lib:  /nix/store/xfhz0rvqfyja4pxnlv29c649cdyx4vdq-openssl-1.1.1i-armv7l-unknown-linux-gnueabihf/lib/libcrypto.so
-- LibCrypto Static Lib:  LibCrypto_STATIC_LIBRARY-NOTFOUND
-- LibCrypto Include Dir: /nix/store/2k6z1k39pfz6yggcnl0xamxnj96qi3m6-openssl-1.1.1i-armv7l-unknown-linux-gnueabihf-dev/include
-- LibCrypto Shared Lib:  /nix/store/xfhz0rvqfyja4pxnlv29c649cdyx4vdq-openssl-1.1.1i-armv7l-unknown-linux-gnueabihf/lib/libcrypto.so
-- LibCrypto Static Lib:  LibCrypto_STATIC_LIBRARY-NOTFOUND
-- Configuring incomplete, errors occurred!
See also "/build/source/build/CMakeFiles/CMakeOutput.log".
See also "/build/source/build/CMakeFiles/CMakeError.log".
error: --- Error --- nix-daemon
builder for '/nix/store/9a2fjmigyqab2qfzi2m84dixi0inpfbn-aws-sdk-cpp-1.8.113-armv7l-unknown-linux-gnueabihf.drv' failed with exit code 1; last 10 log lines:
  -- Found LibCrypto: /nix/store/xfhz0rvqfyja4pxnlv29c649cdyx4vdq-openssl-1.1.1i-armv7l-unknown-linux-gnueabihf/lib/libcrypto.so
  -- LibCrypto Include Dir: /nix/store/2k6z1k39pfz6yggcnl0xamxnj96qi3m6-openssl-1.1.1i-armv7l-unknown-linux-gnueabihf-dev/include
  -- LibCrypto Shared Lib:  /nix/store/xfhz0rvqfyja4pxnlv29c649cdyx4vdq-openssl-1.1.1i-armv7l-unknown-linux-gnueabihf/lib/libcrypto.so
  -- LibCrypto Static Lib:  LibCrypto_STATIC_LIBRARY-NOTFOUND
  -- LibCrypto Include Dir: /nix/store/2k6z1k39pfz6yggcnl0xamxnj96qi3m6-openssl-1.1.1i-armv7l-unknown-linux-gnueabihf-dev/include
  -- LibCrypto Shared Lib:  /nix/store/xfhz0rvqfyja4pxnlv29c649cdyx4vdq-openssl-1.1.1i-armv7l-unknown-linux-gnueabihf/lib/libcrypto.so
  -- LibCrypto Static Lib:  LibCrypto_STATIC_LIBRARY-NOTFOUND
  -- Configuring incomplete, errors occurred!
  See also "/build/source/build/CMakeFiles/CMakeOutput.log".
  See also "/build/source/build/CMakeFiles/CMakeError.log".
error: --- Error --- nix-build
1 dependencies of derivation '/nix/store/1ssspmj2bvf6l7kavpcihlimd47wyx97-nix-2.3.10-armv7l-unknown-linux-gnueabihf.drv' failed to build

@flokli
Copy link
Contributor

flokli commented Jan 21, 2021 •
edited
Loading

This is a recent regression. nix used to cross-compile. I bisected this:

git bisect start
# good: [d897e8e0ebc50943d434fe54a1edae3e53669316] Merge pull request #108351 from LibreCybernetics/update-anbox
git bisect good d897e8e0ebc50943d434fe54a1edae3e53669316
# bad: [5153deedd88a911fd6548b5bedfd77119b8a31ba] Merge pull request #108909 from thiagokokada/libinput-by-device-type
git bisect bad 5153deedd88a911fd6548b5bedfd77119b8a31ba
# bad: [ffdefb66c0ebbc6f7492dc0763508867245ce934] Merge pull request #108966 from pstn/murmurPswitch
git bisect bad ffdefb66c0ebbc6f7492dc0763508867245ce934
# bad: [82115f0c4bf57fb5b1c8c322cd36e92f069fc124] Merge pull request #107958 from ztzg/x-16304-ubuntu-determinism
git bisect bad 82115f0c4bf57fb5b1c8c322cd36e92f069fc124
# bad: [0d880dbc889d3a6a8a00c6e4345581cee665ad7a] starboard: 0.7.1 -> 0.8.0
git bisect bad 0d880dbc889d3a6a8a00c6e4345581cee665ad7a
# bad: [4a7d5be0c98ae3ccfbb8feaa474902a3b190753b] Merge pull request #108468 from prusnak/openhmd
git bisect bad 4a7d5be0c98ae3ccfbb8feaa474902a3b190753b
# bad: [7aed38b8ab76bf326243711d257eafa76f15654f] Merge pull request #107547 from jerith666/openjdk-15
git bisect bad 7aed38b8ab76bf326243711d257eafa76f15654f
# good: [328040d127ff0e0a3f658b2a9fb08e81a082fef3] python3Packages.flask-socketio: enable tests
git bisect good 328040d127ff0e0a3f658b2a9fb08e81a082fef3
# bad: [a9a348e8f5367c5d39fbd1963431136c7973c89d] Merge pull request #108566 from fabaff/python-XXXio
git bisect bad a9a348e8f5367c5d39fbd1963431136c7973c89d
# bad: [de753c64f9d5c67a178fcea016bb9ac3df57a11c] hydraUnstable: patch to support nixUnstable 2.4pre20201205_a5d85d0
git bisect bad de753c64f9d5c67a178fcea016bb9ac3df57a11c
# skip: [d899420b3d0598976605004893f19e32140ecdaf] aws-c-io: init at 0.7.0
git bisect skip d899420b3d0598976605004893f19e32140ecdaf
# skip: [41fee267fd3fd095243c9d65a588a062e479bc99] aws-c-event-stream: 0.1.1 -> 0.2.6
git bisect skip 41fee267fd3fd095243c9d65a588a062e479bc99
# bad: [4cb0e9e1c48e30b3c566384872fd502effb933fc] nix: update aws-sdk-cpp TransferManager ContentEncoding patch for 1.8.113
git bisect bad 4cb0e9e1c48e30b3c566384872fd502effb933fc
# skip: [6bb19efc8200623d6b085a1e23b0e0a0a3e5fb53] aws-c-cal: init at 0.4.5
git bisect skip 6bb19efc8200623d6b085a1e23b0e0a0a3e5fb53
# skip: [b95a9663a0f865e3dcec4b6b5a3b148f947bacac] aws-checksums: 0.1.7 -> 0.1.10
git bisect skip b95a9663a0f865e3dcec4b6b5a3b148f947bacac
# skip: [09fdc4553bb894f3bef77bc3a1097fd38af94df5] s2n: init at 0.10.23
git bisect skip 09fdc4553bb894f3bef77bc3a1097fd38af94df5
# skip: [3619fa90651184f3f0aaeef8bb8ff2cf21827933] aws-c-common: 0.3.11 -> 0.4.63
git bisect skip 3619fa90651184f3f0aaeef8bb8ff2cf21827933
# skip: [ff3f5bb60c7f27c45832ae73eeacce931f77b625] aws-sdk-cpp: 1.7.90 -> 1.8.113
git bisect skip ff3f5bb60c7f27c45832ae73eeacce931f77b625
# only skipped commits left to test
# possible first bad commit: [4cb0e9e1c48e30b3c566384872fd502effb933fc] nix: update aws-sdk-cpp TransferManager ContentEncoding patch for 1.8.113
# possible first bad commit: [ff3f5bb60c7f27c45832ae73eeacce931f77b625] aws-sdk-cpp: 1.7.90 -> 1.8.113
# possible first bad commit: [41fee267fd3fd095243c9d65a588a062e479bc99] aws-c-event-stream: 0.1.1 -> 0.2.6
# possible first bad commit: [d899420b3d0598976605004893f19e32140ecdaf] aws-c-io: init at 0.7.0
# possible first bad commit: [6bb19efc8200623d6b085a1e23b0e0a0a3e5fb53] aws-c-cal: init at 0.4.5
# possible first bad commit: [09fdc4553bb894f3bef77bc3a1097fd38af94df5] s2n: init at 0.10.23
# possible first bad commit: [b95a9663a0f865e3dcec4b6b5a3b148f947bacac] aws-checksums: 0.1.7 -> 0.1.10
# possible first bad commit: [3619fa90651184f3f0aaeef8bb8ff2cf21827933] aws-c-common: 0.3.11 -> 0.4.63

This seems to have been caused by some of these aws changes not have been verified to work in cross, introduced in #107623.

@flokli flokli mentioned this issue Jan 21, 2021
Merged
10 tasks
@misuzu
Copy link
Contributor

misuzu commented Jan 26, 2021

Cross-compiling glibc on aarch64 is broken:

$ nix-build channel:nixos-unstable -A pkgsCross.armv7l-hf-multiplatform.glibc
these 6 derivations will be built:
  /nix/store/3j5j5snlqidzvp8xd51g28rdna4ff5kp-glibc-2.32-25-armv7l-unknown-linux-gnueabihf.drv
  /nix/store/c1k68cb9jppl6sivc5disj3cq7g9jj8i-armv7l-unknown-linux-gnueabihf-binutils-wrapper-2.35.1.drv
  /nix/store/dbiidvkq2drm91i7sgrg540snl1igih6-armv7l-unknown-linux-gnueabihf-stage-final-gcc-debug-10.2.0.drv
  /nix/store/vpfl7h9bif24l81iksmzn4n6rcn63jg6-armv7l-unknown-linux-gnueabihf-stage-final-gcc-debug-wrapper-10.2.0.drv
  /nix/store/4jrv5a948p39q5n7jc1vaq1wy4n8v1m4-stdenv-linux.drv
  /nix/store/as1n8v21m351rlrpqf169grxr5aa4gsa-glibc-2.32-25-armv7l-unknown-linux-gnueabihf.drv
building '/nix/store/3j5j5snlqidzvp8xd51g28rdna4ff5kp-glibc-2.32-25-armv7l-unknown-linux-gnueabihf.drv'...
unpacking sources
unpacking source archive /nix/store/yl0acs9vzng0cd4qfdcgb3zpwjm034nw-glibc-2.32.tar.xz
source root is glibc-2.32
setting SOURCE_DATE_EPOCH to timestamp 1596593820 of file glibc-2.32/wctype/wctype_l.c
patching sources
applying patch /nix/store/43dy13j5z8s53xxyrl763wc14cq98ank-2.32-25.patch.gz

.........

a - elf/dl-profstub.os
a - elf/dl-libc.os
a - elf/dl-sym.os
a - elf/dl-error.os
a - elf/libc_early_init.os
a - elf/aeabi_unwind_cpp_pr1.os
a - elf/find_exidx.os
armv7l-unknown-linux-gnueabihf-gcc dl-lookup.c -c -std=gnu11 -fgnu89-inline  -g -O2 -Wall -Wwrite-strings -Wundef -Werror -fmerge-all-constants -frounding-math -fno-stack-protector -Wstrict-prototypes -Wold-style-definition -fmath-errno     -fno-stack-protector -DSTACK_PROTECTOR_LEVEL=0 -fexceptions -fasynchronous-unwind-tables  -ftls-model=initial-exec      -I../include -I/build/build/elf  -I/build/build  -I../sysdeps/unix/sysv/linux/arm/le  -I../sysdeps/unix/sysv/linux/arm  -I../sysdeps/arm/nptl  -I../sysdeps/unix/sysv/linux/include -I../sysdeps/unix/sysv/linux  -I../sysdeps/nptl  -I../sysdeps/pthread  -I../sysdeps/gnu  -I../sysdeps/unix/inet  -I../sysdeps/unix/sysv  -I../sysdeps/unix/arm  -I../sysdeps/unix  -I../sysdeps/posix  -I../sysdeps/arm/le/armv7/multiarch  -I../sysdeps/arm/armv7/multiarch  -I../sysdeps/arm/le/armv7  -I../sysdeps/arm/armv7  -I../sysdeps/arm/armv6t2  -I../sysdeps/arm/armv6  -I../sysdeps/arm/le  -I../sysdeps/arm/include -I../sysdeps/arm  -I../sysdeps/wordsize-32  -I../sysdeps/ieee754/flt-32  -I../sysdeps/ieee754/dbl-64  -I../sysdeps/ieee754  -I../sysdeps/generic  -I.. -I../libio -I. -nostdinc -isystem /nix/store/8cxq0frw1lwkfv1whzg87xz22yw7mgpk-armv7l-unknown-linux-gnueabihf-stage-static-gcc-debug-10.2.0/lib/gcc/armv7l-unknown-linux-gnueabihf/10.2.0/include -isystem /nix/store/8cxq0frw1lwkfv1whzg87xz22yw7mgpk-armv7l-unknown-linux-gnueabihf-stage-static-gcc-debug-10.2.0/lib/gcc/armv7l-unknown-linux-gnueabihf/10.2.0/include-fixed -isystem /nix/store/3wdvnp51dv9v6r56zadj150x5vbw8i7r-linux-headers-5.10.4/include -D_LIBC_REENTRANT -include /build/build/libc-modules.h -DMODULE_NAME=libc -include ../include/libc-symbols.h  -DPIC     -DTOP_NAMESPACE=glibc -o /build/build/elf/dl-lookup.o -MD -MP -MF /build/build/elf/dl-lookup.o.dt -MT /build/build/elf/dl-lookup.o
armv7l-unknown-linux-gnueabihf-gcc   -nostdlib -nostartfiles -r -o /build/build/libc_pic.os \
 -Wl,-d -Wl,--whole-archive /build/build/libc_pic.a -o /build/build/libc_pic.os
armv7l-unknown-linux-gnueabihf-gcc   -nostdlib -nostartfiles -r -o /build/build/elf/librtld.map.o -Wl,--defsym=calloc=0 -Wl,--defsym=free=0 -Wl,--defsym=malloc=0 -Wl,--defsym=realloc=0 -Wl,--defsym=__stack_chk_fail=0 -Wl,--defsym=__stack_chk_fail_local=0 \
        '-Wl,-(' /build/build/elf/dl-allobjs.os /build/build/libc_pic.a -lgcc '-Wl,-)' -Wl,-Map,/build/build/elf/librtld.mapT
/nix/store/ix71adklpdq04c8hkjks4xyiamx6la9h-armv7l-unknown-linux-gnueabihf-binutils-2.35.1/bin/armv7l-unknown-linux-gnueabihf-ld: /build/build/libc_pic.a: error adding symbols: archive has no index; run ranlib to add one
collect2: error: ld returned 1 exit status
make[2]: *** [Makefile:525: /build/build/elf/librtld.map] Error 1
make[2]: Leaving directory '/build/glibc-2.32/elf'
make[1]: *** [Makefile:470: elf/subdir_lib] Error 2
make[1]: Leaving directory '/build/glibc-2.32'
make: *** [Makefile:9: all] Error 2
error: --- Error --- nix-daemon
builder for '/nix/store/3j5j5snlqidzvp8xd51g28rdna4ff5kp-glibc-2.32-25-armv7l-unknown-linux-gnueabihf.drv' failed with exit code 2; last 10 log lines:
   -Wl,-d -Wl,--whole-archive /build/build/libc_pic.a -o /build/build/libc_pic.os
  armv7l-unknown-linux-gnueabihf-gcc   -nostdlib -nostartfiles -r -o /build/build/elf/librtld.map.o -Wl,--defsym=calloc=0 -Wl,--defsym=free=0 -Wl,--defsym=malloc=0 -Wl,--defsym=realloc=0 -Wl,--defsym=__stack_chk_fail=0 -Wl,--defsym=__stack_chk_fail_local=0 \
    '-Wl,-(' /build/build/elf/dl-allobjs.os /build/build/libc_pic.a -lgcc '-Wl,-)' -Wl,-Map,/build/build/elf/librtld.mapT
  /nix/store/ix71adklpdq04c8hkjks4xyiamx6la9h-armv7l-unknown-linux-gnueabihf-binutils-2.35.1/bin/armv7l-unknown-linux-gnueabihf-ld: /build/build/libc_pic.a: error adding symbols: archive has no index; run ranlib to add one
  collect2: error: ld returned 1 exit status
  make[2]: *** [Makefile:525: /build/build/elf/librtld.map] Error 1
  make[2]: Leaving directory '/build/glibc-2.32/elf'
  make[1]: *** [Makefile:470: elf/subdir_lib] Error 2
  make[1]: Leaving directory '/build/glibc-2.32'
  make: *** [Makefile:9: all] Error 2
error: --- Error ---------------------------------------------------- nix-build
1 dependencies of derivation '/nix/store/as1n8v21m351rlrpqf169grxr5aa4gsa-glibc-2.32-25-armv7l-unknown-linux-gnueabihf.drv' failed to build

I think it was broken at least starting from this evaluation: https://hydra.nixos.org/eval/1643415
Most likely because of this: #108571

@Ericson2314
Copy link
Member

I wouldn't test building glibc like that. For example:

these 6 derivations will be built:
  /nix/store/3j5j5snlqidzvp8xd51g28rdna4ff5kp-glibc-2.32-25-armv7l-unknown-linux-gnueabihf.drv
  ...
  /nix/store/as1n8v21m351rlrpqf169grxr5aa4gsa-glibc-2.32-25-armv7l-unknown-linux-gnueabihf.drv

@misuzu
Copy link
Contributor

misuzu commented Jan 26, 2021

I wouldn't test building glibc like that. For example:

these 6 derivations will be built:
  /nix/store/3j5j5snlqidzvp8xd51g28rdna4ff5kp-glibc-2.32-25-armv7l-unknown-linux-gnueabihf.drv
  ...
  /nix/store/as1n8v21m351rlrpqf169grxr5aa4gsa-glibc-2.32-25-armv7l-unknown-linux-gnueabihf.drv

What do you mean? When i cross-compiling SD image on aarch64 I have exactly the same error. On x86-64 there are no issues:

$ nix-build channel:nixos-unstable -A pkgsCross.armv7l-hf-multiplatform.glibc
this path will be fetched (5.66 MiB download, 26.17 MiB unpacked):
  /nix/store/74axx8fiqavhlpvhvna0akmdwx89jf5f-glibc-2.32-25-armv7l-unknown-linux-gnueabihf
copying path '/nix/store/74axx8fiqavhlpvhvna0akmdwx89jf5f-glibc-2.32-25-armv7l-unknown-linux-gnueabihf' from 'https://cache.nixos.org'...
/nix/store/74axx8fiqavhlpvhvna0akmdwx89jf5f-glibc-2.32-25-armv7l-unknown-linux-gnueabihf

@TredwellGit
Copy link
Member

I doubt #108571 is the cause.
https://sourceware.org/git/?p=glibc.git;a=commitdiff;h=0d9793e82a19bcef10ef7d73a26cd44b7ad30753;hp=1d49bede4d8a5448067e8579b14fedf437e75aee

@misuzu
Copy link
Contributor

misuzu commented Jan 27, 2021 •
edited
Loading

I doubt #108571 is the cause.
https://sourceware.org/git/?p=glibc.git;a=commitdiff;h=0d9793e82a19bcef10ef7d73a26cd44b7ad30753;hp=1d49bede4d8a5448067e8579b14fedf437e75aee

With reverted 9a4e098 it builds on aarch64 so it's definitely is the cause.

EDIT: after proper testing I have found that reverting 9a4e098 does not actually fix anything.
The next possible cause on the list is 306aefd (#108453).

EDIT2: Looks like 306aefd is the real culprit.

@TredwellGit
Copy link
Member

Does it build on staging (glibc 2.32-35)?

@flokli
Copy link
Contributor

flokli commented Jan 28, 2021

@TredwellGit I was able to build pkgsCross.aarch64-multiplatform.glibc on staging (7e0ff70):

checking for references to /build/ in /nix/store/98c900mxzym3q8s1zby1h3ds0kwcknjw-glibc-2.32-35-aarch64-unknown-linux-gnu-bin...
patchelf: cannot find section '.dynamic'. The input file is most likely statically linked
patchelf: cannot find section '.dynamic'. The input file is most likely statically linked
moving /nix/store/98c900mxzym3q8s1zby1h3ds0kwcknjw-glibc-2.32-35-aarch64-unknown-linux-gnu-bin/sbin/* to /nix/store/98c900mxzym3q8s1zby1h3ds0kwcknjw-glibc-2.32-35-aarch64-unknown-linux-gnu-bin/bin
shrinking RPATHs of ELF executables and libraries in /nix/store/63wm3dsfdz9knzlxflycr8awvcdsjyr1-glibc-2.32-35-aarch64-unknown-linux-gnu-dev
patching script interpreter paths in /nix/store/63wm3dsfdz9knzlxflycr8awvcdsjyr1-glibc-2.32-35-aarch64-unknown-linux-gnu-dev
checking for references to /build/ in /nix/store/63wm3dsfdz9knzlxflycr8awvcdsjyr1-glibc-2.32-35-aarch64-unknown-linux-gnu-dev...
could not find build ID of /nix/store/2hh6m77rk1sjkwv5gdrqllv2v6qay0jf-glibc-2.32-35-aarch64-unknown-linux-gnu-static/lib/libmcheck.a, skipping
shrinking RPATHs of ELF executables and libraries in /nix/store/2hh6m77rk1sjkwv5gdrqllv2v6qay0jf-glibc-2.32-35-aarch64-unknown-linux-gnu-static
shrinking /nix/store/2hh6m77rk1sjkwv5gdrqllv2v6qay0jf-glibc-2.32-35-aarch64-unknown-linux-gnu-static/lib/libmcheck.a
patchelf: wrong ELF type
patching script interpreter paths in /nix/store/2hh6m77rk1sjkwv5gdrqllv2v6qay0jf-glibc-2.32-35-aarch64-unknown-linux-gnu-static
checking for references to /build/ in /nix/store/2hh6m77rk1sjkwv5gdrqllv2v6qay0jf-glibc-2.32-35-aarch64-unknown-linux-gnu-static...
patchelf: wrong ELF type
shrinking RPATHs of ELF executables and libraries in /nix/store/3azyqiq3ymdpvf6q9hcwnjr7cbippgvk-glibc-2.32-35-aarch64-unknown-linux-gnu-debug
patching script interpreter paths in /nix/store/3azyqiq3ymdpvf6q9hcwnjr7cbippgvk-glibc-2.32-35-aarch64-unknown-linux-gnu-debug
checking for references to /build/ in /nix/store/3azyqiq3ymdpvf6q9hcwnjr7cbippgvk-glibc-2.32-35-aarch64-unknown-linux-gnu-debug...
/nix/store/q0ziclkbn0s06lnz00db2jracw0vn35n-glibc-2.32-35-aarch64-unknown-linux-gnu

@flokli
Copy link
Contributor

flokli commented Jan 28, 2021

/nix/store/m8wigcw4g7rfxbk0gc8yz7gckw87dwsj-aarch64-unknown-linux-musl-stage-static-gcc-debug-10.2.0.drv seems to have regressed too (which is pulled in for pkgs.pkgsCross.aarch64-multiplatform.nix).

@misuzu
Copy link
Contributor

misuzu commented Jan 28, 2021

Does it build on staging (glibc 2.32-35)?

It doesn't, but with binutils-2.34 it does.
With binutils-2.36 gcc fails to build.

@flokli
Copy link
Contributor

flokli commented Feb 8, 2021

The static gcc regression ssems to have been sorted out, now it's #112173, which already should be fixed on staging-next.

@stigtsp stigtsp mentioned this issue Feb 8, 2021
Closed
6 tasks
@thefloweringash
Copy link
Member

Just dropping in to confirm @misuzu's results. aarch64-linux -> armv7l-linux doesn't compile on master or nixos-21.05

306aefd9fc015c32dfcfcffcc301f362fb996201 is the first bad commit
commit 306aefd9fc015c32dfcfcffcc301f362fb996201
Author: Bernardo Meurer <[email protected]>
Date:   Mon Jan 4 19:52:46 2021 -0800

    binutils: 2.34 -> 2.35.1

 pkgs/development/tools/misc/binutils/default.nix    | 10 ++--------
 .../tools/misc/binutils/no-plugins.patch            | 21 ---------------------
 2 files changed, 2 insertions(+), 29 deletions(-)
 delete mode 100644 pkgs/development/tools/misc/binutils/no-plugins.patch
`git bisect log` 
$ git bisect log
git bisect start
# good: [9f05c50be198a396eb6c555fc239a2e0da42c9a5] libarchive: Fix Windows build
git bisect good 9f05c50be198a396eb6c555fc239a2e0da42c9a5
# bad: [2e88b6f989d35e279b65c5c8831b1a3a667a56dc] Merge pull request #125554 from mweinelt/polkit
git bisect bad 2e88b6f989d35e279b65c5c8831b1a3a667a56dc
# bad: [c3fbb346ae93c6cef3a460b6f6295573a5eb28f7] Merge pull request #111887 from TredwellGit/libreoffice-fresh
git bisect bad c3fbb346ae93c6cef3a460b6f6295573a5eb28f7
# good: [baf2814f48cfb2fec0a045722609bb2c659929d1] redis: disable systemd in musl build
git bisect good baf2814f48cfb2fec0a045722609bb2c659929d1
# good: [3f132c79acc12133589cafacad28ad8d7baccf3e] Merge pull request #108536 from SuperSandro2000/small-fixes
git bisect good 3f132c79acc12133589cafacad28ad8d7baccf3e
# bad: [163d758979ff922327b9fb11cdff965b3933d8dc] Merge pull request #110209 from r-ryantm/auto-update/php-psalm
git bisect bad 163d758979ff922327b9fb11cdff965b3933d8dc
# good: [195c3acf8f846b9b28f75690062d7a284e754bd4] Merge pull request #86912 from iblech/patch-15
git bisect good 195c3acf8f846b9b28f75690062d7a284e754bd4
# bad: [ef2abc20bc08546fa4dc55a14dab96a3d3f62981] Merge master into staging-next
git bisect bad ef2abc20bc08546fa4dc55a14dab96a3d3f62981
# good: [b153e63f07c9f88490c710bede0adc8c4e542774] pythonPackages.sphinxcontrib-bibtex: Disable tests, add maintainer, cleanups
git bisect good b153e63f07c9f88490c710bede0adc8c4e542774
# good: [1afec6062749abd6cb20916e9391a75931deb6ad] Merge pull request #95274 from misuzu/qemu-iscsi
git bisect good 1afec6062749abd6cb20916e9391a75931deb6ad
# good: [abaefba36c185897aca6ac915b633148a45dfb80] python37Packages.google-cloud-bigquery: 2.6.1 -> 2.6.2
git bisect good abaefba36c185897aca6ac915b633148a45dfb80
# bad: [5114061fa1ea2963a1daaa6640d61983ebde5337] Merge pull request #107594 from ehmry/stdenv-cross-llvm
git bisect bad 5114061fa1ea2963a1daaa6640d61983ebde5337
# good: [236384f0a16bccf92f030300ff42d4530fbfeddc] libxkbcommon: 0.10.0 -> 1.0.3
git bisect good 236384f0a16bccf92f030300ff42d4530fbfeddc
# good: [641cbc17b1c3379981bff06d788dba86e98a78a1] Merge pull request #108825 from TredwellGit/openjpeg
git bisect good 641cbc17b1c3379981bff06d788dba86e98a78a1
# good: [38472ca27bf5d333da8acd9162064857a7e14fdb] Merge pull request #102168 from FRidh/qt515
git bisect good 38472ca27bf5d333da8acd9162064857a7e14fdb
# bad: [bfb627d78f25de2ccf9e2af46e500d0030d3abe7] gpgme: 1.15.0 -> 1.15.1
git bisect bad bfb627d78f25de2ccf9e2af46e500d0030d3abe7
# bad: [306aefd9fc015c32dfcfcffcc301f362fb996201] binutils: 2.34 -> 2.35.1
git bisect bad 306aefd9fc015c32dfcfcffcc301f362fb996201
# good: [0934b7077c651bffd9c6c72cc4cc9a13efa25846] qt5-packages.nix: fix title
git bisect good 0934b7077c651bffd9c6c72cc4cc9a13efa25846
# first bad commit: [306aefd9fc015c32dfcfcffcc301f362fb996201] binutils: 2.34 -> 2.35.1

@thefloweringash thefloweringash mentioned this issue Jun 4, 2021
Merged
9 tasks
@Mindavi
Copy link
Contributor

Mindavi commented Jun 16, 2021

Talloc now also cross-builds, as mentioned in the issue.

@Atemu Atemu mentioned this issue Oct 30, 2021
Closed
2 tasks
@Atemu
Copy link
Member Author

Atemu commented Oct 30, 2021

ZFS is now broken.

This builds now though:

{ lib, pkgs, ... }:

{
  imports = [
    <nixpkgs/nixos/modules/installer/sd-card/sd-image-armv7l-multiplatform-installer.nix>
  ];

  boot.kernelPackages = lib.mkForce pkgs.linuxPackages;

  # Everything but ZFS
  boot.supportedFilesystems = lib.mkForce [ "btrfs" "reiserfs" "vfat" "f2fs" "xfs" "ntfs" "cifs" "ext4" "vfat" ];

  nixpkgs = {
    crossSystem = lib.systems.examples.armv7l-hf-multiplatform;
    localSystem = { system = "x86_64-linux"; };
  };
}

@Mindavi
Copy link
Contributor

Mindavi commented Nov 3, 2021 •
edited
Loading

Question: How do you build this? I tried nix-build configuration.nix and nixos-rebuild build -I nixos-config=configuration.nix -I nixpkgs=<path/to/nixpkgs>, but I think I'm doing it wrong.

Edit: my configuration.nix:

{ lib, ... }:

{
  imports = [
    <nixpkgs/nixos/modules/installer/cd-dvd/sd-image-aarch64.nix>
  ];

  nixpkgs = {
    crossSystem = lib.systems.examples.aarch64-multiplatform;
    localSystem = { system = "x86_64-linux"; };
  };
}

Something is probably wrong there, since the configuration.nix atemu posted works fine.

The error I get with nixos-rebuild:

/run/current-system/sw/bin/nixos-rebuild: line 175: /tmp/nixos-rebuild.zSouvL/nix/bin/nix-build: cannot execute binary file: Exec format error

@Atemu
Copy link
Member Author

Atemu commented Nov 3, 2021

The second command should work. Try with --fast, just a hunch though.

@Mindavi
Copy link
Contributor

Mindavi commented Nov 4, 2021

--fast helps 👍

@Mindavi
Copy link
Contributor

Mindavi commented Nov 10, 2021

On latest master (52c1e64), I don't see issues with pkgsCross.aarch64-multiplatform:

{ lib, pkgs, ... }:

{
  imports = [
    <nixpkgs/nixos/modules/installer/sd-card/sd-image-aarch64-installer.nix>
  ];

  nixpkgs = {
    crossSystem = lib.systems.examples.aarch64-multiplatform;
    localSystem = { system = "x86_64-linux"; };
  };
}

Built with: nixos-rebuild -I nixos-config=configuration.nix -I nixpkgs=/path/to/nixpkgs --fast build

@Artturin
Copy link
Member

Artturin commented May 21, 2022 •
edited
Loading

this builds
just run nix build

{
  description = "cross-compile the sd-image-aarch64 on x86_64-linux to aarch64-multiplatform";
  inputs = {
    nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable";
  };

  outputs = { self, nixpkgs }: {

    nixosConfigurations.cross-vm = nixpkgs.lib.nixosSystem {
      system = "x86_64-linux";
      modules = [
        ({ pkgs, lib, modulesPath, ... }: {
          nixpkgs = {
            crossSystem = lib.systems.examples.aarch64-multiplatform;
          };

          imports = [ (modulesPath + "/installer/sd-card/sd-image-aarch64.nix") ];

          users.mutableUsers = false;
          users.users.root = {
            password = "root";
          };
          users.users.user = {
            password = "user";
            isNormalUser = true;
            extraGroups = [ "wheel" ];
          };
          system.stateVersion = "22.05";
        })
      ];
    };

    packages.x86_64-linux.default = self.nixosConfigurations.cross-vm.config.system.build.sdImage;
  };
}

KoviRobi added a commit to KoviRobi/nixos-rpi-cross-compile that referenced this issue Feb 6, 2023
@KoviRobi
Initial commit, from nixpkgs issue 106759
e391c21 
NixOS/nixpkgs#106759 (comment)

input 'nixpkgs':
 'github:NixOS/nixpkgs/0ea7a8f1b939d74e5df8af9a8f7342097cdf69eb' (2022-07-02)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
0.kind: bug Something is broken 6.topic: cross-compilation Building packages on a different platform than they will be used on
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests