Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bintools-wrapper: eliminate build references when cross-compiling #173952

Closed

Conversation

lopsided98
Copy link
Contributor

@lopsided98 lopsided98 commented May 22, 2022

Description of changes

The wrapper had references to build platform bash (stdenv.shell) and expand-response-params. I'm not sure if this is the best way to fix it, but it seems to work. I was able to build both the native x86_64 stdenv and cross-compiled armv6l stdenv.

A similar fix is likely needed for cc-wrapper.

Things done
  • Built on platform(s)
    • x86_64-linux
    • aarch64-linux
    • x86_64-darwin
    • aarch64-darwin
  • For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
  • Tested, as applicable:
  • Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
  • Tested basic functionality of all binary files (usually in ./result/bin/)
  • 22.05 Release Notes (or backporting 21.11 Release notes)
    • (Package updates) Added a release notes entry if the change is major or breaking
    • (Module updates) Added a release notes entry if the change is significant
    • (Module addition) Added a release notes entry if adding a new NixOS module
    • (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes
  • Fits CONTRIBUTING.md.

The wrapper had references to build platform bash and expand-response-params.
@ofborg ofborg bot added 10.rebuild-linux-stdenv This PR causes stdenv to rebuild 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin 10.rebuild-linux: 501+ 10.rebuild-linux: 5001+ labels May 22, 2022
@Mindavi Mindavi added the 6.topic: cross-compilation Building packages on a different platform than they will be used on label May 22, 2022
@Mindavi
Copy link
Contributor

Mindavi commented Jun 2, 2022

The shell change seems ok, but not sure about the buildPackages change. Can't that be changed in the derivation file itself?

@lopsided98
Copy link
Contributor Author

Can't that be changed in the derivation file itself?

That would also require several more complex changes in the bootstrapping code for each platform.

@lopsided98
Copy link
Contributor Author

cc @Ericson2314, since you might have some familiarity with this

@AlexandreTunstall
Copy link
Contributor

AlexandreTunstall commented Jul 12, 2023

I think I've encountered the issue this PR is trying to fix while trying to cross-compile GHC.

A similar fix is needed for cc-wrapper.

$ nix build nixpkgs#pkgsCross.riscv64.gcc -o result-gcc
$ head -n1 result-gcc/bin/gcc
#! /nix/store/51sszqz1d9kpx480scb1vllc00kxlx79-bash-5.2-p15/bin/bash
$ nix run nixpkgs#file -- -L /nix/store/51sszqz1d9kpx480scb1vllc00kxlx79-bash-5.2-p15/bin/bash
/nix/store/51sszqz1d9kpx480scb1vllc00kxlx79-bash-5.2-p15/bin/bash: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /nix/store/3n58xw4373jp0ljirf06d8077j15pc4j-glibc-2.37-8/lib/ld-linux-x86-64.so.2, BuildID[sha1]=4fa92b70b17d9f0c3f7715fdd952e56cc1efc118, for GNU/Linux 3.10.0, not stripped

As shown above, pkgsCross.riscv64.gcc produces a cc-wrapper that does not run on riscv64.

@leifhelm
Copy link
Contributor

leifhelm commented Aug 4, 2023

@Mindavi buildPackages is only used to compile expand-response-params (in both bintools-wrapper and cc-wrapper). I implemented it in the derivation. It’s just an if that uses pkgs instead of buildPackages when cross compiling.
With a lot of if statements the big rebuild can be avoided. i.e. Only change stuff when cross-compiling.
Is that neater?

@leifhelm
Copy link
Contributor

leifhelm commented Aug 4, 2023

After adding

    shell = runtimeShell;
    buildPackages = pkgs;

to wrapCCWith as well, nix path-info -r .#pkgsCross.riscv64.gcc lists only riscv64 packages.

/nix/store/372zfza5hk7zhwiyjky382zv15y42cgw-riscv64-unknown-linux-gnu-stage-static-gcc-12.3.0-libgcc
/nix/store/q5v876nk37wxryqwkbp7a4pi3swgp1kq-glibc-riscv64-unknown-linux-gnu-2.37-8
/nix/store/zrjx5jdyj382cqy0xr98bwdc6s2k0hh5-zlib-riscv64-unknown-linux-gnu-1.2.13
/nix/store/19gsi9r8r2qznv2s909f77r0ammv9hwg-binutils-riscv64-unknown-linux-gnu-2.40-lib
/nix/store/243qy2cc408hbhzgl2d03wcrr5ydzxsv-pcre2-riscv64-unknown-linux-gnu-10.42
/nix/store/5d10w6w90k0kkz7cs94zn6sprkwlf81i-linux-headers-6.4
/nix/store/628wlhmn912wp9ssl8827mnidyhls15k-glibc-riscv64-unknown-linux-gnu-2.37-8-bin
/nix/store/6vpbh764zwa41sv69ih0dl2ln1zjz6qb-glibc-riscv64-unknown-linux-gnu-2.37-8-dev
/nix/store/yv5y6x78vkwj8iarfdadqwj280nyx3ci-riscv64-unknown-linux-gnu-stage-final-gcc-12.3.0-libgcc
/nix/store/cw2x43p6qr0a1m6jy3l6vd9v07zl3d4x-riscv64-unknown-linux-gnu-stage-final-gcc-12.3.0-lib
/nix/store/9zmm4qw8jls9b0xmz31x1dnfrh3xb0jy-gmp-with-cxx-riscv64-unknown-linux-gnu-6.2.1
/nix/store/ib8rbz77wp7q29qb66cp5xmw1fsxd123-isl-riscv64-unknown-linux-gnu-0.20
/nix/store/v4cb7pk25qc3wfrsdqvi4m290yyspz7h-mpfr-riscv64-unknown-linux-gnu-4.2.0
/nix/store/ijc0s193yr6dspzd1b5gp2ix862xxffi-libmpc-riscv64-unknown-linux-gnu-1.3.1
/nix/store/879knymhp35xd1l7x7nw08rawmygspcb-gcc-riscv64-unknown-linux-gnu-12.3.0-libgcc
/nix/store/r2wp6szfjqr9bfz2j7lgp7kny4avj48k-gcc-riscv64-unknown-linux-gnu-12.3.0-lib
/nix/store/7k9mg81z8jd28q5r6w3k0zmsgjz59kq0-gcc-riscv64-unknown-linux-gnu-12.3.0
/nix/store/qcs9wc8ncrdmn1kpa6f9z69a74zr2303-gcc-riscv64-unknown-linux-gnu-12.3.0-man
/nix/store/9qdvszlfyd9d6pg3w3k4xskmvz54yxgi-gcc-wrapper-12.3.0-man
/nix/store/kabjm6bgbg20bjp75p16zqy8laggl141-gnugrep-riscv64-unknown-linux-gnu-3.11
/nix/store/q93y53hhh2rrn5kw8m1aq0dnv066cvhc-binutils-riscv64-unknown-linux-gnu-2.40
/nix/store/n6vnp7381idsaki1hym1mmq6d67mg8bb-attr-riscv64-unknown-linux-gnu-2.5.1
/nix/store/hq8rh39fgxya8aypdhlpm8g62xasks82-acl-riscv64-unknown-linux-gnu-2.3.1
/nix/store/ssgsrr7yb9salalx0hw4f5rfw5dypnvx-coreutils-riscv64-unknown-linux-gnu-9.3
/nix/store/v84ifs4qqa32xw7y0vfdmw6fccfr2pr9-bash-5.2-p15-riscv64-unknown-linux-gnu
/nix/store/x6wr3wskk10vv5m1pmf9q0szxhgvsmdf-expand-response-params-riscv64-unknown-linux-gnu
/nix/store/rc5hkf5axsfgmqwwy8is532v8srwnx90-binutils-wrapper-2.40
/nix/store/cbq45bn0csibjx97fzdkrjsarl254jlj-gcc-wrapper-12.3.0

@wegank wegank added the 2.status: stale https://github.com/NixOS/nixpkgs/blob/master/.github/STALE-BOT.md label Mar 19, 2024
@sternenseemann
Copy link
Member

Best solution is to fix the incorrect logic in the wrappers themselves and adjust the stdenv bootstrapping accordingly: #304861

@stale stale bot removed the 2.status: stale https://github.com/NixOS/nixpkgs/blob/master/.github/STALE-BOT.md label Apr 18, 2024
@sternenseemann
Copy link
Member

Should be resolved on master!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
6.topic: cross-compilation Building packages on a different platform than they will be used on 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin 10.rebuild-linux: 501+ 10.rebuild-linux: 5001+ 10.rebuild-linux-stdenv This PR causes stdenv to rebuild
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants