stdenv: add rebootstrap parameter #209459
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This was referenced Jan 7, 2023
ofborg
bot
added
10.rebuild-darwin: 0
ghost
marked this pull request as ready for review
ghost
marked this pull request as draft
The command
```
nix-build -A tests.trivial-builders.references --show-trace
```
fails eval with
```
in job ‘nixpkgs.tests.trivial-builders.references’:
error: The option `meta.description' does not exist. Definition values:
- In `makeTest parameters': "Run the Nixpkgs trivial builders tests"
```
because `meta.description` and `meta.license` are not valid for
`nixosTest`s (they are valid for `mkDerivation` of course).
This has been causing Hydra eval failures:
https://hydra.nixos.org/jobset/nixos/pr-209870-gcc-external-bootstrap#tabs-errors
Let's fix eval by removing these attributes.
The command at the top of this file fails to evaluate: ``` $ nix-build -A tests.pkg-config.defaultPkgConfigPackages in job ‘nixpkgs.tests.pkg-config.defaultPkgConfigPackages.tests-combined.x86_64-linux’: error: pkg-config module `recurseForDerivations` is not defined to be a derivation. Please check the attribute value for `recurseForDerivations` in `pkgs/top-level/pkg-config-packages.nix` in Nixpkgs. ``` This is also causing eval errors on Hydra: https://hydra.nixos.org/jobset/nixos/pr-209870-gcc-external-bootstrap#tabs-errors Let's filter out `recurseForDerivations=true` from the attrset, since it exists mainly as a flag to signal special handling when recursing.
When wrapping `clang` and using a `gccForLibs` whose `libgcc` is in
its own output (rather than the `lib` output), this commit will adds
`-L${gccForLibs.libgcc}/lib` to `cc-ldflags`.
If that flag is not added, `firefox` will fail to compile because it
invokes `clang-wrapper` with `-fuse-ld=lld` and passes `-lgcc_s` to
`lld`, but does not tell `lld` where to find `libgcc_s.so`. In that
situation, firefox will fail to link.
The Nix-driven bootstrap of gcc expects that `stdenv.cc.cc` has a `version` attribute, except if `stdenv.cc` is the `bootstrapFiles`. This commit causes `ccache-links`'s `cc-wrapper` to `inherit version` in order to satisfy that requirement. Without this commit, ofborg CI will fail. Co-authored-by: Sandro <[email protected]>
This commit has no effect on eval. It simply factors out a common subexpression.
This commit has no effect on eval. It simply reorganizes the `gcc11` and `gcc12` expressions so they apply a list of `overrideAttrs`. The list is currently empty.
#### Summary
By default, when you type `make`, GCC will compile itself three
times. This PR inhibits that behavior by configuring GCC with
`--disable-bootstrap`, and reimplements the triple-rebuild using
Nix rather than `make`/`sh`.
#### Immediate Benefits
- Allow `gcc11` and `gcc12` on `aarch64` (without needing new
`bootstrapFiles`)
- Faster stdenv rebuilds: the third compilation of gcc
(i.e. stageCompare) is no longer a `drvInput` of the final stdenv.
This allows Nix to build stageCompare in parallel with the rest of
nixpkgs instead of in series.
- No more copying `libgcc_s` out of the bootstrap-files or other
derivations
- No more Frankenstein compiler: the final gcc and the libraries it
links against (mpfr, mpc, isl, glibc) are all built by the same
compiler (xgcc) instead of a mixture of the bootstrapFiles'
compiler and xgcc.
- No more [static lib{mpfr,mpc,gmp,isl}.a hack]
- Many other small `stdenv` hacks eliminated
- `gcc` and `clang` share the same codepath for more of `cc-wrapper`.
#### Future Benefits
- This should allow using a [foreign] `bootstrap-files` so long as
`hostPlatform.canExecute bootstrapFiles`.
- This should allow each of the libraries that ship with `gcc`
(lib{backtrace, atomic, cc1, decnumber, ffi, gomp, iberty,
offloadatomic, quadmath, sanitizer, ssp, stdc++-v3, vtv}) to be
built in separate (one-liner) derivations which `inherit src;`
from `gcc`, much like #132343
#### Incorporates
- #210004
- #36948 (unreverted)
- #210325
- #210118
- #210132
- #210109
- #213909
- #216136
- #216237
- #210019
- #216232
- #216016
- #217977
- #217995
#### Closes
- Closes #108305
- Closes #108111
- Closes #201254
- Closes #208412
#### Credits
This project was made possible by three important insights, none of
which were mine:
1. @Ericson2314 was the first to advocate for this change, and
probably the first to appreciate its advantages. Nix-driven
(external) bootstrap is "cross by default".
2. @trofi has figured out a lot about how to get gcc to not mix up
the copy of `libstdc++` that it depends on with the copy that it
builds, by moving the `bootstrapFiles`' `libstdc++` into a
[versioned directory]. This allows a Nix-driven bootstrap of gcc
without the final gcc would still having references to the
`bootstrapFiles`.
3. Using the undocumented variable [`user-defined-trusted-dirs`]
when building glibc. When glibc `dlopen()`s `libgcc_s.so`, it
uses a completely different and totally special set of rules for
finding `libgcc_s.so`. This trick is the only way we can put
`libgcc_s.so` in its own separate outpath without creating
circular dependencies or dependencies on the bootstrapFiles. I
would never have guessed to use this (or that it existed!) if it
were not for a [comment in guix] which @Mic92 [mentioned].
My own role in this PR was basically: being available to go on a
coding binge at an opportune moment, so we wouldn't waste a
[crisis].
[aarch64-compare-ofborg]: https://github.com/NixOS/nixpkgs/pull/209870/checks?check_run_id=10662822938
[amd64-compare-ofborg]: https://github.com/NixOS/nixpkgs/pull/209870/checks?check_run_id=10662825857
[nonexistent sysroot]: #210004
[versioned directory]: #209054
[`user-defined-trusted-dirs`]: https://sourceware.org/legacy-ml/libc-help/2013-11/msg00026.html
[comment in guix]: https://github.com/guix-mirror/guix/blob/5e4ec8218142eee8e6e148e787381a5ef891c5b1/gnu/packages/gcc.scm#L253
[mentioned]: #210112 (comment)
[crisis]: #108305
[foreign]: #170857 (comment)
[static lib{mpfr,mpc,gmp,isl}.a hack]: https://github.com/NixOS/nixpkgs/blob/2f1948af9c984ebb82dfd618e67dc949755823e2/pkgs/stdenv/linux/default.nix#L380
This commit adds `gcc/common/checksum.nix`, which contains code common to both gcc11 and gcc12, implementing the `enableChecksum` feature. When gcc's built-in bootstrap (`--enable-bootstrap`) is used, gcc compiles itself three times and compares a hash of the unlinked `.o` files from the second and third compilation. The `enableChecksum=true` parameter performs the same comparison as part of the `postInstall` phase. Notably, `enableChecksum=true` can be used with `enableBootstrap=false`. Co-authored-by: Sandro <[email protected]>
This commit adds a derivation `gcc-stageCompare` to `pkgs/test/stdenv/default.nix`. It is important to always build this derivation whenever building `stdenv`! Because we are using a Nix-driven bootstrap instead of gcc's built-in `--enable-bootstrap`, the `gcc` derivation no longer performs the post-self-compilation sanity check. You must build this derivation in order to perform that sanity check. The major benefit of this new approach is that the sanity check (which involves a third compilation of gcc) can be performed *concurrently* with all packages that depend on `stdenv`, rather than serially. Since `stdenv` has very little derivation-level parallelism it cannot take advantage of more than one or perhaps two builders. If you have three or more builders this commit will reduce the time-to-rebuild-stdenv by around 20% (one of three gcc rebuilds is removed from the critical path, and stdenv's build time is dominated by roughly 3*gcc + 1*binutils + 1*bison-test-suite). Co-authored-by: Sandro <[email protected]>
The Nix-driven bootstrap of gcc resulted in some changes to the structure of the `libgccjit` outpaths, and also added an additional output (`libgcc`) to `gcc`. This commit makes the corresponding changes in the `emacs` derivation in order to not break emacs. Emacs is the only user of `libgccjit` in nixpkgs at the moment.
This commit implements @SuperSandro2000's suggestion here: #209870 (review) I am submitting it as a separate PR in order to avoid a full rebuild on Hydra.
This reverts commit 8841434.
Now that we've dropped the
gcc-links-statically-to-lib{isl,mpfr,mpc,gmp} hack, our gcc needs
libisl.so. Let's add it to the bootstrap-files.
We do not want to preserve the symlinks from libgcc_s, since they point to another outpath. We want to copy from that outpath rather than link to it.
…ibstdc++} Our bootstrap-files unpacker has always relied on a lot of unstated assumptions, one of them being that every library has a DT_NEEDED for librt.so, so patchelf'ing something into the RUNPATH into librt.so means that it will be searched for every library load in all of the bootstrap-files. Unfortunately that assumption is not true for libgcc. This causes problems, because patchelf links against libgcc (and against libstdc++, which links against libgcc). So we can't use patchelf on libgcc, because it needs libgcc, so patchelf doesn't work until libgcc is patchelfed. The robust solution here is to use static linking for the copy of patchelf that is shipped with the bootstrap-files. We don't have to go all the way to a statically linked libc; just -static-libgcc and -static-libstdc++ are enough to break the circular dependency.
github-actions
bot
added
6.topic: emacs
ghost
mentioned this pull request
4 tasks
ofborg
bot
added
10.rebuild-linux-stdenv
ghost
closed this
This pull request was closed.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description of changes
This is just for testing/development purposes. It should never be enabled for production builds.
This makes it very easy to find out if stale bootstrapFiles are the cause of some problem.
Building with
rebootstrap=stdenv.hostPlatform.isFoowould be a useful thing to have in the "large" Hydra jobset. It's basically likenix build -f pkgs/top-level/release.nix stdenvBootstrapTools.x86_64-linux.testbut it builds all of stdenv using the normal nixpkgs expressions instead of just buildinghellowith a custom expression.Things done