-
Notifications
You must be signed in to change notification settings - Fork 12.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
ci: Build gccjit from a git archive #122658
Conversation
A full `git clone` of GCC includes quite a lot of history, and it's completely unnecessary for building it in CI. We can use a GitHub archive URL to get a simple tarball that is much faster to download. Also, the `gcc-build` directory can be removed after install to reduce the image size even further.
rustbot has assigned @Mark-Simulacrum. Use r? to explicitly pick a reviewer |
# This commit hash needs to be updated to use a more recent gcc fork version. | ||
git checkout 78dc50f0e50e6cd1433149520bd512a4e0eaa1bc | ||
curl -L "$GIT_REPO/archive/$GIT_COMMIT.tar.gz" | | ||
tar -xz --transform "s/gcc-$GIT_COMMIT/gcc-src/" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If for some reason we do want git metadata on disk, AFAIK shallow clones are not possible with specific commit hashes, but you can get there with git init; git fetch --depth 1 $repo $commit; git checkout $commit
.
Co-authored-by: Urgau <[email protected]>
Co-authored-by: Urgau <[email protected]>
cd gcc-src | ||
# This commit hash needs to be updated to use a more recent gcc fork version. | ||
git checkout 78dc50f0e50e6cd1433149520bd512a4e0eaa1bc | ||
curl -L "$GIT_REPO/archive/$GIT_COMMIT.tar.gz" | |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We should probably also retry
the download if it fails for some reason.
curl -L "$GIT_REPO/archive/$GIT_COMMIT.tar.gz" | | |
retry curl -L "$GIT_REPO/archive/$GIT_COMMIT.tar.gz" | |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think it's not that simple -- if a partial download fails and restarts, the latter | tar
is not likely to fare well. We could download to disk first, or wrap the pipeline in a bash function to be retried. But I wonder if this is useful, because if a GitHub CI runner can't download from github.com
, we likely have bigger issues...
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The issue here is not so much the download but the transient DNS failures, they happened in the past, even for github.com
(can't find a example right now).
Sure, we may want to download first to disk; I didn't look to closely at the retry function.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I wouldn't worry about it. The whole docker build is already retried 5 times IIRC. This is maybe nice cleanup in the future but isn't worth it for now.
@bors r+ rollup |
@bors rollup=iffy (actually) |
…ulacrum ci: Build gccjit from a git archive A full `git clone` of GCC includes quite a lot of history, and it's completely unnecessary for building it in CI. We can use a GitHub archive URL to get a simple tarball that is much faster to download. Also, the `gcc-build` directory can be removed after install to reduce the image size even further.
…iaskrgr Rollup of 9 pull requests Successful merges: - rust-lang#116016 (Soft-destabilize `RustcEncodable` & `RustcDecodable`, remove from prelude in next edition) - rust-lang#122460 (Rework rmake support library API) - rust-lang#122658 (ci: Build gccjit from a git archive) - rust-lang#122698 (Cancel `cargo update` job if there's no updates) - rust-lang#122878 (Use `arch::wasm::unreachable` instead of `arch::wasm32::unreachable`) - rust-lang#122915 (Delay a bug if no RPITITs were found) - rust-lang#122916 (docs(sync): normalize dot in fn summaries) - rust-lang#122921 (Enable more mir-opt tests in debug builds) - rust-lang#122922 (-Zprint-type-sizes: print the types of awaitees and unnamed coroutine locals.) r? `@ghost` `@rustbot` modify labels: rollup
…acrum ci: Build gccjit from a git archive A full `git clone` of GCC includes quite a lot of history, and it's completely unnecessary for building it in CI. We can use a GitHub archive URL to get a simple tarball that is much faster to download. Also, the `gcc-build` directory can be removed after install to reduce the image size even further.
💥 Test timed out |
@bors retry apple-aarch64 timed out |
☀️ Test successful - checks-actions |
Finished benchmarking commit (0824b30): comparison URL. Overall result: ✅ improvements - no action needed@rustbot label: -perf-regression Instruction countThis is a highly reliable metric that was used to determine the overall result at the top of this comment.
Max RSS (memory usage)ResultsThis is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.
CyclesThis benchmark run did not return any relevant results for this metric. Binary sizeThis benchmark run did not return any relevant results for this metric. Bootstrap: 670.98s -> 670.209s (-0.11%) |
A job failed! Check out the build log: (web) (plain) Click to see the possible cause of the failure (guessed by this bot)
|
A full
git clone
of GCC includes quite a lot of history, and it'scompletely unnecessary for building it in CI. We can use a GitHub
archive URL to get a simple tarball that is much faster to download.
Also, the
gcc-build
directory can be removed after install to reducethe image size even further.