Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Upgrade pre-built Clang used in MSVC and MacOS builds, move MSVC builds to Server 2022 #124850

Merged
merged 1 commit into from
May 10, 2024

Conversation

dpaoliello
Copy link
Contributor

@dpaoliello dpaoliello commented May 7, 2024

Fixes #92948

Example working MacOS and Windows builds: https://github.com/rust-lang/rust/actions/runs/8989360201

There is a bug in Clang 18 that causes issues when building for Arm64 in later parts of the build (specifically libgit2). As a workaround, we will still use the pre-built Clang to build LLVM but will use MSVC for the rest of the Arm64 build.

@rustbot
Copy link
Collaborator

rustbot commented May 7, 2024

r? @Kobzol

rustbot has assigned @Kobzol.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

@rustbot rustbot added A-testsuite Area: The testsuite used to check the correctness of rustc S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-infra Relevant to the infrastructure team, which will review and decide on the PR/issue. labels May 7, 2024
@dpaoliello dpaoliello marked this pull request as ready for review May 7, 2024 19:24
@Kobzol
Copy link
Contributor

Kobzol commented May 7, 2024

I wonder why the build is slower now.

Master:
image

Windows 2022:
image

Could you please add the jobs to the try build section so that we can run them on rust-lang-ci, twice in a row, just to see if maybe some caching will start kicking in?

@dpaoliello
Copy link
Contributor Author

Could you please add the jobs to the try build section so that we can run them on rust-lang-ci, twice in a row, just to see if maybe some caching will start kicking in?

Done

@Kobzol
Copy link
Contributor

Kobzol commented May 7, 2024

@bors try

@bors
Copy link
Contributor

bors commented May 7, 2024

⌛ Trying commit 768e382 with merge 00258dc...

bors added a commit to rust-lang-ci/rust that referenced this pull request May 7, 2024
Upgrade pre-built Clang used in MSVC and MacOS builds, move MSVC builds to Server 2022

Fixes rust-lang#92948

Example working MacOS and Windows builds: <https://github.com/rust-lang/rust/actions/runs/8989360201>

Using Server 2022 for AArch64 MSVC is blocked by <llvm/llvm-project#81849>. I'll see if I can work around the issue in a follow up PR.
@dpaoliello
Copy link
Contributor Author

I wonder why the build is slower now.

Looks like a little from git, but the majority was building LLVM. Comparing the x86-64 builds:

Part Before After Delta
Total 1:29:12 1:56:52 +27:40
"checkout the source code" 0:00:22 0:01:54 +01:32
"checkout submodules" 0:01:47 0:04:48 +03:01
"run the build" 1:24:41 1:47:44 +22:57
LLVM 0:05:22 0:27:58 +22:36

The LLVM build slowdown makes sense, given that we wrap Clang in SCCache to speed up the build, but we wouldn't have a cached build using the updated version of Clang.

@Kobzol
Copy link
Contributor

Kobzol commented May 7, 2024

Yeah, that makes sense. Let's see if sccache helps.

@dpaoliello
Copy link
Contributor Author

dpaoliello commented May 7, 2024

First run is complete, can we try again and see if it picks up the cached build from the last attempt (assuming that try builds update the cache?)

@Kobzol
Copy link
Contributor

Kobzol commented May 8, 2024

@bors try

@bors
Copy link
Contributor

bors commented May 8, 2024

⌛ Trying commit f8770d8 with merge 21e0471...

bors added a commit to rust-lang-ci/rust that referenced this pull request May 8, 2024
Upgrade pre-built Clang used in MSVC and MacOS builds, move MSVC builds to Server 2022

Fixes rust-lang#92948

Example working MacOS and Windows builds: <https://github.com/rust-lang/rust/actions/runs/8989360201>

Using Server 2022 for AArch64 MSVC is blocked by <llvm/llvm-project#81849>. I'll see if I can work around the issue in a follow up PR.
@bors
Copy link
Contributor

bors commented May 8, 2024

☀️ Try build successful - checks-actions
Build commit: 21e0471 (21e04717e36d43996a01589abec8505839fea807)

@Kobzol
Copy link
Contributor

Kobzol commented May 8, 2024

Ok, it's fast again, great. Could you please remove the CI modifications?

Otherwise PR looks good to me. - &job-windows-2019-16c seems to be unused after this PR (?), if it is unused, I would just remove it from jobs.yml.

@dpaoliello
Copy link
Contributor Author

Ok, it's fast again, great. Could you please remove the CI modifications?

Otherwise PR looks good to me. - &job-windows-2019-16c seems to be unused after this PR (?), if it is unused, I would just remove it from jobs.yml.

Done - thanks so much for helping out with this :)

@Kobzol
Copy link
Contributor

Kobzol commented May 8, 2024

@bors r+ rollup=never

@bors
Copy link
Contributor

bors commented May 8, 2024

📌 Commit 4aaadd1 has been approved by Kobzol

It is now in the queue for this repository.

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels May 8, 2024
@Kobzol
Copy link
Contributor

Kobzol commented May 8, 2024

Thanks for preparing and testing this! Let's see if it works.

@bors
Copy link
Contributor

bors commented May 8, 2024

⌛ Testing commit 4aaadd1 with merge 87f5aa7...

@rust-log-analyzer

This comment has been minimized.

@bors
Copy link
Contributor

bors commented May 9, 2024

💔 Test failed - checks-actions

@bors bors added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels May 9, 2024
@dpaoliello
Copy link
Contributor Author

Ah, interesting, it's Clang 18 that breaks aarch64-msvc, not VS 2022's STL - I guess I'll have to apply the workaround to this PR...

@dpaoliello dpaoliello marked this pull request as draft May 9, 2024 17:03
@rustbot rustbot added the T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap) label May 9, 2024
@rust-log-analyzer

This comment has been minimized.

@dpaoliello
Copy link
Contributor Author

@Kobzol Workaround for aarch64-msvc has been implemented: please have a look at builder.rs

@Kobzol
Copy link
Contributor

Kobzol commented May 9, 2024

@bors r+

@bors
Copy link
Contributor

bors commented May 9, 2024

📌 Commit 5212e07 has been approved by Kobzol

It is now in the queue for this repository.

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels May 9, 2024
@dpaoliello dpaoliello marked this pull request as ready for review May 9, 2024 21:59
@bors
Copy link
Contributor

bors commented May 10, 2024

⌛ Testing commit 5212e07 with merge a6e87c5...

@bors
Copy link
Contributor

bors commented May 10, 2024

☀️ Test successful - checks-actions
Approved by: Kobzol
Pushing a6e87c5 to master...

@bors bors added the merged-by-bors This PR was explicitly merged by bors. label May 10, 2024
@bors bors merged commit a6e87c5 into rust-lang:master May 10, 2024
7 checks passed
@rustbot rustbot added this to the 1.80.0 milestone May 10, 2024
@dpaoliello dpaoliello deleted the clang2022 branch May 10, 2024 02:53
@rust-timer
Copy link
Collaborator

Finished benchmarking commit (a6e87c5): comparison URL.

Overall result: ✅ improvements - no action needed

@rustbot label: -perf-regression

Instruction count

This is a highly reliable metric that was used to determine the overall result at the top of this comment.

mean range count
Regressions ❌
(primary)
- - 0
Regressions ❌
(secondary)
- - 0
Improvements ✅
(primary)
- - 0
Improvements ✅
(secondary)
-0.3% [-0.3%, -0.3%] 1
All ❌✅ (primary) - - 0

Max RSS (memory usage)

This benchmark run did not return any relevant results for this metric.

Cycles

This benchmark run did not return any relevant results for this metric.

Binary size

This benchmark run did not return any relevant results for this metric.

Bootstrap: 673.028s -> 675.734s (0.40%)
Artifact size: 315.78 MiB -> 315.98 MiB (0.06%)

bors added a commit to rust-lang-ci/rust that referenced this pull request Oct 16, 2024
(ci) Update macOS Xcode to 15

This updates the macOS builders to Xcode 15. The aarch64 images will be removing Xcode 14 and 16 very soon (actions/runner-images#10703), so we will need to make the switch to continue operating. The linked issue also documents GitHub's new policy for how they will be updating Xcode in the future. Also worth being aware of is the future plans for x86 runners documented in actions/runner-images#9255 and actions/runner-images#10686, which will impact our future upgrade behaviors.

I decided to also update the Xcode in the x86_64 runners, even though they are not being removed. It felt better to me to have all macOS runners on the same (major) version of Xcode. However, note that the x86_64 runners do not have the latest version of 15 (15.4), so I left them at 15.2 (which is currently the default Xcode of the runner).

Xcode 15 was previously causing problems (see rust-lang#121058) which seem to be resolved now. `@bjorn3` fixed the `invalid r_symbolnum` issue with cranelift. The issue with clang failing to link seems to be fixed, possibly by the update of the pre-built LLVM from 14 to llvm 15 in rust-lang#124850, or an update in our source version of LLVM. I have run some try builds and at least LLVM seems to build (I did not run any tests).

Closes rust-lang#121058
bors added a commit to rust-lang-ci/rust that referenced this pull request Oct 21, 2024
(ci) Update macOS Xcode to 15

This updates the macOS builders to Xcode 15. The aarch64 images will be removing Xcode 14 and 16 very soon (actions/runner-images#10703), so we will need to make the switch to continue operating. The linked issue also documents GitHub's new policy for how they will be updating Xcode in the future. Also worth being aware of is the future plans for x86 runners documented in actions/runner-images#9255 and actions/runner-images#10686, which will impact our future upgrade behaviors.

I decided to also update the Xcode in the x86_64 runners, even though they are not being removed. It felt better to me to have all macOS runners on the same (major) version of Xcode. However, note that the x86_64 runners do not have the latest version of 15 (15.4), so I left them at 15.2 (which is currently the default Xcode of the runner).

Xcode 15 was previously causing problems (see rust-lang#121058) which seem to be resolved now. `@bjorn3` fixed the `invalid r_symbolnum` issue with cranelift. The issue with clang failing to link seems to be fixed, possibly by the update of the pre-built LLVM from 14 to llvm 15 in rust-lang#124850, or an update in our source version of LLVM. I have run some try builds and at least LLVM seems to build (I did not run any tests).

Closes rust-lang#121058
github-actions bot pushed a commit to rust-lang/miri that referenced this pull request Oct 22, 2024
(ci) Update macOS Xcode to 15

This updates the macOS builders to Xcode 15. The aarch64 images will be removing Xcode 14 and 16 very soon (actions/runner-images#10703), so we will need to make the switch to continue operating. The linked issue also documents GitHub's new policy for how they will be updating Xcode in the future. Also worth being aware of is the future plans for x86 runners documented in actions/runner-images#9255 and actions/runner-images#10686, which will impact our future upgrade behaviors.

I decided to also update the Xcode in the x86_64 runners, even though they are not being removed. It felt better to me to have all macOS runners on the same (major) version of Xcode. However, note that the x86_64 runners do not have the latest version of 15 (15.4), so I left them at 15.2 (which is currently the default Xcode of the runner).

Xcode 15 was previously causing problems (see #121058) which seem to be resolved now. `@bjorn3` fixed the `invalid r_symbolnum` issue with cranelift. The issue with clang failing to link seems to be fixed, possibly by the update of the pre-built LLVM from 14 to llvm 15 in rust-lang/rust#124850, or an update in our source version of LLVM. I have run some try builds and at least LLVM seems to build (I did not run any tests).

Closes #121058
lnicola pushed a commit to lnicola/rust-analyzer that referenced this pull request Oct 22, 2024
(ci) Update macOS Xcode to 15

This updates the macOS builders to Xcode 15. The aarch64 images will be removing Xcode 14 and 16 very soon (actions/runner-images#10703), so we will need to make the switch to continue operating. The linked issue also documents GitHub's new policy for how they will be updating Xcode in the future. Also worth being aware of is the future plans for x86 runners documented in actions/runner-images#9255 and actions/runner-images#10686, which will impact our future upgrade behaviors.

I decided to also update the Xcode in the x86_64 runners, even though they are not being removed. It felt better to me to have all macOS runners on the same (major) version of Xcode. However, note that the x86_64 runners do not have the latest version of 15 (15.4), so I left them at 15.2 (which is currently the default Xcode of the runner).

Xcode 15 was previously causing problems (see #121058) which seem to be resolved now. `@bjorn3` fixed the `invalid r_symbolnum` issue with cranelift. The issue with clang failing to link seems to be fixed, possibly by the update of the pre-built LLVM from 14 to llvm 15 in rust-lang/rust#124850, or an update in our source version of LLVM. I have run some try builds and at least LLVM seems to build (I did not run any tests).

Closes #121058
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-testsuite Area: The testsuite used to check the correctness of rustc merged-by-bors This PR was explicitly merged by bors. S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap) T-infra Relevant to the infrastructure team, which will review and decide on the PR/issue.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

CI: Migrate to windows-2022
7 participants