Enable new pass manager with LLVM 13 #88243

nikic · 2021-08-22T20:58:35Z

The new pass manager is enabled by default in clang since Clang/LLVM 13. Per the recent discussion on llvm-dev (https://lists.llvm.org/pipermail/llvm-dev/2021-August/152305.html) the legacy pass manager will be unmaintained in LLVM 14 and removed entirely in LLVM 15.

This switches us to use the new pass manager if LLVM >= 13 is used. It's possible to still use the old pass manager using -Z new-llvm-pass-manager=no.

nikic · 2021-08-22T20:58:49Z

@bors try @rust-timer queue

rust-timer · 2021-08-22T20:58:51Z

Awaiting bors try build completion.

@rustbot label: +S-waiting-on-perf

bors · 2021-08-22T20:58:56Z

⌛ Trying commit 2b70908ae47b386d22aefa2cd445a26e1a600916 with merge 273c8e11ecc63776c5fe5d98ad6a74ef3a37a7f3...

bors · 2021-08-22T21:48:47Z

☀️ Try build successful - checks-actions
Build commit: 273c8e11ecc63776c5fe5d98ad6a74ef3a37a7f3 (273c8e11ecc63776c5fe5d98ad6a74ef3a37a7f3)

rust-timer · 2021-08-22T21:48:48Z

Queued 273c8e11ecc63776c5fe5d98ad6a74ef3a37a7f3 with parent 91f9806, future comparison URL.

rust-timer · 2021-08-22T23:41:07Z

Finished benchmarking try commit (273c8e11ecc63776c5fe5d98ad6a74ef3a37a7f3): comparison url.

Summary: This change led to very large relevant mixed results 🤷 in compiler performance.

Very large improvement in instruction counts (up to -32.4% on incr-patched: println builds of inflate)
Large regression in instruction counts (up to 2.1% on full builds of match-stress-enum)

If you disagree with this performance assessment, please file an issue in rust-lang/rustc-perf.

Benchmarking this pull request likely means that it is perf-sensitive, so we're automatically marking it as not fit for rolling up. While you can manually mark this PR as fit for rollup, we strongly recommend not doing so since this PR led to changes in compiler perf.

Next Steps: If you can justify the regressions found in this perf run, please indicate this with @rustbot label: +perf-regression-triaged along with sufficient written justification. If you cannot justify the regressions please fix the regressions and do another perf run. If the next run shows neutral or positive results, the label will be automatically removed.

@bors rollup=never
@rustbot label: +S-waiting-on-review -S-waiting-on-perf +perf-regression

klensy · 2021-08-23T16:27:45Z

But for max-rss it's not so green, sadly https://perf.rust-lang.org/compare.html?start=91f9806208834de3fb5f62712356b0d84ec388fd&end=273c8e11ecc63776c5fe5d98ad6a74ef3a37a7f3&stat=max-rss

nikic · 2021-08-28T11:11:48Z

Fix for wasm-panic-small regression: https://reviews.llvm.org/D108866

nikic · 2021-08-30T18:13:42Z

@klensy Yeah, that's a known tradeoff in the new pass manager design. It keeps analysis results around unless invalidated. That means less recalculations, but also more memory usage.

cuviper · 2021-08-31T00:36:50Z

Are there ways to mitigate that memory usage if needed? We already see rustc pushing address limits on 32-bit hosts...

nikic · 2021-08-31T15:54:14Z

r? @cuviper or @nagisa

Are there ways to mitigate that memory usage if needed? We already see rustc pushing address limits on 32-bit hosts...

I'm at least not aware of any options that can changed to reduce memory usage :/

wooster0 · 2021-08-31T16:04:09Z

Maybe a flag could be added that enables the old PM to trade longer compile times for less memory? --old-pm. If it's added, it should explicitly be a temporary flag though that gets removed in coming versions because at some point the old PM will probably be removed from newer LLVM versions. This could temporarily help with the memory usage for people that really need it (such as people on 32 bit systems) until more memory optimizations arrive in rustc or LLVM itself.

nikic · 2021-08-31T16:07:09Z

@r00ster91 This flag already exists in the form of -Z new-llvm-pass-manager=no :)

bors · 2021-09-01T14:37:02Z

☔ The latest upstream changes (presumably #88395) made this pull request unmergeable. Please resolve the merge conflicts.

bors · 2021-09-05T10:14:43Z

☔ The latest upstream changes (presumably #88603) made this pull request unmergeable. Please resolve the merge conflicts.

nagisa · 2021-09-25T11:30:18Z

@bors r+

bors · 2021-09-25T11:30:19Z

📌 Commit 51203dc has been approved by nagisa

bors · 2021-09-25T13:43:55Z

⌛ Testing commit 51203dc with merge 63cc2bb...

Amanieu · 2021-09-25T14:36:26Z

It is my understanding that SEH longjmp does not count as an "unwind" for the purposes of LLVM's nounwind. Clang always emits nounwind for C code even on MSVC targets and yet longjmp still works correctly. Since it is UB for longjmp to skip over the execution of any destructors (source: C++ standard), LLVM can just assume that longjmp won't execute any landing pads and optimize them away (which is main purpose of nounwind).

nikic · 2021-09-25T14:51:21Z

Oooh, I wasn't aware of that! LangRef for nounwind does indeed say:

This function attribute indicates that the function never raises an exception. If the function does raise an exception, its runtime behavior is undefined. However, functions marked nounwind may still trap or generate asynchronous exceptions. Exception handling schemes that are recognized by LLVM to handle asynchronous exceptions, such as SEH, will still provide their implementation defined semantics.

So nounwind is really noexcept, and this use of nounwind is fine.

nikic · 2021-09-25T16:18:12Z

I submitted https://bugs.llvm.org/show_bug.cgi?id=51969 for the optimization problem under NewPM.

bors · 2021-09-25T17:14:33Z

☀️ Test successful - checks-actions
Approved by: nagisa
Pushing 63cc2bb to master...

rust-timer · 2021-09-25T18:49:38Z

Finished benchmarking commit (63cc2bb): comparison url.

Summary: This change led to very large relevant mixed results 🤷 in compiler performance.

Very large improvement in instruction counts (up to -31.7% on incr-patched: println builds of inflate)
Large regression in instruction counts (up to 1.6% on full builds of inflate)

If you disagree with this performance assessment, please file an issue in rust-lang/rustc-perf.

Next Steps: If you can justify the regressions found in this perf run, please indicate this with @rustbot label: +perf-regression-triaged along with sufficient written justification. If you cannot justify the regressions please open an issue or create a new PR that fixes the regressions, add a comment linking to the newly created issue or PR, and then add the perf-regression-triaged label to this PR.

@rustbot label: +perf-regression

rylev · 2021-09-28T18:10:07Z

This is such a large performance gain that the few, small regressions are clearly worth the price. As noted this does regression rss, but that's an explicit tradeoff made by enabling the pass manager.

@rustbot label +perf-regression-triaged

2858: update rust to nightly-2021-10-03 r=bradjc a=hudson-ayers ### Pull Request Overview This pull request updates Rust to yesterday's nightly, and adds an `allow()` for 14 dead code warnings that surface in the tbf header module. I was motivated to submit this because upstream Rust recently switched over to LLVMs new pass manager (rust-lang/rust#88243) which seems to be able to make better decisions about inlining functions thanks to its bottom-up pass approach ensuring that callees are always maximally optimized by the time callers are being optimized. This change reduces code size by 1232 bytes on Imix. I realize that two rust updates in a month is more frequently than we normally update, so if we want to wait a while I understand. ### Testing Strategy This pull request was tested by compiling. ### TODO or Help Wanted N/A ### Documentation Updated - [x] Updated the relevant files in `/docs`, or no updates are required. ### Formatting - [x] Ran `make prepush`. Co-authored-by: Hudson Ayers <[email protected]>

Pkgsrc changes: * Adapt a couple of patches Upstream changes: Version 1.57.0 (2021-12-02) ========================== Language -------- - [Macro attributes may follow `#[derive]` and will see the original (pre-`cfg`) input.][87220] - [Accept curly-brace macros in expressions, like `m!{ .. }.method()` and `m!{ .. }?`.][88690] - [Allow panicking in constant evaluation.][89508] Compiler -------- - [Create more accurate debuginfo for vtables.][89597] - [Add `armv6k-nintendo-3ds` at Tier 3\*.][88529] - [Add `armv7-unknown-linux-uclibceabihf` at Tier 3\*.][88952] - [Add `m68k-unknown-linux-gnu` at Tier 3\*.][88321] - [Add SOLID targets at Tier 3\*:][86191] `aarch64-kmc-solid_asp3`, `armv7a-kmc-solid_asp3-eabi`, `armv7a-kmc-solid_asp3-eabihf` \* Refer to Rust's [platform support page][platform-support-doc] for more information on Rust's tiered platform support. Libraries --------- - [Avoid allocations and copying in `Vec::leak`][89337] - [Add `#[repr(i8)]` to `Ordering`][89507] - [Optimize `File::read_to_end` and `read_to_string`][89582] - [Update to Unicode 14.0][89614] - [Many more functions are marked `#[must_use]`][89692], producing a warning when ignoring their return value. This helps catch mistakes such as expecting a function to mutate a value in place rather than return a new value. Stabilised APIs --------------- - [`[T; N]::as_mut_slice`][`array::as_mut_slice`] - [`[T; N]::as_slice`][`array::as_slice`] - [`collections::TryReserveError`] - [`HashMap::try_reserve`] - [`HashSet::try_reserve`] - [`String::try_reserve`] - [`String::try_reserve_exact`] - [`Vec::try_reserve`] - [`Vec::try_reserve_exact`] - [`VecDeque::try_reserve`] - [`VecDeque::try_reserve_exact`] - [`Iterator::map_while`] - [`iter::MapWhile`] - [`proc_macro::is_available`] - [`Command::get_program`] - [`Command::get_args`] - [`Command::get_envs`] - [`Command::get_current_dir`] - [`CommandArgs`] - [`CommandEnvs`] These APIs are now usable in const contexts: - [`hint::unreachable_unchecked`] Cargo ----- - [Stabilize custom profiles][cargo/9943] Compatibility notes ------------------- Internal changes ---------------- These changes provide no direct user facing benefits, but represent significant improvements to the internals and overall performance of rustc and related tools. - [Added an experimental backend for codegen with `libgccjit`.][87260] [86191]: rust-lang/rust#86191 [87220]: rust-lang/rust#87220 [87260]: rust-lang/rust#87260 [88243]: rust-lang/rust#88243 [88321]: rust-lang/rust#88321 [88529]: rust-lang/rust#88529 [88690]: rust-lang/rust#88690 [88952]: rust-lang/rust#88952 [89337]: rust-lang/rust#89337 [89507]: rust-lang/rust#89507 [89508]: rust-lang/rust#89508 [89582]: rust-lang/rust#89582 [89597]: rust-lang/rust#89597 [89614]: rust-lang/rust#89614 [89692]: rust-lang/rust#89692 [cargo/9943]: rust-lang/cargo#9943 [`array::as_mut_slice`]: https://doc.rust-lang.org/std/primitive.array.html#method.as_mut_slice [`array::as_slice`]: https://doc.rust-lang.org/std/primitive.array.html#method.as_slice [`collections::TryReserveError`]: https://doc.rust-lang.org/std/collections/struct.TryReserveError.html [`HashMap::try_reserve`]: https://doc.rust-lang.org/std/collections/hash_map/struct.HashMap.html#method.try_reserve [`HashSet::try_reserve`]: https://doc.rust-lang.org/std/collections/hash_set/struct.HashSet.html#method.try_reserve [`String::try_reserve`]: https://doc.rust-lang.org/alloc/string/struct.String.html#method.try_reserve [`String::try_reserve_exact`]: https://doc.rust-lang.org/alloc/string/struct.String.html#method.try_reserve_exact [`Vec::try_reserve`]: https://doc.rust-lang.org/std/vec/struct.Vec.html#method.try_reserve [`Vec::try_reserve_exact`]: https://doc.rust-lang.org/std/vec/struct.Vec.html#method.try_reserve_exact [`VecDeque::try_reserve`]: https://doc.rust-lang.org/std/collections/struct.VecDeque.html#method.try_reserve [`VecDeque::try_reserve_exact`]: https://doc.rust-lang.org/std/collections/struct.VecDeque.html#method.try_reserve_exact [`Iterator::map_while`]: https://doc.rust-lang.org/std/iter/trait.Iterator.html#method.map_while [`iter::MapWhile`]: https://doc.rust-lang.org/std/iter/struct.MapWhile.html [`proc_macro::is_available`]: https://doc.rust-lang.org/proc_macro/fn.is_available.html [`Command::get_program`]: https://doc.rust-lang.org/std/process/struct.Command.html#method.get_program [`Command::get_args`]: https://doc.rust-lang.org/std/process/struct.Command.html#method.get_args [`Command::get_envs`]: https://doc.rust-lang.org/std/process/struct.Command.html#method.get_envs [`Command::get_current_dir`]: https://doc.rust-lang.org/std/process/struct.Command.html#method.get_current_dir [`CommandArgs`]: https://doc.rust-lang.org/std/process/struct.CommandArgs.html [`CommandEnvs`]: https://doc.rust-lang.org/std/process/struct.CommandEnvs.html

Pkgsrc changes: * Adjust line numbers in a number of patches * remove the --disable-dist-src option, so that we produce the rust-src rust component, which we upload to LOCALSRC to allow the rust-src package to build, which is needed for rust-analyzer. * Cargo checksum for vendor/cc no longer needs patching; checksum for vendor/libc updated Upstream changes: Version 1.57.0 (2021-12-02) ========================== Language -------- - [Macro attributes may follow `#[derive]` and will see the original (pre-`cfg`) input.][87220] - [Accept curly-brace macros in expressions, like `m!{ .. }.method()` and `m!{ .. }?`.][88690] - [Allow panicking in constant evaluation.][89508] Compiler -------- - [Create more accurate debuginfo for vtables.][89597] - [Add `armv6k-nintendo-3ds` at Tier 3\*.][88529] - [Add `armv7-unknown-linux-uclibceabihf` at Tier 3\*.][88952] - [Add `m68k-unknown-linux-gnu` at Tier 3\*.][88321] - [Add SOLID targets at Tier 3\*:][86191] `aarch64-kmc-solid_asp3`, `armv7a-kmc-solid_asp3-eabi`, `armv7a-kmc-solid_asp3-eabihf` \* Refer to Rust's [platform support page][platform-support-doc] for more information on Rust's tiered platform support. Libraries --------- - [Avoid allocations and copying in `Vec::leak`][89337] - [Add `#[repr(i8)]` to `Ordering`][89507] - [Optimize `File::read_to_end` and `read_to_string`][89582] - [Update to Unicode 14.0][89614] - [Many more functions are marked `#[must_use]`][89692], producing a warning when ignoring their return value. This helps catch mistakes such as expecting a function to mutate a value in place rather than return a new value. Stabilised APIs --------------- - [`[T; N]::as_mut_slice`][`array::as_mut_slice`] - [`[T; N]::as_slice`][`array::as_slice`] - [`collections::TryReserveError`] - [`HashMap::try_reserve`] - [`HashSet::try_reserve`] - [`String::try_reserve`] - [`String::try_reserve_exact`] - [`Vec::try_reserve`] - [`Vec::try_reserve_exact`] - [`VecDeque::try_reserve`] - [`VecDeque::try_reserve_exact`] - [`Iterator::map_while`] - [`iter::MapWhile`] - [`proc_macro::is_available`] - [`Command::get_program`] - [`Command::get_args`] - [`Command::get_envs`] - [`Command::get_current_dir`] - [`CommandArgs`] - [`CommandEnvs`] These APIs are now usable in const contexts: - [`hint::unreachable_unchecked`] Cargo ----- - [Stabilize custom profiles][cargo/9943] Compatibility notes ------------------- Internal changes ---------------- These changes provide no direct user facing benefits, but represent significant improvements to the internals and overall performance of rustc and related tools. - [Added an experimental backend for codegen with `libgccjit`.][87260] [86191]: rust-lang/rust#86191 [87220]: rust-lang/rust#87220 [87260]: rust-lang/rust#87260 [88243]: rust-lang/rust#88243 [88321]: rust-lang/rust#88321 [88529]: rust-lang/rust#88529 [88690]: rust-lang/rust#88690 [88952]: rust-lang/rust#88952 [89337]: rust-lang/rust#89337 [89507]: rust-lang/rust#89507 [89508]: rust-lang/rust#89508 [89582]: rust-lang/rust#89582 [89597]: rust-lang/rust#89597 [89614]: rust-lang/rust#89614 [89692]: rust-lang/rust#89692 [cargo/9943]: rust-lang/cargo#9943 [`array::as_mut_slice`]: https://doc.rust-lang.org/std/primitive.array.html#method.as_mut_slice [`array::as_slice`]: https://doc.rust-lang.org/std/primitive.array.html#method.as_slice [`collections::TryReserveError`]: https://doc.rust-lang.org/std/collections/struct.TryReserveError.html [`HashMap::try_reserve`]: https://doc.rust-lang.org/std/collections/hash_map/struct.HashMap.html#method.try_reserve [`HashSet::try_reserve`]: https://doc.rust-lang.org/std/collections/hash_set/struct.HashSet.html#method.try_reserve [`String::try_reserve`]: https://doc.rust-lang.org/alloc/string/struct.String.html#method.try_reserve [`String::try_reserve_exact`]: https://doc.rust-lang.org/alloc/string/struct.String.html#method.try_reserve_exact [`Vec::try_reserve`]: https://doc.rust-lang.org/std/vec/struct.Vec.html#method.try_reserve [`Vec::try_reserve_exact`]: https://doc.rust-lang.org/std/vec/struct.Vec.html#method.try_reserve_exact [`VecDeque::try_reserve`]: https://doc.rust-lang.org/std/collections/struct.VecDeque.html#method.try_reserve [`VecDeque::try_reserve_exact`]: https://doc.rust-lang.org/std/collections/struct.VecDeque.html#method.try_reserve_exact [`Iterator::map_while`]: https://doc.rust-lang.org/std/iter/trait.Iterator.html#method.map_while [`iter::MapWhile`]: https://doc.rust-lang.org/std/iter/struct.MapWhile.html [`proc_macro::is_available`]: https://doc.rust-lang.org/proc_macro/fn.is_available.html [`Command::get_program`]: https://doc.rust-lang.org/std/process/struct.Command.html#method.get_program [`Command::get_args`]: https://doc.rust-lang.org/std/process/struct.Command.html#method.get_args [`Command::get_envs`]: https://doc.rust-lang.org/std/process/struct.Command.html#method.get_envs [`Command::get_current_dir`]: https://doc.rust-lang.org/std/process/struct.Command.html#method.get_current_dir [`CommandArgs`]: https://doc.rust-lang.org/std/process/struct.CommandArgs.html [`CommandEnvs`]: https://doc.rust-lang.org/std/process/struct.CommandEnvs.html

rustbot added the S-waiting-on-perf Status: Waiting on a perf run to be completed. label Aug 22, 2021

rustbot added perf-regression Performance regression. S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-perf Status: Waiting on a perf run to be completed. labels Aug 22, 2021

joshtriplett added relnotes-perf Performance improvements that should be mentioned in the release notes. and removed perf-regression Performance regression. labels Aug 23, 2021

inquisitivecrystal added A-LLVM Area: Code generation parts specific to LLVM. Both correctness bugs and optimization-related issues. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. labels Aug 24, 2021

nikic mentioned this pull request Aug 29, 2021

[NewPM] Add missing LTO ArgPromotion pass rust-lang/llvm-project#115

Merged

nikic force-pushed the newpm-2 branch from 2b70908 to 964b371 Compare August 31, 2021 15:35

nikic changed the title ~~Enable new pass manager with LLVM 13 (WIP)~~ Enable new pass manager with LLVM 13 Aug 31, 2021

rust-highfive assigned cuviper Aug 31, 2021

nikic force-pushed the newpm-2 branch from 964b371 to ee6fbd1 Compare September 1, 2021 15:38

nikic force-pushed the newpm-2 branch from ee6fbd1 to 595c741 Compare September 5, 2021 11:20

Pin panic-in-drop=abort test to old pass manager

51203dc

nikic force-pushed the newpm-2 branch from 3ad2fdf to 51203dc Compare September 25, 2021 10:40

bors removed the S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. label Sep 25, 2021

bors added the merged-by-bors This PR was explicitly merged by bors. label Sep 25, 2021

bors merged commit 63cc2bb into rust-lang:master Sep 25, 2021

rustbot added this to the 1.57.0 milestone Sep 25, 2021

rustbot added the perf-regression Performance regression. label Sep 25, 2021

dtolnay mentioned this pull request Sep 26, 2021

Incompatible with nightly-2021-09-26: unknown pass name 'sancov' rust-fuzz/cargo-fuzz#276

Closed

rustbot added the perf-regression-triaged The performance regression has been triaged. label Sep 28, 2021

taiki-e mentioned this pull request Oct 2, 2021

Update for new LLVM pass manager rust-fuzz/afl.rs#193

Merged

This was referenced Oct 4, 2021

Tracking issue for new LLVM pass manager (-Znew-llvm-pass-manager) #74705

Closed

update rust to nightly-2021-10-03 tock/tock#2858

Merged

ehuss mentioned this pull request Oct 5, 2021

toml 0.4.9 to 0.5.3 compile for a long time on 2021-10-02 nightly #89524

Closed

hellow554 mentioned this pull request Dec 3, 2021

same functions are not merged in library #91490

Closed

ldm0 mentioned this pull request May 26, 2022

Infinite recursion optimized away with opt-level=z #97428

Closed

bjorn3 mentioned this pull request May 24, 2023

Is longjmp through rust code UB rust-lang/unsafe-code-guidelines#404

Open

gurry mentioned this pull request Dec 3, 2023

ice: ProjectionKinds Deref and Field were mismatched #118144

Closed

elsandosgrande mentioned this pull request Sep 3, 2024

Discussion: Polly in Gentoo LLVM Stack xarblu/xarblu-overlay#345

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Enable new pass manager with LLVM 13 #88243

Enable new pass manager with LLVM 13 #88243

nikic commented Aug 22, 2021 •

edited

Loading

nikic commented Aug 22, 2021

rust-timer commented Aug 22, 2021

bors commented Aug 22, 2021

bors commented Aug 22, 2021

rust-timer commented Aug 22, 2021

rust-timer commented Aug 22, 2021

klensy commented Aug 23, 2021

nikic commented Aug 28, 2021

nikic commented Aug 30, 2021

cuviper commented Aug 31, 2021

nikic commented Aug 31, 2021

wooster0 commented Aug 31, 2021 •

edited

Loading

nikic commented Aug 31, 2021

bors commented Sep 1, 2021

bors commented Sep 5, 2021

nagisa commented Sep 25, 2021

bors commented Sep 25, 2021

bors commented Sep 25, 2021

Amanieu commented Sep 25, 2021

nikic commented Sep 25, 2021 •

edited

Loading

nikic commented Sep 25, 2021

bors commented Sep 25, 2021

rust-timer commented Sep 25, 2021

rylev commented Sep 28, 2021

Enable new pass manager with LLVM 13 #88243

Enable new pass manager with LLVM 13 #88243

Conversation

nikic commented Aug 22, 2021 • edited Loading

nikic commented Aug 22, 2021

rust-timer commented Aug 22, 2021

bors commented Aug 22, 2021

bors commented Aug 22, 2021

rust-timer commented Aug 22, 2021

rust-timer commented Aug 22, 2021

klensy commented Aug 23, 2021

nikic commented Aug 28, 2021

nikic commented Aug 30, 2021

cuviper commented Aug 31, 2021

nikic commented Aug 31, 2021

wooster0 commented Aug 31, 2021 • edited Loading

nikic commented Aug 31, 2021

bors commented Sep 1, 2021

bors commented Sep 5, 2021

nagisa commented Sep 25, 2021

bors commented Sep 25, 2021

bors commented Sep 25, 2021

Amanieu commented Sep 25, 2021

nikic commented Sep 25, 2021 • edited Loading

nikic commented Sep 25, 2021

bors commented Sep 25, 2021

rust-timer commented Sep 25, 2021

rylev commented Sep 28, 2021

nikic commented Aug 22, 2021 •

edited

Loading

wooster0 commented Aug 31, 2021 •

edited

Loading

nikic commented Sep 25, 2021 •

edited

Loading