-
Notifications
You must be signed in to change notification settings - Fork 12.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Failed to compile a project with LTO + PGO #115344
Comments
Were you able to reproduce this without using cargo pgo, by using the compiler flags manually? (-Cprofile-generate etc.) |
I haven't tried to do it yet but I will (and then report the results here) |
My project was done using the flags directly. See https://gitlab.com/kevincox/ricochetrobots/-/blob/6bbc44ddbf2ab8ae9edcbdca62e3035572f9c1ed/default.nix#L41-57 # Cargo.toml
[profile.release]
codegen-units = 1
lto = true
module flag identifiers must be unique (or of 'require' type) !"CG Profile" LLVM ERROR: Broken module found, compilation aborted! error: could not compile `ricochetrobots` (bin "ricochetrobots-debug-precompute") warning: build failed, waiting for other jobs to finish... module flag identifiers must be unique (or of 'require' type) !"CG Profile" LLVM ERROR: Broken module found, compilation aborted! error: could not compile `ricochetrobots` (bin "solve") [naersk] cargo returned with exit code 101, exiting |
I'm using the compiler flag directly (via
The file is confirmed to exist via
My PGO+LTO setup worked with Rust 1.66.1, but doesn't work with 1.71.1. |
This "file does not exist" error seems like a different one from the OP ("LLVM ERROR: Broken module found, compilation aborted!"), and it would be good to have a small self-contained example for your new error -- and possibly a dedicated issue since it also looks different from #63609. It feels possible to investigate in |
I got the same issue during the ruff optimization with PGO with
Note that Ruff also enables LTO as a part of its Release profile. When I switch off LTO, Additional details -
Tested the same scenario on my Linux machine (Fedora 38, x86-64) for the same scenario with the same Rust version and got the same error. |
Another affected project - https://github.com/foundry-rs/foundry Got the same error during
|
|
Indeed. I have a workaround for now ( |
In my case the compilation was happening clean, so incremental doesn't seem like the problem for me. |
Another example of hitting this error with
Also, I've checked cleaning the build directory before the PGO optimization build - didn't help. Dust also enables LTO in its Release profile. |
Uhh, so I was able to reproduce this on Linux (with just |
I need to recheck but AFAIK I remember some Rust projects there LTO + PGO worked fine... :) |
Yeah it's something specific in the project causing it, I'm trying to minimize |
@Kobzol Were you able to reproduce the issue? I have reproduced it once again with the latest Fedora 38 and Rust stable 1.73 and Ruff project on the latest commit |
Another affected project: https://github.com/googleforgames/quilkin . When
OS: Fedora 38 |
Another affected project - https://github.com/ImageOptim/gifski . On Linux with Rust 1.73 I get the same error:
Additional information:
|
Thanks for use clap::Command;
fn main() {
Command::new("a").about("a");
} in combination with this in [lib]
crate-type = ["lib", "cdylib"] and LTO=thin. I'll probably post this as a new issue and try to investigate more. |
Another project - ocrs. After adding
Rustc version: 1.75 |
One more - needletail. After
Rustc version:
needletail version: |
Another one - uv. Error is the same:
Rustc: 1.75 |
Another project - namada. The error is the same:
Rustc version: 1.76 |
Another project - boa. Error:
Rustc version: 1.76 |
Project - pylyzer. After adding the following lines to the
I get the following errors during LTO + PGO optimization build:
Rustc version: 1.77.2 |
I am having the same issue with my image compression program flaca, but have narrowed it down to a single (direct) dependency: mozjpeg-sys. So long as my instrumentation runs avoid triggering those parts of the program, PGO+LTO work just fine. Otherwise:
I don't know if the problem originates with that crate or just the way I'm using it. I tried stripping out all its extra features and even tested an old version without the 🤷 For what it's worth, I'm running each step manually (rather than automating via
|
Project - https://github.com/iggy-rs/iggy The error:
OS: Fedora 40 |
It seems to be happening with sccache Using:
Rust version: 1.81.0-nightly (6b0f4b5 2024-06-24) |
Project - https://github.com/infinyon/fluvio-jolt Using Error:
OS: Fedora 40 |
Project - https://github.com/anacrolix/possum When I enable LTO (add
OS: Fedora 40 |
Do anyone have any idea what and how we may fix this issue, as it seem to affect a lot of projects? @Kobzol? |
Also happens when trying to compile polars with PGO Edit: |
Project - https://github.com/mwlon/pcodec After enabling LTO with
Fedora 40 |
Project - https://git.picodata.io/picodata/picodata/picodata Error:
Fedora 41 |
@DianQK It would be incredible if you managed to fix this |
[WIP] The embedded bitcode should always be prepared for LTO/ThinLTO Fixes rust-lang#115344. Fixes rust-lang#117220. r? ghost
[WIP] The embedded bitcode should always be prepared for LTO/ThinLTO Fixes rust-lang#115344. Fixes rust-lang#117220. r? ghost
A candidate PR: #133250. |
@DianQK you are an amazing human being! 💟 |
In vectordotdev/vector#15631 (comment) I tried to optimize Vector with LTO + PGO enabled. But during the optimization phase (done with
cargo pgo optimize
) I get the following error: Kobzol/cargo-pgo#32 (comment) . I am not the only person with this error - llvm/llvm-project#57501 (comment) (@kevincox probably can provide more details about they project).The only way to avoid the bug is disable LTO completely (switching from Fat to Thin mode does not help).
I expected to see this happen: Vector compiles successfully with LTO and PGO enabled.
Instead, this happened: Vector failed to compile with LTO + PGO.
Meta
rustc --version --verbose
(from https://github.com/vectordotdev/vector/blob/master/rust-toolchain.toml):Other links:
cargo-pgo
: LLVM ERROR: Broken module found, compilation aborted Kobzol/cargo-pgo#32The text was updated successfully, but these errors were encountered: