Huge performance gap between lto="fat",cgu=1 and default release profile

[Robbepop]

I was told on Rust's Discord that I should maybe post this issue here together with my research so far which can be found following this GitHub issue link. The GitHub issue links to other sources of information such as the GitHub Gist that accumulates a lot of benchmark results.

The tl;dr is that we see slowdowns of up to x4.21 with the default release profile compared to using

[profile.release]
lto = "fat"
codegen-units = 1

Which is a lot more than the 10-15% that I usually see between both profiles.

People told me that maybe this is due to some edge cases within the optimization pipeline. I personally ran out of ideas since whatever I do I usually see slowdowns of at least 75-100% which is unacceptable since this is a very performance critical library. The major problem with the custom profile using lto="fat",cgu=1 is that it compiled massively slow and probably won't be used by all library users by default which will lead to very poor performance in the default case for those users. The only thing I can do rn is to spam potential users with warnings about using the correct profile settings.

---

I also asked this in Rust's Zulip but didn't gain a lot of really useful information out of it unfortunately.