Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

LTO ignored for all crate types when building multiple crate types and one doesn't support it #51009

Open
fitzgen opened this issue May 23, 2018 · 7 comments
Labels
A-LTO Area: Link-time optimization (LTO) C-bug Category: This is a bug. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue.

Comments

@fitzgen
Copy link
Member

fitzgen commented May 23, 2018

https://github.com/rustwasm/rust_wasm_template has

[lib]
crate-type = [
  # Build a cdylib to make a `.wasm` library.
  "cdylib",
  # Build an rlib for testing and benching.
  "rlib"
]

[profile.release]
lto = true

The lto = true is ignored for everything because rlibs don't support LTO. I would expect that the cdylib has LTO and that only the rlib ignores the LTO options.

@alexcrichton tells me this is a rust compiler bug, not a cargo bug.

FWIW, this command does not work either:

cargo rustc -- -C lto=fat --crate-type cdylib
@Mark-Simulacrum Mark-Simulacrum added T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. C-bug Category: This is a bug. labels May 24, 2018
@alexcrichton
Copy link
Member

The error originates from this location when we're executing LTO

The general idea is that if you emit both a cdylib and and rlib then rustc should "do the right thing" in terms of object files. This means that the rlib is created with object files that weren't LTO'd together. Right now the backend is structured in such a way that it only sends one set of object files to the linking phase, but a change like this would require some refactorings to send a set of object files per crate type. That way the rlib crate type would receive pre-LTO object files and the cdylib would receive the post-LTO object files. (afterwards everything would get cleaned up)

bltavares added a commit to bltavares/crossgen that referenced this issue Nov 18, 2018
s command `cargo` subcommand is exclusively intended to be used to help
with [working arround](rust-lang/rust#51009)
how crate types are defined, in order to help with cross-platform
builds.

It is currently [not possible](rust-lang/cargo#6160) to define a
single `crate-type` override on `cargo build`, which causes libs
intended to be used on other languages to compile more than one type of
crate.

This commit adds a dependency on this [new CLI app](https://github.com/bltavares/cargo-crate-type) to be able to workaround the limitations of Cargo.

We are now able to enable the `-C lto` optimization again, as we will
only compile a single type of artifact

Signed-off-by: Bruno Tavares <[email protected]>
yoshuawuyts pushed a commit to yoshuawuyts/crossgen that referenced this issue Dec 18, 2018
* Improve lib template on Travis

This commit contains changes on the generated lib template to build
cross-platform libs. With this changes, I was able to setup a [Travis]
build with GitHub [Releases] of a cross-platform lib.

It is not possible yet to build a different crate type only using
command line args. It requires a modification on `Cargo.toml` to include
new types of library outputs. I've already opened an issue on [cargo](rust-lang/cargo#6160) to see what should be the case here.

Meanwhile, lib authors must change `Cargo.toml` and include the extra
`crate-type` attribute with all the libs. Sadly, this also means that
`LTO` optimisation is not available if the lib uses `lib` or `rlib`
attributes.

To avoid modifying the `Cargo.toml`, the sugestion would be that on a
future PR we could add to the deploy script a modification on the
`Cargo.toml` manifest to include the corresponding crate-type, and only
such crate type for that target. This means we would be able to bring
LTO and output smaller libs.

Realated to:
- #11

[Travis]: https://travis-ci.org/bltavares/rust-over-jna-example/builds/439439854
[Releases]: https://github.com/bltavares/rust-over-jna-example/releases/tag/initial

Signed-off-by: Bruno Tavares <[email protected]>

* Define the type of artifact to be produced by cargo.

s command `cargo` subcommand is exclusively intended to be used to help
with [working arround](rust-lang/rust#51009)
how crate types are defined, in order to help with cross-platform
builds.

It is currently [not possible](rust-lang/cargo#6160) to define a
single `crate-type` override on `cargo build`, which causes libs
intended to be used on other languages to compile more than one type of
crate.

This commit adds a dependency on this [new CLI app](https://github.com/bltavares/cargo-crate-type) to be able to workaround the limitations of Cargo.

We are now able to enable the `-C lto` optimization again, as we will
only compile a single type of artifact

Signed-off-by: Bruno Tavares <[email protected]>
@thefallentree
Copy link

do we have an fix for this issue?

@thefallentree
Copy link

A possible work around is to keep the original library as rlib, create an wrapper library that outputs staticlib, and have your lto required targets link to that.

@matklad
Copy link
Member

matklad commented Jul 12, 2021

Minimized example:

$ cat main.rs 
pub extern "C" fn f() -> i32 { 92 }
$ rustc -C lto=fat --crate-type rlib main.rs
$ rustc -C lto=fat --crate-type cdylib main.rs
$ rustc -C lto=fat --crate-type rlib,cdylib main.rs
error: lto can only be run for executables, cdylibs and static library outputs

error: aborting due to previous error
$ rustc --version
rustc 1.54.0-beta.1 (bf62f4de3 2021-06-23)

Both crate types work in isolation, but break together.

bors bot pushed a commit to xaynetwork/xayn_discovery_engine that referenced this issue Feb 15, 2022
Ticket:

- [TY-2366]


**Update:** lto error was addressed in #140 

---

can't use `lto` because of:
rust-lang/rust#51009

running:
`RUSTFLAGS="-Ccodegen-units=1 -Clto=on -Cembed-bitcode=yes" just compile-ios-local`

gives the error:
```
error: lto can only be run for executables, cdylibs and static library outputs

error: could not compile `hyper` due to previous error
```

but `-Ccodegen-units=1` alone still decreases the size from 221mb to 144mb

[TY-2366]: https://xainag.atlassian.net/browse/TY-2366?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ
morrisonlevi added a commit to DataDog/libdatadog that referenced this issue Aug 19, 2022
Due to a [rust bug][1], LTO was not being applied, and we need the lib
crate type in order to run tests and clippy lints. For now, keep these
as separate crates.

  [1]: rust-lang/rust#51009
@jasonsewall
Copy link

I've hit this issue myself and I would like to know if there is any inclination to address this or not.

@Enselic Enselic added the A-LTO Area: Link-time optimization (LTO) label May 29, 2024
@danielsn
Copy link

Same, we're hitting this issue, would be interested in knowing if a fix is planned.

@yurivish
Copy link

yurivish commented Aug 8, 2024

I also encountered this issue.

A workaround is to use cargo rustc instead of cargo build for builds, passing an explicit crate-type to override the ["rlib", "cdylib"] options in Cargo.toml:

cargo rustc --crate-type cdylib --target wasm32-unknown-unknown

There might be some unanticipated downsides to this approach but it seems to work and builds the .wasm output without ignoring the LTO setting.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-LTO Area: Link-time optimization (LTO) C-bug Category: This is a bug. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue.
Projects
None yet
Development

No branches or pull requests

9 participants