Support feeding rust-lld into gcc (and clang)

[Gankra]

This is a subtask of #71515

summary of discussion

On linux/unix platforms, you aren't supposed to directly invoke ld/lld. You're supposed to invoke the linker through your system c compiler (i.e. gcc), whose responsibility it is to discover system symbols like crt1.o and provide them to ld. This means we can't "just" use rust-lld; we must feed it into gcc/clang/whatever. (We really don't want to implement this system symbol logic ourselves.)

In general, you can't provide the linker as a path, you must inject it into the C compiler's search path as "ld". ("ld.lld" with -fuse-ld=lld would also be possible, but requires newer versions of GCC, so plain "ld" is preferable if possible.)

Solution: add a new -C linker-flavor mode, gcc-lld, that would place rust-lld on the PATH as "ld" and invoke it through the system cc.

Note that distros do not build rust's private rust-lld binary. They make their rust packages depend on lld and symlink it in. So if you have installed rustc via your package manager, using rust-lld is using your system lld. This is all to say that it's generally unnecessary to "check" for a system lld, as rust-lld will already be pointing at it for the folks who care about using system toolchains.

Should this mode become the default on linux, anyone slipping through the cracks can similarly replace rust-lld with a symlink, or explicitly pass a linker-flavor.

Note: the private rust-lld binary can be found at:

$(rustc --print sysroot)/lib/rustlib/$HOST_TARGET_TRIPLE/bin/rust-lld

which for desktop linux would be:

$(rustc --print sysroot)/lib/rustlib/x86_64-unknown-linux-gnu/rust-lld

original comment:

On linux/unix platforms, you aren't supposed to directly invoke ld/lld. You're supposed to invoke the linker through your system c compiler (i.e. gcc), whose responsibility it is to discover system symbols like crt1.o and provide them to ld. This means we can't "just" use rust-lld; we must feed it into gcc/clang/whatever. (We really don't want to implement this system symbol logic ourselves.)

In general, you can't provide the linker as a path, you must inject it into the C compiler's search path as "ld". Alternatively, you can do the same thing but inject it as "ld.lld", and pass "-fuse-ld=lld" to gcc, which may be important as apparently lld does clang-style binary name detection to select different behaviour based on whether it's "ld" or "ld.lld". (needs investigation)

Unfortunately -fuse-ld=lld is only part of GCC 9, so we may require feature/version detection to use it (clang has had it for a long time). Doing this detection may be fairly expensive (execing gcc, possibly multiple times), so we may not want to do this in rustc itself. Ideally something higher up like cargo or maybe even rustup or something would do this analysis and cache the result. (needs design discussion)

It's also not clear to me how one would manually opt into this compilation mode, and/or force it on. Should it be controlled by -C linker-flavor? (needs design discussion)

[petrochenkov]

Ideally something higher up like cargo or maybe even rustup or something would do this analysis and cache the result. (needs design discussion)

CMake (or autotools, etc) do that in C++ world.

It would be nice to have this for other things, like detection of -static-pie or -no-pie, which are currently done in rustc via interpreting error output from the linker (!) and rerunning it.

(Then it can be fed to rustc with --env-capability x=y or something like that, or perhaps environment variables.)

[Gankra]

FWIW, firefox's build system does exactly this analysis in its configure step. If we're lucky we'll be able to copy the logic they use for detecting linkers.

(Except we would be injecting our own ld and just validating that it worked, and presumably we won't bother with checking for the presence of Gold)

[Gankra]

Ok did a bit of analysis on the binary name stuff, and it's pretty clearly laid out in the main entry point for the binary.

lld is basically 4 independent implementations of the targets it supports, which it calls Drivers: gnu (ELF), darwin (Mach-O), winlink (COFF), wasm. Binary name is used to select the Driver, and it can be overloaded with the -flavor flag (for whatever reason the binary name approach is "preferred" by the docs).

The file name "ld" is special in that it's behaviour depends on the host: on macos it will pick the darwin driver, while on all other platforms it will pick the gnu driver. So for non-cross builds it will "do the right thing" on unix-like platforms (where you would expect a thing called "ld" to exist) (but also using lld for targetting macos is currently Always Wrong regardless).

So this is all to say that the most portable option -- putting rust-lld on the PATH with the name "ld" during our linking -- should work perfectly fine. At least for the common case of targetting your host platform.

For now I'll be ignoring the distro-friendly usecase of detecting installed lld's and selecting that version, for the purpose of getting an MVP working so we can ask people to test out how well this configuration works on their projects.

TODO: figure out whether the "rule" that you must invoke ld through your cc holds for cross-compilation (and for both hosting on and targetting on linux (what would that even mean for windows => linux??)).

TODO: figure out what distro maintainers currently do about the rustc-private rust-lld binary (in rustup installs it's not on the PATH, and can be found at $HOME/.rustup/toolchains/$TOOLCHAIN/lib/rustlib/$HOST-TRIPLE/bin/rust-lld).

TODO: is PATH the right mechanism for this? Should/can we use something else to make rust-lld shadow the system ld? Should we keep that shadowing enabled for the whole build (so e.g. build.rs scripts which go off and build some C/C++ code will also use lld?)

PROPOSAL: add two new modes to -C linker-flavor:

• gcc-lld: places rust-lld on the PATH as "ld" and invokes it through the system cc
• gcc-lld-system: (this is future work that is out of scope for this bug, but the existence of this distinction is worth discussion now) invokes the system cc with -fuse-linker=lld, telling it to use the native lld install (my opinion: should not do detection, will just fail if gcc is too old to understand the flag, or if ld.lld isn't on the PATH)

BIKESHED: gcc-lld deviates from the lld-link naming scheme, but I believe in that case we do actually directly invoke lld with the link driver. Whereas the configuration we are concerned with is invoke (g)cc and tell it to use lld (with the gnu driver).

Automatically enabling these modes is out of scope for this bug, for now they will never be selected without manual intervention, although the ultimate goal is to automatically use these modes in some cases.

[Gankra]

@cuviper does the proposed distinction and mechanism for "use rust-lld" and "use the system lld" make sense to you? Would love any insights you have here, so that I can be confident in this approach.

[mati865]

TODO: figure out what distro maintainers currently do about the rustc-private rust-lld binary

At least Alpine, Arch, Fedora, OpenSUSE and Solus don't build it.
Debian/Ubuntu provides it as /usr/bin/rust-ldd for the repo Rust package.

[Gankra]

Oh!!

Is that just a symlink to a system install of ld.lld, implying package managers consider lld a dependency of rustc? And therefore if you're using a version of cargo/rust installed via your package manager, "using" rust-lld is using the system lld?

If so, we may not need to care about making a distinction between the system and bundled lld, since the way you installed rust basically defines your preference..?

[mati865]

Is that just a symlink to a system install of ld.lld, implying package managers consider lld a dependency of rustc? And therefore if you're using a version of cargo/rust installed via your package manager, "using" rust-lld is using the system lld?

Indeed

$ file /usr/bin/rust-lld
/usr/bin/rust-lld: symbolic link to lld-9

If so, we may not need to care about making a distinction between the system and bundled lld, since the way you installed rust basically defines your preference..?

As long as system LLD is recent enough it doesn't differ much from rust-lld.

[Hello71]

I think this -C linker-flavor=gcc-lld-system is unnecessary complication if it is exactly identical to -C linker-flavor=gcc -C link-args=-fuse-ld=lld. I'm kind of waffling on whether injecting ld is a good idea though. I think it might cause confusion in some cases, such as if the user has set -C link-arg=-fuse-ld manually. And along those lines, what do we do with -C linker-flavor=gcc-lld -C link-arg=-fuse-ld=not-lld?

[cuviper]

@cuviper does the proposed distinction and mechanism for "use rust-lld" and "use the system lld" make sense to you? Would love any insights you have here, so that I can be confident in this approach.

I feel if anything the neutral option should mean the system lld, and rust-lld should be a more specific choice. But if we're willing to probe around a little, I'd also be happy with a single option that uses rust-lld when present, or else uses the system lld. When we head further toward using lld by default, that order would go rust-lld, system lld, or just default ld.

[luser]

TODO: is PATH the right mechanism for this? Should/can we use something else to make rust-lld shadow the system ld?

GCC (and clang) support a -Bprefix option which lets you set the search path for compiler binaries.

We used to use that for cross-mac Firefox builds:
https://hg.mozilla.org/mozilla-central/file/eb20068ba6ea62c0dd38aba6e3e7b5e2d0bcd78c/build/macosx/cross-mozconfig.common#l24

Should we keep that shadowing enabled for the whole build (so e.g. build.rs scripts which go off and build some C/C++ code will also use lld?)

This is a much harder problem to solve and I think it's fine to declare it out of scope at first. Linking Rust code is a much more tightly scoped problem since we control the linker invocation from rustc. Also if your build requires a C compiler so a build script can compile something odds are you already have a linker that works for your target anyway. Making pure-Rust builds less dependent on external toolchains is a very good and useful goal without solving arbitrary C compilation.

[mati865]

-B would work for shadowing system linker but I think Rust would have to shadow all linker names or pass -fuse-ld=bfd since the compiler could be configured to default to different linkers.

I'd like to push this issue forward, is it waiting on something specific?

[Hello71]

I took a look at it a while ago (April) and I don't see any technical blockers. I think we just need to agree on the exact method of selecting lld, and what happens in conflicting cases. As I said earlier, it's not at all clear to me that the correct behavior when both lld and gold are specified (for example) is to silently use lld instead.

[joshtriplett]

I'm really interested in seeing this happen as well.

The proposed -C linker-flavor=gcc-lld makes sense to me.

[Gankra]

This is the last known blocker for using lld on linux, so I've updated the first comment with a summary of the discussion for anyone who would like to take a crack at implementing it.

[g2p]

I'm starting work on this.

Besides adding a linker flavor and relevant documentation, I am customizing bootstrap/dist.rs to prepare a directory ending in ld@rust-lld that contains a single file symlinking ld to ../…/rust-lld. That should make it easy to customize (if distros already customize rust-lld to symlink to the system lld, they don't need to do anything else).

[g2p]

About target specs:
They tend to customize flags passed to the linker (pre_link_args in most cases), so that at least their default linker flavor works out of the box.
There is one test (compiler/rustc_target/src/spec/tests/tests_impl.rs) that enforces that such customizations are shared between the msvc and lld-link flavors.
Enforcing similar sharing for gcc and gcc-lld would require rewriting about 65 targets.
Either I rework the way LinkArgs customization is done to make it easier to share / default to share / enforce sharing, or I use a flag that is not -C linker-flavor (but might require the gcc flavor).

[bstrie]

Either I rework the way LinkArgs customization is done to make it easier to share / default to share / enforce sharing, or I use a flag that is not -C linker-flavor (but might require the gcc flavor).

@g2p do you have any thoughts on which would be the better approach? If not then we can ask for guidance from the compiler team.

[sledgehammervampire]

Is this issue being worked on? I'm new to the "guts" of Rust, and this seems like something useful to work on, but I'll need someone who can answer my questions about this stuff.

[bstrie]

@1000teslas I would ask the compiler team directly in one of the following Zulip channels:

• #compiler
• #compiler/help

[sledgehammervampire]

@bstrie I've already done so and I'm blocked on this https://rust-lang.zulipchat.com/#narrow/stream/182449-t-compiler.2Fhelp/topic/building.20lld.20fails

[sledgehammervampire]

@bstrie I made a PR. Is there anyone in particular you think should review my PR?
Edit: Never mind, bjorn3 reviewed my PR.