Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Link error when using native_link_modifiers_bundle and ld.bfd #87541

Closed
12101111 opened this issue Jul 28, 2021 · 1 comment · Fixed by #88227
Closed

Link error when using native_link_modifiers_bundle and ld.bfd #87541

12101111 opened this issue Jul 28, 2021 · 1 comment · Fixed by #88227
Labels
C-bug Category: This is a bug.

Comments

@12101111
Copy link
Contributor

This is found in #85600 (comment)

code: https://github.com/12101111/native_link_modifiers_bundle

export RUSTFLAGS="-Ctarget-feature=+crt-static -Clinker=ld.bfd"
cargo run --target=x86_64-unknown-linux-musl
   Compiling cc v1.0.69
   Compiling foo v0.1.0 (/tmp/musl-link)
error: linking with `ld.bfd` failed: exit status: 1
  |
  = note: "ld.bfd" "/home/han/.rustup/toolchains/nightly-x86_64-unknown-linux-musl/lib/rustlib/x86_64-unknown-linux-musl/lib/self-contained/rcrt1.o" "/home/han/.rustup/toolchains/nightly-x86_64-unknown-linux-musl/lib/rustlib/x86_64-unknown-linux-musl/lib/self-contained/crti.o" "/home/han/.rustup/toolchains/nightly-x86_64-unknown-linux-musl/lib/rustlib/x86_64-unknown-linux-musl/lib/self-contained/crtbeginS.o" "/tmp/musl-link/target/x86_64-unknown-linux-musl/debug/deps/foo-656e6e56d7971844.15pu1hl0kal84f1y.rcgu.o" "/tmp/musl-link/target/x86_64-unknown-linux-musl/debug/deps/foo-656e6e56d7971844.1h68etnkmxrn6yzz.rcgu.o" "/tmp/musl-link/target/x86_64-unknown-linux-musl/debug/deps/foo-656e6e56d7971844.1pe73trdgmny6a8t.rcgu.o" "/tmp/musl-link/target/x86_64-unknown-linux-musl/debug/deps/foo-656e6e56d7971844.1zms9s2k5rtx003u.rcgu.o" "/tmp/musl-link/target/x86_64-unknown-linux-musl/debug/deps/foo-656e6e56d7971844.2re1n6nelkxha2og.rcgu.o" "/tmp/musl-link/target/x86_64-unknown-linux-musl/debug/deps/foo-656e6e56d7971844.36sds9xqyeuceja1.rcgu.o" "/tmp/musl-link/target/x86_64-unknown-linux-musl/debug/deps/foo-656e6e56d7971844.3nlkyw0e8uutnyey.rcgu.o" "/tmp/musl-link/target/x86_64-unknown-linux-musl/debug/deps/foo-656e6e56d7971844.8q3ae5zw7l4m4j0.rcgu.o" "/tmp/musl-link/target/x86_64-unknown-linux-musl/debug/deps/foo-656e6e56d7971844.i3svhsqj32n5bqa.rcgu.o" "/tmp/musl-link/target/x86_64-unknown-linux-musl/debug/deps/foo-656e6e56d7971844.j7ech079p6rmvfx.rcgu.o" "/tmp/musl-link/target/x86_64-unknown-linux-musl/debug/deps/foo-656e6e56d7971844.wktolicz9f1e7px.rcgu.o" "/tmp/musl-link/target/x86_64-unknown-linux-musl/debug/deps/foo-656e6e56d7971844.46l3v0trxis9yeee.rcgu.o" "--as-needed" "-L" "/tmp/musl-link/target/x86_64-unknown-linux-musl/debug/deps" "-L" "/tmp/musl-link/target/debug/deps" "-L" "/tmp/musl-link/target/x86_64-unknown-linux-musl/debug/build/foo-c6c7efd0b76b3015/out" "-L" "/home/han/.rustup/toolchains/nightly-x86_64-unknown-linux-musl/lib/rustlib/x86_64-unknown-linux-musl/lib" "-Bstatic" "/tmp/musl-link/target/x86_64-unknown-linux-musl/debug/deps/libfoo-cb4158256382c543.rlib" "--start-group" "/home/han/.rustup/toolchains/nightly-x86_64-unknown-linux-musl/lib/rustlib/x86_64-unknown-linux-musl/lib/libstd-f1cbe3fb941687fb.rlib" "/home/han/.rustup/toolchains/nightly-x86_64-unknown-linux-musl/lib/rustlib/x86_64-unknown-linux-musl/lib/libpanic_unwind-6774d3dce98216e2.rlib" "/home/han/.rustup/toolchains/nightly-x86_64-unknown-linux-musl/lib/rustlib/x86_64-unknown-linux-musl/lib/libminiz_oxide-2b70a69428b67501.rlib" "/home/han/.rustup/toolchains/nightly-x86_64-unknown-linux-musl/lib/rustlib/x86_64-unknown-linux-musl/lib/libadler-7c0d8eb24f55575d.rlib" "/home/han/.rustup/toolchains/nightly-x86_64-unknown-linux-musl/lib/rustlib/x86_64-unknown-linux-musl/lib/libobject-39efbcf3ca573d24.rlib" "/home/han/.rustup/toolchains/nightly-x86_64-unknown-linux-musl/lib/rustlib/x86_64-unknown-linux-musl/lib/libaddr2line-65df03b5b3a62814.rlib" "/home/han/.rustup/toolchains/nightly-x86_64-unknown-linux-musl/lib/rustlib/x86_64-unknown-linux-musl/lib/libgimli-6c5352065190f68a.rlib" "/home/han/.rustup/toolchains/nightly-x86_64-unknown-linux-musl/lib/rustlib/x86_64-unknown-linux-musl/lib/libstd_detect-7a40648f799302f6.rlib" "/home/han/.rustup/toolchains/nightly-x86_64-unknown-linux-musl/lib/rustlib/x86_64-unknown-linux-musl/lib/librustc_demangle-255ccea4ef40f4d4.rlib" "/home/han/.rustup/toolchains/nightly-x86_64-unknown-linux-musl/lib/rustlib/x86_64-unknown-linux-musl/lib/libhashbrown-e6cbd0b0da1406a9.rlib" "/home/han/.rustup/toolchains/nightly-x86_64-unknown-linux-musl/lib/rustlib/x86_64-unknown-linux-musl/lib/librustc_std_workspace_alloc-6be43c7f9d293971.rlib" "/home/han/.rustup/toolchains/nightly-x86_64-unknown-linux-musl/lib/rustlib/x86_64-unknown-linux-musl/lib/libunwind-a8a6eb096906f234.rlib" "/home/han/.rustup/toolchains/nightly-x86_64-unknown-linux-musl/lib/rustlib/x86_64-unknown-linux-musl/lib/libcfg_if-84f9c9d47d8869d8.rlib" "/home/han/.rustup/toolchains/nightly-x86_64-unknown-linux-musl/lib/rustlib/x86_64-unknown-linux-musl/lib/liblibc-1d56f1e04d9d51b1.rlib" "/home/han/.rustup/toolchains/nightly-x86_64-unknown-linux-musl/lib/rustlib/x86_64-unknown-linux-musl/lib/liballoc-b292e64c32db2ac6.rlib" "/home/han/.rustup/toolchains/nightly-x86_64-unknown-linux-musl/lib/rustlib/x86_64-unknown-linux-musl/lib/librustc_std_workspace_core-d144bec3bbe14c26.rlib" "/home/han/.rustup/toolchains/nightly-x86_64-unknown-linux-musl/lib/rustlib/x86_64-unknown-linux-musl/lib/libcore-58e3a51e774ec0f7.rlib" "--end-group" "/home/han/.rustup/toolchains/nightly-x86_64-unknown-linux-musl/lib/rustlib/x86_64-unknown-linux-musl/lib/libcompiler_builtins-5added8ca6387776.rlib" "-lfoo" "-Bdynamic" "--eh-frame-hdr" "-znoexecstack" "-L" "/home/han/.rustup/toolchains/nightly-x86_64-unknown-linux-musl/lib/rustlib/x86_64-unknown-linux-musl/lib" "-L" "/home/han/.rustup/toolchains/nightly-x86_64-unknown-linux-musl/lib/rustlib/x86_64-unknown-linux-musl/lib/self-contained" "-o" "/tmp/musl-link/target/x86_64-unknown-linux-musl/debug/deps/foo-656e6e56d7971844" "--gc-sections" "-static" "-pie" "--no-dynamic-linker" "-z" "text" "-zrelro" "-znow" "/home/han/.rustup/toolchains/nightly-x86_64-unknown-linux-musl/lib/rustlib/x86_64-unknown-linux-musl/lib/self-contained/crtendS.o" "/home/han/.rustup/toolchains/nightly-x86_64-unknown-linux-musl/lib/rustlib/x86_64-unknown-linux-musl/lib/self-contained/crtn.o"
  = note: ld.bfd: /tmp/musl-link/target/x86_64-unknown-linux-musl/debug/build/foo-c6c7efd0b76b3015/out/libfoo.a(foo.o): in function `bar':
          /tmp/musl-link/foo.c:3: undefined reference to `printf'
          
  = help: some `extern` functions couldn't be found; some native libraries may need to be installed or have their path specified
  = note: use the `-l` flag to specify native libraries to link
  = note: use the `cargo:rustc-link-lib` directive to specify the native libraries to link with Cargo (see https://doc.rust-lang.org/cargo/reference/build-scripts.html#cargorustc-link-libkindname)

error: could not compile `foo` due to previous error

This is the part of link command line:

"/tmp/musl-link/target/x86_64-unknown-linux-musl/debug/deps/libfoo-cb4158256382c543.rlib" 
"--start-group" 
"/home/han/.rustup/toolchains/nightly-x86_64-unknown-linux-musl/lib/rustlib/x86_64-unknown-linux-musl/lib/libstd-f1cbe3fb941687fb.rlib" 
"/home/han/.rustup/toolchains/nightly-x86_64-unknown-linux-musl/lib/rustlib/x86_64-unknown-linux-musl/lib/libpanic_unwind-6774d3dce98216e2.rlib" 
"/home/han/.rustup/toolchains/nightly-x86_64-unknown-linux-musl/lib/rustlib/x86_64-unknown-linux-musl/lib/libminiz_oxide-2b70a69428b67501.rlib" 
"/home/han/.rustup/toolchains/nightly-x86_64-unknown-linux-musl/lib/rustlib/x86_64-unknown-linux-musl/lib/libadler-7c0d8eb24f55575d.rlib"
"/home/han/.rustup/toolchains/nightly-x86_64-unknown-linux-musl/lib/rustlib/x86_64-unknown-linux-musl/lib/libobject-39efbcf3ca573d24.rlib"
"/home/han/.rustup/toolchains/nightly-x86_64-unknown-linux-musl/lib/rustlib/x86_64-unknown-linux-musl/lib/libaddr2line-65df03b5b3a62814.rlib"
"/home/han/.rustup/toolchains/nightly-x86_64-unknown-linux-musl/lib/rustlib/x86_64-unknown-linux-musl/lib/libgimli-6c5352065190f68a.rlib"
"/home/han/.rustup/toolchains/nightly-x86_64-unknown-linux-musl/lib/rustlib/x86_64-unknown-linux-musl/lib/libstd_detect-7a40648f799302f6.rlib"
"/home/han/.rustup/toolchains/nightly-x86_64-unknown-linux-musl/lib/rustlib/x86_64-unknown-linux-musl/lib/librustc_demangle-255ccea4ef40f4d4.rlib"
"/home/han/.rustup/toolchains/nightly-x86_64-unknown-linux-musl/lib/rustlib/x86_64-unknown-linux-musl/lib/libhashbrown-e6cbd0b0da1406a9.rlib"
"/home/han/.rustup/toolchains/nightly-x86_64-unknown-linux-musl/lib/rustlib/x86_64-unknown-linux-musl/lib/librustc_std_workspace_alloc-6be43c7f9d293971.rlib"
"/home/han/.rustup/toolchains/nightly-x86_64-unknown-linux-musl/lib/rustlib/x86_64-unknown-linux-musl/lib/libunwind-a8a6eb096906f234.rlib"
"/home/han/.rustup/toolchains/nightly-x86_64-unknown-linux-musl/lib/rustlib/x86_64-unknown-linux-musl/lib/libcfg_if-84f9c9d47d8869d8.rlib"
"/home/han/.rustup/toolchains/nightly-x86_64-unknown-linux-musl/lib/rustlib/x86_64-unknown-linux-musl/lib/liblibc-1d56f1e04d9d51b1.rlib"
"/home/han/.rustup/toolchains/nightly-x86_64-unknown-linux-musl/lib/rustlib/x86_64-unknown-linux-musl/lib/liballoc-b292e64c32db2ac6.rlib"
"/home/han/.rustup/toolchains/nightly-x86_64-unknown-linux-musl/lib/rustlib/x86_64-unknown-linux-musl/lib/librustc_std_workspace_core-d144bec3bbe14c26.rlib"
"/home/han/.rustup/toolchains/nightly-x86_64-unknown-linux-musl/lib/rustlib/x86_64-unknown-linux-musl/lib/libcore-58e3a51e774ec0f7.rlib"
"--end-group"
"/home/han/.rustup/toolchains/nightly-x86_64-unknown-linux-musl/lib/rustlib/x86_64-unknown-linux-musl/lib/libcompiler_builtins-5added8ca6387776.rlib" 
"-lfoo"

libfoo.a need symbols from libc, but libc is include in liblibc-*.rlib, which appear in the command line before libfoo.a. ld.lld is "smart" enough to find those symbols from liblibc-*.rlib, but ld.bfd use the traditional unix linker behavior.

The logic of assemble linker command line is at

// FIXME: Move this below to other native libraries
// (or alternatively link all native libraries after their respective crates).
// This change is somewhat breaking in practice due to local static libraries being linked
// as whole-archive (#85144), so removing whole-archive may be a pre-requisite.
if sess.opts.debugging_opts.link_native_libraries {
add_local_native_libraries(cmd, sess, codegen_results);
}
// Rust libraries.
add_upstream_rust_crates::<B>(cmd, sess, codegen_results, crate_type, tmpdir);
// Native libraries linked with `#[link]` attributes at and `-l` command line options.
// If -Zlink-native-libraries=false is set, then the assumption is that an
// external build system already has the native dependencies defined, and it
// will provide them to the linker itself.
if sess.opts.debugging_opts.link_native_libraries {
add_upstream_native_libraries(cmd, sess, codegen_results, crate_type);
}

Currently libfoo.a is added to linker command line by add_upstream_native_libraries

Possible solution:

  1. Add native libraries with kindNativeLibKind::Static { bundle: Some(false), .. } from upstream rlib crate to linker command line right after that crate is added in add_upstream_rust_crates.
  2. Move all static native libraries and rlib into one link group
  3. (Maybe just a workaround): also link libc with kind = "static", modifiers = "-bundle": Use link modifiers -bundle on musl and wasi target libc#2272

Meta

rustc --version --verbose:

rustc 1.56.0-nightly (2faabf579 2021-07-27)
binary: rustc
commit-hash: 2faabf579323f5252329264cc53ba9ff803429a3
commit-date: 2021-07-27
host: x86_64-unknown-linux-musl
release: 1.56.0-nightly
LLVM version: 12.0.1
@12101111 12101111 added the C-bug Category: This is a bug. label Jul 28, 2021
@petrochenkov
Copy link
Contributor

cc #85805

I don't think this is a bug.
libunwind does not indicate in any way that it depends on libc, so rustc builds the linker invocation without knowing about this dependency, thus we have linking errors.

We should support specifying all dependencies explicitly (similarly to how Rust crates form a dependency tree with Cargo) instead of making everything automatically depend on everything by using mechanisms like groups (which lld does by default).

Dylan-DPC-zz pushed a commit to Dylan-DPC-zz/rust that referenced this issue Aug 26, 2021
…rochenkov

Adjust linking order of static nobundle libraries

Link the static libraries with "-bundle" modifier from upstream rust crate right after linking this rust crate.
Some linker such as GNU linker `ld.bdf` treat order of linking as order of dependency.

After this change, static libraries with "-bundle" modifier is linked in the same order as "+bundle" modifier.
So we can change the value of "bundle" modifier without causing linking error.

fix: rust-lang#87541

r? `@petrochenkov`
Manishearth added a commit to Manishearth/rust that referenced this issue Aug 27, 2021
…rochenkov

Adjust linking order of static nobundle libraries

Link the static libraries with "-bundle" modifier from upstream rust crate right after linking this rust crate.
Some linker such as GNU linker `ld.bdf` treat order of linking as order of dependency.

After this change, static libraries with "-bundle" modifier is linked in the same order as "+bundle" modifier.
So we can change the value of "bundle" modifier without causing linking error.

fix: rust-lang#87541

r? ``@petrochenkov``
@bors bors closed this as completed in 4a6547c Aug 27, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
C-bug Category: This is a bug.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants