Cannot link to third-party Rust libraries in the same prefix rustc was installed to

[ghost]

I'm working on a repository for the Paludis package manager, and Rust has trouble with the standard system paths.

<snip for clarity>
root@jurily ~# cave contents dev-rust/sdl
/usr/lib64/libsdl-e351513a-0.3.2.so

root@jurily ~# cave contents dev-lang/rust
/usr/lib64/rustlib/x86_64-unknown-linux-gnu/lib/liblibc-3574b280-0.11-pre.so
/usr/lib64/rustlib/x86_64-unknown-linux-gnu/lib/libstd-aad93cea-0.11-pre.so
/usr/lib64/liblibc-3574b280-0.11-pre.so
/usr/lib64/libstd-aad93cea-0.11-pre.so

With standard invocation:

rustc --opt-level=3 --out-dir /var/tmp/paludis/build/dev-rust-sprocketnes-scm/work/sprocketnes-scm -L /var/tmp/paludis/build/dev-rust-sprocketnes-scm/work/sprocketnes-scm nes.rs
nes.rs:11:1: 11:18 error: can't find crate for `sdl`
nes.rs:11 extern crate sdl;
          ^~~~~~~~~~~~~~~~~

So then I had the wonderful idea to feed it $LDPATH manually:

erustc () { # happy fun times
    OIFS="$IFS"
    IFS=":"
    local LDHACK=""
    local LDTOKENS=""
    read -a LDTOKENS <<< "${LDPATH}"
    for i in "${LDTOKENS[@]}"; do
        LDHACK="$LDHACK -L $i"
    done

    IFS="$OIFS"
    edo rustc --opt-level=3 --out-dir "${WORK}" -L "${WORK}" "${@}" ${LDHACK}
}

---

rustc --opt-level=3 --out-dir /var/tmp/paludis/build/dev-rust-sprocketnes-scm/work/sprocketnes-scm -L /var/tmp/paludis/build/dev-rust-sprocketnes-scm/work/sprocketnes-scm nes.rs -L /usr/local/lib -L /usr/lib64 -L /lib64 -L /usr/lib64/nss -L /usr/lib64/qt4 -L /usr/lib64/mysql -L /usr/lib64/xulrunner
nes.rs:1:1: 1:1 error: multiple dylib candidates for `std` found
nes.rs:1 //
         ^
nes.rs:1:1: 1:1 note: candidate #1: /usr/lib64/rustlib/x86_64-unknown-linux-gnu/lib/libstd-aad93cea-0.11-pre.so
nes.rs:1 //
         ^
nes.rs:1:1: 1:1 note: candidate #2: /usr/lib64/libstd-aad93cea-0.11-pre.so
nes.rs:1 //
         ^
nes.rs:1:1: 1:1 error: multiple dylib candidates for `libc` found
nes.rs:1 //
         ^
nes.rs:1:1: 1:1 note: candidate #1: /usr/lib64/liblibc-3574b280-0.11-pre.so
nes.rs:1 //
         ^
nes.rs:1:1: 1:1 note: candidate #2: /usr/lib64/rustlib/x86_64-unknown-linux-gnu/lib/liblibc-3574b280-0.11-pre.so
nes.rs:1 //

Lose-lose. I assume there are valid reasons for this, but it's still confusing:

1. Why are there multiple shared libraries installed? If rustc needs its own set, wouldn't it make sense to link them statically?
2. Why can't rustc figure out which one to use?
3. Would it actually make a difference? They're the same version, right down to the hash.
4. If I put everything next to the "good" libstd, will it keep working? Can I count on that path to not change?

I could use a completely different directory of course, but that's an ugly workaround and would likely hurt upstream chances.

[thestinger]

1. Why are there multiple shared libraries installed? If rustc needs its own set, wouldn't it make sense to link them statically?

They're built in an earlier stage, so in theory this could break due to either bugs or stage markers. In practice, it's fine to link them and I do it in my Arch Linux package.

https://projects.archlinux.org/svntogit/community.git/tree/trunk?h=packages/rust
https://github.com/thestinger/packages/tree/master/rust-git

2. Why can't rustc figure out which one to use?

Only one of these should be in the library search path. I'm not quite sure what's going wrong, but perhaps it has to do with you adding the rustlib directory to the search path.

[ghost]

but perhaps it has to do with you adding the rustlib directory to the search path.

No, that's the rustc default. I added /usr/lib64. rustc is linked to the ones in /usr/lib64:

jurily@jurily ~> ldd /usr/bin/rustc | grep libstd-
    libstd-aad93cea-0.11-pre.so => /usr/bin/../lib/libstd-aad93cea-0.11-pre.so (0x00007ff49174f000)

jurily@jurily ~> ldd /usr/lib64/libsplay-c8a1a9db-0.0.so | grep libstd-
    libstd-aad93cea-0.11-pre.so => /usr/lib64/../../../../../../../usr/lib64/rustlib/x86_64-unknown-linux-gnu/lib/libstd-aad93cea-0.11-pre.so (0x00007f94d1fad000)

(libsplay was compiled without the extra paths)

[alexcrichton]

This is indeed a bit of a problem with our directory layout. And I've brought it up with @brson before. The gist of it is that:

1. The compiler's runtime libraries are in /usr/lib64.
2. The target libraries (the ones that are linked to) are in /usr/lib64/rustlib/$target/lib

This means the host library (the one in /usr/lib64) was not generated by the compiler living at /usr/bin/rustc, meaning the compiler doesn't link to it by default. When you pass -L /usr/lib64, it's found as a candidate, however.

That's the situation, and one possible solution is to assume convergence and not distribute target artifacts for the host architecture (have it all in /usr/lib64). I think we'd want to think a bit about doing that, however.

[ghost]

make snap-stage3 already produces a static (where it counts) executable. Use that and all problems go away.

I imagine it would also increase sanity in the build system to move to an all-static-ish bootstrap.

[lilyball]

This issue is becoming problematic for me now, because OS X 10.10 no longer searches /usr/local/lib by default for libraries. I can no longer compile rust-lua because it can't find -llua. If I add -L/usr/local/lib then I get the multiple matching dylibs error. I can compile successfully with #[link_args = "-L/usr/local/lib -llua"] but unfortunately these arguments aren't propagated to downstream dependencies of my rlib, so they don't even know to link against -llua (much less where to find it).

How did rustc handle this issue before? Did it simply not search /usr/local/lib for crates, but the system linker would go ahead and search it when linking the final result to find third-party libs?

I think rustc needs to simply prioritize dylibs found inside the target library root over anything else.

[lilyball]

An alternative might be to add a metadata bit to the crate that says "this is a host crate", and use that to skip the crate when looking for target crates.

[ghost]

I still think the solution is to link rustc statically. There are no advantages of dynamic linking for rustc: we're not loading LLVM lazily so startup times are not improved and the libraries must not be linked to by more than one executable. Also, rustc defaults to building rlibs, this is explicitly worked around in the build system up to and including a special compiler flag.

Moving to a prefer-static build would likely do good things to the cross-platform story as well.

For a workaround, FHS says:

Applications may use a single subdirectory under /usr/lib. If an application uses a subdirectory, all architecture-dependent data exclusively used by the application must be placed within that subdirectory.

[lilyball]

@Jurily Syntax extensions need to link against libsyntax, and with the upcoming pluggable lint stuff they may need to link against librustc as well. rustc dynamically loads these things, so dynamic linking is required for both rustc and syntax extensions.

[lilyball]

Maybe we should just use rustlib for both target and host crates from the stdlib? We can use separate directories for host and target libraries.

[lilyball]

FWIW I'm currently testing code to do the prioritize-target-lib-path approach right now.

[ghost]

Hmm. In that case, rustc should be built one more time on top of the set we want to install, but that would make a lot of people unhappy. Can we separate install-fast and install-careful targets?

[lilyball]

@Jurily The compiler is built on top of what we install already. The difference is there's target crates and host crates. We install both, just in separate directories.

[lilyball]

Prioritizing the target lib path makes -L/usr/local/lib work, but unfortunately Rust doesn't embed search paths like that into rlib crate metadata, so rust-lua still has a problem (which is that any client of the liblua rlib has to independently pass the -L/usr/local/lib flag to find -llua).

[ghost]

I'm confused. I thought @thestinger's Arch build was just an ugly hack that happens to work.

Is there a good reason to have a separate target set for host == target?

[lilyball]

This is still a problem, and still prevents me from using rust-lua on OS X 10.10. Is there any plan to resolve this?

[GP4cK]

Hi guys, I don't know if your still working on this. I also had a linkage problem since I updated to 10.10. My programs couldn't find my third parties libraries... I solved it by adding: "set(CMAKE_EXE_LINKER_FLAGS -L/usr/local/lib)" to my CMakeLists.txt
HTH

[lilyball]

I think that the problem with OS X 10.10 is that there appears to be a -syslibroot flag passed to ld by default, which I believe means the default library search path will only contain paths in the SDK and not the actual /usr/local/lib on disk. Rust could fix this by passing -L/usr/local/lib to cc. It also seems I can say env LIBRARY_PATH=/usr/lib:/usr/local/lib cargo test and it will properly find the library inside /usr/local/lib, but of course that's not a solution to the problem.

[lilyball]

@alexcrichton Any chance Rust could simply pass -L/usr/local/lib when linking on OS X?

[alexcrichton]

I'd be somewhat wary of adding an arbitrary path for one OS to a place that may not be wanted. We'd have to add in the default -L/usr/local/lib but we would also have to add some ability to disable the new action as well.

[mvdnes]

Using the new cargo build scripts, I ran into this issue. When using the pkg-config crate, it may pass -L<path/to/rustlib> to cargo, which will then produce the afformentioned errors.

Is there any workaround or fix for this?

[eternaleye]

A note that may be relevant, considering the original motivation for this was an Exheres repository: Exherbo, the distro that uses that package format (package manager is Paludis), has transitioned to a multiarch setup in which each target's libraries live under /usr/$TRIPLE/lib, /usr/host is a symlink to /usr/$HOST, and both /lib and /usr/lib are symlinks to /usr/host/lib - the way -L works (a manner unlike -L in any other language/compiler I've encountered...) is deeply incompatible with such a filesystem layout.

[aidanhs]

Edit: disregard everything below, it's not related to this issue - I'm trying to link to native libraries, which is resolved by -L native=....

---

This is biting me with a musl-enabled build of rust installed to a custom prefix and trying to build an rlib.

Musl itself and musl-rust are installed in $PREFIX, but rustc does not search $PREFIX/lib for libc.a when building an rlib. strace tells me these locations are checked:

/home/aidanhs/Desktop/rust/musldist/lib/rustlib/x86_64-unknown-linux-musl/lib/libc.a
/tmp/tmp.FsbyOoqA0o/rustful/.rust/lib/x86_64-unknown-linux-musl/libc.a
/tmp/tmp.FsbyOoqA0o/rustful/lib/x86_64-unknown-linux-musl/libc.a

As a result, anything trying to do #![crate_type = "rlib"] and #[link(name = "c", kind = "static")] (as rust libc does for musl-rust) says "cannot find native static library c [...]". Adding a -L to rustc fails for the reasons covered in the rest of this issue.

Though in my case I think -L would be a hack and the 'correct' way to do it is either via LIBRARY_PATH or compiling the path in.

[martinraison]

I'm having the same issue. I can't link a C library installed in /usr/local/lib on OSX (because of the "multiple dylib candidates" error when the -L flag is specified).

[Chuongv]

I am not sure if helps, but I had similar issues with linking a C library to usr/local/lib. I the approaches above, even creating a build script but had the same problems. Turns out that my mac didn't have LIBRARY_PATH defined. Once I had it defined as usr/local/lib, the -L was being passed correctly.

To find out if LIBRARY_PATH is defined do:
echo $LIBRARY_PATH. If it's not, run
export LIBRARY_PATH="/usr/local/lib"

[brson]

Assuming this is still a problem. Any updates?

[steveklabnik]

Triage: it's been over a year since the last comment asking for updates; the last real comment here is from 2015. The original reporter has deleted their GitHub account. As such, I'm going to give this a close; if anyone is still seeing this issue, please let me know and I'm happy to re-open.