-
Notifications
You must be signed in to change notification settings - Fork 2.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add a way to set arbitrary rpaths. #5077
Comments
Does |
Hi Alex. I'm looking at the Even if there were, it would be useful if we could control this stuff in |
Any thoughts on this Alex? |
Ah sorry, but Currently rustc itself basically doesn't have great support for this unfortunately, the story around passing linker flags and such hasn't been fully fleshed out there. |
Thanks Alex. I'm aware of Indeed, linkage options are a little limited and somewhat inconsistent in Rust. I notice that |
Just to elaborate on this, I think software authors should be able to specify their own link flags in |
I've been thinking about this again. I think the best thing we could do is allow This should be inheritable, like
or similar in |
Being able to set a custom rpath is also important when bundling a binary as a Mac OS X app with embedded frameworks -- in this case, you need to encode an rpath of e.g. |
I'm having the exact same issue. In order to get this to work I have to do this in a
Problem is that I either want:
But none of these works for me. In my case I would be fine with edit: Now that I think about it. Maybe |
This change makes it such that during bindgen, we will look for the library in build/. If the user chooses the move (or copy) it to the system default search paths then we will pick it up during runtime. If the user does _not_ move the library, (who would?) we must specify the rpath during linking phase. There are various ways to do this, either by `RUSTFLAGS=.. cargo build` or by creating a `./cargo/config` and specify per target the desired flags. Within the `config` file however, we can not anticipate the absolute path see: rust-lang/cargo#5077 Instead, Mayastor now has a Makefile that does this for people, so whoever wants to give it a swirl can do so without having to install "stuff" outside of the repo. Lastly, as people might want to use Ubuntu 18.04, the default in the Makefile is to not include ISA-L and crypto as those require a NASM version not available in 18.04 Signed-off-by: Jeffry Molanus <[email protected]>
This change makes it such that during bindgen, we will look for the library in build/. If the user chooses the move (or copy) it to the system default search paths then we will pick it up during runtime. If the user does _not_ move the library, (who would?) we must specify the rpath during linking phase. There are various ways to do this, either by `RUSTFLAGS=.. cargo build` or by creating a `./cargo/config` and specify per target the desired flags. Within the `config` file however, we can not anticipate the absolute path see: rust-lang/cargo#5077 Instead, Mayastor now has a Makefile that does this for people, so whoever wants to give it a swirl can do so without having to install "stuff" outside of the repo. Lastly, as people might want to use Ubuntu 18.04, the default in the Makefile is to not include ISA-L and crypto as those require a NASM version not available in 18.04 Signed-off-by: Jeffry Molanus <[email protected]>
This change makes it such that during bindgen, we will look for the library in build/. If the user chooses the move (or copy) it to the system default search paths then we will pick it up during runtime. If the user does _not_ move the library, (who would?) we must specify the rpath during linking phase. There are various ways to do this, either by `RUSTFLAGS=.. cargo build` or by creating a `./cargo/config` and specify per target the desired flags. Within the `config` file however, we can not anticipate the absolute path see: rust-lang/cargo#5077 Instead, Mayastor now has a Makefile that does this for people, so whoever wants to give it a swirl can do so without having to install "stuff" outside of the repo. Lastly, as people might want to use Ubuntu 18.04, the default in the Makefile is to not include ISA-L and crypto as those require a NASM version not available in 18.04 Signed-off-by: Jeffry Molanus <[email protected]>
This change makes it such that during bindgen, we will look for the library in build/. If the user chooses the move (or copy) it to the system default search paths then we will pick it up during runtime. If the user does _not_ move the library, (who would?) we must specify the rpath during linking phase. There are various ways to do this, either by `RUSTFLAGS=.. cargo build` or by creating a `./cargo/config` and specify per target the desired flags. Within the `config` file however, we can not anticipate the absolute path see: rust-lang/cargo#5077 Instead, Mayastor now has a Makefile that does this for people, so whoever wants to give it a swirl can do so without having to install "stuff" outside of the repo. Lastly, as people might want to use Ubuntu 18.04, the default in the Makefile is to not include ISA-L and crypto as those require a NASM version not available in 18.04 Signed-off-by: Jeffry Molanus <[email protected]>
I'm willing to work on this if we can find an inoffesive way for the user to pass down the flags. What did people think of:
In build.rs? |
@vext01 this is exactly what we're looking for as well (customizing the |
CMake could set the rpath of an executable to any location defined by a variable and reset the rpath of an executable to any other locations when running Hope cargo could implement something similar, or at least support using some variables to set the content of link-arg |
Is this issue working now? I do not want to set |
Hitting this problem as well. For us it's pretty common to build dynamic libraries (it is often the case with Linux-like-embedded, openwrt, buildroot etc) and link other binaries to them. I also thought of copying the library into the path where tests, Rust binaries are, but there's no way of finding out all paths. There's a better way of doing this currently? Thanks! |
Ditto. I'm thinking setting the rpath to $ORIGIN would help, if your (on windows, you can sort of get around it by copying the dll to the crate root, as long as you're issuing your |
I've finally found a way to add an rpath in a build script. If you invoke nightly cargo with Using that you can do stuff like:
This seems to work, and you could load I suspect the reason we've all missed this is because What's slightly annoying is that you have to remember to manually type The best I've come up with so far is a cargo wrapper like this: #!/bin/sh
cargo -Z extra-link-arg $* Then run I also looked at making cargo's I see there is another issue relating to arbitrary flags already. |
Some of the "don't want to set // build.rs
// make cargo run and cargo test work without setting LD_LIBRARY_PATH
println!("cargo:rustc-env=LD_LIBRARY_PATH={}", p.display()); Not sure why this works or was ever intended to work, but it does make things bearable for me. Regardless, I would like to see this feature happen as well because it might make my use case a little easier. My use case:
Notes:
|
That'll work if you run your program via cargo, but not if you want your binary to be stand-alone.
Agreed, and this is particularly acute for embedded applications, where folks often have to do linker gymnastics. |
I need this quite often. Yes, |
@vext01 I'm trying to do something similar. I've found that this works for example if you build a binary directly out of a |
Relatedly, if my last comment is correct, then I think that's a reason not to do this with generic linker flags, but to first-class the idea of rpaths instead. Say you have a crate A that depends on |
I'm having the exact same problem. A build script for OpenCV uses libclang, which on macOS lives where ever you installed Xcode. So the chain of opencv build.rs -> clang -> clang-sys doesn't work. clang-sys finds libclang.dylib just fine but has no way of propagating rpath back up to clang and OpenCV's build.rs. As a library maintainer, the idea that your library's environment variables requirement are propagated to all the library users sucks. I would expect that setting rpath on the non-dylib targets causes it to be picked up by the target, and if the target is not a dylib/executable I'd expect it to go up another level until it's finds a dylib/executable the exact same way link libs do. I don't see a way of mimicking that with any flag wizardry—cargo has to be aware of rpaths for its targets and propagate accordingly. I'd love to take a stab at this. I didn't understand from the Contributor's Guide what's the procedure for approving feature requests? |
Is there a technical reason why this hasn't been addressed yet by the Cargo team? It's been 4 years now, |
The unstable However, looking into how C/C++ build systems work, I don't think Cargo build scripts should be setting rpaths. There are several distinct use cases for setting rpaths:
I don't think use cases 1 & 2 are relevant for Rust because Rust dylibs aren't very useful without a stable ABI and Rust crates can't link to cdylib crates like normal Rust crates. If a Rust crate is built as a cdylib, that's probably the only artifact that's being distributed. Please correct me if I'm misunderstanding, but I think most people commenting here are interested in use case 3, as I am.
This seemed to me like a good idea at first, but the more I think about it, I don't think Cargo build scripts setting arbitrary rpaths would be a great solution. Cargo already has the information it needs to set rpaths; it was given via There are two problems with this:
Currently it seems the only practical way to use C/C++ libraries with Rust is to link statically, rely on Linux system package managers, or only link DLLs (Windows)/frameworks (macOS) which are parts of the OS that are guaranteed to be there. To change this, I propose:
Unfortunately Windows does not have rpaths or anything directly analogous, so setting PATH is the best that could be done unless a feature is added to copy linked DLLs into the build directory and do this recursively for all those DLLs' dependencies. |
Great response by @Be-ing - I wholeheartedly agree with the problems highlighted, and the solution proposal given. To expand on one point:
Indeed, and I would much rather recommend the latter of the two options, since the typical use-case/design-pattern for dynamic libraries on Windows is to have all the required .dll files in the same folder as the .exe which uses them - because the Windows dynamic linker will always look in |
Yes, copying the DLLs to the same directory as the executable would be ideal for Windows, but I think it would be harder to implement than using the PATH environment variable. I am baffled that neither Windows nor the MSVC toolchain provides a tool to get the DLLs linked to an executable and recursively all the DLLs those DLLs are linked to considering this is the way Microsoft designed Windows to handle DLLs. Such a tool does exist, but it's written in C# so that doesn't really help Cargo. :/ So if someone wants this feature in Cargo, I suggest starting by writing a Rust library which can recursively find all the DLLs linked to an executable. |
The closest analogy to rpath that Windows has is setting a probing path (see Application Configuration Files). However, it's limited to nine paths and is relative to the application directory. Also Cargo does not support application configuration files at this time. |
@Be-ing I think that might leave out several important use cases. (I apologize in advance if I've misunderstood something!)
Can you say more about this?
I think there are other use cases:
Cross-compilation is another use case that brings these issues into sharp relief. One might want to build a Rust crate against a set of headers and libraries for a target system that's wholly different than the build system. In this case, the build-time library search path is definitely different than the runtime one. To me, the high order bit is that only the top-level executable builder necessarily knows where the native shared library should be found, both at build time and runtime. Admittedly, some people don't care and don't want to have to think about any of this. They want to be able to install libfoo with the system package manager and depend on
To be clear, you mean that: when
I mentioned above why I think this has problems -- the runtime path is not the search path, and the I don't necessarily recommend the following but we've done this with some success:
I mention this by way of example, not because I think Rust should first-class this approach. It's annoying in a few obvious ways. But it has the effect that the top-level builder can specify where to find the library and the right rpath entries get added. And we can still use the discovery mechanism built into the |
What if the traditional In MacPorts we work around this kind of problem by writing compiler and linker wrapper scripts that get injected into the path ahead of whatever would otherwise be used. Kludgy, but it's often the only way to make certain our choice of compiler and options are used... |
Just a note that for build scripts
|
…urrent project. According to `https://github.com/rust-lang/cargo/issues/5077` Signed-off-by: HouXiaoxuan <[email protected]>
Hi,
Consider this minimal example project:
https://github.com/vext01/rust_link_test
The project has a dependency written in C which is built by the
build.rs
.For the examples or tests to work, the shared object for the C code needs to be in the linker path:
If I set
LD_LIBRARY_PATH
toc_lib
then everything works. The standard way to work around having to set theLD_LIBRARY_PATH
is to encode an rpath into the consuming binary (in this case the example or test binary).Whilst the example be fixed using, e.g.:
At the top of the file, the same doesn't appear to be true of tests.
Another thing that bothers me about using
link_args
is that it assumes that the path is known, but typically that's a detail that only the build script will know.Cargo build scripts already have
cargo:rustc-link-lib
which allows this kind of thing to add linkage to a given library:Why not have a more general interface whereby arbitrary linker flags can be passed?
Hope this make sense.
The text was updated successfully, but these errors were encountered: