Cross compilation is broken when host and target architectures are the same. #1513
Comments
I don't think this is related to
|
|
See the similar issue with linuxbrew rust-lang/rust#58394 and LibreELEC LibreELEC/LibreELEC.tv#4719 You might be able to use the same workaround, adding the content below to [target.x86_64-unknown-linux-gnu]
linker = "/usr/bin/cc" |
I don't see why that should be causing an issue here, the buildroot toolchain is designed to work when the target glibc is newer than the host glibc toolchain from my understanding. Keep in mind that this compiles perfectly fine without errors when using an aarch64 or i386 target with the same newer buildroot glibc target version so I don't understand why this would be an issue only when the host is the same as the target architecture otherwise.
Pretty sure that's being set already here. |
It's totally fine to build with a newer glibc version but I don't think it guarantees that the built artifact will run on the host machine right? In the case of proc-macro, it needs to run on the host machine. aarch64 or i386 target works because proc-macro is built with host toolchain, it would be great if you can provide the build log for aarch64 or i386.
Can you double check that? Pasting the file content on your local machine to here would be very helpful. |
That's correct from my understanding, but if it needs to run on the host why would it be trying to build/link for the newer target glibc then for something that needs to use the host toolchain? aarch64 build log:
So I guess the question is then why would that not be happening with x86_64?
|
If you [target.x86_64-unknown-linux-gnu]
linker = "x86_64-buildroot-linux-gnu-gcc"Since the host and the target is the same, and there is no way to set different linkers for proc-macro and regular Rust crates |
Yep
Is this missing from pyo3 or other places as well? From the looks of it in pyo3 |
|
Let me try to summarize what's happening. Rust has proc-macro and build script, they can generate code when compiling and they are linked with host toolchain and needs to be able to run on host. When cross compiling for a different target, even though you specified But when host and target is the same, say So this has nothing to do with I think there are several ways to resolve this:
|
So does this mean that we would need to add a force cross compilation override of some sort to cargo?
Heh, well a lot it seems since the architecture being the same effectively is entirely separate from whether or not one is cross compiling.
Ok, so we need to fix this in addition to the cargo issue above still though right?
Ok, so keeping in mind I'm trying to get this working in a way that's acceptable for upstream buildroot, I don't think any of these would work for the following reasons:
The host OS toolchain is invalid for compiling target binaries with buildroot so we can't do this, it can only be used for compiling buildroot host packages(build tools), not buildroot target packages(stuff that runs on the target system we are cross compiling for). The built in OS toolchain is effectively only usable for bootstrapping/running the build tools(host packages in buildroot terminology) which buildroot then uses to compile everything for the target. We can't directly compile target binaries from the OS toolchain since buildroot needs to effectively fully control the build tools/process used for target binary compilation.
Buildroot doesn't control the host OS toolchain glibc version so this isn't possible.
I thought we are already doing this effectively. |
This was referenced Mar 22, 2021
|
Sorry I have been slow to join this conversation. @jameshilliard it looks to me from the issues where you have referenced this thread that the fundamental problem is Unfortunately I don't think we're able to offer a universal solution for this without upstream As a possible workaround, @jameshilliard can you try setting (I imagine that should work as long as the buildroot libs aren't compiled with additional instruction sets which your host machine doesn't support.) |
Yeah, that's my best understanding of the issue at the moment.
Yeah, I figured I would at least try and fix the obvious cross compilation detection issue in pyo3 in #1514 to make it easier to test potential cargo fixes.
Not sure how that's related to this, best I can tell this isn't a dependency issue so much as cargo not providing critical auto-detection logic override hooks needed for cross compilation.
Yeah...didn't get very far there as I'm totally unfamiliar with the cargo codebase, but this capability should already be there for the most part since it does actually do that when the host and target architecture are different, I just can't figure out how that's actually controlled internally but this should be a fairly simple fix for whoever is familiar with this area of the codebase unless I'm missing something.
This might work for my own builds but I don't think it's maintainable enough to upstream as it seems highly likely to cause a bunch of problematic side effects. We do have a lot of workarounds/hacks in buildroot for cross compilation issues(we do try and fix issues in upstream dependencies when possible though) but this looks like it's highly likely to have a bunch of broken edge cases.
Well I do think anything upstreamable will need to handle this case properly as this is a supported configuration in buildroot. |
I was more trying to point that there's been a number of issues where Cargo did the wrong thing for build scripts / proc macros and they're steadily being fixed.
Completely agree that |
Yeah, at least it seems to cross build for supported targets that have different architectures than the host without too much trouble, it's probably less complex than our python integrations as well which tend to be fairly involved as we also manage system package dependency resolution for all the supported python packages. |
|
I may have a potential fix for the cargo issue in #9322. |
|
I see that the cargo PR has now been merged (whew, looks like it was a meaty one), so I'm going to close this here as presumably it's now possible to achieve this on nightly. Please re-open if that's not the case. |
|
Should be possible to use the fix in stable like this in |
|
Hmmm. It's possible to enable unstable features via a cargo config file (https://doc.rust-lang.org/cargo/reference/unstable.html#unstable-features), so I would suggest it's easy enough for users like buildroot who need this to add the appropriate env and cargo config without needing code changes to setuptools-rust. |
|
Well you also have to set |
|
Yes, that's one of the env vars I meant users can set. 👍 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
When I tried cross compiling for a
x86_64target from ax86_64host I get a build failure that seems to be caused by broken cross compilation detection logic which results inPYO3_CROSS_LIB_DIRbeing ignored. My build setup appears to work fine when I cross compile fori386oraarch64targets from ax86_64host.This comment seems to indicate cross compilation mode will not function when the host and target are the same:
pyo3/build.rs
Lines 885 to 886 in 190eb72
How should one ensure that pyo3 knows to always run in cross compilation mode even if the host and target architectures are the same?
The text was updated successfully, but these errors were encountered: