1.47 beta cross-compiling broken: tries to run the reverse cross-compile

[infinity0]

config.toml:

[..]
build = "x86_64-unknown-linux-gnu"
host = ["armv5te-unknown-linux-gnueabi"]
target = ["armv5te-unknown-linux-gnueabi"]
[..]

build log tail:

RUST_BACKTRACE=1 python3 src/bootstrap/bootstrap.py build --config debian/config.toml -vvv --on-fail env
[..]
Dirty - /<<PKGBUILDDIR>>/build/armv5te-unknown-linux-gnueabi/stage1-std
  c Sysroot { compiler: Compiler { stage: 1, host: TargetSelection { triple: "armv5te-unknown-linux-gnueabi", file: None } } }
Building stage1 std artifacts (armv5te-unknown-linux-gnueabi -> x86_64-unknown-linux-gnu)
running: "/usr/bin/cargo" "build" "--target" "x86_64-unknown-linux-gnu" "-Zbinary-dep-depinfo" "-j" "4" "-v" "-v" "--release" "--features" "panic-unwind backtrace" "--manifest-path" "/<<PKGBUILDDIR>>/library/test/Cargo.toml" "--message-format" "json-render-diagnostics"
warning: config profiles require the `-Z config-profile` command-line option (found profile `release` in environment variable `CARGO_PROFILE_RELEASE`)
warning: Patch `backtrace v0.3.50 (/<<PKGBUILDDIR>>/library/backtrace)` was not used in the crate graph.
Check that the patched package version and available features are compatible
with the dependency requirements. If the patch has a different version from
what is locked in the Cargo.lock file, run `cargo update` to use the new
version. This may also occur with an optional dependency that is not enabled.
error: process didn't exit successfully: `/<<PKGBUILDDIR>>/build/bootstrap/debug/rustc -vV` (exit code: 101)
--- stderr
rustc command: "LD_LIBRARY_PATH"="/<<PKGBUILDDIR>>/build/armv5te-unknown-linux-gnueabi/stage1/lib" "/<<PKGBUILDDIR>>/build/armv5te-unknown-linux-gnueabi/stage1/bin/rustc" "-vV" "-Wrust_2018_idioms" "-Wunused_lifetimes" "-Dwarnings" "-Clinker=arm-linux-gnueabi-gcc" "--remap-path-prefix" "/<<PKGBUILDDIR>>=/usr/src/rustc-1.47.0" "-Z" "force-unstable-if-unmarked"
sysroot: "/<<PKGBUILDDIR>>/build/armv5te-unknown-linux-gnueabi/stage1"
libdir: "/<<PKGBUILDDIR>>/build/armv5te-unknown-linux-gnueabi/stage1/lib"
thread 'main' panicked at '
Failed to run:
"/<<PKGBUILDDIR>>/build/armv5te-unknown-linux-gnueabi/stage1/bin/rustc" "-vV" "-Wrust_2018_idioms" "-Wunused_lifetimes" "-Dwarnings" "-Clinker=arm-linux-gnueabi-gcc" "--remap-path-prefix" "/<<PKGBUILDDIR>>=/usr/src/rustc-1.47.0" "-Z" "force-unstable-if-unmarked"
-------------: Os { code: 8, kind: Other, message: "Exec format error" }', src/bootstrap/bin/rustc.rs:152:22
stack backtrace:
[.. etc ..]

That line Building stage1 std artifacts (armv5te-unknown-linux-gnueabi -> x86_64-unknown-linux-gnu) does not look like it should be there.

[Mark-Simulacrum]

Can you gist the full log somewhere? It should have enough information to know what caused this.

[infinity0]

@Mark-Simulacrum thanks for taking a look, it's here, 4MB: https://gist.github.com/infinity0/f0100323593bfc86247c7d8f86a1a3e9

[Mark-Simulacrum]

@infinity0 Unrelated, but https://blog.rust-lang.org/inside-rust/2020/08/30/changes-to-x-py-defaults.html is something you should probably be aware of.

I think the problem is that you're not explicitly setting the targets, so we're including hosts and the build triple in them. Is this a regression from the previous release? We can probably adjust the behavior back if so.

Edit: It should be fixable with --target armv5te-unknown-linux-gnueabi explicitly passed on the command line.

[nielx]

Hit the same issue, it turns out that there is a change in rustbuild that ./x.py build no longer implies building stage 2, which leads to this behaviour. (./x.py dist still implies --stage 2).

To resolve this, use ./x.py --stage 2 build.

I am unsure whether this is a bug in the strict sense, maybe cross-compiling should imply --stage 2, but I know too little about rustbuild to argue for or against it.

[Mark-Simulacrum]

Hm, so passing --stage 2 fixes this? That's surprising to me, I would expect us to build a similar or equivalent set regardless of stage. Could you gist a log with stage 2?

[infinity0]

@Mark-Simulacrum --target $host does not work, I will try --stage 2 next. With --target $host (and the above config.toml) we get Building stage1 std artifacts (armv5te-unknown-linux-gnueabi -> armv5te-unknown-linux-gnueabi) which is also unwanted.

This is a regression, it was working fine for us on 1.46 and earlier versions. However I do note that the documentation for config.toml says:

# In addition to the build triple, other triples to produce full compiler
# toolchains for. Each of these triples will be bootstrapped from the build
# triple and then will continue to bootstrap themselves. This platform must
# currently be able to run all of the triples provided here.
#
# Defaults to just the build triple
#host = ["x86_64-unknown-linux-gnu"]

When cross-compiling in Debian we do not assume that "this [the build] platform must currently be able to run all of the triples provided here", and this worked fine on 1.46 and earlier. I am not sure if the change in 1.47 was simply making this documentation actually effective - if so it would be nice to revert it and reverse what the documentation states.

When cross-compiling in general, I think it's reasonable not to assume that one can actually run the resulting binaries, and I'm not sure what the intention behind this statement in the documentation is supposed to be for. If you are cross-compiling for a specific platform that you can run, e.g. x86-64 -> i686, you can run the tests explicitly afterwards for that target, there is no need to make this assumption for the build step.

[infinity0]

OK, --stage 2 works and (if I understood that PR correctly) should be identical to the old behaviour.

It's possible that --target $host might also have worked if I had emptied host and target in config.toml, however we need --stage 2 regardless so I will just go with this option rather than screw around with the other settings.

@Mark-Simulacrum it seems that the documentation for config.toml could be updated? At least "This platform must currently be able to run all of the triples provided here." is empirically not true, and the previous sentence also seems like it could do with fixing. As far as I can tell the host compiler is never actually run (on build --stage 2) so it is not accurate to say "each of these triples [..] then will continue to bootstrap themselves". I am not sure what exactly actually happens for the general stage-N case.