Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Can't build on Windows 10 with x86_64-pc-windows-gnu; GHA not actually cross-compiling either? #1193

Closed
EndilWayfare opened this issue Aug 14, 2021 · 35 comments
Labels
O-windows Operating system: Windows

Comments

@EndilWayfare
Copy link

Following the instructions in the README, everything seemed to be working fine until ./y.rs build.

The MSVC toolchain is not yet supported by rustc_codegen_cranelift.
Switch to the MinGW toolchain for Windows support.
Hint: You can use `rustup set default-host x86_64-pc-windows-gnu` to
set the global default target to MinGW

Ok, fair enough. Compatibility layer time!

error: linking with `x86_64-w64-mingw32-gcc` failed: exit code: 1
  |
  = note: "x86_64-w64-mingw32-gcc" "-Wl,C:\\Users\\{redacted}\\AppData\\Local\\Temp\\rustczVob4a\\list.def" "-fno-use-linker-plugin" "-Wl,--dynamicbase" "-Wl,--disable-auto-image-base" "-m64" "-Wl,--high-entropy-va"
// lots and lots of paths
  = note: ld: cannot find -lcfgmgr32


warning: `rustc_codegen_cranelift` (lib) generated 1 warning
error: could not compile `rustc_codegen_cranelift` due to previous error; 1 warning emitted

Hmm, dang it. I can't seem to find helpful references to -lcfgmgr32 online. It sounds like mingw can't find the library it needs for cfgmgr32.dll. I tried reinstalling my mingw header package with pacman to no avail. I found out, as I understand it, that the x86_64-pc-windows-gnu toolchain uses a bundled mingw in a rust-mingw component, so my existing installation shouldn't matter anyway. What if I copy libcfgmgr32.a from my local mingw into the "lib" folder of the cargo source for the winapi it's referencing (that doesn't sound like the best idea...)? Huh, that... fixes it?

But now there's a new problem:

          C:\Users\{redacted}\rustc_codegen_cranelift\build_sysroot\target\x86_64-pc-windows-gnu\release\deps\std-f372f6b22d805bdf.std.53bbqtws-cgu.0.rcgu.o:std.53bbqtws-cgu.0:(.debug_info+0x40): relocation truncated to fit: IMAGE_REL_AMD64_ADDR32 against `.debug_str'

and other variations on that theme until

          C:\Users\{redacted}\rustc_codegen_cranelift\build_sysroot\target\x86_64-pc-windows-gnu\release\deps\std-f372f6b22d805bdf.std.53bbqtws-cgu.0.rcgu.o:std.53bbqtws-cgu.0:(.debug_info+0x56): additional relocation overflows omitted from the output
          collect2.exe: error: ld returned 1 exit status

I found this (https://github.com/bjorn3/rustc_codegen_cranelift/issues/977#issuecomment-830650448), which seems to be related, except I'm not using MSVC. (How do I disable DWARF debuginfo?) Based on that comment/thread, I tried to comment out the bit in y.rs that blows up on MSVC, just for laughs. No luck; not yet implemented: Unimplemented TLS model in x64 backend: Coff Yep, that's what the July report said, and it seems to still be the case. From what I can gather, the ADDR32 bit of it has something to do with jumps into relocatable code being too large to fit in a 32-bit pointer. I can confirm that my installed gnu toolchain compiles other crates successfully (I have nothing nearly as large as a sysroot, though).

I played around with a bunch of RUSTFLAGS combination (both to rustc and passthrough to gcc codegen/linking) based on tangential information (very little info on IMAGE_REL_AMD64_ADDR32 specifically) to no avail. It's possible that this is an effective direction, and I just don't know enough about gcc to make it work. It seems odd to me though, because there's no mention (that I could find) of special compiler args being necessary.

I tried a few scattershot previous commits, before and after the transition to a Rust-based build system. Same problem.

I tried cross-compiling with Ubuntu on WSL2. No luck:

❯ ./y.rs build
[BUILD] y.rs
info: syncing channel updates for 'nightly-2021-08-07-x86_64-pc-windows-gnu'
info: latest update on 2021-08-07, rust version 1.56.0-nightly (5ad7389bd 2021-08-06)
info: downloading component 'cargo'
info: downloading component 'clippy'
info: downloading component 'llvm-tools-preview'
info: downloading component 'rust-docs'
info: downloading component 'rust-mingw'
info: downloading component 'rust-src'
info: downloading component 'rust-std'
info: downloading component 'rustc'
info: downloading component 'rustc-dev'
info: downloading component 'rustfmt'
info: installing component 'cargo'
info: installing component 'clippy'
info: installing component 'llvm-tools-preview'
info: installing component 'rust-docs'
info: installing component 'rust-mingw'
info: installing component 'rust-src'
info: installing component 'rust-std'
info: installing component 'rustc'
info: installing component 'rustc-dev'
info: installing component 'rustfmt'
error: the 'rustc' binary, normally provided by the 'rustc' component, is not applicable to the 'nightly-2021-08-07-x86_64-pc-windows-gnu' toolchain

What are you talking about? It says info: downloading component 'rustc' and then info: installing component 'rustc' RIGHT THERE. I have no idea what's going on, as I typically don't mess with nightly/betas.

I gave up on trying to compile from source and checked out the GHA artifacts. Looks like the uploading artifacts from the windows CI build is disabled? https://github.com/bjorn3/rustc_codegen_cranelift/blob/d498e6d6971575714353f11aa4d3e63a9d2030b2/.github/workflows/main.yml#L155-L162

Oh well, cross-x86_64-mingw sounds promising.

$ ./cargo
bash: ./cargo: cannot execute binary file: Exec format error

Dang it.

$ file cargo 
cargo: ELF 64-bit LSB shared object x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, BuildID[sha1]=b8e72eb3363cc311d848656c88fa225cbaf411a7, for GNU/Linux 3.2.0, with debug_info, not stripped

Well, that definitely doesn't sound right.

./y.rs build --sysroot none did compile, although it (understandably) does nothing and hangs forever when invoked. However:

$ file build/cargo
build/cargo: PE32+ executable (console) x86-64, for MS Windows

Unless I'm doing something wrong (quite possible), it seems that the cross-compile CI target isn't measuring what it's supposed to be measuring (successfully producing a Windows-runnable artifact from Linux).

TWIR sent me here. I've been following this on-and-off for a while, and it's really awesome to see how close it is to basically-feature-complete. I wanted to get in on it (faster debug builds? Yes please!), but I've tried as hard as I can on my own.

@EndilWayfare
Copy link
Author

Ayyyy, got it to work.

TL;DR: Compiling on Windows == still broken for me. Cross-compiling from Linux works if 1) target/host overrides are set correctly and 2) you make sure target_triple is propagated to compiling the cargo wrapper.

Cross-compiling from WSL wasn't working because somehow in all my faffing about I overrode the default host triple in addition to the default target. Oof. So, it compiled then, but the resulting cargo failed in Windows with the same error (and file output) as the GHA artifact did. After digging around in the source code a bit, since the problem seemed to hinge on the cargo binary, I decided to try altering this:
https://github.com/bjorn3/rustc_codegen_cranelift/blob/d498e6d6971575714353f11aa4d3e63a9d2030b2/build_system/build_sysroot.rs#L46-L53
to this:

    // Build and copy cargo wrapper
    let mut build_cargo_wrapper_cmd = Command::new("rustc");
    build_cargo_wrapper_cmd
        .arg("--target")
        .arg(target_triple)
        .arg("scripts/cargo.rs")
        .arg("-o")
        .arg(target_dir.join("cargo"))
        .arg("-g");
    spawn_and_wait(build_cargo_wrapper_cmd);

, to match https://github.com/bjorn3/rustc_codegen_cranelift/blob/d498e6d6971575714353f11aa4d3e63a9d2030b2/build_system/build_sysroot.rs#L179-L181

That would also explain why CI doesn't appear to be producing a viable cross-compiled artifact!

@EndilWayfare
Copy link
Author

Dammit, just kidding. That was my system cargo.exe, not ./build/cargo.exe. The new cross-compiled cargo hangs forever like the ./y.rs build --sysroot none version...

@bjorn3
Copy link
Member

bjorn3 commented Aug 14, 2021

Cross-compiling from WSL wasn't working because somehow in all my faffing about I overrode the default host triple in addition to the default target. Oof. So, it compiled then, but the resulting cargo failed in Windows with the same error (and file output) as the GHA artifact did. After digging around in the source code a bit, since the problem seemed to hinge on the cargo binary, I decided to try altering this:

I have never actually tried to cross-compile cg_clif itself for windows. CI tests compiling cg_clif on windows using the mingw toolchain and it tests compiling programs using cg_clif for windows using the mingw toolchain.

But now there's a new problem:

          C:\Users\{redacted}\rustc_codegen_cranelift\build_sysroot\target\x86_64-pc-windows-gnu\release\deps\std-f372f6b22d805bdf.std.53bbqtws-cgu.0.rcgu.o:std.53bbqtws-cgu.0:(.debug_info+0x40): relocation truncated to fit: IMAGE_REL_AMD64_ADDR32 against `.debug_str'

and other variations on that theme until

          C:\Users\{redacted}\rustc_codegen_cranelift\build_sysroot\target\x86_64-pc-windows-gnu\release\deps\std-f372f6b22d805bdf.std.53bbqtws-cgu.0.rcgu.o:std.53bbqtws-cgu.0:(.debug_info+0x56): additional relocation overflows omitted from the output
          collect2.exe: error: ld returned 1 exit status

Maybe removing https://github.com/bjorn3/rustc_codegen_cranelift/blob/d498e6d6971575714353f11aa4d3e63a9d2030b2/build_sysroot/Cargo.toml#L23 helps?

@EndilWayfare
Copy link
Author

I have never actually tried to cross-compile cg_clif itself for windows. CI tests compiling cg_clif on windows using the mingw toolchain and it tests compiling programs using cg_clif for windows using the mingw toolchain.

Yeah, I figured that out yesterday, I just ran out of time trying weird combinations of compiling different bits on Windows and WSL to post about it. I was like, OOOOOH got it, the host/target are operating on a different layer of indirection than I assumed. It's not that the codegen is cross-compiled to run on windows, it's that the codegen does cross-compilation (running on Linux, producing cranelift debug builds that run on windows). That's awesome, but I don't feel like doubling up on dependency sources/build-artifacts between WSL and Windows...

@EndilWayfare
Copy link
Author

Maybe removing

https://github.com/bjorn3/rustc_codegen_cranelift/blob/d498e6d6971575714353f11aa4d3e63a9d2030b2/build_sysroot/Cargo.toml#L23

helps?

Ahhhh, the Cargo.toml in "build_sysroot", ok. I messed around with the root one, but I missed that there was another. Well, it helps a little, in the sense that it actually compiles successfully. But, it also doesn't because the resulting "cargo.exe" hangs indefinitely still.

@EndilWayfare
Copy link
Author

While I'm at it, is there a reason that the windows CI target is disabled from uploading its artifacts? Later today, I might get a chance to try compiling on a fresher, less battle-worn Windows dev environment and we'll see if that changes anything.

@bjorn3
Copy link
Member

bjorn3 commented Aug 14, 2021

I can't remember. Maybe a leftover from trying to set up windows ci?

@EndilWayfare
Copy link
Author

Ok, I decided to let cargo.exe sit there for a while and find out if it ever actually does anything. It does. It allocates a lot of memory until it gets OOM'd.

$ build/cargo.exe
memory allocation of 1766464 bytes failed
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Os { code: 1455, kind: Uncategorized, message: "The paging file is too small for this operation to complete." }', scripts/cargo.rs:76:50

Huh. Ok, once more with RUST_BACKTRACE=1. While we're waiting for it to explode, let's pull up Task Manager and see if that tells us anything.
Screenshot_081421_092735_PM
Hmmm, cargo.exe is... multiplying? Those are all subprocesses under the terminal emulator that initially invoked cargo.exe...
Screenshot_081421_092945_PM
!?

It caps out at over 9,000 [yes, really] subprocesses consuming 19 GB of RAM before it goes down.
Interestingly, it doesn't actually give me a backtrace. It just goes

$ build/cargo.exe
memory allocation of 2269140 bytes failed

It seems to slowly unwind the subprocesses after it prints the failure message, and it doesn't "return" to the shell prompt until all the subprocesses are gone.

@EndilWayfare
Copy link
Author

Interestingly, it doesn't actually give me a backtrace

oooooh, could that be a side-effect of debug = true?

@EndilWayfare
Copy link
Author

It's also very probable that the windows CI build suffers from the same issue. The action only tests whether it compiles; test.sh isn't run, because windows.

I tried running test.sh locally with my MinGW bash, and it exploded. First, I had to add "./build/bin" and "/c/Users/{user}/.rustup/toolchains/nightly-2021-08-07-x86_64-pc-windows-gnu/bin" to $PATH, because otherwise windows can't find the .dll's. Then, [BUILD] mini_core failed with the same relocation truncated to fit: IMAGE_REL_AMD64_ADDR32 nonsense, so I added -C debuginfo=0 to the line in "scripts/tests.sh". Now mini_core gets into rustc proper, but it ICEs on an unresolved symbol.

thread 'rustc' panicked at 'can't resolve symbol _ZN9mini_core8A_STATIC17h1e90a401893c503aE', C:\Users\pormand\.cargo\git\checkouts\wasmtime-41807828cb3a7a7e\9c550fc\cranelift\jit\src\backend.rs:288:21
stack backtrace:
   0:         0x6f2d2d2f - std::backtrace_rs::backtrace::dbghelp::trace::h15703146ef9749a7
                               at /rustc/5ad7389bdd1abe7d2c6f73a233af1a7a69e96285\/library\std\src\..\..\backtrace\src\backtrace/dbghelp.rs:98:5
   1:         0x6f2d2d2f - std::backtrace_rs::backtrace::trace_unsynchronized::hf8d56c17f1e136e9
                               at /rustc/5ad7389bdd1abe7d2c6f73a233af1a7a69e96285\/library\std\src\..\..\backtrace\src\backtrace/mod.rs:66:5
   2:         0x6f2d2d2f - std::sys_common::backtrace::_print_fmt::hab3c52876e76d449
                               at /rustc/5ad7389bdd1abe7d2c6f73a233af1a7a69e96285\/library\std\src\sys_common/backtrace.rs:67:5   
   3:         0x6f2d2d2f - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::hab9c1cc6e7fc385b   
                               at /rustc/5ad7389bdd1abe7d2c6f73a233af1a7a69e96285\/library\std\src\sys_common/backtrace.rs:46:22  
   4:         0x6f33e39a - core::fmt::write::hae96daedf1d6be29
                               at /rustc/5ad7389bdd1abe7d2c6f73a233af1a7a69e96285\/library\core\src\fmt/mod.rs:1117:17
   5:         0x6f2c6262 - std::io::Write::write_fmt::hafe5401ad6ee5626
                               at /rustc/5ad7389bdd1abe7d2c6f73a233af1a7a69e96285\/library\std\src\io/mod.rs:1667:15
   6:         0x6f2d64e0 - std::sys_common::backtrace::_print::hbe08c5b6a843015d
                               at /rustc/5ad7389bdd1abe7d2c6f73a233af1a7a69e96285\/library\std\src\sys_common/backtrace.rs:49:5   
   7:         0x6f2d64e0 - std::sys_common::backtrace::print::h674aa25aba0c71a3
                               at /rustc/5ad7389bdd1abe7d2c6f73a233af1a7a69e96285\/library\std\src\sys_common/backtrace.rs:36:9   
   8:         0x6f2d64e0 - std::panicking::default_hook::{{closure}}::h5c12d9d9abbca0f9
                               at /rustc/5ad7389bdd1abe7d2c6f73a233af1a7a69e96285\/library\std\src/panicking.rs:210:50
   9:         0x6f2d6094 - std::panicking::default_hook::hbc4ccd03703b88a3
                               at /rustc/5ad7389bdd1abe7d2c6f73a233af1a7a69e96285\/library\std\src/panicking.rs:227:9
  10:     0x7ff6672d28fe - cg_clif::DEFAULT_HOOK::{{closure}}::{{closure}}::h4825952fd8172fa7
  11:         0x6f2d6dc1 - std::panicking::rust_panic_with_hook::h1ecdc969fd899f2f
                               at /rustc/5ad7389bdd1abe7d2c6f73a233af1a7a69e96285\/library\std\src/panicking.rs:628:17
  12:         0x6f2d6899 - std::panicking::begin_panic_handler::{{closure}}::h36573f2e57c65019
                               at /rustc/5ad7389bdd1abe7d2c6f73a233af1a7a69e96285\/library\std\src/panicking.rs:521:13
  13:         0x6f2d373f - std::sys_common::backtrace::__rust_end_short_backtrace::hbff34e368d9c5643
                               at /rustc/5ad7389bdd1abe7d2c6f73a233af1a7a69e96285\/library\std\src\sys_common/backtrace.rs:141:18 
  14:         0x6f2d67f9 - rust_begin_unwind
                               at /rustc/5ad7389bdd1abe7d2c6f73a233af1a7a69e96285\/library\std\src/panicking.rs:517:5
  15:         0x6f2d67ac - std::panicking::begin_panic_fmt::h4b4cd9d4d947fbb4
                               at /rustc/5ad7389bdd1abe7d2c6f73a233af1a7a69e96285\/library\std\src/panicking.rs:460:5
  16:     0x7ffba855d144 - cranelift_jit::backend::JITModule::get_address::ha71330b8095d55ca
  17:     0x7ffba85639fa - cranelift_jit::compiled_blob::CompiledBlob::perform_relocations::h209dee35f48b61c5
  18:     0x7ffba855de36 - cranelift_jit::backend::JITModule::finalize_definitions::hca5b226777528b55
  19:     0x7ffba849e66f - rustc_codegen_cranelift::driver::jit::run_jit::h9a1e46bdfa469fae
  20:     0x7ffba8503f4d - <rustc_codegen_cranelift::CraneliftCodegenBackend as rustc_codegen_ssa::traits::backend::CodegenBackend>::codegen_crate::h495a51801e90fb82
  21:           0xadd650 - rustc_interface::passes::QueryContext::enter::hd0572fc530ac941d
  22:           0xacdc5b - rustc_interface::queries::Queries::ongoing_codegen::h2f921abc637d3d75
  23:           0x9df8ed - rustc_interface::queries::<impl rustc_interface::interface::Compiler>::enter::h1cb4948d439e9483        
  24:           0x99a85c - rustc_span::with_source_map::h6cc506dfc70b42ef
  25:           0x9e1120 - rustc_interface::interface::create_compiler_and_run::hf8038fe8f954301a
  26:           0x9a7b3d - std::sys_common::backtrace::__rust_begin_short_backtrace::hb1ec0507bfe8704c
  27:           0x99397e - core::ops::function::FnOnce::call_once{{vtable.shim}}::h5d71226d451acc81
  28:         0x6f2e4557 - <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once::h525f0061b701c388
                               at /rustc/5ad7389bdd1abe7d2c6f73a233af1a7a69e96285\library\alloc\src/boxed.rs:1636:9
  29:         0x6f2e4557 - <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once::h5e31eae84f71f0a7
                               at /rustc/5ad7389bdd1abe7d2c6f73a233af1a7a69e96285\library\alloc\src/boxed.rs:1636:9
  30:         0x6f2e4557 - std::sys::windows::thread::Thread::new::thread_start::h1ed0a385a606a823
                               at /rustc/5ad7389bdd1abe7d2c6f73a233af1a7a69e96285\/library\std\src\sys\windows/thread.rs:57:17
  31:     0x7ffc41e57034 - <unknown>
  32:     0x7ffc43782651 - <unknown>

error: internal compiler error: unexpected panic

note: the compiler unexpectedly panicked. this is a bug.

note: we would appreciate a bug report: https://github.com/bjorn3/rustc_codegen_cranelift/issues/new

note: rustc 1.56.0-nightly (5ad7389bd 2021-08-06) running on x86_64-pc-windows-gnu

note: compiler flags: -Z unstable-options -C debuginfo=2 -C llvm-args=mode=jit -C prefer-dynamic

query stack during panic:
end of query stack

Sounds suspiciously like https://github.com/bjorn3/rustc_codegen_cranelift/issues/1137?

@bjorn3
Copy link
Member

bjorn3 commented Aug 15, 2021

Hmmm, cargo.exe is... multiplying? Those are all subprocesses under the terminal emulator that initially invoked cargo.exe...

I see what is going on. On Unix systems the directory containing the current executabe is not in PATH, so trying to spawn "cargo" will attempt to run the original cargo. On windows it is in PATH, so it will attempt to run the wrapper itself.

https://github.com/bjorn3/rustc_codegen_cranelift/blob/d498e6d6971575714353f11aa4d3e63a9d2030b2/scripts/cargo.rs#L76

@bjorn3
Copy link
Member

bjorn3 commented Aug 15, 2021

I tried running test.sh locally with my MinGW bash, and it exploded.

Looks like the jit tests don't work. Could you try if disabling those tests will make the rest of the tests pass?

Then, [BUILD] mini_core failed with the same relocation truncated to fit: IMAGE_REL_AMD64_ADDR32 nonsense, so I added -C debuginfo=0 to the line in "scripts/tests.sh".

Todo:

  • Disable dwarf debuginfo generation on Windows.

@EndilWayfare
Copy link
Author

I see what is going on. On Unix systems the directory containing the current executabe is not in PATH, so trying to spawn "cargo" will attempt to run the original cargo. On windows it is in PATH, so it will attempt to run the wrapper itself.

Confirmed. Renaming from cargo.exe to cargo-clif.exe works. It successfully substitutes for cargo to build another crate, except I have to set RUSTFLAGS="-C debuginfo=0" or IMAGE_REL_AMD64_ADDR32 happens.

@EndilWayfare
Copy link
Author

Looks like the jit tests don't work. Could you try if disabling those tests will make the rest of the tests pass?

It looks like disabling jit tests and tests that invoke cargo works, and the other tests succeed. I say "looks", because I haven't gotten to the end of "tests.sh" with modification yet. It's a lot harder to kill the runaway-process phenomenon when it's a binary invoking cargo rather than bash invoking cargo; it's enough to SIGINT in bash, but you kind of have to wait for OOM in the other case because Task Manager gets overwhelmed and unusable.

@EndilWayfare
Copy link
Author

EndilWayfare commented Aug 15, 2021

base_sysroot_tests completes successfully, under above caveats

@EndilWayfare
Copy link
Author

Circling back to no_sysroot_tests, it also works under above caveats

@bjorn3
Copy link
Member

bjorn3 commented Aug 16, 2021

I disabled debuginfo on windows in a branch: https://github.com/bjorn3/rustc_codegen_cranelift/tree/windows_fixes This still needs the jit tests disabled and the recursive cargo problem to be fixed.

@EndilWayfare
Copy link
Author

EndilWayfare commented Aug 16, 2021

Disabling the jit tests should probably be as easy as setting JIT_SUPPORTED=0 for windows triples in ext_config.sh

@bjorn3
Copy link
Member

bjorn3 commented Aug 27, 2021

Pushed a commit to disable jit tests on windows to that branch.

I am not quite sure how to proceed with the cargo though that doesn't hurt non-windows usage. Any ideas?

@EndilWayfare
Copy link
Author

Hmm, I've been using the "rename the binary" strat for regular use (stuck the .exe in a folder in my PATH and renamed it to cargo-clif). It seems to work alright, but I haven't stress-tested it extensively. This cargo only needs to spawn instances of the "real" cargo and never another of itself, right?

Maybe rename for the tests? If you don't want to rename on all platforms, you could set a CRANELIFT_CARGO_NAME, or something, in ext_config.sh where other platform-specific stuff happens.

Maybe a global rename (test, dev, and release builds) wouldn't be such a bad idea. I can't imagine someone wanting to give up their regular cargo all the time for the cranelift one, or having to always type the full path for one of them (or set a custom alias or something). Better, imo, to opinionatedly stabilize on something so everyone's not doing something subtlely different.

Ideally, I think, it wouldn't be a full cargo replacement wrapper. I don't know how practical this would be, but maybe it could work as a cargo command (like cargo-edit, cargo-expand, or cargo-audit), and you would cargo cranelift build or something. Or maybe something more along the lines of +nightly, but for codegen instead of toolchain selection. The downside there is, there'd (probably) have to be some changes on cargo's side. We've never had an alternative backend before, so this is kinda new UI design space... 😕

@bjorn3
Copy link
Member

bjorn3 commented Aug 29, 2021

In the (hopefully) near future it will become possible to select cg_clif using an option in Cargo.toml. The cargo side of this has already landed in rust-lang/cargo#9118. rust-lang/rust#81746 will ship cg_clif as rustup component to make it actually useful.

[profile.dev]
codegen-backend = "cranelift"

The cargo wrapper will be kept for local development of cg_clif, but it won't be shipped with rustc.

@bjorn3 bjorn3 mentioned this issue Aug 29, 2021
@bjorn3
Copy link
Member

bjorn3 commented Aug 29, 2021

Renamed the cargo wrapper to cargo-clif. Opened https://github.com/bjorn3/rustc_codegen_cranelift/pull/1195 for the fixes so far. @EndilWayfare can you please test if it fixes all problems you have?

@EndilWayfare
Copy link
Author

Sure! Replying inline on the pull request.

@EndilWayfare
Copy link
Author

Hmm, noticing some interesting things with cargo test

RUSTFLAGS="-C debuginfo=0" cargo test
// ...
running 4 tests
test model::tests::part_number::parses_doublet ... ok
test model::tests::part_number::parses_singlet ... ok
test model::tests::part_number::parses_doublet_element_a ... ok
test model::tests::part_number::parses_doublet_element_b ... FAILED

failures:

---- model::tests::part_number::parses_doublet_element_b stdout ----
thread 'model::tests::part_number::parses_doublet_element_b' panicked at 'assertion failed: `(left == right)`
  left: `Ok(PartNumber { sequence: 1, kind: Doublet, element: Some(0) })`,
 right: `Ok(PartNumber { sequence: 1, kind: Doublet, element: Some(1) })`', src\model.rs:244:13
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
RUSTFLAGS="-C debuginfo=0" cargo-clif test
// ...
running 4 tests
test model::tests::part_number::parses_singlet ... FAILED
test model::tests::part_number::parses_doublet ... FAILED
test model::tests::part_number::parses_doublet_element_b ... FAILED
test model::tests::part_number::parses_doublet_element_a ... FAILED

failures:

---- model::tests::part_number::parses_singlet stdout ----
---- model::tests::part_number::parses_singlet stderr ----
note: got unexpected return code -1073741819
---- model::tests::part_number::parses_doublet stdout ----
---- model::tests::part_number::parses_doublet stderr ----
note: got unexpected return code -1073741819
---- model::tests::part_number::parses_doublet_element_b stdout ----
---- model::tests::part_number::parses_doublet_element_b stderr ----
note: got unexpected return code -1073741819
---- model::tests::part_number::parses_doublet_element_a stdout ----
---- model::tests::part_number::parses_doublet_element_a stderr ----
note: got unexpected return code -1073741819

failures:
    model::tests::part_number::parses_doublet
    model::tests::part_number::parses_doublet_element_a
    model::tests::part_number::parses_doublet_element_b
    model::tests::part_number::parses_singlet

test result: FAILED. 0 passed; 4 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.19s

Only one error, but all tests fail on cargo-clif

Fixed the failing test, all tests still fail on cargo-clif.

@EndilWayfare
Copy link
Author

On a lark, specifying the gnu toolchain doesn't help. It's worse 🤣

RUSTFLAGS="-C debuginfo=0" cargo-clif +nightly-2021-08-25-x86_64-pc-windows-gnu test
error: process didn't exit successfully: `C:\Users\{redacted}\bin\bin/cg_clif.exe -vV` (exit code: 0xc0000135, STATUS_DLL_NOT_FOUND)

@EndilWayfare
Copy link
Author

Don't know how to determine if it can't find a clif .dll, a gnu toolchain .dll, or some other .dll entirely...

@prantor19

This comment has been minimized.

@bjorn3
Copy link
Member

bjorn3 commented Aug 31, 2021

What project are those failing tests on? They are probably a problem with cg_clif independent of the windows support.

@bjorn3
Copy link
Member

bjorn3 commented Aug 31, 2021

On a lark, specifying the gnu toolchain doesn't help. It's worse 🤣

RUSTFLAGS="-C debuginfo=0" cargo-clif +nightly-2021-08-25-x86_64-pc-windows-gnu test
error: process didn't exit successfully: `C:\Users\{redacted}\bin\bin/cg_clif.exe -vV` (exit code: 0xc0000135, STATUS_DLL_NOT_FOUND)

I think you need to build cg_clif with the gnu toolchain too then. cg_clif needs to be built with the exact same compiler as which it will be loaded into.

@EndilWayfare
Copy link
Author

I'm 98% sure that version of cargo-clif was build with gnu. I'll double check, though...

@EndilWayfare
Copy link
Author

Actually wait, doesn't this project force a pinned nightly toolchain? Because I only have that one installed in gnu flavor.

@EndilWayfare
Copy link
Author

Those failing tests are from one of my own projects. I was checking if cg_clif could help with test compile times.

@bjorn3 bjorn3 added the O-windows Operating system: Windows label Mar 30, 2022
@bjorn3
Copy link
Member

bjorn3 commented Sep 1, 2022

I believe this should work fine now given recent improvements. There is still a case where it can crash on windows though which will be fixed by bytecodealliance/wasmtime#4747.

@bjorn3
Copy link
Member

bjorn3 commented Sep 27, 2022

https://github.com/bjorn3/rustc_codegen_cranelift/pull/1284 has added actual native Windows testing to the CI. @EndilWayfare does it work for you now?

@bjorn3
Copy link
Member

bjorn3 commented Dec 15, 2022

Closing. If you still have problems please open a new issue.

@bjorn3 bjorn3 closed this as completed Dec 15, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
O-windows Operating system: Windows
Projects
None yet
Development

No branches or pull requests

3 participants