WASI: Cannot open paths with nightly >= 2021-03-11 when linked with LLD 11.1

[abbec]

I have code opening a path like std::fs::read("/absolute/path.txt") in WASI. All the surrounding code stays the same but when I upgrade nightly from 2021-03-10 to 2021-03-11 it does not work anymore.

Version details:

# opening a path with leading "/" does _NOT_ work
rustc 1.52.0-nightly (f98721f88 2021-03-10)
binary: rustc
commit-hash: f98721f886ab52d32d622ad0a46216ad03f3e525
commit-date: 2021-03-10
host: x86_64-apple-darwin
release: 1.52.0-nightly

# opening a path with leading "/" _DOES_ work
rustc 1.52.0-nightly (3a5d45f68 2021-03-09)
binary: rustc
commit-hash: 3a5d45f68cadc8fff4fbb557780f92b403b19c19
commit-date: 2021-03-09
host: x86_64-apple-darwin
release: 1.52.0-nightly
LLVM version: 12.0.0

Looking at the history, the only thing I can see that looks WASI-related is this but I have no idea here and any help would be appreciated!

EDIT: This description turned out to be somewhat incorrect, see below for a good repro case.

[abbec]

I was wrong, actually it does not work at all on nightly 2021-03-11 and onwards (I tested nightly 2021-05-11 with the same result), with or without the leading "/":

failed to find a pre-opened file descriptor through which "relative/path.txt" could be opened

and

failed to find a pre-opened file descriptor through which "/absolute/path.txt" could be opened

whereas on nightly 2021-03-10 the same code works fine both with and without the leading "/".

[apiraino]

hi @abbec thanks for filing a report. Do you imply that even a snippet like this returns that error?

use std::fs;
fn repro() {
    fs::read("/absolute/path.txt").unwrap();
}
fn main() {
    repro();
}

Can you attach a small but fully actionable and reproducible snippet (along with the command to compile)? There might be more to investigate and that would be useful. Thanks!

[apiraino]

Which command and rustc flags do you use to compile the sources? Also, do you get an actual error stack? If yes, can you expand that error with RUST_BACKTRACE=full? Are you able to test against the latest LLVM 12?

[abbec]

Yeah, it does work with LLVM 12 but not with LLVM 11.1.

REPRO CASE:

use std::fs;

fn repro() {
    let s = fs::read("/absolute/path.txt").unwrap();
    println!("{:#?}", s);
}

fn main() {
    repro();
}

Run this

$ rustup toolchain install nightly-2021-03-11
$ rustup default nightly-2021-03-11
$ rustup target add wasm32-wasi
$ export CARGO_TARGET_WASM32_WASI_LINKER=/path/to/llvm-11/bin/lld
$ cargo build --target wasm32-wasi
$ echo "YES" > path.txt
$ wasmtime run --mapdir=absolute::. target/wasm32-wasi/debug/rust-repro.wasm
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Custom { kind: Other, error: "failed to find a pre-opened file descriptor through which \"/absolute/path.txt\" could be opened" }', src/main.rs:4:44

The same thing works fine with

$ rustup toolchain install nightly-2021-03-10
$ rustup default nightly-2021-03-10
$ rustup target add wasm32-wasi
$ export CARGO_TARGET_WASM32_WASI_LINKER=/path/to/llvm-11/bin/lld
$ cargo build --target wasm32-wasi
$ echo "YES" > path.txt
$ wasmtime run --mapdir=absolute::. target/wasm32-wasi/debug/rust-repro.wasm
[
    89,
    69,
    83,
    10,
]

[apiraino]

great work @abbec for creating a reproducible example.

pinging ICE breakers to try to identify where this started (need to be triaged with both LLVM installed), so also adding LLVM label

@rustbot ping icebreakers-cleanup-crew

@rustbot label +A-LLVM

[rustbot]

Hey Cleanup Crew ICE-breakers! This bug has been identified as a good
"Cleanup ICE-breaking candidate". In case it's useful, here are some
instructions for tackling these sorts of bugs. Maybe take a look?
Thanks! <3

cc @AminArria @camelid @chrissimpkins @contrun @DutchGhost @elshize @h-michael @HallerPatrick @hdhoang @hellow554 @henryboisdequin @imtsuki @JamesPatrickGill @kanru @KarlK90 @LeSeulArtichaut @MAdrianMattocks @matheus-consoli @mental32 @nmccarty @Noah-Kennedy @pard68 @PeytonT @pierreN @Redblueflame @RobbieClarken @RobertoSnap @robjtede @SarthakSingh31 @shekohex @sinato @smmalis37 @steffahn @Stupremee @tamuhey @turboladen @woshilapin @yerke

[rustbot]

Hey Cleanup Crew ICE-breakers! This bug has been identified as a good
"Cleanup ICE-breaking candidate". In case it's useful, here are some
instructions for tackling these sorts of bugs. Maybe take a look?
Thanks! <3

cc @AminArria @camelid @chrissimpkins @contrun @DutchGhost @elshize @h-michael @HallerPatrick @hdhoang @hellow554 @henryboisdequin @imtsuki @JamesPatrickGill @kanru @KarlK90 @LeSeulArtichaut @MAdrianMattocks @matheus-consoli @mental32 @nmccarty @Noah-Kennedy @pard68 @PeytonT @pierreN @Redblueflame @RobbieClarken @RobertoSnap @robjtede @SarthakSingh31 @shekohex @sinato @smmalis37 @steffahn @Stupremee @tamuhey @turboladen @woshilapin @yerke

[pietroalbini]

This seems to have been introduced in 1.52.0 nightly given the affected version. Relabeling as a stable-to-stable regression.

[joshtriplett]

cc @alexcrichton

[alexcrichton]

@abbec could you describe a bit why you're linking with a non-Rust LLD, and specifically an older-than-LLVM-12 LLD?

Given your range of nightlies for the regression this seems most likely due to #82924 which I could imagine relies on features and/or bug fixes in LLVM 12 to work.

In general the wasm targets right now are intended to work best with the LLD that Rust ships itself, and otherwise if you're looking to interop with C/C++ code it's recommended to say pretty up-to-date because there's still a good deal of feature development and bug fixes happening.

[abbec]

Yeah to be fair, this issue popped up in passing and I am now using LLVM 12 which works fine.

Reason for using an external linker is a custom libc and other C libraries.

[alexcrichton]

Ah ok, if that's the case I'd recommend closing this because the wasm target and toolchain are undergoing enough changes that not a lot of effort is put into making all versions of objects work with past and future toolchains. This will naturally evolve over time as wasm stabilizes, but at this time I think the guarantee of Rust is that if you want to intermix Rust & C on wasm you need to have matching LLVM versions.

[apiraino]

ok, closing this as per Alex's suggestion. @abbec please feel free to action this issue again in case something relevant happens again. Thanks again for your time on this issue!