Status of rust+C bindings for wasm targets

[elichai]

Hi,
There are a lot of important dependencies in the rust ecosystem that use ffi to C fia the cc crate or similiar.
which tools and targets currently support that?
in the past asmjs-unknown-emscripten worked and together with cargo-web you could just run cargo web test to know if your library compiles+run on wasm/emscripten.
Sadly emscripten is broken on stable for a while now rust-lang/rust#66916.

wasm32-unknown-unknown works in combination with clang-8 only(probably a bug), not before and not later(doesn't work on clang-9 and clang-10) rust-lang/cc-rs#378
so wasm-pack can't be used here.

wasi seem to have the same problem, running cargo wasi test fails to find the sysroot(fatal error: 'string.h' file not found) although it seems like you can download sysroots for wasi or compile them yourself https://bytecodealliance.github.io/wasmtime/wasm-c.html

Is there any tool out there that can test rust+C code out of the box? (We had emscripten tests in the CI until it broke and we want to continue testing for wasm target in the CI)

[kripken]

@tlively has made a lot of progress on the emscripten side of things, and I think can update on the status there.

[tlively]

Currently the wasm32-unknown-emscripten and asmjs-unknown-emscripten targets are the only targets that can be linked with C code because all the other wasm targets use a different ABI for compatibility with wasm-bindgen. Unfortunately, I’m the only one maintaining the emscripten targets in Rust and I don’t have a lot of time to spend on it, so I’ve mostly just been trying to fix critical bugs as I become aware of them. I’ll take a look at the one you linked to above.

[DzmitryFil]

I am also struggling to compile simple c libraries like imgui to wasm.
@alexcrichton maybe you could help us, and share couple of words on how to build wasm32-unknown-unknown with wasi sysroot, or some other way? Or building rust+C to wasm is a lost cause for near future?

[alexcrichton]

If you want to integrate Rust and C on the web your best bet is likely going to be Emscripten. The next best bet is going to be using WASI (and wasi-sdk), but that requires getting a WASI implementation working on the best which isn't always as available. Finally you can try to get wasm32-unknown-unknown working. While that's technically possible for C/C++ there aren't really any maintained toolchains targeting that for C/C++, so you're on your own making such a toolchain.

[DzmitryFil]

Well, wasm32-emscripten target doesn't get much love from rust community either. Anyway, thanks for reply, it's nice to know status of these things.

[RReverser]

Hi, I'm a WebAssembly DevRel at Google and thought I'd chime as I've actively worked with both Rust (including wasm-bindgen) and Emscripten at various points.

Admittedly, this is a simple demo, but, as far as I can tell, wasm32-unknown-emscripten target works fine for combined Rust + C code.

Of course, you can't expect to use both #[wasm_bindgen] and Emscripten bindings at the same time - you need to choose a single tool that will be providing runtime bindings and generating the JavaScript wrapper - but as long as one side is a "driver" and another contains pure portable code, things should work.

Is there a specific API or minimal code sample you could show that you're struggling with?

[DzmitryFil]

Wow, thanks for offering help. My problem is compiling imgui-rs crate to wasm32-unknown-unknown target, i have no idea how to solve this, because my c++ knowledge is pretty limited.
As of emsripten, stuff like rust-lang/rust#66916 doesn't give much confidence. Also stdweb, which supports emscripten target, seems abandoned (no activity for 8 months).
Also, I do you use wasm_bindgen quite extensively. For example, how would i replace wasm-bindgen-futures crate and work with js promises from inside rust, if i choose to use wasm32-unknown-emscripten?

[RReverser]

I can't say much about cargo-web TBH because while I've seen it, personally never used it. In example above you can see that I'm just using regular cargo build command instead.

As of emsripten, stuff like rust-lang/rust#66916 doesn't give much confidence.

I haven't seen that error, and given that 1) it's from half a year ago and 2) examples I've tried work fine, I suspect it might have been some temporary issue that has been resolved since.

Also, I do you use wasm_bindgen quite extensively. For example, how would i replace wasm-bindgen-futures crate and work with js promises from inside rust, if i choose to use wasm32-unknown-emscripten?

Right, unfortunately, you can't, since these are two different toolchains. You need to either deal with Web APIs and JS values from Emscripten, or from Rust, but not both.

I've experimented in the past with using Embind in Rust (shameless plug: https://www.youtube.com/watch?v=zxIbTfsOJZE), but not too long afterwards wasm-bindgen appeared and it wasn't clear if there are benefits to using both, so never made it into a full package. Either way, if really necessary, it should be possible to build your own bindings from Rust to Emscripten's APIs or vice versa, but that would be complicated.

It might be easier to understand what issues you're running into with wasm32-unknown-unknown for your particular library, but that's probably best discussed in an issue on imgui-rs repo where author could chime in too and probably help out.

[DzmitryFil]

Yeah, i probably could successfully compile rust with emscripten, but wasm-bindgen absence seems like a showstopper.

It might be easier to understand what issues you're running into with wasm32-unknown-unknown for your particular library, but that's probably best discussed in an issue on imgui-rs repo where author could chime in too and probably help out.

The thing is, my issues aren't specific to this library. When you compile any c library that uses standard headers like stdio.h or string.h you get compilation errors.
servo/rust-stb-image#94
rust-lang/cc-rs#378 (comment)
I tried manually inserting headers from wasi-libc, and including them into cc build, but then i got linking errors like

note: rust-lld: error: unable to find library -lstdc++

Imgui doesn't really need any os-specific stuff to work, i've seen it compiled with emscripten on the web, but i have no idea how to make it work with wasm32-unknown-unknown.

[RReverser]

When you compile any c library that uses standard headers like stdio.h or string.h you get compilation errors.

Yeah that's true, for that you need some minimal stdlib and it's quite unfortunate that wasm32-unknown-unknown in Clang doesn't include any.

That said, this situation is not unique to WebAssembly, but also happens in embedded targets which often don't have full stdlib either. I've found that often enough you can send PRs and make upstream code rely only on portable headers. E.g. you can see this commit on libdeflate where I made some of the headers-related changes: ebiggers/libdeflate@27d5a74

Alternatively, in other cases, you can build C / C++ against wasm32-wasi target instead. If you know that you won't be calling any actual I/O APIs anyway and only need the basics like string.h, then this is often a simple enough option that just works.

[DzmitryFil]

I've found that often enough you can send PRs and make upstream code rely only on portable headers.

It's probably possible, but it seems like too much work to do for the whole imgui.

If you know that you won't be calling any actual I/O APIs anyway and only need the basics like string.h, then this is often a simple enough option that just works.

Well, i just tried to build imgui-rs to wasm32-wasi, and i get exactly the same errors about not having string.h and friends :)
Also if i could compile to wasm32-wasi, could i run it inside browser?

[RReverser]

Well, i just tried to build imgui-rs to wasm32-wasi, and i get exactly the same errors about not having string.h and friends :)

That's weird, wasm32-wasi definitely has these libraries. Did you install and use WASI SDK that was mentioned above?

Also if i could compile to wasm32-wasi, could i run it inside browser?

As I said, only if you're sure that no I/O will be called and you only need basic headers and functions, otherwise things get a bit more complicated.

[DzmitryFil]

That's weird, wasm32-wasi definitely has these libraries. Did you install and use WASI SDK that was mentioned above?

I simply used cargo wasi build, and got the same stdlib errors.
Forgive my ignorance, but i'm not sure how would i use wasi sdk to build rust+c code?

[tlively]

Well, i just tried to build imgui-rs to wasm32-wasi, and i get exactly the same errors about not having string.h and friends :)

That's weird, wasm32-wasi definitely has these libraries. Did you install and use WASI SDK that was mentioned above?

So does Emscripten, for that matter. The tricky bit here seems to be that the rust toolchain driver projects do not know how to properly build C/C++ libraries targeting Emscripten (or WASI??). It's more effort, but it should work to build the C/C++ static library separately using Emscripten then pass that library to rustc to link in.

[RReverser]

So does Emscripten, for that matter.

Yeah but the difference is that Emscripten also generates surrounding runtime JS code, which we explicitly don't need or want in this case, so WASI SDK is a more lightweight option.

[DzmitryFil]

It's more effort, but it should work to build the C/C++ static library separately using Emscripten then pass that library to rustc to link in

according to this rustwasm/wasm-pack#621 (comment)
emscripten compiler would be incompatible with wasm32-unknown-unknown target
my goal is to build wasm32-unknown-unknown (or wasi?) target, use wasm-bindgen, and still have the ability to link to c library.

WASI SDK is a more lightweight option.

As far as i understand, i should use clang directly to build imgui cpp source code with sysroot from wasi-sdk, and later link that to rust?
Can you give me more specific instructions please?

[DzmitryFil]

I just tried to build wasm32-wasi target passing --sysroot=[path_to_my_build_of_wasi_sysroot] flag to cc::Build, and still i get

• rust-lld: error: unable to find library -lstdc++

[tlively]

my goal is to build wasm32-unknown-unknown (or wasi?) target, use wasm-bindgen, and still have the ability to link to c library.

You cannot do this.

The only way to mix Rust and C code when targeting WebAssembly is to use the wasm32-unknown-emscripten target. You cannot use wasm-bindgen when you do that because wasm-bindgen is fundamentally incompatible with C. Using WASI will not work. There is no workaround. There are long term plans to make wasm32-unknown-unknown and wasm32-wasi compatible with C, but there is no ETA for that.

Source: https://github.com/rust-lang/rust/blob/master/src/librustc_target/abi/call/mod.rs#L599-L601

[RReverser]

The only way to mix Rust and C code when targeting WebAssembly is to use the wasm32-unknown-emscripten target. You cannot use wasm-bindgen when you do that because wasm-bindgen is fundamentally incompatible with C.

This is definitely not true. As I mentioned above, you can go the other way around as well, as long as your C / C++ is not doing any I/O that requires corresponding JS bindings - this is the only limitation that is hard to work around.

See the libdeflate example above as well which is written in C, but I've used it in a Rust project with wasm-bindgen in Squoosh.

[RReverser]

Here's a slightly modified variation of my example above, that uses wasm-bindgen instead of Emscripten for JS interactions, but still combines Rust + C in the same way:

[tlively]

@RReverser only the wasm32-unknown-emscripten target uses the real C ABI for FFI. All the other targets, including WASI, use the different wasm32_bindgen_compat ABI. You've been lucky that your C interop has happened to work, but in general this will not be the case.

[RReverser]

@tlively Ah, I've missed that nuance. From looking at implementation, it seems that the primary difference is in layout of aggregate parameters / results (structs by value and such)? That's probably why I've never run into issues with it before, but I agree it's risky.

[tlively]

I have a PR to document this problem that points back to this issue here: rustwasm/wasm-bindgen#2209.

[TheBlueMatt]

Yes, it has nothing to do with malloc/no_std/etc, it has to do with the ABI of passing structs by ownership to functions.

[GregoryConrad]

For compiling to wasm32-wasi, I think it would be worthwhile to recommend the use of the wasi-sdk directly:

# Script assumes you have clang, cmake, and ninja available locally
# Also note I needed mmap support, so I added -D_WASI_EMULATED_MMAN and -lwasi-emulated-mman below

# Set up wasi-sdk
WASI_SDK_VERSION_TAG=wasi-sdk-17
WASI_SDK_ROOT=wasi-sdk
git clone --recursive --branch $WASI_SDK_VERSION_TAG https://github.com/WebAssembly/wasi-sdk $WASI_SDK_ROOT
(cd $WASI_SDK_ROOT && make package)

# Configure environment variables for cargo build
WASI_SDK_PATH=$WASI_SDK_ROOT/build/install/opt/wasi-sdk
WASI_SDK_PATH=`realpath $WASI_SDK_PATH`
export CC="$WASI_SDK_PATH/bin/clang -D_WASI_EMULATED_MMAN --sysroot=$WASI_SDK_PATH/share/wasi-sysroot"
export LLD="$WASI_SDK_PATH/bin/lld -lwasi-emulated-mman"
export LD="$LLD"
export AR="$WASI_SDK_PATH/bin/llvm-ar"
export NM="$WASI_SDK_PATH/bin/llvm-nm"

# Build for web
TARGET=wasm32-wasi
rustup target add $TARGET
cargo build -r --target=$TARGET

[NicholasLYang]

I've been trying to get a Rust codebase that uses tree-sitter to compile to Wasm. I eventually got it to work using Zig's toolchain for C compilation:

CC="zig cc" cargo build --target wasm32-wasi

However, it appears that the wasm file expects the tree-sitter code to be imported via the env import:

Error: failed to run main module `target/wasm32-wasi/debug/vicuna.wasm`

Caused by:
    0: failed to instantiate "target/wasm32-wasi/debug/vicuna.wasm"
    1: unknown import: `env::ts_parser_parse` has not been defined

Does anybody know if there's some way to link it all together so that the .wasm file doesn't need this import?

[chrysn]

You may want to check whether that problem also arises when just using C code. My experience (at least with using the cc crate to build) is that things Just Work if all the symbols were provided are inside object files that are made known to cargo. In parallel, you could check which precise symbols you are missing in the wasm file (running wasm-dis from binaryen on your wasm files and searching for "env" will show them) -- maybe things do work generally anyway, and you're just missing an individual symbol.

[NicholasLYang]

Thanks for the response! I'm not sure what you mean by just using C code. Do you mean when running natively? Because it does work then. I suppose I can try building this with a stub of C code and see if it works with webassembly.

[amckinney]

@NicholasLYang I’m actually trying to do something very similar with rust and tree-sitter - were you able to figure anything out?

[kothavade]

@amckinney did you ever figure it out? currently struggling with rust wasm and tree-sitter

[shadaj]

@vedkothavade try with https://github.com/shadaj/tree-sitter-c2rust! This is how rust-sitter supports wasm32-unknown-unknown.

[kothavade]

thanks @shadaj! i was able to get tree-sitter-c2rust itself to build, but languages themselves fail to compile (eg. tree-sitter-c). i locally cloned the language repo and replaced the tree-sitter dep in Cargo.toml and build.rs with tree-sitter-c2rust and still am getting errors such as (stdbool.h not found) at build time--any suggestions?

edit: actually, i think that the clang errors i'm getting might be in part due to some nix weirdness going on, will have to investigate further--let me know if you've seen any CC errors when building languages though, maybe it will help

[shadaj]

@vedkothavade sorry for the late response! Yeah, we've seen this in Rust Sitter. We get around this by including minimal stub implementations of those headers, see https://github.com/hydro-project/rust-sitter/blob/main/tool/src/lib.rs#L67.

[kothavade]

@shadaj thanks!

[rafaelbeckel]

If you arrived here while searching how to compile Rust+C for the wasm32-unknown-unknown target, it's currently possible with Rust nightly with the --Z wasm_c_abi=spec flag they introduced in April. Here's a minimal example.