Add support for wasm32-unknown-unknown

[K4rakara]

At the moment, attempting to build for wasm32-unknown-unknown causes the following panic: don't know how to build Lua for wasm32-unknown-unknown.

I know running Lua in wasm isn't the most efficient thing in the world, but I feel like the functionality should be added.

I'd be happy to look into this, I just want to get the OK before I go and implement it.

[khvzak]

The panic is generated by the lua-src crate. So it should be a first place for a fix.
Personally, I don't have any objections having lua for the wasm32 target, but have feeling that this could be quite complicated.

[lenscas]

There seem to be 3 main projects to get lua working on wasm
https://github.com/vvanders/wasm_lua
https://github.com/serprex/luwa
https://github.com/ceifa/wasmoon

with https://github.com/ceifa/wasmoon seeming to be the only one that recently had changes

[makspll]

I would love to see this implemented! Could somebody summarise the necessary tasks needing to be done for this to run on wassm ?

[lenscas]

I would love to see this implemented! Could somebody summarise the necessary tasks needing to be done for this to run on wassm ?

Step 1: get Lua to compile to WASM or use one of the existing projects for that.

Step 2: find a way to correctly compile said lua VM to wasm together with mlua. Presumably you want them to get compiled in such a way that they share the same WASM memory (So... probably need to end up with 1 wasm file?) to reduce the overhead of FFI.

Step 3: Find some way to get mlua to communicate with the bundled lua. (I am assuming that the normal API's won't work, or that at least some extra plumbing is required.)

Also, there are JS implementations of Lua. Those might be easier to get working but are probably slower than a version compiled in wasm, especially if the wasm version can use the same memory as the rest of the program. https://github.com/fengari-lua/fengari-web is used by the teal playground.

Also, assuming you indeed want to reduce FFI overhead as much as possible and thus make the luavm use the same memory as the rest of the program, then I would imagine that at most only 1 instance of the VM is running. As a result, the functions to create a new Lua instance should probably be disabled and instead replaced with one to get a global instance.

[makspll]

Thanks @lenscas!

Hmm would having many lua vm's be very bad in terms of efficiency? ideally the API stays the same between targets, sharing VM's between scripts is very messy and foot-gunny, unless it's possible to somehow blackbox each script within the vm

[lenscas]

I would expect that if you want to spawn new VM's that they all have their own memory. Which means that the only way to "share" memory is by copying it over. I'm also not sure if 2 WASM modules can communicate directly with each other. If they can't, you are looking at going through JS. Which means that every value needs to be copied twice.

Lua itself has ways to do sandboxing. Though doing that properly is of course not the easiest.

[coderedart]

I will try to add some info.
as @lenscas mentioned, fengari is just a lua vm implemented in JS. wasmoon is a wrapper around lua54 C VM compiled into wasm, and drives it using JS.

anyway, both of them are useless for our usecase. if you separate out the lua vm from the main rust app as two individual wasm modules, there's nothing for mlua to do. both modules need to share memory and communicate with each other smh, but its more like a lua app and rust app talking to each other and not "embedding" lua into rust app.

I cannot know for sure, but you should be able to embed a lua vm inside rust wasm (no JIt though). there's two ways:
First, use a rust lua VM like Hematita. this should allow us to use wasm32-unknown-unknown target + wasm_bindgen (which allows rust <-> web JS interactions. used by most tools like trunk or eframe).
BUT apparently, the ABI used is not C ABI compatible. and fixing it would be breaking change for wasm_bindgen and its not happening anytime soon. more info rustwasm/wasm-bindgen#2209 . i am not completely sure if hematita will help, gotta ask the developer himself, but the project is being put on hold for the moment.

Second, use a C lua VM (bindings) and compile it to wasm. like i said, wasm32-unknown-unknown toolchain is not available for C code. so, we must use wasm32-unknown-emscripten which brings us to the issue rustwasm/team#291 . this seems like the most doable thing at the moment.

now, my C/C++ (or cmake) skills are absolute garbage, so, dealing with lua54's building process (by copying from wasmoon) is too hard for me. BUT, fortunately, luau_src crate used cmake and i was able to successfully do a hello world example using mlua + luau using emscripten target with basically zero effort.

1. just clone mlua repo.
2. add a basic main.rs in the src folder

use mlua::Lua;

fn main() {
    println!("inside main");
    for i in 0..5 {
        Lua::new()
            .load(&format!("print('hello lua, times: {i}')"))
            .exec()
            .expect("failed to execute lua");
    }
}

3. compile the above with cargo build --features=luau,vendored --target=wasm32-unknown-emscripten to produce a mlua.js and mlua.wasm files.
4. create a basic html page in crate root folder with python -m http.server. don't forget to use the right path for the mlua.js script!!!

<script src="./target/wasm32-unknown-emscripten/debug/mlua.js"></script>
<h1>Hello World</h1>

5. check console logs to see if there's the hello lua output

now, technically, lua54 is also doable because wasmoon can do it. but i will leave that task to someone who can understand what's happening in https://github.com/ceifa/wasmoon/blob/main/build.sh .

just to point out again, this issue is about compilation to wasm32-unknown-unknown which is not possible due to ABI issues (which will be fixed at some point according to the linked issue). i don't really understand these things completely, but if you want to know more, there's zulip for wasmtime which has nice helpful people, and there's rust-wasm team channel on officlal rust discord server.

PS: #40 seems to be a duplicate of this.

[coderedart]

a lot of crates usually only support wasm32-unknown-unknown and seems like winit does the same 😿 which makes this useless for embedding in apps / games.
but emscripten already supports GLFW / SDL to a decent extent https://emscripten.org/docs/porting/multimedia_and_graphics/OpenGL-support.html#opengl-es-extensions , so if rust SDL2 library gets emscripten support Rust-SDL2/rust-sdl2#884 this will be tremendously useful for me everyone who wants lua scripting on rust web apps / games.

[makspll]

@coderedart fantastic and thorough exploration, thank you!

winit emscripten support seems to be a no-go, and so for engines like bevy I imagine it's an unlikely solution, so it looks like the only way for this to be useful in engines like bevy is via this issue

• Document wasm-bindgen incompatibility with C/C++ rustwasm/wasm-bindgen#2209

[coderedart]

reply in that thread

Nope, there hasn't been any progress :( Everyone working in this space has been busy with other things.

i guess this issue will go back to hibernation. i will just keep my eye on hematita.

[khvzak]

I plan soon to start porting Lua 5.3 from C to Rust which should bring wasm32-unknown-unknown as well as no_std (hopefully) support to mlua.
Seems it's the only way (apart from rewriting from scratch).
Why Lua 5.3? Because it's fairly modern, fast and less complex than Lua 5.4 (20k vs 24k LoC).
Also the goal is to keep C API layer backward compatible.

i will just keep my eye on hematita.

hematita is mostly a POC than a real implementation. It even does not has GC support.

[coderedart]

I am happy to get anything, but just wanted to know why lua 5.3 instead of 5.1 (which works on luau / luajit)? the whole internet including apps like mpv seem to stick to 5.1 / 5.2 because of backwards compatibility.

[lenscas]

I rather see 5.3 on the Web than 5.1.

There js a compatible library that brings 5.1 and luaJIT to 5.3 compatibility wise. I don't think something similar exists for 5.3 to 5.1.

[makspll]

That would be interesting, but also sounds like a big project,

Are there any attempts at bringing lua to pure rust which are somewhat complete? It's probably worth reaching out to people who have tried this before.

[coderedart]

https://github.com/cjneidhart/lua-in-rust

Seems to be a wip lua impl

[khvzak]

There are many attempts to implement Lua on pure Rust but unfortunately all of them are not finished and/or ready for production use.
Currently I work on my own implementation of Lua 5.3 and hope next version of mlua will include it. I take a look of what is done to get some ideas.

Unfortunately Lua is not very easy to port/rewrite. There are lots of macros in the source code and tricks.

[coderedart]

tracking issue for C ABI and Wasm ABI merge. rust-lang/rust#83788

it seems the work is primarily blocked by clang's multi value return types support in C ABI for wasm.

[Auxnon]

There are many attempts to implement Lua on pure Rust but unfortunately all of them are not finished and/or ready for production use. Currently I work on my own implementation of Lua 5.3 and hope next version of mlua will include it. I take a look of what is done to get some ideas.

Unfortunately Lua is not very easy to port/rewrite. There are lots of macros in the source code and tricks.

I was eager enough to get this fixed I started trying to translate lua 5.4.4 to rust before seeing this. Do you welcome PRs to help with the drudgery where possible?

[coderedart]

FWIW, I was able to get both sdl2 working along with mlua on the emscripten target :)

a very minimal example https://github.com/coderedart/rust-sdl2-wasm is now live at https://coderedart.github.io/rust-sdl2-wasm/
you should see a rect with changing color and some lua code on top.
you can edit the lua code by typing and using backspace. no support for newlines, text wrapping or anything.
click anywhere on canvas to run lua code.

1. go to the above link.
2. open web developer tools -> console
3. click anywhere on canvas.
4. you should see print('hello') as that is the default lua code. change it to something else and print again.
5. the state is retained, so you can use local x = 5; print(x) etc.. as a repl

sdl2 basically works without us doing much.
glfw crate is harder because emscripten uses older versions of glfw and is missing polyfills for a lot of newer apis. will work as long as you don't use the newer api functions (which will trigger linker error for undefined symbols).

I used Luau backend as it works perfectly. this will be fine until we get a pure lua implementation of rust.

[coderedart]

Finally, It is done.
egui + sdl2 + emscripten + mlua + Luau + glow
all of them wrapped with a tiny little example inside 50 lines of main.rs
here's a sample:

Untitled.mp4

you can play with it too. live version @ https://coderedart.github.io/rust-sdl2-wasm/
source code @ https://github.com/coderedart/rust-sdl2-wasm . much simpler than you would expect :D
another live version with glfw @ https://coderedart.github.io/egui_glow_glfw_emscripten/

the next step is to wait for mlua 0.9 when we can actually script egui with lua live inside a browser.

[tari]

mlua-rs/lua-src-rs#7 made lua-src build for emscripten targets, and using the changes in #337 I've been able to build an application embedding Lua 5.4 for wasm32-unknown-emscripten. Luau worked without any changes as well.

[Andrewp2]

Does rust-lang/rust#117919 -Zwasm-c-abi matter at all for this issue?