Parcel Integration

[kazimuth]

The Parcel project recently released Rust support; it's now extremely easy to bundle rust into a production application. See the announcement of the release.

They have a good story for the calling-into-rust end:
add.rs:

#[no_mangle]
pub fn add(x: i32, y: i32) -> i32 {
   x + y
}

index.js:

import {add} from './add.rs';

console.log(add(1,2));

However, they don't have a good story for calling-out-of-rust the way stdweb does. Currently, when you use the js! macro, TypeError: second argument must be an object is logged to the console, and nothing happens - I suspect because of missing glue code.

Parcel seems to be using wasm32-unknown-unknown - the generated loading code doesn't do anything emscripten-y. I don't know what magic cargo-web does, but it's currently missing from parcel. If it could be integrated, that would be amazing.

[koute]

Yes, closer integration with JavaScript bundlers is something I plan to actively pursue Very Soon™. For the last two weeks I've been refactoring and rewriting a lot of cargo-web to make this feasible.

[kazimuth]

You can see the code they use to load rust code here, it's quite straightforward: https://github.com/parcel-bundler/parcel/blob/master/src/assets/RustAsset.js

It should be pretty easy to extend that to support cargo-web. The only problem I can forsee is detecting when to use cargo-web rather than cargo, because Parcel prefers a no-configuration approach. Maybe it should just look for Web.toml, or a dependency on stdweb failing that.

I'd be happy to make that PR, but if changes to cargo-web are incoming I can wait :)

[koute]

Yeah, that certainly looks pretty straightforward.

Hm... yep, there are a few ways you could detect that a crate needs cargo-web:

1. See if there is a Web.toml next to Cargo.toml.
2. Run cargo metadata --format-version 1, parse the JSON it spits out, and see if there is stdweb anywhere in the dependencies.

On a side note, something would probably have to be done with parcel-bundler/parcel#647 as we'll need to pass a bunch of imports into the WASM module for anything to work. (Unless we'd use a cargo-web-specific WASM loader instead of the generic one? I'm not familiar with Parcel's internals.)

[kazimuth]

Looking through their generated code, it seems to build a graph of modules that use simple require and module.exports calls that are then bundled up with a small require shim.

Mulling this over, it's probably best to generate both a js file and a wasm file, where the js file is responsible for loading the wasm file, and then recursively run the js file through their preprocessor.

That way you'll be able to have something like:

cow.js

module.exports = {
   moo: function() { console.log('moo!'); }
}

src/lib.rs

#[no_mangle]
pub fn do_thing() {
    js! {
        var moo = require("../cow.js").moo;
        moo();
    }
}

For this code, we can spit out a small file that looks like:

function initialize(instance) {
   var __imports = {
      __thing_needed_by_rust_runtime: function(x,y) { /* ... */ }
      __moo_js_code: function() {
        var moo = require("../cow.js").moo;
        moo();
     }
   };
   instance.instantiate(__imports);
   instance.do_initialization_things();
}

var instance = require('src/lib.rs-wasm-artifact.wasm')
initialize(instance);
module.exports = instance;

and then jury-rig Parcel to recursively run its JS processing on this file, which will be what you get when you require("src/lib.rs").

This would need parcel-bundler/parcel#647 to be implemented so that we would have access to the instantiate method of the WebAssembly module. We'd also need cargo-web to generate the initialize function. Maybe a command-line flag that changes the format of the cargo-web-generated js code? We can then create the rest of the template in Parcel. As a bonus, that flag could also be used in a WebPack loader or similar.

[koute]

I took a closer look at this, and it seems that right now it would only be possible to generate a module with async exports without having to modify Parcel.

We can easily work around Parcel's lack of support for specifying WASM imports by simply instantiating the WASM module ourselves (we can rename the .wasm to .wasm.raw, import it so that we get an URL to it, and then simply fetch and instantiate), however from what I can see there isn't really a way to specify that a given .js should be loaded asynchronously.

However, it looks like Parcel supports plugins which, from what I can tell, should allow us to get around the problem. I'll try to whip up an experimental plugin and see how it goes.

[koute]

I've just added a --runtime parameter to the build subcommand in cargo-web with which a non-standalone runtime can be generated to make it possible to use it with JS bundlers.

I've also started an experimental Parcel plugin which, while still missing some essential features, seems to work. I'll probably push it tomorrow once I'll get it a little more fleshed out.

[koute]

I'm happy to announce that I've made an experimental Parcel plugin which, surprisingly, seems to work! (Or at least I hope so...)

You can find it here:

https://github.com/koute/parcel-plugin-cargo-web

To use it you should just have to run npm install --save parcel-plugin-cargo-web in your project, and then you can use a normal JavaScript import on a Cargo.toml from a crate you want to include. The plugin assumes you have Rust installed, but otherwise it will install the nightly toolchain and cargo-web for you automatically. (And cargo-web will install the necessary target, so besides having to have Rust installed it's mostly zero-config.)

Unfortunately I had to resort to a quite... creative hack to make it work. (You can look at the sources if you're interested; the plugin itself is very small.) So please treat it as something experimental for now. Hopefully Parcel with support this kind of a thing natively in the future.

[kazimuth]

@koute, there actually is a way to make that work! See https://github.com/parcel-bundler/parcel/blob/master/src/Bundler.js#L38-L46, you can add a "bundle loader" that preloads certain kinds of modules.

I'm working on a PR against the main parcel repository now, and using that. The problem I'm running up against is that dynamically changing the output type of my module is causing Parcel some confusion (you've sidestepped this problem by having the user import Cargo.toml). Will probably open once I work that issue out.

[koute]

@koute, there actually is a way to make that work! See https://github.com/parcel-bundler/parcel/blob/master/src/Bundler.js#L38-L46, you can add a "bundle loader" that preloads certain kinds of modules.

Yes, I know. Unfortunately these are precisely the problem. The point is that (as far as I know) those bundle loaders must be preregistered (so you can't really add new ones dynamically), they have to be static (you register one bundle loader per artifact type and you can't change it), and they only have the access to .wasm's URL (which is not enough; you also need the .js runtime generated by cargo-web to instantiate the .wasm module.).

I've worked around this issue by replacing bundler.bundleLoaders with a Proxy object with a getter which dynamically generates new bundle loaders (one bundle loader per imported crate) with a hardcoded require for the .js runtime, and I add the .js runtime itself as a normal dependency.

[bgourlie]

Is there any existing parcel issue for addressing the rust specific concerns that we're hacking around? If not, I can try to get this on their radar and potentially implement a proper fix.

[koute]

@bgourlie AFAIK the issue isn't really Rust specific, and Emscripten folks have (AFAIK) exactly the same issues as we do.

The whole issue boils down to the fact that Parcel expects the WASM bundle loader (or any other one for that matter) to be static, but we can't use a static one since we need to supply our own imports for the WASM module instantiation and we need to do stuff like manually calling main after the WASM is instantiated. (For more details you can probably read both the comments in this issue and in the Parcel issue we've linked to in this thread.)

If you'd like to take a stab at fixing this properly you might want to ask @kazimuth how far he managed to get and perhaps talk with the Parcel folks how they would like it to be fixed.