Support init function without parameters

[ibaryshnikov]

Motivation

When used without a bundler, there is a need to call init with a .wasm file name:

await init('../pkg/my_project_bg.wasm');

and again

await init('../pkg/another_project_bg.wasm');

Proposed Solution

Make it possible to use

await init();

which will default to path with generated .wasm file

[fitzgen]

Do you have an idea of how to implement this?

While we know that the wasm is next to the JS glue by default, and we know how to go from the JS glue to the wasm with a relative URL path, we don't know where the whole pkg directory is hosted relative to the web page. And, IIUC, all requests are relative to the web page, not relative to the JS script's source that is making the request.

[ibaryshnikov]

@fitzgen what about import.meta?

function init(module) {
    let result;
    const imports = { './webgl_engine': __exports };

    if (typeof module === "undefined") {
        module = import.meta.url.replace(/\.js$/, '_bg.wasm');
    }

    if (module instanceof URL || typeof module === 'string' || module instanceof Request) {

[alexcrichton]

I think that's just a node-ism and wouldn't work in other locations? (and it's also only an es6 node-ism I think which is still somewhat experimental)

[ibaryshnikov]

What is node-ism? It's a part of es-modules, browser support according to MDN is Chrome 64, Firefox 62, Opera 51, Safari 11.1 (no information about Edge, but it will be based on Chromium anyway soon). It's possible to use import.meta for <script type="module" and document.currentScript otherwise. Inside <script type="module" document.currentScript is always null

[fitzgen]

https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Statements/import.meta

Looks like it is stage 3 right now. Yeah, with import.meta we could do this for the --web target.

[alexcrichton]

Oh nice! I didn't realize that and that sounds reasonable to me!

[jljorgenson18]

I'm running into problems parsing the module with import.meta. If you are using target --web but still using babel to compile (like if you want to pass the url yourself for the wasm dist inside of a Webpack dist), then this fails. Babel prompts you to install a syntax plugin but it still fails in Webpack. Ideally, there would be an option to turn the import.meta fallback off since it is not fully supported yet and it may cause parsing errors for a feature that may not even be used.

[jljorgenson18]

I was able to get around this by adding

@babel/plugin-syntax-import-meta and babel-plugin-bundled-import-meta (with the importStyle set to cjs)

[Pauan]

@jljorgenson18 The web target is intended to be used without a bundler.

I think you want the bundler target instead.

[jljorgenson18]

@Pauan The bundler option doesn't work at the moment because it relies on Webpack's internal Wasm parsing (which is extremely buggy and I have yet to see it work). Nearly all of the workarounds I've seen for Webpack failing to parse the wasm file end up using file-loader and then they pass the url to WebAssembly.instantiate themselves. If you opt for a solution that loads in a wasm distribution via url, then the web target is the best bet since you can just pass in the url string. I think ideally, there would be a middle ground target that is more agnostic to pure web modules and pure bundler options.

[Pauan]

I haven't had any issues with Webpack + Wasm. There used to be some bugs in the past, but they've been fixed. Make sure everything is up to date (Webpack, wasm-bindgen, wasm-pack, etc.)

If you're still running into issues, then you should file a bug report.

[jljorgenson18]

The initial issues I ran into on the wasm-bindgen side had to do with #1246 and #1564 which seemed to be fixed by webpack/webpack#8786.

However, with using tools like Emscripten, Webpack cannot handle the wasm files webpack/webpack#7352. So the solution is just to instantiate the module yourself with a bit of glue code and to use file-loader. If you are using wasm files generated from various tools, chances are at this stage that Webpack won't work well with all of them. You could dynamically change the loader using inline loader syntax or a combination of oneOf + resourceQueries, but at that point, what is the point of using Webpack to instantiate the wasm modules? If you want to have greater control over how the wasm bytes are imported while still using some of the nice javascript bindings, then the web option actually works really well other than the import.meta condition.

[Pauan]

Ah, right, Rust + emscripten has a lot of issues (in general, not just with Webpack). Well, I'm glad you were able to find a workaround at least.

[jljorgenson18]

Well emscripten was compiling a C module, it was just that the general wasm + emscripten compiliation does not get along well with Webpack (I'm guessing it's because of the extra env options needed). Because of that, you have to add a rule to overwrite the webassembly/experimental type with file-loader and then potentially create your own instantiate functionality.

[suyoshi]

Hello,

I run into one issue after the other trying to use wasm together with Angular.

target --web doesn't work, because webpack doesn't support import.meta
target --bundler doesn't work either, see above
target --no-modules doesn't work out of the box either, only with workarounds which is hacky

Please, consider adding an option to omit import.meta from --target web it would make things a lot easier.

[suyoshi]

FWIW, I was using target --web fine until I upgraded yesterday and #1579 broke it for me, will just downgrade for now.

[jljorgenson18]

@suyoshi If you are using babel, you can add @babel/plugin-syntax-import-meta and babel-plugin-bundled-import-meta (with the importStyle set to cjs)

[ibaryshnikov]

Hi @suyoshi , thanks for your report! I'm sorry for breaking things. Can you tell more details about your case so we could probably find a good solution? In particular, why the --target web is not working for you? Do you use a specific file-loader in webpack? Do you use both emscripten and wasm-bindgen on the same page?

[suyoshi]

Sorry for the late reply, I didn't get any notifications!

@ibaryshnikov
I'm using Angular for my frontend which is using webpack. Now webpack has the following unresolved issue webpack/webpack#6719 which is kinda stagnant and the reason why --target web does not work anymore since #1579. In particular this error webpack/webpack#6719 (comment).

It would be really great if wasm-bindgen could ship a workaround until it's fully supported.

[ibaryshnikov]

@suyoshi --target bundler should be fine, but as you mentioned here, webpack can't resolve .wasm extensions during the build because of the default angular config. I tried to manually add .wasm inside pkg, and it worked. I think it's a good time to revisit #1441, since it may become a proper fix

[suyoshi]

@ibaryshnikov Yeah, I know that. But even if webpack resolves it properly, I noticed that there's an even worse issue plaguing the bundler target.

Leaving alone the circular dependency warnings - the This is not allowed, because WebAssembly download and compilation must happen asynchronous. error makes it totally useless for me. 😞

I know how to work around it, but it prevents you from importing any types from the generated package.
E.g. this will blow up with the error from above:
import { SomeInterface } from "my-wasm-package"

The web target is overall just working better and more suitable for webpack, because of the aforementioned issues. It would be really great if you could consider a workaround for webpack/webpack#6719 since upstream is not making any progress. 🙏

[ibaryshnikov]

@suyoshi

error makes it totally useless for me

what makes it useless for your case? I mean, await init() still goes asynchronously... Will it be fine with await import('../path/to/module') instead?

[suyoshi]

Sorry, I should have explained it better. The problem is not that it's asynchronous.

This is possible with --target web:

import init, { SomeClass, SomeInterface} from "my-wasm-package";

private someClass: SomeClass;
private someObject: SomeInterface;
...
await init("path/to/module"); // only works without the `import.meta` change
...
this.someClass = new SomeClass();
this.someObject = this.someClass.someFunction();
...

But it's not possible with --target bundler because the error prevents importing SomeClass and SomeInterface which is what makes it useless in my eyes.

import { SomeClass, SomeInterface} from "my-wasm-package";
^
|-- This is not allowed, because WebAssembly download and compilation must happen asynchronous.

private someClass: SomeClass;
private someObject: SomeInterface;
...
import("path/to/module").then(module => {
    ...
});
...
...

I hope that explanation makes more sense!

[Pauan]

@suyoshi You should be able to do something like this:

private someClass: import("path/to/module").SomeClass;
private someObject: import("path/to/module").SomeInterface;
...
import("path/to/module").then(module => {
    ...
});

Ugly, but it will work. You could also tidy it up a bit like this:

type Foo = typeof import("path/to/module");

...

private someClass: Foo["SomeClass"];
private someObject: Foo["SomeInterface"];
...
import("path/to/module").then(module => {
    ...
});

[Pauan]

There's also another solution, which I think is much better.

You can import the .wasm file synchronously...

import { SomeClass, SomeInterface } from "path/to/module";

private someClass: SomeClass;
private someObject: SomeInterface;
...
this.someClass = new SomeClass();
this.someObject = this.someClass.someFunction();
...

(Note that there's no dynamic import() here)

...and then you can asynchronously import the main module in your src/main.ts:

import { enableProdMode } from '@angular/core';
import { platformBrowserDynamic } from '@angular/platform-browser-dynamic';

import { environment } from './environments/environment';

if (environment.production) {
  enableProdMode();
}

import('./app/app.module')
  .then(AppModule => platformBrowserDynamic().bootstrapModule(AppModule))
  .catch(err => console.error(err));

This should make Webpack happy (because there is a dynamic import()), but it allows you to access the .wasm file synchronously (no need for dynamic import() to load the .wasm file).

I recommend doing this rather than using --target web, because --target web is intended for use without a bundler, and so we don't make any guarantees that it will work with bundlers. So using --target bundler is recommended.

[suyoshi]

Hello @Pauan,

I tried the synchronous example, but just cannot get it to work:

WebAssembly module is included in initial chunk.
This is not allowed, because WebAssembly download and compilation must happen asynchronous.
Add an async splitpoint (i. e. import()) somewhere between your entrypoint and the WebAssembly module:

It happens because of this line (path/to/module being my wasm package) since it triggers
import * as wasm from './my_wasm_package.wasm'.

What is path/to/module in your example?
import { SomeClass, SomeInterface } from "path/to/module";

About the first example, it kinda works, but is really ugly as you said. As long as the web target works, I don't see why this should be preferred.

[Pauan]

@suyoshi In your code example you used path/to/module, so you know better than me what it means. The path should be the path to the wasm-bindgen generated code.

Did you change src/main.ts as I suggested, making the './app/app.module' import asynchronous? You should be able to just copy-paste the code entirely, it's standard Angular boilerplate.

As long as the web target works, I don't see why this should be preferred.

Because 1) it doesn't work, as you have noticed, and 2) there's no guarantee it will work in the future, because it wasn't designed for this use case.

In addition, my suggestion of importing the .wasm file synchronously is even better than the web target, since you don't need to use await init("path/to/module")

[suyoshi]

@Pauan Yes, path/to/module is the generated wasm-bindgen code. I asked, because it doesn't work for me (error above).

And I also changed the src/main.ts. It only works if I completely omit the import { ... } from "path/to/module" (which isn't part of src/main.ts).

[ibaryshnikov]

@suyoshi since #1646 was merged, you should be able to try wasm-pack build --target bundler on next release (or simply wasm-pack build, because bundler is a default target), and check if it works for you

[Pauan]

@suyoshi Ah, there was a small error in my code. Here is the correct code:

import { enableProdMode } from '@angular/core';
import { platformBrowserDynamic } from '@angular/platform-browser-dynamic';

import { environment } from './environments/environment';

if (environment.production) {
  enableProdMode();
}

import('./app/app.module')
  .then(x => platformBrowserDynamic().bootstrapModule(x.AppModule))
  .catch(err => console.error(err));

I created a new Angular project and verified that the above does indeed work perfectly: I can load the wasm-bindgen files synchronously and everything works great:

import { Component } from '@angular/core';
import { add } from "./wasm/index";

@Component({
  selector: 'app-root',
  templateUrl: './app.component.html',
  styleUrls: ['./app.component.css']
})
export class AppComponent {
  title = '' + add(1, 5);
}

[suyoshi]

@Pauan @ibaryshnikov You guys rock! The last approach seems to work fairly well. Many thanks. 👍

Just one minor thing (I hope). It results in two warnings:

WARNING in Circular dependency detected:
../../../../git/my-wasm-pack/pkg/my_wasm_pack.js -> ../../../../git/my-wasm-pack/pkg/my_wasm_pack_bg.wasm -> ../../../../git/my-wasm-pack/pkg/my_wasm_pack.js

WARNING in Circular dependency detected:
../../../../git/my-wasm-pack/pkg/my_wasm_pack_bg.wasm -> ../../../../git/my-wasm-pack/pkg/my_wasm_pack.js -> ../../../../git/my-wasm-pack/pkg/my_wasm_pack_bg.wasm

WARNING in Lazy routes discovery is not enabled. Because there is neither an entryModule nor a statically analyzable bootstrap code in the main file.
** Angular Live Development Server is listening on localhost:4200, open your browser on http://localhost:4200/ **
ℹ ｢wdm｣: Compiled with warnings.

But those are negligible for now I guess, since everything seems to work as it should.

Sorry for all the noise!

[tonimitrevski]

Hi guys, I'm wondering In Angular when we are importing wasm package, can we get rid of these warnings

WARNING in Circular dependency detected:
src\app\pkg\wasm_game_of_life.js -> src\app\pkg\wasm_game_of_life_bg.wasm -> src\app\pkg\wasm_game_of_life.js

WARNING in Circular dependency detected:
src\app\pkg\wasm_game_of_life_bg.wasm -> src\app\pkg\wasm_game_of_life.js -> src\app\pkg\wasm_game_of_life_bg.wasm

Thanks

[tonimitrevski]

And btw I received another error on prod building, od develop building everything is working fine

ERROR Error: Uncaught (in promise): TypeError: Failed to execute 'compile' on 'WebAssembly': Incorrect response MIME type. Expected 'application/wasm'.
TypeError: Failed to execute 'compile' on 'WebAssembly': Incorrect response MIME type. Expected 'application/wasm'.

[ibaryshnikov]

@tonimitrevski it's a false positive about circular dependencies since it's not possible to import javascript from wasm atm, it just happened that imports object used during instantiateStreaming contains the js file name as a key, and the same key is used inside wasm to address the functions. I'm not sure if there's a need to fix it on the wasm-bindgen side, but it's definitely worth opening a separate issue.

As a workaround, is it possible to skip certain imports in circular dependency detector? I had the same warning while building my app with webpack, but I simply ignored it.

Incorrect mime type is a common error. As error says, the file server should use a correct mime type for .wasm files. What do you use to serve statics?

[tonimitrevski]

@ibaryshnikov Thank you for the answer.
For circular dependency detector, I already skip, in angular.json architect: {build: {options: "showCircularDependencies": true}}, but This is a bit insecure because for all cases will be ignored (even if exist real circular dependency)
For the issue "Incorrect mime type is a common error", the problem was because I used PHP server, and with Node server, it's working excellent
Thanks for the help

[ibaryshnikov]

@tonimitrevski to continue about circular dependencies, if you use something like this plugin https://github.com/aackerman/circular-dependency-plugin there's an option to exclude certain files, for example

// webpack.config.js
const CircularDependencyPlugin = require('circular-dependency-plugin')

module.exports = {
  entry: "./src/index",
  plugins: [
    new CircularDependencyPlugin({
      exclude: /\.wasm/,

Angular doesn't expose webpack config, but maybe there's an equivalent option

[Pauan]

It's possible to create a custom Webpack config for Angular: https://netbasal.com/customize-webpack-configuration-in-your-angular-application-d09683f6bd22

[tonimitrevski]

Guys thank you very much I fixed the problem, appreciate your help 💪 💪 💪

[arjanvaneersel]

I ran into the same issue and followed the instructions of @Pauan. This does allow me to compile the app, but the app won't start, because of the following error: TypeError: "process.version is undefined" js _stream_writable.js:57 Webpack 33 main.ts:16:24

My main.ts looks like this:

import { enableProdMode } from '@angular/core';
import { platformBrowserDynamic } from '@angular/platform-browser-dynamic';

// import { AppModule } from './app/app.module';
import { environment } from './environments/environment';

if (environment.production) {
  enableProdMode();
}

// platformBrowserDynamic().bootstrapModule(AppModule)
//   .catch(err => console.error(err));

import('./app/app.module')
  .then(x => platformBrowserDynamic().bootstrapModule(x.AppModule))
  .catch(err => console.error(err));

[ibaryshnikov]

@arjanvaneersel could you also attach the version of wasm-bindgen? What target do you use when building with wasm-pack? Since there was an update in wasm-bindgen where it started to use extension while importing wasm like

import wasm from './my_module_bg.wasm';

there is no need in workarounds, it should work out of the box with wasm-pack build --target bundler

[arjanvaneersel]

@ibaryshnikov The version of bindgen was 0.2.34. I now updated it to 0.2.58 to see if that has any effect. I didn't specify any target, so it should use bundler as that's the default if I'm not mistaking.

The new version indeed takes care of the import problem, but I do still get this error:

WebAssembly module is included in initial chunk.
This is not allowed, because WebAssembly download and compilation must happen asynchronous.
Add an async splitpoint (i. e. import()) somewhere between your entrypoint and the WebAssembly module

Hence my impression that I still need to change main.ts. For that I used the above mentioned instructions of @Pauan. However, that way doesn't seem to work anymore. I'm using Angular 8 BTW.

[arjanvaneersel]

@ibaryshnikov The problem is somewhere else. I created a fresh Angular project and imported the wasm module created via the latest bindgen and followed the instructions to change main.ts as described above. And it works like a charm.

Hence something else in my code is causing this error.

[Pauan]

@arjanvaneersel Sounds like you (or one of your dependencies) is incorrectly assuming that it's running in Node, so it's trying to look up process.version and failing.

[minagawah]

Although it is absolutely irrelevant to the thread for I use wasm-pack to integrate with React, as @jljorgenson18 mentioned, I am using babel-plugin-bundled-import-meta to handle import.meta syntax, and it works:

# babel.config.js

  plugins: [
    [
      'babel-plugin-bundled-import-meta',
      {
        bundleDir: 'server/static/dist/wasm',
        importStyle: 'cjs',
      },
    ],
  ],

I have a working example, and it uses wasm-loader for .wasm files (while the example is CRA, it also works with manually wired React apps as long as I bundle with Webpack and use --target web). It may not solve your problem with Angular, but may help some of you who are having the same issue with React.