Most performant way to pass data JS/WASM context

[kyr0]

Hi,

lets imagine I have some 1024 f32 values stored in some Float32Array buffer, waiting to be processed by WASM based DSP code :)

I'm still pretty new to WebAssembly. I understood that you can pass only typed numeric values as arguments to exported WASM functions. That makes sense to me and that's why I decided to pass my data using memory. I'm fine with that as well...

So, to pass my 1024 values, I assign them to the .memory directly. Like:

const mem = exports.memory.buffer;
const F32 = new Float32Array(mem);
F32[0] = 31337.777;

This is fun, but to assign all my values, I have to loop over all of the values to assign them all to the memory. Well, somehow that feels wrong performance-wise. I would have expected a native impl. to do that for me. e.g. a key like data in the memoryDescriptor argument of the WebAssembly.Memory constructor allowing to initialize the memory with an ArrayBuffer.

So, well, then I do my WASM function call and when the WASM impl. did it's magic, it's writing the result values back to the memory. This is happening inside of the DSP loop, so there is no overhead inside my WASM code to do that - as far as I can see.

But now, after I'm back in JS context, I have to iterate over the whole memory again just to read all the values and construct another JS based data representation. And somehow I expected a native impl. to be present for this purpose as well. Maybe something like memory.read(Float32Array) to return me the buffer data as a Float32Array, abstracting the pointer and iteration maze.

Am I missing something?
Is there a better way to pass large amounts of data from/to WASM that I just overlooked?

Thanks in advance and best,
Aron

[binji]

You have to decide whether you want to copy data between JavaScript and WebAssembly, or whether you want WebAssembly to own the data.

If you want to copy the data, you can use TypedArray.prototype.set() around rather than writing the for loop yourself:

let instance = ...;
let myJSArray = new Float32Array(...);
let length = myJSArray.length;
let myWasmArrayPtr = instance.exports.allocateF32Array(length);
let myWasmArray = new Float32Array(instance.exports.memory.buffer, myWasmArrayPtr, length);

// Copy data in to be used by WebAssembly.
myWasmArray.set(myJSArray);

// Process the data in the array.
instance.exports.processF32Array(myWasmArrayPtr, length);

// Copy data out to JavaScript.
myJSArray.set(myWasmArray);

If WebAssembly owns the data, you can create a view over the WebAssembly.Memory buffer and pass that around to your JavaScript functions too:

let instance = ...;
let length = ...;
let myArrayPtr = instance.exports.allocateF32Array(length);
let myArray = new Float32Array(instance.exports.memory.buffer, myArrayPtr, length);

// Use myArray as a normal Float32Array in JavaScript, fill in data, etc.
...

// Process the data in the array.
instance.exports.processF32Array(myArrayPtr, length);

// No need to copy data back to JavaScript, just use myArray directly.

You can also use tools like embind to make this a little easier to use.

[kyr0]

Wow, thank you @binji this looks awesome. It's exactly what I was looking for. Also thank you for taking the time to write the example code. This is very helpful as well. I will try it this evening and pull request some improvements for the Loader impl. which is in use for interfacing with WASM in https://github.com/AssemblyScript/assemblyscript :) (That's why I didn't came across embind, which looks awesome for use with emscripten)

[kyr0]

Do you see an option to extend the docs on webassembly.org regarding this?

For example here:

https://webassembly.org/docs/web/

I'm willing to do this if possible.

I think, it would be also a good idea to explain it here:

https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/WebAssembly/Memory

[kyr0]

@binji Off-topic, you might like https://github.com/torch2424/wasmBoy if you're not already aware of the project. And this project might also benefit form the solution you proposed, as they use for-loops to set the ROM memory.

[binji]

Do you see an option to extend the docs on webassembly.org regarding this?

Agreed that we should update webassembly.org, with this change and a lot else too. The docs there are still from the design repo, but the spec repo is much more recent and accurate.

Adding some examples to MDN would probably be easier and better, considering many more web developers will use that site than webassembly.org.

Off-topic, you might like https://github.com/torch2424/wasmBoy if you're not already aware of the project.

Yep, I know @torch2424. :-) I have my own wasm-based gameboy emulator too: https://github.com/binji/binjgb

[kyr0]

Adding some examples to MDN would probably be easier and better, considering many more web developers will use that site than webassembly.org.

Alright, I will post an update tonight. Let's see if it gets published.

Yep, I know @torch2424. :-) I have my own wasm-based gameboy emulator too: https://github.com/binji/binjgb

Haha, cool! I will def. try it out :) I bet your emulator doesn't crash in Zelda - Link's Awakening when you get the sword (at the beach) ;))

I hope to find some time to fix the crash in wasmBoy too :)

[torch2424]

Hey! Rad that you found wasmboy! 😄 Yes, @binji 's emulator is definitely wayyy more accurate haha! Something I need to work on for sure. But, Ironically, I use Link's awakening to test wasmboy all the time haha! I'm surprised it crashed on you.

Opened some bugs based on the feedback:
torch2424/wasmboy#141 - Memory Passing
torch2424/wasmboy#142 - Zelda Crash (With screenshots it works for me)

Anyways, Don't want derail the conversation/issue, I'll move over to the issues I opened. But thanks for the feedback! 😄

[Macil]

If WebAssembly owns the data, you can create a view over the WebAssembly.Memory buffer and pass that around to your JavaScript functions too:

One important caveat: the view into the Memory buffer will be invalidated if the WebAssembly instance grows its memory. Avoid storing any references to the view that persist past further calls into the WebAssembly instance.

[fmkang]

@binji Can you tell me what do you put at the AssemblyScript side when you use this? I noticed it is not a standard function.

let myWasmArrayPtr = instance.exports.allocateF32Array(length);

I write as follows and it works, but I still want to know if it's in a correct way.

export function allocateF32Array(length: usize): usize {
    return memory.allocate(length * sizeof<f32>());
}

Do I need to write another AS function to free it with memory.free(), and call it after I transfer the array to JS side?

[torch2424]

@fmkang So what @binji explained was on the JS side (please correct me if I am wrong).

Using Typed Arrays, you can do efficient writing into WASM memory from the JS side using set() with the Wasm Array Buffer.

Writing to Wasm Memory from inside Wasm/AS, I am still using the standard for loop approach, e.g: https://github.com/torch2424/wasmBoy/blob/master/core/memory/dma.ts#L29

Though, perhaps @MaxGraey or @dcodeIO could perhaps help with how to do this most efficiently from inside AS or Wasm land? 😄 Though we may be better off moving this to the AS repo

[dcodeIO]

@fmkang You'd simply

export function allocateF32Array(length: i32): Float32Array {
  return new Float32Array(length);
}

on the AS side and

let myArray = module.getArray(Float32Array, module.allocateF32Array(length));

on the JS side, using the loader.

[fmkang]

@torch2424 Yes, @binji 's code is on the JS side, but let myWasmArrayPtr = instance.exports.allocateF32Array(length) is calling a function named allocateF32Array in the Wasm module (likely compiled by AssembleScript). This function is not mentioned in his snippet nor part of AS built-in functions, so I think it should be implemented by myself. I'm asking how to get this pointer, which points to the corresponding Wasm array.

@dcodeIO Thanks. It seems that this simplifies passing arrays. I'll carefully experiment it before posting any new comments here. But my module crashes when being loaded after I replace my function with yours. The console says

astest.html:1 Uncaught (in promise) TypeError: WebAssembly Instantiation: Import #0 module="env" error: module is not an object or function

It even crashes if I write something like let a = new Float32Array(10); at anywhere in my AS code.

[fmkang]

@dcodeIO The Wasm module crashes when we were trying to initialize an array with expressions (instead of literals, such as let c: f64[] = [a[0] + b[0], a[1] + b[1]];) or assign values to elements of an array (such as let a: f64[] = [0, 0, 0]; a[0] = 1; or let array = new Array<i32>(); array.push(1);).

At first, we thought this is because we did not put memory in env, so we wrote:

var importObject = {
    env: { memory: new WebAssembly.Memory({initial:10}) },
    imports: { imported_func: arg => console.log(arg) }
};
WebAssembly.instantiate(wasmBinary, importObject).then(...);

But the problem still existed. Then we happened to find it is because of the lack of abort. We wrote:

env: {
    abort(msg, file, line, column) {
        console.error("abort called at main.ts:" + line + ":" + column);
    }
}

or simply env: { abort: function(){} }, and the problem solved. However, there is no error message, and the execution of the code did not really "abort". We still do not know the actual cause of this problem.

We are really new to WebAssembly. I write this post just to give an update. You don't really need to reply.

[dcodeIO]

@fmkang It appears that you are not using the AssemblyScript loader to instantiate the module. While you can implement all of this yourself, the loader already adds basic functionality around a module's exports, like the abort function. If you have any more questions regarding AssemblyScript, feel free to ask on our issue tracker and we'll be happy to help :)

[monktastic]

So, I've tried using this technique:

let myArrayPtr = instance.exports.allocateF32Array(length);
let myArray = new Float32Array(instance.exports.memory.buffer, myArrayPtr, length);

// Use myArray as a normal Float32Array in JavaScript, fill in data, etc.
...

// Process the data in the array.
instance.exports.processF32Array(myArrayPtr, length);

But my processF32Array function results in RuntimeError: memory access out of bounds. I'm not using the loader, FWIW. I've also tried:

let myArray = module.getArray(Float32Array, module.allocateF32Array(length));

But my module doesn't have a "getArray," and I'm not sure how it's supposed to get it.

[pwstegman]

I was looking for how to copy arrays to/from WebAssembly and came across this thread. For anyone else, I was able to copy arrays over using AssemblyScript and @torch2424's library as-bind

Here's my simple test:

AssemblyScript

export function sum(arr: Float64Array): f64 {
  let sum: f64 = 0;
  for(let i = 0; i < arr.length; i++) {
    sum += arr[i];
  }
  return sum;
}

JavaScript (Node)

const { AsBind } = require("as-bind");
const fs = require("fs");
const wasm = fs.readFileSync(__dirname + "/build/optimized.wasm");

const asyncTask = async () => {
  const asb = await AsBind.instantiate(wasm);

  // Make a large array
  console.time('Making array');
  let arr = new Float64Array(1e8).fill(1);
  console.timeEnd('Making array');

  // Find the sum using reduce
  console.time('Reduce');
  let sum1 = arr.reduce((acc, val) => acc + val, 0);
  console.timeEnd('Reduce');
  
  // Find the sum with for loops
  console.time('JS For');
  let sum2 = 0;
  for(let i = 0; i < arr.length; i++) sum2 += arr[i];
  console.timeEnd('JS For');

  // Find the sum with WebAssembly
  console.time('Wasm For');
  let sum3 = asb.exports.sum(arr);
  console.timeEnd('Wasm For');

  console.log(sum1, sum2, sum3);
};

asyncTask();

On my machine, I got the following output:

Making array: 789.086ms
Reduce: 2452.922ms
JS For: 184.818ms
Wasm For: 2008.482ms
100000000 100000000 100000000

Looks like there's a good bit of overhead when copying the array over to WebAssembly with this method, although I can imagine that tradeoff will be worth it for a more expensive operation than a sum.

Edit:

I removed the summation from the AssemblyScript to isolate the running time of just the array copy:

export function sum(arr: Float64Array): f64 {
  let sum: f64 = 0;
  return sum;
}

And I reran the benchmarks.

Without Wasm summation

Making array: 599.826ms
Reduce: 2810.395ms
JS For: 188.623ms
Wasm For: 762.481ms
100000000 100000000 0

So looks like the data copy itself took 762.481ms to copy over 100 million f64s.

[MaxGraey]

@pwstegman I created another benchmark which use random input data and prevent js engine optimize away your loop: https://webassembly.studio/?f=5ux4ymi345e

And results is differ for Chrome & FF:

Chrome 81.0.4044.129

js sum: 129.968017578125ms
sum result = 49996811.62100115

js reduce: 1436.532958984375ms
js reduce result = 49996811.62100115

wasm sum: 153.000244140625ms
wasm sum result = 49996811.62100115

wasm reduce: 125.009033203125ms
wasm reduce result = 49996811.62100115

wasm empty: 0.002685546875ms

[toughengineer]

Hello.
I'm basically a newborn regarding familiarity with WASM, and I stumbled upon the same problem as folks here and in #1162. (I'm asking here vs. #1162 since this one has more recent activity.)
After brief research I made a demo while trying to figure it out, and ended up with a method where you expose views into WASM memory as typed arrays to avoid copying data in and out, here is the gist of it from the JS point of view:

// need to know the size in advance, that's OK since the impl makes some init 
const fft = new Module.KissFftReal(/*size=*/N);depending on the size anyway

// get view into WASM memory as Float64Array (of size=N in this case)
const input = fft.getInputTimeDataBuffer();

// fill 'input' buffer

// perform transformation, view into WASM memory is returned here (of size=(N + 2) in this case, +2 for Nyquist bin)
const output = fft.transform();

// use transformation result returned in 'output' buffer

Is this the way to go? Is there any better solution? Would appreciate any comment/refresher on this topic.

p.s. though kinda ugly it works fantastic for me in practice (as far as I can understand and appreciate it)

p.p.s.

here's my (yet unanswered) stackoverflow question regarding similar concern:
https://stackoverflow.com/questions/65566923/is-there-a-more-efficient-way-to-return-arrays-from-c-to-javascript