Allow returning Vec<T>

[rookboom]

It would really be great to be able to return a vector of structs or tuples:

For example. I have the following type:

#[wasm_bindgen]
struct Range {
    offset: u32,
    length: u32
}

that I want to return in my function

#[wasm_bindgen]
pub fn token_ranges(text: &str) -> Vec<Range> 

I am getting this error:

   Compiling picl_wasm_runtime v0.1.0 (file:///A:/Repos/picl_native_runtime/bindings/typescript)
error[E0277]: the trait bound `std::boxed::Box<[Range]>: wasm_bindgen::convert::WasmBoundary` is not satisfied
  --> src\lib.rs:15:1
   |
15 | #[wasm_bindgen]
   | ^^^^^^^^^^^^^^^ the trait `wasm_bindgen::convert::WasmBoundary` is not implemented for `std::boxed::Box<[Range]>`
   |
   = help: the following implementations were found:
             <std::boxed::Box<[u16]> as wasm_bindgen::convert::WasmBoundary>
             <std::boxed::Box<[i16]> as wasm_bindgen::convert::WasmBoundary>
             <std::boxed::Box<[f32]> as wasm_bindgen::convert::WasmBoundary>
             <std::boxed::Box<[i32]> as wasm_bindgen::convert::WasmBoundary>
           and 5 others
   = note: required because of the requirements on the impl of `wasm_bindgen::convert::WasmBoundary` for `std::vec::Vec<Range>`

My workaround is to flatten my data and return a Vec and then splice my token ranges on the JS side. This is unfortunate...

How can I add a WasmBoundary trait for a custom type?
Is there a better way?

[alexcrichton]

Thanks for the report!

Right now there's not a great way to do this in wasm-bindgen itself in terms of the conversion here has to be also accompanied with a conversion on the JS side which may be somewhat tricky. It should certainly be possible though given enough support!

[Hywan]

Same issue here :-).

What kind of support do you need?

[alexcrichton]

@rookboom @Hywan do y'all basically need Vec<T> where T has #[wasm_bindgen] on it?

[Hywan]

@alexcrichton Exactly, that's precisely my usecase.

[rookboom]

Yes, that would be fantastic.

[alexcrichton]

Ok this is then I think pretty similar to #104. We somehow need a way to communicate this to the CLI tool but currently the strategy for doing that is quite limited.

[gabrielcarneiro97]

I'm with a similar issue here, but with a complex struct.

code:

#[derive(Clone)]
#[wasm_bindgen]
pub struct Coords {
    x: usize,
    y: usize
}

#[wasm_bindgen]
pub struct Cell {
    state: State,
    position: Coords,
    neighboors: Vec<Coords>,
    neighboors_alive: i32
}

#[wasm_bindgen]
impl Cell {
    pub fn new(state: State, position: Coords, neighboors: Vec<Coords>) -> Cell {
        Cell {
            state,
            position,
            neighboors,
            neighboors_alive: 0
        }
    }
}

error:

error[E0277]: the trait bound `std::boxed::Box<[Coords]>: wasm_bindgen::convert::FromWasmAbi` is not satisfied
  --> src\lib.rs:36:1
   |
36 | #[wasm_bindgen]
   | ^^^^^^^^^^^^^^^ the trait `wasm_bindgen::convert::FromWasmAbi` is not implemented for `std::boxed::Box<[Coords]>`
   |
   = help: the following implementations were found:
             <std::boxed::Box<[u16]> as wasm_bindgen::convert::FromWasmAbi>
             <std::boxed::Box<[wasm_bindgen::JsValue]> as wasm_bindgen::convert::FromWasmAbi>
             <std::boxed::Box<[u8]> as wasm_bindgen::convert::FromWasmAbi>
             <std::boxed::Box<[f32]> as wasm_bindgen::convert::FromWasmAbi>
           and 5 others
   = note: required because of the requirements on the impl of `wasm_bindgen::convert::FromWasmAbi` for `std::vec::Vec<Coords>`

[David-OConnor]

Gabriel and rook - have y'all found a workaround? Solving or working around this would add much flexibility to WASM in Rust.

Eventually, being able to use HashMaps, or structs from other packages (Like ndarrays) would be nice, but having some type of collection that maps to JS arrays would be wonderful; not sure if I can continue my project without this.

[rookboom]

My workaround is to pass JSON over the wasm boundary... Not ideal but works for now.

[Gisleburt]

For those looking for a workaround on this, if you can turn your data into a Vec<u8> or &[u8]

#[wasm_bindgen]
pub struct ByteStream {
    offset: *const u8,
    size: usize,
}

#[wasm_bindgen]
impl ByteStream {
    pub fn new(bytes: &[u8]) -> ByteStream {
        ByteStream {
            offset: bytes.as_ptr(),
            size: bytes.len(),
        }
    }

    pub fn offset(&self) -> *const u8 {
        self.offset
    }

    pub fn size(&self) -> usize {
        self.size
    }
}

A good example of how to use this is creating a texture in Rust to render in Javascript, so for example:

#[wasm_bindgen]
pub fn render() -> ByteStream {
  let texture = Vec::new();
  // ...
  ByteStream::new(&texture);
}

const texture = render();
const textureRaw = new Uint8ClampedArray(memory.buffer, texture.offset(), texture.size());
const image = new ImageData(textureRaw, width, height);

[fitzgen]

Rather than returning pointers and lengths manually, you can use this, which should be slightly less error prone: https://docs.rs/js-sys/0.3.9/js_sys/struct.Uint8Array.html#method.view

[stefan2718]

If anyone is still looking at this, I was able to work around this using Serde to serialize/deserialize the data. This was the guide I used: https://rustwasm.github.io/docs/wasm-bindgen/reference/arbitrary-data-with-serde.html

Edit: For those wanting to avoid JSON serialization, the guide above also includes a link to serde-wasm-bindgen which "leverages direct APIs for JavaScript value manipulation instead of passing data in a JSON format."

[theashguy]

I'm wanting to take this one step further and return Vec<js_sys::Function> which means no serde serialization for me (Maybe there is some kind of ref I could serialize if I dug into the internals?). The workaround I'm kicking about at the moment is to create the collection on the JS side and expose some methods for appending and cleaning up.

Something like...

window.vecCacheOfAnything = {
  append: function (key, item) {
    if(!window.vecCache.cache[key]) {
      window.vecCache.cache[key] = []
    }

    window.vecCache[key].push(item)
  },
  clear: function (key) {
    delete window.vecCache.cache[key]
  },
  cache: {}
}

Not suitable for prod work and by no means ideal or ergonomic but I'm really just thrashing around to see how far I can take rustwasm at the moment :D. Possibly an idea for anyone blocked by this issue and wanting to do things more complex. Also an extra AC for the team to add to the backlog!

[Zireael07]

This issue is kind of a blocker for pretty much any sort of a bigger project using rustwasm.

[Pauan]

It's not a complete solution, but I created #1749 which adds in FromIterator for Array:

use js_sys::Array;

#[wasm_bindgen]
pub fn token_ranges(text: &str) -> Array {
    get_vec_somehow().into_iter().collect()
}

This means that now you can send Vec<T> to JS, you just have to return an Array and use .into_iter().collect() to convert the Vec<T> into an Array.

[dasoncheng]

Same issue here :-).

#[wasm_bindgen]
#[derive(Debug)]
pub struct HeartBeat {
    pub template: u8,
    pub classify: u8,
    pub index: u32,
    pub tr: u16,
    pub hr: u16,
    pub feature: [[f32; 20]; 3],
}

the trait bound `[[f32; 20]; 3]: wasm_bindgen::convert::traits::IntoWasmAbi` is not satisfied

the trait `wasm_bindgen::convert::traits::IntoWasmAbi` is not implemented for `[[f32; 20]; 3]`

help: the following implementations were found:
        <&'a [f32] as wasm_bindgen::convert::traits::IntoWasmAbi>
        <&'a [f64] as wasm_bindgen::convert::traits::IntoWasmAbi>
        <&'a [i16] as wasm_bindgen::convert::traits::IntoWasmAbi>
        <&'a [i32] as wasm_bindgen::convert::traits::IntoWasmAbi>
      and 20 othersrustc(E0277)
lib.rs(38, 1): the trait `wasm_bindgen::convert::traits::IntoWasmAbi` is not implemented for `[[f32; 20]; 3]`

[dragly]

@Pauan Great! Thanks for looking into this.

However, I am encountering the following error now.

the trait `std::convert::AsRef<wasm_bindgen::JsValue>` is not implemented for `MyObject`
note: required because of the requirements on the impl of `std::iter::FromIterator<MyObject>` for `js_sys::Array`

from the following code:

#[wasm_bindgen]
struct MyObject {
    a: f32,
    b: f32,
}

#[wasm_bindgen]
pub fn test() -> Array {
    let objects = vec![
        MyObject {
            a: 123.0,
            b: 1024.1,
        },
        MyObject {
            a: 456.7,
            b: 1024.8,
        },
    ];
    objects.into_iter().collect()
}

Am I missing something?

This is using wasm-bindgen = "0.2.51". Maybe your change is not in yet? The error is different from before, however, so it seems like something changed.

[Pauan]

@dragly As explained in the PR, you need to use .map(JsValue::from), like this:

objects.into_iter().map(JsValue::from).collect()

This is because structs are a Rust data type, and so you have to manually use JsValue::from to convert them into a JS data type (the same is true for other Rust data types like &str, i32, etc.).

[Kinrany]

As a workaround, is it possible to also specify the TypeScript type returned by the function?

By default fn foo() -> Array is compiled into () => any[] instead of something like () => string[].

[Pauan]

@Kinrany Yes, but it requires a bit of a hack:

#[wasm_bindgen]
extern "C" {
    #[wasm_bindgen(typescript_type = "Array<string>")]
    type MyArray;
}

(The typescript_type attribute can specify any TypeScript type, even complex types like |)

Now you just do fn foo() -> MyArray and use .unchecked_into::<MyArray>() to cast the Array into MyArray.

[bushrat011899]

Is there any work on this issue still? I saw #2734 had a solution to this issue but needed additional help with passing tests. If a full resolution cannot be implemented without a breaking change, could the #[wasm_bindgen] macro be extended with an attribute similar to getter_with_clone which uses serialization to coerce function outputs into something JS compatible?

/// Current Workaround
#[wasm_bindgen]
pub fn manual_serialization() -> JsValue {
    serde_wasm_bindgen::to_value::<Vec<String>>(&vec![]).unwrap()
}

/// Proposed Alternative
#[wasm_bindgen(serialize_output)]
pub fn automatic_serialization() -> Vec<String> {
    vec![]
}

This appears to be the workaround I've seen the most commonly used, but has a massive flaw in that all type information is lost for the developer and wasm-bindgen itself for type annotations. I saw that #2649 was closed because of the potential hidden performance impact from a solution like this. I would think that adding this as a macro option would allow the ergonomics without hiding the issues.

[sinking-point]

@bushrat011899

I have had no response from @chinedufn @alexcrichton regarding the failing test. I spent a lot of time on this PR and this is discouraging to say the least. I wonder if they just never look at PRs unless they pass all tests. In which case, I could assume an interpretation of the comment in the test, and get it to pass. That way I'd get their attention, and if I assumed the wrong interpretation they will tell me to change it. However the PR is quite old now and probably has merge conflicts with master to sort out. If I'm going to do more work on this I want some assurance that I'm not wasting my time.

[bushrat011899]

@bushrat011899

I have had no response from @chinedufn @alexcrichton regarding the failing test. I spent a lot of time on this PR and this is discouraging to say the least. I wonder if they just never look at PRs unless they pass all tests. In which case, I could assume an interpretation of the comment in the test, and get it to pass. That way I'd get their attention, and if I assumed the wrong interpretation they will tell me to change it. However the PR is quite old now and probably has merge conflicts with master to sort out. If I'm going to do more work on this I want some assurance that I'm not wasting my time.

That really is unfortunate, this seems like a really obvious feature to work on in my opinion. My first thought for something useful with WASM was to write a REST API client. That way, I could have type guarantees and unit tests that, for example, my Axum server was interacting with a web UI correctly. Kinda hard to do that if something simple like "give me a list of data" requires type erasure.

[bushrat011899]

Because this issue still doesn't have a good workaround IMO, I've tried to make my own. Below is a single-file procedural attribute macro which modifies a function definition in-place to:

1. Return a JsValue instead of whatever type it is supposed to return.
2. Evaluate the function body (async or sync) and store in a temporary variable __result.
3. Pass the __result to serde_wasm_bindgen to create a JsValue to return.

This doesn't fix the issue entirely. For example, wasm-bindgen will mark the TypeScript definition of this function as returning any (understandable, it is!), and serialization is not always required to cross the FFI boundary.

I haven't published a crate before, and this is small enough that I don't know if it really needs to be, but this is how I've worked around this pretty massive functionality gap in wasm-bindgen. Hope someone finds it useful!

Proc-Macro Crate lib.rs

use proc_macro::TokenStream;
use proc_macro2::TokenStream as TokenStream2;
use quote::quote;
use syn::{parse::Nothing, parse2, parse_quote, ItemFn, ReturnType, Type};

#[proc_macro_attribute]
pub fn wrap_jsvalue(args: TokenStream, input: TokenStream) -> TokenStream {
    let input: TokenStream2 = input.into();

    let function = parse2::<Nothing>(args.into()).and(parse2::<ItemFn>(input.clone()));

    let expanded = match function {
        Ok(function) => expand_macro(function),
        Err(parse_error) => {
            let compile_error = parse_error.to_compile_error();
            quote!(#compile_error #input)
        }
    };

    expanded.into()
}

fn expand_macro(mut function: ItemFn) -> TokenStream2 {
    let ReturnType::Type(arrow, return_type) = function.sig.output else {
        return quote!(#function);
    };

    let stmts = function.block.stmts;

    let result = if function.sig.asyncness.is_some() {
        quote!((move || async move { #(#stmts)* })().await)
    } else {
        quote!((move || { #(#stmts)* })())
    };

    function.block = Box::new(parse_quote!({
        let __result: #return_type = #result;

        serde_wasm_bindgen::to_value(&__result).unwrap()
    }));

    function.sig.output = ReturnType::Type(arrow, Box::new(Type::Verbatim(quote!(JsValue))));

    quote!(#function)
}

Primary Crate lib.rs

use wasm_bindgen::prelude::*;
use my_macros_crate::wrap_jsvalue;

#[wrap_jsvalue]
#[wasm_bindgen]
pub async fn test() -> Vec<String> {
    vec!["Hello".to_string(), "World".to_string()]
}

[fjarri]

@bushrat011899: I hit this issue (and similar ones with argument types) before, and tried to generalize it as https://docs.rs/wasm-bindgen-derive/latest/wasm_bindgen_derive/ . Would be interesting to include your macro there as well.

[bushrat011899]

@fjarri I'm not surprised, it seems like a pretty easy issue to hit! Feel free to take the code I wrote and include it if you like. It's straightforward enough that I'm sure any reasonable person would end up converging on this independently anyway.

[rossng]

Great work @Liamolucko! I look forward to using this once released.

[gabrielDevlog]

Hi guys, great to see a PR has been merged.
Is there any ETA for this feature ? Or is it possible for you to trigger a new release, so we can have this feature using cargo add.

Thanks

[daxpedda]

See #3530.

[samuelOsborne]

@daxpedda I checked that issue, still not released right ?

[daxpedda]

No.
You can find all our releases in the release section.

[Teebor-Choka]

When's the next release? The last one was in June.

[daxpedda]

We don't have a timetable for a next release.
See #3530.

[mnjdhl]

I am also stuck with this issue:

Has the fix been released already?

[mnjdhl]

Even it doesn't work with vector of basic types such as Vec:

[daxpedda]

Has the fix been released already?

Yes it has been.