Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support WIT resource types #197

Open
landonxjames opened this issue Nov 10, 2023 · 7 comments
Open

Support WIT resource types #197

landonxjames opened this issue Nov 10, 2023 · 7 comments

Comments

@landonxjames
Copy link

I appears that having a wit resource type in a component definition causes wasmtime-py to throw an error when generating bindings.

The following .wit file:

package project:runtime

interface clients {
  resource placeholder {
    hello-world: func() -> string
  }
}

world runtime {
  export clients
}

Implemented by this Rust code:

//WIT imports
wit_bindgen::generate!({
world: "runtime",
exports: {
    "project:runtime/clients/placeholder": Placeholder,
}});
use exports::project::runtime::clients::GuestPlaceholder;

pub struct Placeholder;

impl GuestPlaceholder for Placeholder {
    fn hello_world(&self) -> String {
        let val = "Hello World";
        val.to_string()
    }
}

and compiled using:

  • Cargo - 1.73.0
  • Wasm-tools - 1.0.51
  • Wasmtime-py - 14.0.0

Throws the following error:

Full Error

$ wasm-tools component new ./target/wasm32-unknown-unknown/release/wasm_runtime.wasm -o runtime_component_resources.wasm

$ python -m wasmtime.bindgen runtime_component_resources.wasm --out-dir pybind/
Traceback (most recent call last):
  File "/Volumes/workplace/project/.venv/lib/python3.11/site-packages/wasmtime/_func.py", line 260, in enter_wasm
    yield byref(trap)
  File "/Volumes/workplace/project/.venv/lib/python3.11/site-packages/wasmtime/_func.py", line 101, in __call__
    raise WasmtimeError._from_ptr(error)
wasmtime._error.WasmtimeError: error while executing at wasm backtrace:
    0:  0x100 - wit-component:shim!indirect-wasi:cli/terminal-stdin-get-terminal-stdin
    1: 0x254f - wit-component:adapter:wasi_snapshot_preview1!wasi_snapshot_preview1::descriptors::Descriptors::new::ha2fdde51d30a71bd
    2: 0x1640 - wit-component:adapter:wasi_snapshot_preview1!wasi_snapshot_preview1::State::descriptors::h780c546b61bcfc7f
    3: 0x180c - wit-component:adapter:wasi_snapshot_preview1!fd_write
    4:  0x13a - wit-component:shim!adapt-wasi_snapshot_preview1-fd_write
    5: 0x1b080d - <unknown>!wasi::lib_generated::fd_write::hd4964fea612b930f
    6: 0x1acdfa - <unknown>!<std::io::Write::write_fmt::Adapter<T> as core::fmt::Write>::write_str::h375f1d6863bea9df
    7: 0x1ba82b - <unknown>!core::fmt::write::h0eddb54b80b97b9d
    8: 0x1add1c - <unknown>!std::io::Write::write_fmt::h6d46415105134b08
    9: 0x1afaa2 - <unknown>!std::panicking::default_hook::{{closure}}::he04c18047097e21e
   10: 0x1ad3d8 - <unknown>!std::panicking::default_hook::hb03d7fae0dedb715
   11: 0x1b007b - <unknown>!std::panicking::rust_panic_with_hook::hc93abff18edee779
   12: 0x1af73e - <unknown>!std::panicking::begin_panic_handler::{{closure}}::h922bcdd9c6fdedfb
   13: 0x1af6a3 - <unknown>!std::sys_common::backtrace::__rust_end_short_backtrace::h2597d6ecb1d3419e
   14: 0x1afd27 - <unknown>!rust_begin_unwind
   15: 0x1b5689 - <unknown>!core::panicking::panic_fmt::h35d9e7e9c02f9eb5
   16: 0x1b5c5f - <unknown>!core::panicking::panic::h2d50353119445d1c
   17: 0x9662 - <unknown>!bindgen::bindgen::InterfaceGenerator::types::he68d4fe0b27868c4
   18: 0x4fe2 - <unknown>!bindgen::bindgen::WasmtimePy::generate::h662240fac89ac3d8
   19: 0x2f9dd - <unknown>!<bindgen::bindings::PythonBindings as bindgen::bindings::Guest>::generate::h8ccba8d7064427e7
   20: 0x2fbe6 - <unknown>!generate

Caused by:
    python exception

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/Volumes/workplace/project/.venv/lib/python3.11/site-packages/wasmtime/bindgen/__main__.py", line 40, in <module>
    main()
  File "/Volumes/workplace/project/.venv/lib/python3.11/site-packages/wasmtime/bindgen/__main__.py", line 30, in main
    files = generate(name, contents)
            ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Volumes/workplace/project/.venv/lib/python3.11/site-packages/wasmtime/bindgen/__init__.py", line 144, in generate
    result = root.generate(store, name, component)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Volumes/workplace/project/.venv/lib/python3.11/site-packages/wasmtime/bindgen/generated/__init__.py", line 288, in generate
    ret = self.lift_callee0(caller, ptr, len0, ptr1, len2)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Volumes/workplace/project/.venv/lib/python3.11/site-packages/wasmtime/_func.py", line 91, in __call__
    with enter_wasm(store) as trap:
  File "/Users/lnj/.pyenv/versions/3.11.4/lib/python3.11/contextlib.py", line 155, in __exit__
    self.gen.throw(typ, value, traceback)
  File "/Volumes/workplace/project/.venv/lib/python3.11/site-packages/wasmtime/_func.py", line 266, in enter_wasm
    maybe_raise_last_exn()
  File "/Volumes/workplace/project/.venv/lib/python3.11/site-packages/wasmtime/_func.py", line 276, in maybe_raise_last_exn
    raise exn
  File "/Volumes/workplace/project/.venv/lib/python3.11/site-packages/wasmtime/_func.py", line 184, in trampoline
    pyresults = func(*pyparams)
                ^^^^^^^^^^^^^^^
  File "/Volumes/workplace/project/.venv/lib/python3.11/site-packages/wasmtime/bindgen/generated/__init__.py", line 200, in lowering16_callee
    ret = import_object.terminal_stdin.get_terminal_stdin()
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Volumes/workplace/project/.venv/lib/python3.11/site-packages/wasmtime/bindgen/__init__.py", line 95, in get_terminal_stdin
    raise NotImplementedError
NotImplementedError

The NotImplementedError at the end seems to be caused by the resource in the wit. Changing the wit to just nest the function directly under the interface (and making the corresponding changes to the rust code) resolves the issue.

package project:runtime

interface clients {
    hello-world: func() -> string
}

world runtime {
  export clients
}
@alexcrichton
Copy link
Member

Thanks for the report! Resources aren't implemented at all yet and will probably need a good chunk of work to implement them, but luckily I think componentize-py has probably also done a good deal of the hard work so lifting most of the implementation from there would probably work.

@landonxjames
Copy link
Author

My team is investigating dedicating some resources to get this implemented (thankfully we have a Python guy), and just curious if we could get a quick outline of the work required to support this? I see the related PR to componetize-py that you mentioned, would the work be similar to whats present there?

@alexcrichton
Copy link
Member

That'd be great! I'm happy to be around to answer questions, so don't hesitate to reach out either here or on Zulip or on email. Another resource I'd recommend is the implementation of resources in jco. The compilation model of jco is the same as wasmtime-py where it takes an input component and spits out glue using a core wasm API. I believe that this file will be of interest and you should find the high-level organization of jco and wasmtime-py to look pretty similar. For example wasmtime-py will probably want an almost exact copy of this function and probably others. Handling of HandleLift and other related pseudo-instructions is probably also going to be similar.

@ivan-kolesar
Copy link

Hi @alexcrichton @landonxjames , is ti possible to share the status of this feature? We would like to use resource in multiple parts of our stack (specifically to parts that are written in javascript and python), but are currently hold by missing python part.

@alexcrichton
Copy link
Member

Hello! No status on this yet, I don't believe anyone's working on it.

@jamesls
Copy link
Contributor

jamesls commented Apr 10, 2024

Hi, I'm looking into what's needed to implement resources in wasmtime-py and wanted to see if there were any thoughts on what the generated Python API should look like.

If we use the WIT file at the top of this issue (and add a constructor to it):

package component:runtime;

interface clients {
  resource placeholder {
    constructor(name: string);
    hello-world: func() -> string;
  }
}

world runtime {
  export clients;
}

With the Rust API in wasmtime using bindgen!(), you'd consume the generated code with:

// Assuming we've created our store/component/linker, and the world name is
// ``Runtime`.

let (bindings, _) = Runtime::instantiate(&mut store, &component, &linker)?;
let clients = bindings.component_runtime_clients(); // 'interface clients'
let placeholder = clients.placeholder();  // 'resource placeholder'
let resource_instance = placeholder.call_constructor(&mut store, "myname")?;  // constructor(name: string)
let result = placeholder.call_hello_world(&mut store, resource_instance)?; // hello-world: func() -> string

So if we wanted that same API in Python, that would look something like:

from wasmtime import Store
from .generated import Root

store = Store()
runtime = Root(store)
clients = runtime.clients()  #  'interface clients'
placeholder = clients.placeholder(); # 'resource placeholder'
resource_instance = placeholder.constructor(store, "myname")  # 'constructor(name: string)'
result = placeholder.hello_world(store, resource_instance)  # hello-world: func() -> string

However, I think ideally we'd invoke methods on the instantiated resource directly (e.g. resource_instance.hello_world(...), like you'd normally do with an instantiated class in Python.
This is more along the lines of how you'd use the generated jco bindings:

import * as runtime from "./bindings/runtime.js";

// We could just do:
//
//   const instance = new runtime.clients.Placeholder('myname')
//
// But breaking out each step to compare with the Rust bindgen!()

const clients = runtime.clients;  // 'interface clients'
const Placeholder = clients.Placeholder; // 'resource placeholder'
const resourceInstance = new Placeholder('myname'); // 'constructor(name: string);'
const result = resourceInstance.helloWorld(); // 'hello-world: func() -> string'

I'm not sure how feasible this is, but maybe something along these lines could work for Python:

store = Store()
runtime = Root(store)
clients = runtime.clients()  #  'interface clients'
Placeholder = clients.Placeholder; # 'resource placeholder'
resource_instance = Placeholder(store, "myname")  # 'constructor(name: string)'
result = resource_instance.hello_world(store)  # hello-world: func() -> string

@alexcrichton
Copy link
Member

Given how different the ownership idioms are in Rust and Python I'd definitely recommend following jco's footsteps here more than Wasmtime's, and what you have there all looks reasonable to me!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants