Skip to content

Commit

Permalink
feat(host_fn): infer types from annotations (#11)
Browse files Browse the repository at this point in the history
Infer host function Wasm types from type annotations if given. When
inferred, parameters and return values are automatically marshaled and
unmarshaled and "current plugin" access is hidden. Whenever a host
function is declared, add it to a thread-local registry of host
functions. Default `Plugin` creation will use the global list of
plugins.

In the weeds note: because this means we have to hang on to functions at
specific addresses, we always end up using user_data now: in particular
we store the offset of the target host function in the user_data to
determine which implementation to call. I added this because the FFI
call would fail if the target python host function wasn't available at
module-level; this would turn into a stack overflow (it appeared that
the runtime tried to re-invoke a host function infinitely if it couldn't
find it.)

---

Example of inferred decorator + global host func registry:

```python
import typings
import extism
@extism.host_fn()
def incr_json(input: typings.Annotated[dict, extism.Json]) -> str
    return input.get("hello", "world")

with extism.Plugin(some_wasm_file) as plugin:
    plugin.call("incr_json", json.dumps({"hello", "earth"}))
```
  • Loading branch information
chrisdickinson authored Oct 16, 2023
1 parent 860d275 commit 5804356
Show file tree
Hide file tree
Showing 7 changed files with 788 additions and 96 deletions.
24 changes: 8 additions & 16 deletions example.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,18 +5,18 @@
import pathlib

sys.path.append(".")
from extism import Function, host_fn, ValType, Plugin, set_log_file
from extism import Function, host_fn, ValType, Plugin, set_log_file, Json
from typing import Annotated

set_log_file("stderr", "trace")


@host_fn
def hello_world(plugin, input_, output, a_string):
@host_fn(user_data=b"Hello again!")
def hello_world(inp: Annotated[dict, Json], *a_string) -> Annotated[dict, Json]:
print("Hello from Python!")
print(a_string)
print(input_)
print(plugin.input_string(input_[0]))
output[0] = input_[0]
inp["roundtrip"] = 1
return inp


# Compare against Python implementation.
Expand All @@ -36,16 +36,7 @@ def main(args):
hash = hashlib.sha256(wasm).hexdigest()
manifest = {"wasm": [{"data": wasm, "hash": hash}]}

functions = [
Function(
"hello_world",
[ValType.I64],
[ValType.I64],
hello_world,
"Hello again!",
)
]
plugin = Plugin(manifest, wasi=True, functions=functions)
plugin = Plugin(manifest, wasi=True)
print(plugin.id)
# Call `count_vowels`
wasm_vowel_count = plugin.call("count_vowels", data)
Expand All @@ -55,6 +46,7 @@ def main(args):
print("Number of vowels:", j["count"])

assert j["count"] == count_vowels(data)
assert j["roundtrip"] == 1


if __name__ == "__main__":
Expand Down
6 changes: 6 additions & 0 deletions extism/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,9 @@
ValType,
Val,
CurrentPlugin,
Codec,
Json,
Pickle,
)

__all__ = [
Expand All @@ -28,4 +31,7 @@
"Function",
"ValType",
"Val",
"Codec",
"Json",
"Pickle",
]
Loading

0 comments on commit 5804356

Please sign in to comment.