feat(wasmtime-cli): restore support for wasi http module #6878

eduardomourar · 2023-08-22T11:17:52Z

~~Within the wasmtime CLI, the current default behavior is to only inject the synchronous functions to linkers. This will add a flag called --async that will inject the asynchronous one instead.~~

This is required in order to enable back the wasi-http module because it has been implemented only with async functions. Additionally, it will help the async support code path to be tested as async should be enabled by default in the future.

This will enable back the wasi-http module within the wasmtime CLI. We will make the bindgen synchronous through the tokio executor available in wasmtime.

alexcrichton · 2023-08-22T14:24:50Z

Thanks! Could you expand a bit more on the rationale for this? Why would users of the CLI want to control async-vs-not? Or is this instead intended for testing the async implementations on the CLI?

eduardomourar · 2023-08-22T14:47:06Z

Thanks! Could you expand a bit more on the rationale for this? Why would users of the CLI want to control async-vs-not? Or is this instead intended for testing the async implementations on the CLI?

Just added more details to the PR body itself.

pchickey · 2023-08-22T16:50:10Z

This isn't the correct approach - instead, wasmtime-wasi::preview2::in_tokio should be made pub, and and the wasmtime-wasi-http crate should use that to implement a sync bindgen trait by wrapping the async bindgen trait.

futures::executor isnt the right executor to use with wasmtime-wasi, it will fall over if not used with tokio.

github-actions · 2023-08-25T22:45:05Z

Subscribe to Label Action

cc @peterhuene

This issue or pull request has been labeled: "wasi", "wasmtime:api", "wasmtime:c-api"

Thus the following users have been cc'd because of the following labels:

peterhuene: wasmtime:api, wasmtime:c-api

To subscribe or unsubscribe from this label, edit the .github/subscribe-to-label.json configuration file.

Learn more.

Within the wasmtime CLI, the current default behavior is to only inject the synchronous functions to linkers. This will add a flag called `--async` that will inject the asynchronous one instead.

This reverts commit b743ff2.

pchickey

With a few changes we can accept this PR, thanks!

Just as a heads up, I know you put a lot of labor into keeping the core-wasm wasmtime::Linker version of this code working, including the sync side of the bindings. We'll keep those working for the wasmtime 13 release, but after that we are going to swap the implementation over to use wasmtime-wasi::preview2 streams and pollables, as well as component resource types. When we make that switch, we will deprecate the core-wasm interfaces, since we don't want to maintain both.

pchickey · 2023-08-31T21:51:10Z

tests/all/cli_tests/wasi-http.wat

+      i32.const 0 ;; base pointer
+      i32.const 0 ;; length
+    ))
+    (local.set $request_id (call $__wasi_http_types_new_outgoing_request


this is impressive! nice job

pchickey · 2023-08-31T22:07:57Z

src/commands/run.rs

+                    }
+                    #[cfg(feature = "component-model")]
+                    CliLinker::Component(linker) => {
+                        wasmtime_wasi_http::proxy::sync::add_to_linker(linker)?;


Using wasi-http with the component::Linker will fail because it conflicts with the stream methods in the wasmtime-wasi::preview2 implementation. So, we should instead bail in this case.

that is actually not case because we are taking the implementation from the wasmtime-wasi crate when building the bindings:

wasmtime/crates/wasi-http/src/proxy.rs

Lines 53 to 54 in 75f3da7

"wasi:io/streams": preview2::bindings::sync_io::io::streams,

"wasi:poll/poll": preview2::bindings::sync_io::poll::poll,

we should be able to migrate the cli test to a component when the module code is deprecated.

right, but without actually using HostInputStream and HostOutputStream, it will still crash at runtime.

crates/wasmtime/Cargo.toml

crates/wasi-http/src/http_impl.rs

…sync-support

pchickey · 2023-09-05T16:44:13Z

Looks like there is some sort of cargo issue that happens when building the cli, which doesnt happen in the usual set of tests that runs before the merge queue runs the full set. Put the string prtest:full in a commit message to get the full test suite running each time you push.

prtest:full

eduardomourar · 2023-09-05T20:55:08Z

@pchickey, now tests are passing. I had to ignore the wasi-http tests on Windows as they were failing. Can we merge this, please? I plan on further debugging those after PR #6877 lands.

eduardomourar · 2023-09-06T21:38:52Z

@alexcrichton and @pchickey, could wasi-http be enabled back in the wasmtime v13 release branch, please? Otherwise, there will be no release with module support for this.

alexcrichton · 2023-09-06T21:48:53Z

AFAIK that was the plan yeah, so seems reasonable to me. The process for doing so would require a PR to the release-13.0.0 branch of Wasmtime. @eduardomourar are you ok doing that?

…iance#6878) * feat(wasmtime-cli): add async support flag Within the wasmtime CLI, the current default behavior is to only inject the synchronous functions to linkers. This will add a flag called `--async` that will inject the asynchronous one instead. * chore: refactor wasi http crate * feat(wasmtime-wasi): make in_tokio function public * feat(wasi-http): define default feature called sync * Revert "feat(wasmtime-cli): add async support flag" This reverts commit b743ff2. * chore: improve flaky tests for wasi http * feat(wasi-http): expose sync api for components * chore: add tests for sync api of wasi http components * feat(wasmtime-cli): restore support for wasi http module * chore: revert change to outbound http request invalid test * chore: have extra tracing to help debugging * feat(wasi-http): allow modules with sync functions in linker * fix(wasi-http): missing response body in sync api * feat: include blocking for io streams * chore: add tests for wasi http module in cli * chore: disable preview2 flag in wasi http test * chore: use preview2 flag in wasi http test * fix(wasi-http): missing stream output in sync api * chore: fix tests for wasi http * chore: add tracing for poll oneoff call * chore: send exit signal on wasi http test * chore: swap println to tracing debug * chore: set http server timeout to 50 secs by default * chore: add test posting large file * chore: revert formatting in cargo toml * chore: fix wasi-http feature and skip failing tests prtest:full --------- Co-authored-by: Eduardo Rodrigues <[email protected]>

* feat(wasmtime-cli): add async support flag Within the wasmtime CLI, the current default behavior is to only inject the synchronous functions to linkers. This will add a flag called `--async` that will inject the asynchronous one instead. * chore: refactor wasi http crate * feat(wasmtime-wasi): make in_tokio function public * feat(wasi-http): define default feature called sync * Revert "feat(wasmtime-cli): add async support flag" This reverts commit b743ff2. * chore: improve flaky tests for wasi http * feat(wasi-http): expose sync api for components * chore: add tests for sync api of wasi http components * feat(wasmtime-cli): restore support for wasi http module * chore: revert change to outbound http request invalid test * chore: have extra tracing to help debugging * feat(wasi-http): allow modules with sync functions in linker * fix(wasi-http): missing response body in sync api * feat: include blocking for io streams * chore: add tests for wasi http module in cli * chore: disable preview2 flag in wasi http test * chore: use preview2 flag in wasi http test * fix(wasi-http): missing stream output in sync api * chore: fix tests for wasi http * chore: add tracing for poll oneoff call * chore: send exit signal on wasi http test * chore: swap println to tracing debug * chore: set http server timeout to 50 secs by default * chore: add test posting large file * chore: revert formatting in cargo toml * chore: fix wasi-http feature and skip failing tests prtest:full --------- Co-authored-by: Eduardo Rodrigues <[email protected]>

eduardomourar requested review from a team as code owners August 22, 2023 11:17

eduardomourar requested review from jameysharp and removed request for a team August 22, 2023 11:17

eduardomourar force-pushed the feat/wasmtime-cli-async-support branch from fd6cecb to 9148521 Compare August 22, 2023 11:27

eduardomourar changed the title ~~feat(wasmtime-cli): add async support flag~~ feat(wasmtime-cli): add flag for async support Aug 22, 2023

eduardomourar force-pushed the feat/wasmtime-cli-async-support branch from 9148521 to b743ff2 Compare August 22, 2023 11:31

eduardomourar marked this pull request as draft August 23, 2023 00:37

eduardomourar changed the title ~~feat(wasmtime-cli): add flag for async support~~ feat(wasmtime-cli): add back support for wasi http module Aug 23, 2023

github-actions bot added wasi Issues pertaining to WASI wasmtime:api Related to the API of the `wasmtime` crate itself wasmtime:c-api Issues pertaining to the C API. labels Aug 25, 2023

eduardomourar changed the title ~~feat(wasmtime-cli): add back support for wasi http module~~ feat(wasmtime-cli): restore support for wasi http module Aug 25, 2023

eduardomourar added 9 commits August 26, 2023 02:04

feat(wasmtime-cli): add async support flag

783a9ad

Within the wasmtime CLI, the current default behavior is to only inject the synchronous functions to linkers. This will add a flag called `--async` that will inject the asynchronous one instead.

chore: refactor wasi http crate

c21a3b9

feat(wasmtime-wasi): make in_tokio function public

869c2c0

feat(wasi-http): define default feature called sync

d3c5ec4

Revert "feat(wasmtime-cli): add async support flag"

09a16ed

This reverts commit b743ff2.

chore: improve flaky tests for wasi http

61dec00

feat(wasi-http): expose sync api for components

d9e3a44

chore: add tests for sync api of wasi http components

d1b8af2

feat(wasmtime-cli): restore support for wasi http module

9401c72

eduardomourar force-pushed the feat/wasmtime-cli-async-support branch from 08c09dc to 9401c72 Compare August 26, 2023 00:21

eduardomourar added 3 commits August 26, 2023 20:11

chore: revert change to outbound http request invalid test

9175a41

chore: have extra tracing to help debugging

d112b0a

feat(wasi-http): allow modules with sync functions in linker

1eab683

pchickey approved these changes Aug 31, 2023

View reviewed changes

eduardomourar added 5 commits September 1, 2023 11:20

chore: swap println to tracing debug

eaae136

chore: set http server timeout to 50 secs by default

bd7303a

chore: add test posting large file

222cb47

Merge remote-tracking branch 'upstream/main' into feat/wasmtime-cli-a…

2b60ed1

…sync-support

chore: revert formatting in cargo toml

b4ea2cb

pchickey enabled auto-merge September 1, 2023 18:50

pchickey added this pull request to the merge queue Sep 1, 2023

github-merge-queue bot removed this pull request from the merge queue due to failed status checks Sep 1, 2023

alexcrichton added this pull request to the merge queue Sep 5, 2023

github-merge-queue bot removed this pull request from the merge queue due to a conflict with the base branch Sep 5, 2023

Merge remote-tracking branch 'upstream/main' into feat/wasmtime-cli-a…

725bb0c

…sync-support

elliottt added this pull request to the merge queue Sep 5, 2023

github-merge-queue bot removed this pull request from the merge queue due to failed status checks Sep 5, 2023

eduardomourar force-pushed the feat/wasmtime-cli-async-support branch from 9c3d9af to 968da10 Compare September 5, 2023 19:01

chore: fix wasi-http feature and skip failing tests

e56a4dd

prtest:full

eduardomourar force-pushed the feat/wasmtime-cli-async-support branch from 968da10 to e56a4dd Compare September 5, 2023 20:00

pchickey enabled auto-merge September 5, 2023 22:24

pchickey added this pull request to the merge queue Sep 5, 2023

Merged via the queue into bytecodealliance:main with commit 8eefa7c Sep 5, 2023

eduardomourar deleted the feat/wasmtime-cli-async-support branch September 6, 2023 00:26

eduardomourar mentioned this pull request Sep 7, 2023

chore: backport 6878 to release-13.0.0 #6974

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(wasmtime-cli): restore support for wasi http module #6878

feat(wasmtime-cli): restore support for wasi http module #6878

eduardomourar commented Aug 22, 2023 •

edited

Loading

alexcrichton commented Aug 22, 2023

eduardomourar commented Aug 22, 2023

pchickey commented Aug 22, 2023

github-actions bot commented Aug 25, 2023

pchickey left a comment

pchickey Aug 31, 2023

pchickey Aug 31, 2023

eduardomourar Sep 1, 2023

pchickey Sep 1, 2023

pchickey commented Sep 5, 2023

eduardomourar commented Sep 5, 2023 •

edited

Loading

eduardomourar commented Sep 6, 2023

alexcrichton commented Sep 6, 2023

	"wasi:io/streams": preview2::bindings::sync_io::io::streams,
	"wasi:poll/poll": preview2::bindings::sync_io::poll::poll,

feat(wasmtime-cli): restore support for wasi http module #6878

feat(wasmtime-cli): restore support for wasi http module #6878

Conversation

eduardomourar commented Aug 22, 2023 • edited Loading

alexcrichton commented Aug 22, 2023

eduardomourar commented Aug 22, 2023

pchickey commented Aug 22, 2023

github-actions bot commented Aug 25, 2023

Subscribe to Label Action

pchickey left a comment

Choose a reason for hiding this comment

pchickey Aug 31, 2023

Choose a reason for hiding this comment

pchickey Aug 31, 2023

Choose a reason for hiding this comment

eduardomourar Sep 1, 2023

Choose a reason for hiding this comment

pchickey Sep 1, 2023

Choose a reason for hiding this comment

pchickey commented Sep 5, 2023

eduardomourar commented Sep 5, 2023 • edited Loading

eduardomourar commented Sep 6, 2023

alexcrichton commented Sep 6, 2023

eduardomourar commented Aug 22, 2023 •

edited

Loading

eduardomourar commented Sep 5, 2023 •

edited

Loading