syscall/js: Make syscall/js optional when compiling wasm?

[justinclift]

At present, when we compile to wasm syscall/js is automatically included.

This forces the runtime to either be a browser (eg Firefox, Chrome, etc), or at least pretend to be one.

That's kind of non-optimal, as some of the wasm execution environments presently being developed aren't targeted at browser environments. eg:

• https://github.com/Xe/olin
• https://github.com/ewasm/hera
• https://github.com/perlin-network/life
• https://github.com/cervus-v/cervus
• https://github.com/go-interpreter/wagon
• https://github.com/paritytech/wasmi
• https://github.com/nebulet/nebulet
• https://github.com/WAVM/WAVM
• https://github.com/cranestation/wasmtime

In the WebAssembly Gophers Slack channel we have people asking about non-browser use cases fairly often. It seems likely that'll be a fairly standard use case, if it can be catered to as well.

How feasible would it be to have some way to suppress the default syscall/js inclusion, or otherwise make it optional?

[Xe]

I have investigated creating a custom GOOS in the past, but it fails with that attached gnarly linker error. Guidance on it would be helpful, as implementing this would unblock the use of Go for my server side webassembly things.

[sbinet]

Loosely related to #25612 and github.com/go-interpreter/wagon

[peterbourgon]

Little to add except that I also have use cases for Go-to-wasm that don't involve a browser, and I'd be happy to chat about them to anyone interested in the issue.

[bcmills]

CC @neelance @cherrymui

[neelance]

If you don't have syscall/js, then you'd need something else to interact with the environment. Since there currently is no standard API in the WebAssembly ecosystem, I don't see why we should invent yet another API on our own. If such a standard API emerges in the future, I'd be happy to adopt it.

Today, I would implement the backend API of syscall/js as much as needed for the standard libraries, but no further. You don't need to implement full JavaScript for it. Do you see any significant downsides to this approach?

[justinclift]

@Xe @peterbourgon Probably best for you to jump in here. 😄

[Xe]

@neelance for now I have been using https://github.com/CommonWA/cwa-spec as an imaginary system ABI. What calls would we need to add to this for Go? The time call is one of them.

[neelance]

Is there any effort to get https://github.com/CommonWA/cwa-spec adopted by https://github.com/WebAssembly, maybe as an official spec?

[justinclift]

CC @losfair

[peterbourgon]

@neelance

If you don't have syscall/js, then you'd need something else to interact with the environment.

Essentially I want to punt on this question. I want to compile Go down to a WebAssembly module, exporting a set of functions as a sort of public API, and binding to or declaring (somehow) a set of import functions, to consume other modules. I want to load and run this module in a server-side environment, with no relation to JS. CommonWA is not of much interest to me (us) in this context.

[neelance]

@peterbourgon

Would you want to call the functions concurrently or only one at a time?
Which types of parameters and return values do you expect?
Would they do HTTP requests to other services?

[peterbourgon]

@neelance I wouldn't take anything off the table in terms of functionality. If you're looking for some kind of examples, I can try to dig something up; let me know.

edit: To be clear, I would want to be able to answer each of those questions differently, in different modules.

[Xe]

@peterbourgon in this case using CommonWA (note: I am not married to CommonWA, it's just not something made by me and looks like it could easily be the basis we could build off of as a community to make it better for everyone) would us all just be standardizing on the set of things the binaries the Go compiler emits would be depending on. These things would then be implemented by the JS side of things too.

[justinclift]

@peterbourgon

If you're looking for some kind of examples, I can try to dig something up; let me know.

Please do, they'd likely help. 😄

[DavidHuie]

I have a similar use case to @peterbourgon in mind. Wasm on the server creates the opportunity for new types of features, such as user developed, server executed extensions to products. I'd love to use Go for these sorts of things!

[neelance]

Sorry, but I still don't see why syscall/js prevents people from using non-JavaScript hosts. It is only an interface, it should be possible to implement it without having full-blown JavaScript.

[Xe]

What can non-JavaScript hosts do to stub it out or "break" all of its functionality?

[neelance]

@Xe I don't get your question.

[Xe]

@neelance What is the minimal implementation required for every function to have to return an error-like response?

[neelance]

@Xe That of course depends on the function. I would just start with an empty stub of each function which logs its execution and then add the implementation step by step.

[Xe]

Yes, can you please provide these empty stubs? I can't figure them out on my own apparently.

[neelance]

I don't even know which WebAssembly host you are trying to use.

[neelance]

The import functions that you need to stub are the ones you find in

go/misc/wasm/wasm_exec.js

Line 199 in 7a6ccec

this.importObject = {

[Xe]

For all of these functions: https://github.com/golang/go/blob/master/src/syscall/js/js_js.s

• What is the stack layout in relative ordinal bytes?
• What values are arguments?
• What values are returns?
• What is the "zero" return?
• What is a C-like analogue?

These are things that I must know to implement these calls and have not been able to reverse-engineer on my own.

[neelance]

It's the normal Go calling convention. All arguments and return values live on the stack. You can find the function signatures in https://github.com/golang/go/blob/master/src/html/template/js.go and in the wasm_exec.js I linked above. You can also find the stack offsets in the wasm_exec.js if you don't want to calculate them on your own. A quick Google search got me this: https://science.raphael.poss.name/go-calling-convention-x86-64.html

[neelance]

C has a different calling convention than Go, so you won't be able to use simple C function signatures.

[Xe]

I realize the calling convention is different, I would translate that by hand. I want to just know what i should return for the values such that it will get error-like responses, consider go.runtimeNanotime:

The stack layout in relative ordinal bytes is something like this:

sp+0 -> stack pointer
sp+8 -> expected int64 return value

There are no arguments to this function and there is one int64 return that signifies an integer number. The zero return in this function is the value zero, as such:

func runtime.nanotime() int64 {
  return 0
}

The idea is that this kind of basic documentation about what the ABI expects (and what values are zero-like) will provide people who want to use Go in WebAssembly on the server side guidance on how to stub out functionality that they don't want to support (for example, the very reasonable case of being not a browser and not having a javascript VM involved, so Javascript interaction is not needed and thus removable). Can you please help?

[neelance]

@Xe Sorry, I don't have that right now. The os package will also require syscall/js to not return zero all the time. I can only recommend that you start with a simple example, add what's necessary to make it work and then continue from there. I can help you further if you have a specific error that you don't know how to resolve.

[Xe]

Then if it shouldn't return 0 all the time, what is the valid thing to do?

[neelance]

Fulfill the interface good enough that it supports the actions that you want to support or that it at least does not crash on initialization. Specifics depend on the case.

[neelance]

I can help you further if you have a specific error that you don't know how to resolve.

This still holds. I don't see how I can give you more information without implementing it myself. There is no specification that you can fulfill in one go. You'll have to start with some concrete example that you want to make pass and then we can figure out together what is necessary to do so.

[neelance]

I've written a proof of concept since so many people were asking for it: https://github.com/neelance/go_js_wasm_exec

[justinclift]

Thanks heaps @neelance. 😄

[backkem]

Exploring options for supporting different types of IO in neelance/go_js_wasm_exec#1.

[backkem]

There is initial work being done on a CloudABI target at NuxiNL/go. This is the system ABI the CraneStation/wasmtime runtime plans to use.

[justinclift]

This is probably relevant as well: https://hacks.mozilla.org/2019/03/standardizing-wasi-a-webassembly-system-interface/

[neelance]

Awesome! This is exactly what I was waiting for. I'll look into it soon.

[tiborvass]

Does this mean we'll have a GOOS=wasi ?

[peterbourgon]

@neelance It was suggested to me that anyone working on wasi for Go might want to join #wasi on the Mozilla IRC—just passing that info along.

[justinclift]

#31105 sounds related too. 😄

[mfateev]

I believe for execution Go doesn't need any environment support. Even random seed and time can be hardcoded and it is fine for a lot of use cases. Forcing every host to provide a dummy implementation of WASI or syscall/js sounds very artificial. BTW when Rust and C compile to wasi they don't have any required imports.

Ideally I would love to have the compiler options to not include some Go standard library components at all. For example I would prefer that in some cases the code that references any file or socket IO wouldn't even compile.

[neelance]

I believe for execution Go doesn't need any environment support.

If there is no environment, then your Go code can calculate something, but then only discard the result.

when Rust and C compile to wasi they don't have any required imports.

Neither Rust nor C has a runtime/goroutines. You can directly call a Rust/C function from your WebAssembly host. This is not easily possible with Go's semantics.

[mastersingh24]

TinyGo does not have this issue as they do not assume a JS host environment.

[rsc]

This issue is titled "make syscall/js optional" while #31105 is instead to assume wasi.
As noted above, you have to have something or else there's no way to interact with the outside system.
Perhaps this issue should be closed as a duplicate of #31105?

For wasi, #31105 (comment) describes a way to use Go with it.

[justinclift]

Perhaps this issue should be closed as a duplicate of #31105?

Not sure personally. I don't use Go for wasm any longer, so no longer have any err... skin in this issue.

If other people do, I'll close or leave this open as seems appropriate. 😄