Binary alternative to JSON format for Web Assembly

[chicoxyzzy]

Is there any work on binary alternative to JSON which could be decoded and parsed by same parser as .wasm files? Kind of EDN but fitting solid in Web Assembly pipeline.

[lukewagner]

There is not any work on a binary format for JSON in the wasm group; that mostly seems out of scope of the stated goals. Feel free to reopen if I've misunderstood.

[chicoxyzzy]

What I mean is not binary format for JSON but something like EDN or AST representation of nested data as replacement for JSON.
https://github.com/WebAssembly/design/blob/master/BinaryEncoding.md#why-a-binary-encoding-instead-of-a-text-only-representation
It may look like wasm module with definition section (with variables names for View Source functionality) and exported variable with potentially nested data. But why wrap it into module?
Let's say we want to get data using fetch. Response could contain binary data with wasm-like AST to reduce size and parse data fast.

[lukewagner]

With WebAssembly, the AST stays internal to the wasm decoder and is never explicitly reified for user code; it's more of a spec-internal concept that helps in defining the binary format and execution behavior. IIUC what you're saying, you're interested in a new API where wasm-like bits go in and some JS/wasm-accessible AST-like data structure comes out. I see that both involve ASTs, but it's still quite outside the scope of wasm and would require a wholly new proposal that extended existing APIs (such as, in your example, fetch) and defined the form of the resulting data structure.

[qwertie]

I like your idea, but Wasm doesn't have a direct need for the JSON (or EDN, or even Loyc trees) concepts of heterogeneous lists, hashtables with string keys, etc; it is also meant to be independent from JS. So it isn't a natural fit for the wasm group to define a binary encoding for this.

Indeed, to my disappointment, even to define typed (generic, homogeneous) in-memory data structures so that wasm modules (written in different languages) can exchange data between each other is out of scope.

[kg]

You could certainly leverage the same encoding techniques as wasm for structured, strongly-typed data. But it is highly unlikely that you'd ever be able to use wasm infrastructure for it - wasm implementations are going to be engineered with a focus on decoding wasm modules, not as a general-purpose serialization infrastructure.

[matthew-dean]

@kg Could the structure of a .wasm be intepreted as data somehow? 🤔 Does the JS API allow you to decode the tree and use that as a kind of AST?

[binji]

The JS API doesn't directly, but it's possible to extract the contents of a wasm module and convert it to a JSON representation. @jgravelle-google @aardappel and I have discussed this often. I had plans to add functionality like this to wasp, but haven't created a tool for that yet.

[kg]

@matthew-dean this experiment was basically wasm-as-data, with shape transforms performed on it for compression purposes:
#1180
so you can definitely do it.