Add chainHead_unstable_wasmQuery

[tomaka]

Add a JSON-RPC function that allows you to pass a user-chosen Wasm program that has access to the chain storage and builds a return value.

The advantage is that this program could be sent over the network to a full node, which executes it, returns a proof, and sends the proof back to a light client, which then runs the program again to verify the proof.

Things that should be done:

• Add the networking protocol between the light and full node.
• Figure out how to let frontend JS developers easily create these Wasm programs. We probably don't want to ask them to setup a complicated Rust toolchain and let them figure out how to optimize the size of the Wasm program and all.

Neither of these two things strictly needs to be done before we add the JSON-RPC function.
A light client can emulate this function by executing the Wasm program locally and doing a storage query whenever a key is read by the program. This is not great but it works.

[harrysolovay]

Some questions/notes from an offline conversation about this issue:

TL;DR: how can we enable derived queries to be executed in one round trip?

Is WebAssembly the right solution?

Some reasons that it might be:

• Full nodes embed a wasm executor
• The wasm can contain SCALE encoding/decoding
• Conditional logic, error handling, etc.
• Sandboxed
• Relatively small

How do users form the wasm blobs?

• In an out-of-band processing step?
• Within the JS runtime?
• Babel plugin / TypeScript transformer that turns a subset of JS into wasm?

Does this problem require a tiny query language?

• Something like GraphQL
• Aware of runtime types
• Able compile to wasm text

[pepyakin]

Some notes:

1. On the implementation side, this would require something like this Runtime API/executor: limit the execution of runtime call substrate#10648 to prevent endless execution on the remote.
2. There were discussions about something like this in the context of light clients. The logic was that the dApps need to have small latency to be able to provide a good user experience. However, sometimes the dApp needs to access data that depends on another piece of data. In that case, it has to suffer several round trips. However, if a light client could send a wasm blob that executed on the node serving the light client. That would allow collecting the required data in one go thus improving the latency of the app.

[tomaka]

There were discussions about something like this in the context of light clients. The logic was that the dApps need to have small latency to be able to provide a good user experience. However, sometimes the dApp needs to access data that depends on another piece of data. In that case, it has to suffer several round trips. However, if a light client could send a wasm blob that executed on the node serving the light client. That would allow collecting the required data in one go thus improving the latency of the app.

That's precisely the use case why this function is desired!

[jeluard]

Cool stuff!

Pushing this idea a little further, the wasm blob could be long lived. It could then be notified of storage updates and send back some chunk of data (only meaningful when using long lived connection like WebSockets).
Markets could be build by node owner around this capacity.

How technically realistic would that be?

[tomaka]

In order to add this to the API, it could be as simple as adding an additional optional wasmBlob parameter to chainHead_unstable_call. When set, the server has to use wasmBlob to do the call instead of the chain's runtime.

However I don't like the fact that you have to send the whole Wasm bytecode every single time and that the server would have to maintain an LRU cache of the wasmBlobs.
I think that there should be two new functions named something like uploadWasm and releaseWasm, and the extra parameter to chainHead_unstable_call is instead a reference to a Wasm you've uploaded.

[tomaka]

As far as I can tell, there are only two tricky points to solve:

• What's the maximum allowed size of a Wasm blob?
• What's the maximum allowed "gas" when executing the Wasm?

Having limits on the server is pointless if the client doesn't know what these limits are, because the client needs to make sure to be below these limits. Meaning that we have to precisely specify these limits. For the maximum allowed gas, this is tricky.