Upload JSON schema alongside code contract

[fadeev]

The proposal is to upload JSON schema to the chain alongside code contract, so clients can use this information to dynamically scaffold user interfaces. Having a contract.json file that includes a schema as well, as perhaps, a description of what this code does can be very useful for frontend clients. Even if it can't be securely verified on chain (that this schema was in fact generated from the same source), it's still very valuable source of introspection.

In practice, what we want is for frontend components to dynamically generate UIs (inputs, buttons) based on schema without developers having to manually hard code these things for each code/contract.

Had a call about this with @ethanfrey. Please, feel free to make corrections 🙂 this is just an initial proposal.

[webmaster128]

So far, the existing tooling extracts the JSON schema from the folder ./schema/*.json of the source code tarball URL stored in the source meta field. So the data is available already. However, I find it quite inconvenient to work with source code publications, especially during development since the development loop is quite long:

1. Create new Cosmwasm release and deploy to crates.io
2. Update contract to the new version
3. Choose a new version number for the contract, recompile, check-in, deploy
4. Copy the contract to the development folder, update source and builder

Do I understand correctly that this is about moving the schemas from the source code onto the blockchain directly?

And if we do that, what about storing the entire source on chain? The source of https://crates.io/api/v1/crates/cw-nameservice/0.4.0/download has 26 KB (compressed), which is quite small compared to the 118 KB Wasm (uncompressed) (https://github.com/CosmWasm/cosmwasm-examples/blob/nameservice-0.4.0/nameservice/contract.wasm).

[fadeev]

Do I understand correctly that this is about moving the schemas from the source code onto the blockchain directly?

Yes, exactly. The idea is that a few KB on chain is worth having a luxury not downloading and unziping the whole source to get to the schema (and you may want to download schema of many contracts, depending on UI).

And if we do that, what about storing the entire source on chain?

From the front-end side of things that's definitely interesting. You won't be running contract code on the client, but inspecting it might prove useful.

[ethanfrey]

This came up in a discussion with @fadeev this morning and I fully support adding the schema as some JSON on chain. This is super useful to help figure out how to connect, even if it might not be the "fully correct" api, it will still be helpful.

To make it useful, the uploading tools should easily provide this data. Maybe just point them to the schema directory and they format the schemas into one json and upload it with the code.

[ethanfrey]

As to uploading the code... I have reservations, as this is security critical and people will trust it implicitly. If we do have it, we would want to add an on-chain vouching system...

Actually, this would make a good smart contract use case ;) Let us think that one a bit more before adding it.

[ethanfrey]

One other thought, the wasm data is not written to the iavl store, just the hash is. And the wasm is written directly to disk by the rust code. Storing 6kb (schema json) or 28kb (source code) in the iavl tree may be a bit much for it... it will definitely cost 7000 / 30000 gas to read it (and 2x to write it) and should be stored separately where it is only loaded for those external queries that need it, not on any tx processing (besides uploading code).

But yeah, let's figure out what behavior we want before worrying too much about how to implement it. I do agree easy access to the schema (from upload to usage) would be a benefit.

[webmaster128]

On the 0.10 demonet we see this issue again: codes from cosmjs master (which are there for local development not for deploying to anywhere else) have been deployed to the 0.10 demonet, with outdated or placeholder meta like

• Source: https://crates.io/api/v1/crates/cw-nameservice/not-yet-released/download (placeholder value)
• Builder: cosmwasm/rust-optimizer:0.8.0 (wrong; this was 0.9.0)

(Don't get me wrong, I want to point out the source publication route is very hard to get right.)

As a consequence the contract cannot be verified, which is okay. But we also don't get the interfaces which require code verification for the source code <-> checksum connection.

If you look at the JSON schema as a build result, it would even make sense to remove it from the source code publication and repo.

The source code package from https://crates.io/api/v1/crates/cw-nameservice/0.4.0/download has (uncompressed) 97 854 bytes total including 2712 bytes schema. When you use a compact JSON notation (no compression or change of the document, just no whitespace), you get 1461 bytes of schema. So it is probably reasonable to store that along with the Wasm blob.

[okwme]

Also interested in this as a feature but agree it's maybe tricky to do it well. Just for reference Ethereum has some aspects of this feature in that the final line of the deployed contract code is the SWARM hash code. That means that assuming the source code got uploaded to Swarm, you should be able to find it by this hash. The reality is that no one uses swarm and none of the tooling even attempts to upload compiled contracts to the network. I think that is a real missed opportunity and might be a method that we could adopt and actually utilize. Whether this would take place with IPFS or some other system is more of a question of implementation but would be curious as to your opinions on that method in general.

This issue also reminds me of the discussion on the SDK about similar introspection that would allow for the scaffolding of frontends: cosmos/cosmos-sdk#4322. It was closed with reference to these capabilities coming out of the new updates to gRPC. To be honest I don't fully understand what that solution looks like but wonder if there's something similarly available for CosmWasm code.

[ethanfrey]

Our json schema files are the equivalent of publishing the proto definitions behind the grpc. The question is how to update all this.

1. Clearly the build/upload tooling needs to be better to prevent human error / laziness.
2. We allow arbitrary source urls, which is a superset of swarm hash, and try to point to crates.io or github releases, which are already integrated in the normal dev flow so this can work. But again, if I forget to set those fields....
3. I agree the schema is quite small. If take all files from the directory and combine into one blob, then export in compressed (not pretty) json, I think this will be a great artifact to upload. (We could even gzip if we care, but still not too big).
4. Uploading 2 files can quickly get out of sync. What about simply adding a schema() call to any contract that should return this blob, and auto-generating the static json string in the build step and injecting this export into the binary with some simple macro.

This would increase the wasm size by ~2kb but guarantee we can quickly get the info out.

[ethanfrey]

It seems rather inflexible to keep sticking all this functionality inside the wasmd engine.

Some internal discussions proposed producing an open-source web service that works like etherscan and will validate the contracts for you and produce all the schema bindings, etc. Like etherscan. It is a "trusted 3rd party" but would give us a whole lot more flexibility to iterate on this, and then maybe a future version of wasmd could support it via a plugin or contract that allows the features we chose.