Try-runtime Revamp and Facelift

[kianenigma]

Changelog

remote-externalities

• (CHANGED) pallets in OnlineConfig no longer has the interpretation of an empty list implying everything. pallets: vec![foo] is simply a shorthand for injecting twox(foo) into the list of hashed prefixes that we want to scrape. To remain backwards compatible in most cases, if the list of hashed prefixed is empty, we inject vec![] into it, which implies "download everything".
• (CHANGED) snapshot format has changed and now is a single file, and contains block hash as well. closes try-runtime: make snapshot file contain more data #12194
• (ADDED) Value download is multi-threaded for both the main and child-tree. Significantly improving performance. The number of threads used can be set in OnlineConfig. closes remote-externalities: parallelize value download #12044
• (ADDED) state_version, if read from remote, will by default be set to whatever the remote node has. If snapshot is created, the state_version is kept and restored. The user can still overwrite this, but there's really no reason to use this option.

Try-runtime CLI

• (ADDED) new create_snapshot command. Should help us with https://github.com/paritytech/devops/issues/2083 and such.
• (ADDED) the state_version of the local node and the remote node are checked.
• (CHANGED) on-runtime-upgrade can now optionally (--checks) call into pre_upgrade/post_upgrade/try_state hooks. The old behavior was basically passing in --checks by default, no longer the case. closes try-runtime: fix proof size #12195 and closes TryState: automatically execute after each migration #12130.
• (ADDED) everything in try-runtime CLI now uses WasmExecutor. Thus, we can specify enum Runtime { existing, path } which simplifies a lot of things. Closes try-runtime - specifying runtime & cutting off unnecessary configuration burden #12156
• Make execute-block more reproducible.

The main change that needs update in tools and CLIs is that a wasm file compiled with feature=try-runtime need to be provided separately. So a "test migration" process is as follows:

cargo build --release --feature try-runtime 
cargo run --release --feature try-runtime try-runtime --runtime ./target/release/... -lruntime=debug on-runtime-upgrade ...
cargo clean 

Also be aware of:

• (CHANGED) on-runtime-upgrade can now optionally (--checks) call into pre_upgrade/post_upgrade/try_state hooks. The old behavior was basically passing in --checks by default, no longer the case.

and if you want to do. full migration testing, run with --checks, and without it if you prefer having a more accurate sense of PoV.

---

polkadot companion: paritytech/polkadot#6187

• cumulus companion:
• Update docs.

[kianenigma]

from @ggwpez

Do you think we could add some state-diff to the try-runtime which outputs which keys got modified by a migration?
Some summary like: The following pallets had these storage elements touched y times: …

I can integrate this with a whole new Externalities type that:

1. tracks writes in a more readable way.
2. can load data on demand.

[crystalin]

Adding my request:
We would want to re-use try-runtime in the process of testing more manual element. For exemple, to demonstrate what a new pallet or feature would do on an existing chain state, or to verify the impact of a given change would be on multiple blocks.
For that, the try-runtime could support a command fork which would basically start producing blocks on top of the best block and have some checks (like authoring) disabled to allow the node to keep producing.

Additionally, we could submit any extrinsic without it being verified. That could allow additional scenarios.
Also it would allow to have the RPC services enabled.

As a security requirement, I think try-runtime should disable sending produced blocks (avoiding to mess with the real chain)

[librelois]

It would be nice to have an option in follow-chain that allows to lazily download the onchain storage, download a key/value only when we need it in the block execution.
The problem with follow-chain at the moment is that it downloads the whole storage while a large part of it might not be needed to execute the blocks.

[moonbeam-build-bot]

Additionally, try-runtime could also support exporting some useful information, One of those is the state "Get" and "Put" (that is available through state=trace logs) but providing the actual pallet/method name in addition to the key.
This would be exported as a json file (an array of all the state actions taken during a block) and available through a cli argument

[crystalin]

@kianenigma also it needs to improve the memory footprint, currently I can't try-runtime execute block with some of our Moonbeam networks, using a 2.5GB state (12M keys, https://alan-stuff.s3.amazonaws.com/3069691.top) takes > 64GB

[kianenigma]

It would be nice to have an option in follow-chain that allows to lazily download the onchain storage, download a key/value only when we need it in the block execution. The problem with follow-chain at the moment is that it downloads the whole storage while a large part of it might not be needed to execute the blocks.

Indeed, this will be addressed.