ZombieNet: ensure that asynchronous backing logic is backwards compatible with v1

[rphmeier]

The test is simple: start a 2-node testnet with 1 parachain, where one node runs master and the other runs the asynchronous backing feature branch. Set the group size to 2 in genesis and ensure that the parachain progresses at the expected rate.

[eskimor]

@sandreim Can someone on Engineering take that?

[sandreim]

Sure, is the branch in healthy state right now ?

[eskimor]

No, but I am working on it.

[bredamatt]

Just to confirm, is this https://github.com/paritytech/polkadot/tree/rh-async-backing-statement-distribution the branch?

[eskimor]

No actually it is: rh-async-backing-feature
#5022

The branch referenced by you will be merged into rh-async-backing-feature once ready.

[eskimor]

Correction, there seems to be an issue with prospective parachains.

[eskimor]

Ok, branch should be ready - I believe the failing Zombienet tests are unrelated.

[bredamatt]

Not sure why this happens, but I did a checkout of the branch and attempted building - then I saw this issue:

error: `sp_trie::recorder::Recorder<H>::as_trie_recorder::{opaque#0}<'_>` does not live long enough
   --> /Users/mattia/.cargo/git/checkouts/substrate-7e08433d4c370a21/1f720c1/primitives/state-machine/src/trie_backend_essence.rs:181:44
    |
181 |         let recorder = recorder.as_mut().map(|r| r as _);
    |                                                  ^

error: `sp_trie::recorder::Recorder<H>::as_trie_recorder::{opaque#0}<'_>` does not live long enough
   --> /Users/mattia/.cargo/git/checkouts/substrate-7e08433d4c370a21/1f720c1/primitives/state-machine/src/trie_backend_essence.rs:219:44
    |
219 |         let recorder = recorder.as_mut().map(|r| r as _);
    |                                                  ^

   Compiling sp-weights v4.0.0 (https://github.com/paritytech/substrate?branch=master#1f720c11)
error: could not compile `sp-state-machine` due to 2 previous errors

I had a similar issue on a branch related to a different PR, so not sure if its solely my repo. What solved it was a merge of the latest commits on master to the branch. @eskimor do you have this issue when you try cargo check or cargo build at all?

[bkchr]

@bredamatt this is fine. The latest rustc nightly has some regression. We fixed this for now in Substrate

[bredamatt]

@bkchr indeed a toolchain thing, thanks for pointing me in that direction.

[bredamatt]

Seems to be working just fine with this zombienet configuration:

[settings]
timeout = 1000

[relaychain]
chain = "rococo-local"

[relaychain.genesis.runtime.runtime_genesis_config.configuration.config]
  max_validators_per_core = 2

  [[relaychain.nodes]]
  name = "alice"
  command = "polkadot-async" // This is the binary from the async backing branch
  extra_args = [ "--alice" ]

  [[relaychain.nodes]]
  name = "bob"
  command = "polkadot" // this is the master branch binary
  extra_args = [ "--bob" ]

[[parachains]]
id = 100
cumulus_based = true

  [parachains.collator]
  name = "collator01"
  command = "parachain-template-node"
  args = ["-lparachain=debug"]

What is the best way to share the findings for this type of issue? Should we simply share the logs here, or use some other form of visibility, or potentially write a zombienet feature test and add it to the branch?

[sandreim]

I think it makes sense to keep the CI test in the async branch for now, just to make sure we dont break things as we go fwd.

[rphmeier]

Yeah, @bredamatt please open a PR against the feature branch and file a follow-up issue to remove the test once synchronous backing is obsolete.

[bredamatt]

@rphmeier @eskimor @sandreim waiting for the above before finishing up the CI test on the async branch.

[bredamatt]

@eskimor seems there are some conflicts on the rh-async-backing-feature branch wrt. to the CI tests. @pepoviola, could you mention which files need to be checked out from the master branch into the rh-async-backing-feature branch? If they are not too scary, I could take them on myself so I can sync them into the branch I am adding the integration tests into.

[rphmeier]

We should just merge rh-async-backing-feature with master. I believe @slumber has plans to do this in the near future

[sandreim]

done in #6314