Skip to content
This repository has been archived by the owner on Nov 15, 2023. It is now read-only.

beefy: initialize voter from genesis and fix initial sync #11959

Merged
merged 14 commits into from
Sep 5, 2022

Conversation

acatangiu
Copy link
Contributor

@acatangiu acatangiu commented Aug 2, 2022

Now that we have justifications import (added in #11821), we can drop the "lean beefy" behaviour and start building justifications chain from Genesis with containing all past sessions' mandatory blocks justifications.

For each finality notification also walk finality tree_route to catch session changes.

During initial block import blocks are not finalized, so trying to validate and append justifications within block import fails (for initial network sync imported blocks). Fix this by:

  • Move justification validation to after inner.block_import(), so block is imported in backend and runtime api can be called to get the BEEFY authorities for said block.
  • Move append-to-backend for imported BEEFY justification to voter, because it already has the required logic to BEEFY-finalize blocks only after GRANDPA finalized them.
  • Mark voting rounds as concluded when finalizing through imported justifications as well as when finalizing through voting.

Fixes #11954

Regression test in zombienet: paritytech/polkadot#5855

Now that we have justifications import, we can drop the "lean beefy"
behaviour and start building justifications chain from Genesis with
containing all past sessions' mandatory blocks justifications.
@acatangiu acatangiu added A0-please_review Pull request needs code review. B0-silent Changes should not be mentioned in any release notes C1-low PR touches the given topic and has a low impact on builders. labels Aug 2, 2022
@acatangiu acatangiu self-assigned this Aug 2, 2022
During initial block import blocks are not finalized, so trying to
validate and append justifications within block import fails (for
initial network sync imported blocks).

Changes:

- Move justification validation to _after_ `inner.block_import()`,
  so block is imported in backend and runtime api can be called to
  get the BEEFY authorities for said block.
- Move append-to-backend for imported BEEFY justification to voter,
  because it already has the required logic to BEEFY-finalize blocks
  only after GRANDPA finalized them.
- Mark voting rounds as concluded when finalizing through
  imported justifications as well as when finalizing through voting.

Signed-off-by: acatangiu <[email protected]>
The only way we'd get _different_ _validated_ justifications for same
block number is if authorities are double voting, which will be handled
later.
@acatangiu acatangiu changed the title beefy: initialize voter from genesis beefy: initialize voter from genesis and fix initial sync Aug 2, 2022
@acatangiu
Copy link
Contributor Author

I will follow-up with a zombienet regression test for this.

Copy link
Contributor

@svyatonik svyatonik left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

client/beefy/src/import.rs Show resolved Hide resolved
backend
.blockchain()
.expect_header(BlockId::hash(*hash))
.expect("just finalized block should be available; qed.")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd also avoid calling expect here - there are db issues. concurrency, caches, mocked backends, ... they may cause their own issues. Otoh BEEFY worker may actually stop working because of that, so I can't say that logging + ignoring an error is any better :)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, that is my thinking as well, if this errors out on a session boundary the voter is dead in the water anyway.

This way at least you can see the beefy-gadget task crashing and restart it..

@acatangiu acatangiu added the D2-notlive 💤 PR contains changes in a runtime directory that is not deployed to a chain that requires an audit. label Aug 4, 2022
@bkchr bkchr requested a review from davxy August 4, 2022 12:42
BEEFY voter should resume voting from either:
  - last BEEFY finalized block,
  - session start,
whichever is closest to head.

Signed-off-by: acatangiu <[email protected]>
Copy link
Member

@davxy davxy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall LGTM. I've just added a nitpick (not super relevant) and a question

client/beefy/src/worker.rs Outdated Show resolved Hide resolved
client/beefy/src/worker.rs Show resolved Hide resolved
Copy link
Contributor Author

@acatangiu acatangiu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thank you for reviewing @davxy !

client/beefy/src/worker.rs Show resolved Hide resolved
client/beefy/src/worker.rs Outdated Show resolved Hide resolved
@acatangiu acatangiu merged commit 58d3bc6 into paritytech:master Sep 5, 2022
ark0f pushed a commit to gear-tech/substrate that referenced this pull request Feb 27, 2023
…#11959)

* client/beefy: use backend instead of client where possible

* client/beefy: initialize voter from genesis

Now that we have justifications import, we can drop the "lean beefy"
behaviour and start building justifications chain from Genesis with
containing all past sessions' mandatory blocks justifications.

* client/beefy: walk finality tree_route to catch session changes

* client/beefy: fix block import

During initial block import blocks are not finalized, so trying to
validate and append justifications within block import fails (for
initial network sync imported blocks).

Changes:

- Move justification validation to _after_ `inner.block_import()`,
  so block is imported in backend and runtime api can be called to
  get the BEEFY authorities for said block.
- Move append-to-backend for imported BEEFY justification to voter,
  because it already has the required logic to BEEFY-finalize blocks
  only after GRANDPA finalized them.
- Mark voting rounds as concluded when finalizing through
  imported justifications as well as when finalizing through voting.

* client/beefy: valid justifications are one per block number

The only way we'd get _different_ _validated_ justifications for same
block number is if authorities are double voting, which will be handled
later.

* client/beefy: process incoming justifs during major sync

* client/beefy: correct voter initialization

BEEFY voter should resume voting from either:
  - last BEEFY finalized block,
  - session start,
whichever is closest to head.

* client/beefy: test voter initialization

* client/beefy: impl review suggestions

Signed-off-by: acatangiu <[email protected]>
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
A0-please_review Pull request needs code review. B0-silent Changes should not be mentioned in any release notes C1-low PR touches the given topic and has a low impact on builders. D2-notlive 💤 PR contains changes in a runtime directory that is not deployed to a chain that requires an audit.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

client/beefy: ensure mandatory blocks on session changes by explicitly walking finality tree_route
3 participants