Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

State sync 1 - Blocks backfilling option #353

Closed
wants to merge 59 commits into from

Conversation

abi87
Copy link
Contributor

@abi87 abi87 commented Oct 5, 2023

Why this should be merged

Counterpart of State sync - Blocks backfilling option

How this works

Missing stuff:

  • UTs
  • Add ETA to complete backfilling
  • What if state sync is re-run again before block backfilling completes? Current implementation would redownload a bunch of already downloaded blocks. Avoid that

How this was tested

Extra UTs + Fuji run (ongoing)

Comment on lines 98 to 99
BackfillBlocksEnabled(ctx context.Context) (ids.ID, uint64, error)
BackfillBlocks(ctx context.Context, blocks [][]byte) (ids.ID, uint64, error)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not sure where these methods should be implemented. They are part of the StateSync interface so this is the natural place; but their implementation needs the full vm (to parse and store blocks) which is not passed to the state sync client.
For now I end up providing functions to parse and get blocks via the config, not sure if we have a different canonical way to do this in coreth

break
}

if err := blk.Accept(ctx); err != nil {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is most likely wrong. Accepting a past block seems to mess up a whole bunch of metadata.
@darioush I'd love a feedback on how the indexing of past block should be handled (brand new methods? Upgrade block.Accept to handle past blocks? something else?).

@abi87 abi87 force-pushed the backfill_blocks branch 2 times, most recently from 7d716b6 to b7d5280 Compare November 5, 2023 10:48
@abi87 abi87 self-assigned this Nov 13, 2023
@abi87 abi87 changed the title State sync - Blocks backfilling option State sync 1 - Blocks backfilling option Nov 20, 2023
@abi87 abi87 changed the base branch from master to backfill_blocks_0 November 20, 2023 11:49
@darioush darioush closed this Aug 22, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Archived in project
Status: Backlog 🗄
Development

Successfully merging this pull request may close these issues.

2 participants