Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Discover and download incremental snapshots during bootstrap #19885

Closed
4 tasks done
Tracked by #17088
brooksprumo opened this issue Sep 14, 2021 · 2 comments
Closed
4 tasks done
Tracked by #17088

Discover and download incremental snapshots during bootstrap #19885

brooksprumo opened this issue Sep 14, 2021 · 2 comments
Assignees

Comments

@brooksprumo
Copy link
Contributor

brooksprumo commented Sep 14, 2021

Proposed Implementation

Firstly:

  • If snapshot fetch is disabled (--no-snapshot-fetch), then use existing behavior (no changes)
  • If incremental snapshots are not enabled (no --incremental-snapshots flag), then use existing behavior (no changes)
  • If incremental snapshot fetch is disabled (--no-incremental-snapshot-fetch flag), then use existing behavior (no changes)
  1. Get all incremental snapshot hashes from CRDS for trusted nodes
  2. Get all incremental snapshot hashes from CRDS for all nodes
  3. Retain only the hashes from (2) that match a value from (1) (and their associated rpc addrs/ids)
  4. Retain only the hashes from (3) that have a full snapshot slot that is a multiple of our full snapshot interval
    - We want to avoid downloading full snapshots when possible
  5. Retain only the hashes from (4) that have the highest full snapshot slot
  6. Retain only the hashes from (5) that have the highest incremental snapshot slot
  7. Download the full snapshot (if we don't already have it)
  8. Download the incremental snapshot (if we don't already have it)
@brooksprumo brooksprumo mentioned this issue Sep 14, 2021
29 tasks
@brooksprumo brooksprumo changed the title Figure out how a new node discovers and then downloads full+incremental snapshots [Incremental Snapshots] Figure out how a new node discovers and then downloads full+incremental snapshots Sep 14, 2021
@brooksprumo brooksprumo self-assigned this Sep 14, 2021
@solana-labs solana-labs deleted a comment from lcgogo Sep 16, 2021
@brooksprumo brooksprumo changed the title [Incremental Snapshots] Figure out how a new node discovers and then downloads full+incremental snapshots Discover and download incremental snapshots during bootstrap Sep 30, 2021
@brooksprumo
Copy link
Contributor Author

DONE!

@github-actions
Copy link
Contributor

This issue has been automatically locked since there has not been any activity in past 7 days after it was closed. Please open a new issue for related bugs.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Mar 30, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

1 participant