Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Make validator startup aware of Incremental Snapshots #19297

Closed
Tracked by #17088
brooksprumo opened this issue Aug 18, 2021 · 2 comments · Fixed by #19550 or #19600
Closed
Tracked by #17088

Make validator startup aware of Incremental Snapshots #19297

brooksprumo opened this issue Aug 18, 2021 · 2 comments · Fixed by #19550 or #19600
Assignees
@brooksprumo
Labels
locked issue

Comments

@brooksprumo
Copy link
Contributor

brooksprumo commented Aug 18, 2021 •
edited
Loading

(1) When a validator starts up, there is a call stack of:

1. Bank::exhaustively_free_unused_resource()
2. blockstore_processor::load_frozen_banks()
3. blockstore_processor::do_process_blockstore_from_root()
4. blockstore_processor::process_blockstore_from_root()
5. bank_forks_utils::load_from_snapshot()
6. bank_forks_utils::load()
7. validator::new_banks_from_ledger()
8. Validator::new()
9. validator/src/main::main()

...and Bank::exhaustively_free_unused_resource() needs to have the last_full_snapshot_slot parameter set correctly (right now it is None). So I need to figure out how to get that value and pass it through the call stack.

This is related to #18973

(2) Additionally, when loading the frozen banks, if a root crosses a full snapshot boundary, we need to take a full snapshot at this point, otherwise the clean from (1) maybe we wrong, and also when ABS starts up, its first incremental snapshots may be wrong.

So need to take full snapshots, plus pass the last full snapshot slot to ABS when it starts up.
@brooksprumo brooksprumo mentioned this issue Aug 18, 2021
Closed
29 tasks
@brooksprumo brooksprumo self-assigned this Aug 18, 2021
@brooksprumo brooksprumo mentioned this issue Aug 30, 2021
Merged
brooksprumo added a commit to brooksprumo/solana that referenced this issue Sep 1, 2021
@brooksprumo
Make startup aware of Incremental Snapshots
ba8aeb8 
Now that the background services are aware of incremental snapshots,
they need (1) the corect last full snapshot slot in order to clean
accounts correctly, and (2) all expected full snapshots to be available
when created an incremental snapshot based on them.

This commit fixes startup so both requirements from above are met.

At startup, the blockstore processor loads frozen banks.  Some of these
banks may be roots, and some of these roots may cross the full snapshot
interval.  If/when that happens, take a bank snapshot and queue the full
snapshot in the AccountsPackageSender.  And at the end of startup,
return the last full snapshot slot to pass into the background services.

Fixes solana-labs#19297
@brooksprumo brooksprumo mentioned this issue Sep 1, 2021
Merged
brooksprumo added a commit to brooksprumo/solana that referenced this issue Sep 1, 2021
@brooksprumo
Make startup aware of Incremental Snapshots
e808092 
Now that the background services are aware of incremental snapshots,
they need (1) the corect last full snapshot slot in order to clean
accounts correctly, and (2) all expected full snapshots to be available
when created an incremental snapshot based on them.

This commit fixes startup so both requirements from above are met.

At startup, the blockstore processor loads frozen banks.  Some of these
banks may be roots, and some of these roots may cross the full snapshot
interval.  If/when that happens, take a bank snapshot and queue the full
snapshot in the AccountsPackageSender.  And at the end of startup,
return the last full snapshot slot to pass into the background services.

Fixes solana-labs#19297
brooksprumo added a commit to brooksprumo/solana that referenced this issue Sep 2, 2021
@brooksprumo
Make startup aware of Incremental Snapshots
7ab8410 
Now that the background services are aware of incremental snapshots,
they need (1) the corect last full snapshot slot in order to clean
accounts correctly, and (2) all expected full snapshots to be available
when created an incremental snapshot based on them.

This commit fixes startup so both requirements from above are met.

At startup, the blockstore processor loads frozen banks.  Some of these
banks may be roots, and some of these roots may cross the full snapshot
interval.  If/when that happens, take a bank snapshot and queue the full
snapshot in the AccountsPackageSender.  And at the end of startup,
return the last full snapshot slot to pass into the background services.

Fixes solana-labs#19297
@brooksprumo
Copy link
Contributor Author

I merged the wrong PR...

@brooksprumo brooksprumo reopened this Sep 3, 2021
@brooksprumo brooksprumo mentioned this issue Sep 3, 2021
Merged
@mergify mergify bot closed this as completed in #19600 Sep 7, 2021
@github-actions
Copy link
Contributor

This issue has been automatically locked since there has not been any activity in past 7 days after it was closed. Please open a new issue for related bugs.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Mar 31, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees

@brooksprumo brooksprumo

Labels
locked issue
Projects
None yet
Milestone
No milestone
1 participant
@brooksprumo