Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bank snapshots #3202

Closed
wants to merge 1 commit into from
Closed

Conversation

sakridge
Copy link
Member

@sakridge sakridge commented Mar 8, 2019

Problem

Full node startup is slow when many transactions have been processed.

Summary of Changes

Serialize a bunch of bank state into a snapshot and try to restore from that on boot.

TODO:

  • error handling
  • test more complicated forkings
  • StatusCache::merges
  • BankForkInfo generation
  • snapshot flag plumbing

Fixes #2475

@sakridge sakridge force-pushed the bank-snapshots branch 6 times, most recently from 1c0e9a2 to 400e39c Compare March 9, 2019 00:09
@codecov
Copy link

codecov bot commented Mar 9, 2019

Codecov Report

Merging #3202 into master will decrease coverage by 5.9%.
The diff coverage is 84.4%.

@@           Coverage Diff            @@
##           master   #3202     +/-   ##
========================================
- Coverage    81.3%   75.4%     -6%     
========================================
  Files         129     129             
  Lines       19543   21537   +1994     
========================================
+ Hits        15894   16245    +351     
- Misses       3649    5292   +1643

@garious
Copy link
Contributor

garious commented Mar 9, 2019

This looks like a pretty substantial maintenance burden. Every time we add a field to anything in solana_runtime, we'll need to update some custom serialization routines? This stuff looks pretty mechanical at first glance. Think we could write a custom derive macro or something like that? https://doc.rust-lang.org/edition-guide/rust-2018/macros/custom-derive.html

@sakridge
Copy link
Member Author

sakridge commented Mar 9, 2019

@garious that stuff is really hairy, but let me look into it.

@sakridge
Copy link
Member Author

@sambley are you able to take over this PR/issue? #2475

@sambley
Copy link
Contributor

sambley commented Mar 11, 2019

@sakridge, sure I can take a look. Are all the TODOs listed in this pending to be done in addition to the custom serialization comment from Greg.

@sakridge
Copy link
Member Author

@sambley yes, that's right.

@sambley sambley self-assigned this Mar 17, 2019
@sakridge
Copy link
Member Author

@sambley any chance to look at bank snapshots?

@sambley
Copy link
Contributor

sambley commented Mar 22, 2019

@sambley any chance to look at bank snapshots?

@sakridge, just started taking a look at it. currently looking at #3425 and will work it after that.

@sakridge
Copy link
Member Author

@sambley cool. thanks

@aeyakovenko
Copy link
Member

Close this one?

@aeyakovenko
Copy link
Member

AccountsDB supports a scan method that reads from storage. Here is an example. https://github.com/solana-labs/solana/blob/master/runtime/src/accounts.rs#L305

Using the latest write version for each account/fork will allow you to rebuild the index.

@sakridge sakridge closed this Apr 22, 2019
@sakridge sakridge deleted the bank-snapshots branch April 22, 2019 16:32
brooksprumo pushed a commit to brooksprumo/solana that referenced this pull request Oct 17, 2024
ci: use macos-13 for client_targets pipeline
willhickey pushed a commit to willhickey/solana that referenced this pull request Oct 29, 2024
) (solana-labs#3207)

ci: remove deprecated macos-12 image (solana-labs#3202)

ci: use macos-13 for client_targets pipeline
(cherry picked from commit c6e8239)

Co-authored-by: Yihau Chen <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Fullnode startup is slow
4 participants