Add snapshot_utils::bank_from_latest_snapshot_archives()

[brooksprumo]

While reviewing PR #18565, as issue was brought up to refactor some code
around verifying the bank after rebuilding from snapshots. A new
top-level function has been added to get the latest snapshot archives
and load the bank then verify. Additionally, new tests have been
written and existing tests have been updated to use this new function.

Fixes #18973

While resolving the issue, it became clear there was some additional
low-hanging fruit this change enabled. Specifically, the functions
bank_to_xxx_snapshot_archive() now return their respective
SnapshotArchiveInfo. And on the flip side,
bank_from_snapshot_archives() now takes SnapshotArchiveInfos instead
of separate paths and archive formats. This bundling simplifies bank
rebuilding.

[codecov]

Codecov Report

Merging #18983 (ffc397c) into master (592013e) will increase coverage by 0.0%.
The diff coverage is 86.1%.

@@           Coverage Diff            @@
##           master   #18983    +/-   ##
========================================
  Coverage    82.8%    82.9%            
========================================
  Files         449      449            
  Lines      128281   128344    +63     
========================================
+ Hits       106280   106410   +130     
+ Misses      22001    21934    -67     

[lijunwangs]

Looks good to me overall.

Pull request has been modified.

[brooksprumo]

@carllin @ryoqun Let me know if you'd like more time to review this PR. If yes, leave a comment here. If not, no worries! I won't merge before tomorrow mid-day.

[carllin]

Side note, I think a good way to reduce this code duplication between FullSnapshotArchiveInfo and IncrementalSnapshotArchiveInfo is to create a trait SnapshotArchiveInfo like so:

trait SnapshotArchiveInfo {
   // constructor 
    fn from_snapshot_archive_info(snapshot_archive_info: SnapshotArchiveInfo) -> Self;

    // getter
    fn snapshot_archive_info(&self) -> &SnapshotArchiveInfo;

  // default methods to avoid duplication
    fn hash(&self) -> &Hash {
        &self.snapshot_archive_info().hash
    }

    fn archive_format(&self) -> &ArchiveFormat {
      &self.snapshot_archive_info().archive_format
     }
}

and have FullSnapshotArchiveInfo and IncrementalSnapshotArchiveInfo implement this trait

[brooksprumo]

I like this idea! One thing is that the IncrementalSnapshotArchiveInfo needs a base_slot parameter too. Do you know if there's a way to handle this within the Trait?

[brooksprumo]

I'm implemented the trait but without a constructor. I think for now that should be good.

The next thing I'm working on is adding in incremental snapshot support to the background services (AccountsBackgroundService, AcccountsHashVerifier, SnapshotPackagerService). I'll by building out the SnapshotArchiveInfo more. In particular, moving Full and Incremental under a single enum, so that way when a background service makes a package, it'll contain the info about full vs incremental, and it'll carry through to the next service.

So yeah, is this OK for now? I did change slot() and archive_format() to not return references anymore, since they are Copy and small, and always used as values anyway.

[carllin]

heh just clarifying, the real hashing/verification of the account state actually happens in here, when we call down to

verify_snapshot_bank() -> verify_bank_hash() -> verify_bank_hash_and_lamports() -> calculate_accounts_hash_helper_with_verify(), and we verify the
calculated_hash of the accounts state against the expected serialized bank_hashes in the snapshot

Pull request has been modified.