[TieredStorage] Use IndexOffset in TieredStorageMeta and get_account() #35046
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Codecov ReportAttention:
Additional details and impacted files@@ Coverage Diff @@
## master #35046 +/- ##
=======================================
Coverage 81.6% 81.6%
=======================================
Files 830 830
Lines 224949 224947 -2
=======================================
+ Hits 183627 183652 +25
+ Misses 41322 41295 -27 |
yhchiang-sol
added a commit
to yhchiang-sol/solana
that referenced
this pull request
Feb 13, 2024
…labs#33964) [TieredStorage] Improve param naming of IndexBlockFormat (solana-labs#34033) [TieredStorage] HotStorageReader::get_account_offset (solana-labs#34031) [TieredStorage] Rename owners_offset to owners_block_offset (solana-labs#34047) [TieredStorage] HotStorageReader::get_account_address (solana-labs#34032) [TieredStorage] OwnersBlock (solana-labs#34052) [TieredStorage] HotStorageReader::get_owner_address (solana-labs#34053) [TieredStorage] Define OwnerOffset as u32 (solana-labs#34105) [TieredStorage] Use OwnerOffset type in TieredAccountMeta (solana-labs#34106) Refactors TieredStorageFile read/write methods (solana-labs#34147) [TieredStorage] Make IndexBlock persist u32 offsets (solana-labs#34133) [TieredStorage] Make IndexOffset use u32 (solana-labs#34152) Move MatchAccountOwnerError from append_vec to accounts_file (solana-labs#34187) [TieredStorage] Make AccountOffset use u32 (solana-labs#34151) [TieredStorage] Allow HotStorage to handle more account data (solana-labs#34155) [TieredStorage] Make AccountOffset a trait, introduce HotAccountOffset (solana-labs#34335) [TieredStorage] Improve comments for HOT_ACCOUNT_ALIGNMENT (solana-labs#34404) [TieredStorage] Unit-tests for checking invalid HotAccountOffset (solana-labs#34376) [TieredStorage] Boundary check for accessing hot account meta (solana-labs#34349) [TieredStorage] boundary check for get_account_address() (solana-labs#34529) Sanitizes tiered storage footer after reading from disk (solana-labs#34200) Adds read/write/get_pod() fns to tiered storage (solana-labs#34415) Uses consistent error types in tiered storage (solana-labs#34110) [TieredStorage] Boundary check for get_account_offset() (solana-labs#34531) [TieredStorage] HotStorageReader::account_matches_owners (solana-labs#34350) [TieredStorage] Fix typos in index.rs (solana-labs#34546) [TieredStorage] HotAccountsReader::get_account (solana-labs#34499) [TieredStorage] Rename AddressAndBlockOffsetOnly to AddressesThenOffsets (solana-labs#34658) [TieredStorage] HotStorageWriter::new() (solana-labs#34659) [TieredStorage] Include executable field into AccountMetaFlags (solana-labs#34724) [TieredStorage] Code refactoring for OwnersBlock (solana-labs#34854) [TieredStorage] In-memory struct for writing OwnersBlock (solana-labs#34853) [TieredStorage] writing hot account blocks and index blocks (solana-labs#34828) [TieredStorage] Use RENT_EXEMPT_RENT_EPOCH in HotStorageWriter (solana-labs#34950) [TieredStorage] Write owners block for HotAccountStorage (solana-labs#34927) [TieredStorage] Avoid AccountHash copy in AccountMetaOptionalFields (solana-labs#34969) [TieredStorage] Correct the HotStorage API for account_matches_owners (solana-labs#34967) [TS] Add get_account() and account_matches_owner() to TieredStorageReader (solana-labs#34968) [TieredStorage] Have HotStorageWriter::write_account() return Vec<StoredAccountInfo> (solana-labs#34929) [TieredStorage] Use IndexOffset in TieredStorageMeta and get_account() (solana-labs#35046) [TieredStorage] TieredStorageReader:: and HotStorageReader:: accounts() (solana-labs#35031) [TieredStorage] Enable hot-storage in TieredStorage::write_accounts() (solana-labs#35049) [TieredStorage] Put commonly used test functions into test_utils.rs (solana-labs#35065)
yhchiang-sol
added a commit
to yhchiang-sol/solana
that referenced
this pull request
Feb 13, 2024
solana-labs#35046) #### Problem TieredStorageMeta and TieredStorageReader::get_account API uses u32 to represent IndexOffset. However, within the TieredStorage scope, IndexOffset should be used, it is not until working with AccountsFile API when u32 representation of offset is needed. #### Summary of Changes Have TieredStorageMeta and TieredStorageReader to use IndexOffset. #### Test Plan Existing unit-tests.
yhchiang-sol
added a commit
to yhchiang-sol/solana
that referenced
this pull request
Feb 13, 2024
…labs#33964) [TieredStorage] Improve param naming of IndexBlockFormat (solana-labs#34033) [TieredStorage] HotStorageReader::get_account_offset (solana-labs#34031) [TieredStorage] Rename owners_offset to owners_block_offset (solana-labs#34047) [TieredStorage] HotStorageReader::get_account_address (solana-labs#34032) [TieredStorage] OwnersBlock (solana-labs#34052) [TieredStorage] HotStorageReader::get_owner_address (solana-labs#34053) [TieredStorage] Define OwnerOffset as u32 (solana-labs#34105) [TieredStorage] Use OwnerOffset type in TieredAccountMeta (solana-labs#34106) Refactors TieredStorageFile read/write methods (solana-labs#34147) [TieredStorage] Make IndexBlock persist u32 offsets (solana-labs#34133) [TieredStorage] Make IndexOffset use u32 (solana-labs#34152) Move MatchAccountOwnerError from append_vec to accounts_file (solana-labs#34187) [TieredStorage] Make AccountOffset use u32 (solana-labs#34151) [TieredStorage] Allow HotStorage to handle more account data (solana-labs#34155) [TieredStorage] Make AccountOffset a trait, introduce HotAccountOffset (solana-labs#34335) [TieredStorage] Improve comments for HOT_ACCOUNT_ALIGNMENT (solana-labs#34404) [TieredStorage] Unit-tests for checking invalid HotAccountOffset (solana-labs#34376) [TieredStorage] Boundary check for accessing hot account meta (solana-labs#34349) [TieredStorage] boundary check for get_account_address() (solana-labs#34529) Sanitizes tiered storage footer after reading from disk (solana-labs#34200) Adds read/write/get_pod() fns to tiered storage (solana-labs#34415) Uses consistent error types in tiered storage (solana-labs#34110) [TieredStorage] Boundary check for get_account_offset() (solana-labs#34531) [TieredStorage] HotStorageReader::account_matches_owners (solana-labs#34350) [TieredStorage] Fix typos in index.rs (solana-labs#34546) [TieredStorage] HotAccountsReader::get_account (solana-labs#34499) [TieredStorage] Rename AddressAndBlockOffsetOnly to AddressesThenOffsets (solana-labs#34658) [TieredStorage] HotStorageWriter::new() (solana-labs#34659) [TieredStorage] Include executable field into AccountMetaFlags (solana-labs#34724) [TieredStorage] Code refactoring for OwnersBlock (solana-labs#34854) [TieredStorage] In-memory struct for writing OwnersBlock (solana-labs#34853) [TieredStorage] writing hot account blocks and index blocks (solana-labs#34828) [TieredStorage] Use RENT_EXEMPT_RENT_EPOCH in HotStorageWriter (solana-labs#34950) [TieredStorage] Write owners block for HotAccountStorage (solana-labs#34927) [TieredStorage] Avoid AccountHash copy in AccountMetaOptionalFields (solana-labs#34969) [TieredStorage] Correct the HotStorage API for account_matches_owners (solana-labs#34967) [TS] Add get_account() and account_matches_owner() to TieredStorageReader (solana-labs#34968) [TieredStorage] Have HotStorageWriter::write_account() return Vec<StoredAccountInfo> (solana-labs#34929) [TieredStorage] Use IndexOffset in TieredStorageMeta and get_account() (solana-labs#35046) [TieredStorage] TieredStorageReader:: and HotStorageReader:: accounts() (solana-labs#35031) [TieredStorage] Enable hot-storage in TieredStorage::write_accounts() (solana-labs#35049) [TieredStorage] Put commonly used test functions into test_utils.rs (solana-labs#35065)
yhchiang-sol
added a commit
to yhchiang-sol/solana
that referenced
this pull request
Feb 18, 2024
…labs#33964) [TieredStorage] Improve param naming of IndexBlockFormat (solana-labs#34033) [TieredStorage] HotStorageReader::get_account_offset (solana-labs#34031) [TieredStorage] Rename owners_offset to owners_block_offset (solana-labs#34047) [TieredStorage] HotStorageReader::get_account_address (solana-labs#34032) [TieredStorage] OwnersBlock (solana-labs#34052) [TieredStorage] HotStorageReader::get_owner_address (solana-labs#34053) [TieredStorage] Define OwnerOffset as u32 (solana-labs#34105) [TieredStorage] Use OwnerOffset type in TieredAccountMeta (solana-labs#34106) Refactors TieredStorageFile read/write methods (solana-labs#34147) [TieredStorage] Make IndexBlock persist u32 offsets (solana-labs#34133) [TieredStorage] Make IndexOffset use u32 (solana-labs#34152) Move MatchAccountOwnerError from append_vec to accounts_file (solana-labs#34187) [TieredStorage] Make AccountOffset use u32 (solana-labs#34151) [TieredStorage] Allow HotStorage to handle more account data (solana-labs#34155) [TieredStorage] Make AccountOffset a trait, introduce HotAccountOffset (solana-labs#34335) [TieredStorage] Improve comments for HOT_ACCOUNT_ALIGNMENT (solana-labs#34404) [TieredStorage] Unit-tests for checking invalid HotAccountOffset (solana-labs#34376) [TieredStorage] Boundary check for accessing hot account meta (solana-labs#34349) [TieredStorage] boundary check for get_account_address() (solana-labs#34529) Sanitizes tiered storage footer after reading from disk (solana-labs#34200) Adds read/write/get_pod() fns to tiered storage (solana-labs#34415) Uses consistent error types in tiered storage (solana-labs#34110) [TieredStorage] Boundary check for get_account_offset() (solana-labs#34531) [TieredStorage] HotStorageReader::account_matches_owners (solana-labs#34350) [TieredStorage] Fix typos in index.rs (solana-labs#34546) [TieredStorage] HotAccountsReader::get_account (solana-labs#34499) [TieredStorage] Rename AddressAndBlockOffsetOnly to AddressesThenOffsets (solana-labs#34658) [TieredStorage] HotStorageWriter::new() (solana-labs#34659) [TieredStorage] Include executable field into AccountMetaFlags (solana-labs#34724) [TieredStorage] Code refactoring for OwnersBlock (solana-labs#34854) [TieredStorage] In-memory struct for writing OwnersBlock (solana-labs#34853) [TieredStorage] writing hot account blocks and index blocks (solana-labs#34828) [TieredStorage] Use RENT_EXEMPT_RENT_EPOCH in HotStorageWriter (solana-labs#34950) [TieredStorage] Write owners block for HotAccountStorage (solana-labs#34927) [TieredStorage] Avoid AccountHash copy in AccountMetaOptionalFields (solana-labs#34969) [TieredStorage] Correct the HotStorage API for account_matches_owners (solana-labs#34967) [TS] Add get_account() and account_matches_owner() to TieredStorageReader (solana-labs#34968) [TieredStorage] Have HotStorageWriter::write_account() return Vec<StoredAccountInfo> (solana-labs#34929) [TieredStorage] Use IndexOffset in TieredStorageMeta and get_account() (solana-labs#35046) [TieredStorage] TieredStorageReader:: and HotStorageReader:: accounts() (solana-labs#35031) [TieredStorage] Enable hot-storage in TieredStorage::write_accounts() (solana-labs#35049) [TieredStorage] Put commonly used test functions into test_utils.rs (solana-labs#35065) [TieredStorage] Make TieredStorage::write_accounts() thread-safe (solana-labs#35143)
yhchiang-sol
added a commit
to yhchiang-sol/solana
that referenced
this pull request
Mar 4, 2024
…labs#33964) [TieredStorage] Improve param naming of IndexBlockFormat (solana-labs#34033) [TieredStorage] HotStorageReader::get_account_offset (solana-labs#34031) [TieredStorage] Rename owners_offset to owners_block_offset (solana-labs#34047) [TieredStorage] HotStorageReader::get_account_address (solana-labs#34032) [TieredStorage] OwnersBlock (solana-labs#34052) [TieredStorage] HotStorageReader::get_owner_address (solana-labs#34053) [TieredStorage] Define OwnerOffset as u32 (solana-labs#34105) [TieredStorage] Use OwnerOffset type in TieredAccountMeta (solana-labs#34106) Refactors TieredStorageFile read/write methods (solana-labs#34147) [TieredStorage] Make IndexBlock persist u32 offsets (solana-labs#34133) [TieredStorage] Make IndexOffset use u32 (solana-labs#34152) Move MatchAccountOwnerError from append_vec to accounts_file (solana-labs#34187) [TieredStorage] Make AccountOffset use u32 (solana-labs#34151) [TieredStorage] Allow HotStorage to handle more account data (solana-labs#34155) [TieredStorage] Make AccountOffset a trait, introduce HotAccountOffset (solana-labs#34335) [TieredStorage] Improve comments for HOT_ACCOUNT_ALIGNMENT (solana-labs#34404) [TieredStorage] Unit-tests for checking invalid HotAccountOffset (solana-labs#34376) [TieredStorage] Boundary check for accessing hot account meta (solana-labs#34349) [TieredStorage] boundary check for get_account_address() (solana-labs#34529) Sanitizes tiered storage footer after reading from disk (solana-labs#34200) Adds read/write/get_pod() fns to tiered storage (solana-labs#34415) Uses consistent error types in tiered storage (solana-labs#34110) [TieredStorage] Boundary check for get_account_offset() (solana-labs#34531) [TieredStorage] HotStorageReader::account_matches_owners (solana-labs#34350) [TieredStorage] Fix typos in index.rs (solana-labs#34546) [TieredStorage] HotAccountsReader::get_account (solana-labs#34499) [TieredStorage] Rename AddressAndBlockOffsetOnly to AddressesThenOffsets (solana-labs#34658) [TieredStorage] HotStorageWriter::new() (solana-labs#34659) [TieredStorage] Include executable field into AccountMetaFlags (solana-labs#34724) [TieredStorage] Code refactoring for OwnersBlock (solana-labs#34854) [TieredStorage] In-memory struct for writing OwnersBlock (solana-labs#34853) [TieredStorage] writing hot account blocks and index blocks (solana-labs#34828) [TieredStorage] Use RENT_EXEMPT_RENT_EPOCH in HotStorageWriter (solana-labs#34950) [TieredStorage] Write owners block for HotAccountStorage (solana-labs#34927) [TieredStorage] Avoid AccountHash copy in AccountMetaOptionalFields (solana-labs#34969) [TieredStorage] Correct the HotStorage API for account_matches_owners (solana-labs#34967) [TS] Add get_account() and account_matches_owner() to TieredStorageReader (solana-labs#34968) [TieredStorage] Have HotStorageWriter::write_account() return Vec<StoredAccountInfo> (solana-labs#34929) [TieredStorage] Use IndexOffset in TieredStorageMeta and get_account() (solana-labs#35046) [TieredStorage] TieredStorageReader:: and HotStorageReader:: accounts() (solana-labs#35031) [TieredStorage] Enable hot-storage in TieredStorage::write_accounts() (solana-labs#35049) [TieredStorage] Put commonly used test functions into test_utils.rs (solana-labs#35065) [TieredStorage] Make TieredStorage::write_accounts() thread-safe (solana-labs#35143) [TieredStorage] rent_epoch() returns 0 for zero-lamport accounts (solana-labs#35344)
yhchiang-sol
added a commit
to yhchiang-sol/solana
that referenced
this pull request
Mar 9, 2024
…labs#33964) [TieredStorage] Improve param naming of IndexBlockFormat (solana-labs#34033) [TieredStorage] HotStorageReader::get_account_offset (solana-labs#34031) [TieredStorage] Rename owners_offset to owners_block_offset (solana-labs#34047) [TieredStorage] HotStorageReader::get_account_address (solana-labs#34032) [TieredStorage] OwnersBlock (solana-labs#34052) [TieredStorage] HotStorageReader::get_owner_address (solana-labs#34053) [TieredStorage] Define OwnerOffset as u32 (solana-labs#34105) [TieredStorage] Use OwnerOffset type in TieredAccountMeta (solana-labs#34106) Refactors TieredStorageFile read/write methods (solana-labs#34147) [TieredStorage] Make IndexBlock persist u32 offsets (solana-labs#34133) [TieredStorage] Make IndexOffset use u32 (solana-labs#34152) Move MatchAccountOwnerError from append_vec to accounts_file (solana-labs#34187) [TieredStorage] Make AccountOffset use u32 (solana-labs#34151) [TieredStorage] Allow HotStorage to handle more account data (solana-labs#34155) [TieredStorage] Make AccountOffset a trait, introduce HotAccountOffset (solana-labs#34335) [TieredStorage] Improve comments for HOT_ACCOUNT_ALIGNMENT (solana-labs#34404) [TieredStorage] Unit-tests for checking invalid HotAccountOffset (solana-labs#34376) [TieredStorage] Boundary check for accessing hot account meta (solana-labs#34349) [TieredStorage] boundary check for get_account_address() (solana-labs#34529) Sanitizes tiered storage footer after reading from disk (solana-labs#34200) Adds read/write/get_pod() fns to tiered storage (solana-labs#34415) Uses consistent error types in tiered storage (solana-labs#34110) [TieredStorage] Boundary check for get_account_offset() (solana-labs#34531) [TieredStorage] HotStorageReader::account_matches_owners (solana-labs#34350) [TieredStorage] Fix typos in index.rs (solana-labs#34546) [TieredStorage] HotAccountsReader::get_account (solana-labs#34499) [TieredStorage] Rename AddressAndBlockOffsetOnly to AddressesThenOffsets (solana-labs#34658) [TieredStorage] HotStorageWriter::new() (solana-labs#34659) [TieredStorage] Include executable field into AccountMetaFlags (solana-labs#34724) [TieredStorage] Code refactoring for OwnersBlock (solana-labs#34854) [TieredStorage] In-memory struct for writing OwnersBlock (solana-labs#34853) [TieredStorage] writing hot account blocks and index blocks (solana-labs#34828) [TieredStorage] Use RENT_EXEMPT_RENT_EPOCH in HotStorageWriter (solana-labs#34950) [TieredStorage] Write owners block for HotAccountStorage (solana-labs#34927) [TieredStorage] Avoid AccountHash copy in AccountMetaOptionalFields (solana-labs#34969) [TieredStorage] Correct the HotStorage API for account_matches_owners (solana-labs#34967) [TS] Add get_account() and account_matches_owner() to TieredStorageReader (solana-labs#34968) [TieredStorage] Have HotStorageWriter::write_account() return Vec<StoredAccountInfo> (solana-labs#34929) [TieredStorage] Use IndexOffset in TieredStorageMeta and get_account() (solana-labs#35046) [TieredStorage] TieredStorageReader:: and HotStorageReader:: accounts() (solana-labs#35031) [TieredStorage] Enable hot-storage in TieredStorage::write_accounts() (solana-labs#35049) [TieredStorage] Put commonly used test functions into test_utils.rs (solana-labs#35065) [TieredStorage] Make TieredStorage::write_accounts() thread-safe (solana-labs#35143) [TieredStorage] rent_epoch() returns 0 for zero-lamport accounts (solana-labs#35344) [TieredStorage] Deprecate the use of account-hash in HotStorage (solana-labs#93)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Problem
TieredStorageMeta and TieredStorageReader::get_account API uses
u32 to represent IndexOffset. However, within the TieredStorage scope,
IndexOffset should be used, it is not until working with AccountsFile API
when u32 representation of offset is needed.
Summary of Changes
Have TieredStorageMeta and TieredStorageReader to use IndexOffset.
Test Plan
Existing unit-tests.