-
Notifications
You must be signed in to change notification settings - Fork 4.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[TieredStorage] Have the reader until TieredStorage lazy initialized #35063
Conversation
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #35063 +/- ##
=======================================
Coverage 81.6% 81.6%
=======================================
Files 830 830
Lines 224946 224966 +20
=======================================
+ Hits 183674 183709 +35
+ Misses 41272 41257 -15 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Also, bigger question, is wirte_accounts()
here thread-safe? What happens if different threads call this method on the same instance?
@@ -49,6 +49,7 @@ pub struct TieredStorageFormat { | |||
#[derive(Debug)] | |||
pub struct TieredStorage { | |||
reader: OnceLock<TieredStorageReader>, | |||
read_only: OnceLock<bool>, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Seems like overkill here. Wdyt about using an AtomicBool
instead?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sounds good. I am also thinking about using u64 (i.e., AtomicU64) that represents file size instead so that we can have a faster file_size() as well as checking whether is read-only.
Making this PR draft again as the thread-safety part has been move to a dedicate PR #35143 |
This repository is no longer in use. Please re-open this pull request in the agave repo: https://github.com/anza-xyz/agave |
Problem
Currently, right after TieredStorage::write_accounts() is called, it
immediately mmap the file in case it is the hot storage. It might
be more suitable to delay the mmap until the file is actually read.
Summary of Changes
Have the reader under the TieredStorage to be lazy initialized.
Test Plan
Existing unit-tests.
Dependency
#35049