-
Notifications
You must be signed in to change notification settings - Fork 20.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
core/rawdb: implement sequential reads in freezer_table #23117
Conversation
I made a little benchmark tool for this, in this branch. Using the old reads, where each header is read separately, and causing 3 syscalls per header (so
It speeds up continously (it's on a NUC with pretty hefty RAM, so the disk reads will be better cached) but after three and a half minute, it's reached If the sequential-read mechanism in this PR is used instead, which does
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Mostly nitpicks, but there's one corner case that I believe is broken.
8f1453f
to
821c696
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
SGTM
* core/rawdb: implement sequential reads in freezer_table * core/rawdb, ethdb: add sequential reader to db interface * core/rawdb: lint nitpicks * core/rawdb: fix some nitpicks * core/rawdb: fix flaw with deferred reads not being performed * core/rawdb: better documentation
This PR is a bit of a successor to #20308, which implemented sequential access to freezer data via an iterator interface.
This PR instead returns a slice of data. It does not defer reads until
Next()
, so instead does all the reads while holding the readlock.It also switches the implementation of
Retrieve
to be a just a short-cut for doing a sequential read with max items set to1
. This PR doesn't really add any uses of the new method (except theRetrieve
), but does actually improve one thing as-is: previously, two syscalls were performed to read two index items from the index-file. This PR instead reads all (both) index items using only one syscall.Later on, this can be used to serve
eth64
data such as headers directly from ancients with a minimum number of syscalls. If that is done, we can get by with2
syscalls per "192-header-delivery" instead of576
.