cmd, core, eth, light, trie: dump clean cache periodically #20391
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
rjl493456442
requested review from
holiman,
karalabe and
zsfelfoldi
as code owners
rjl493456442
force-pushed
the
cache-backup-file
branch
from
292bbd9
to
cf62192
Compare
rjl493456442
changed the title
holiman
approved these changes
Nov 27, 2019
|
@holiman nope, the cache dump and reload is enabled by default. And also users can specify the flags to change the behaviors. |
karalabe
requested changes
Jan 14, 2020
trie/database.go
Outdated
| log.Info("Failed to save clean cache into file", "error", err) | ||
| } | ||
| }(time.Now()) | ||
| err = db.cleans.SaveToFileConcurrent(dir, threads) |
There was a problem hiding this comment.
Is there a reason why we'd not want to use max concurrency here? I'd just set this to 0 and let fastcache max out the performance.
There was a problem hiding this comment.
How much time does it take to persist a 1GB cache?
There was a problem hiding this comment.
I can't remember the concrete number, but a few seconds.
rjl493456442
force-pushed
the
cache-backup-file
branch
from
5d9b421
to
4ad8852
Compare
rjl493456442
force-pushed
the
cache-backup-file
branch
from
4ad8852
to
04f6056
Compare
This was referenced Feb 24, 2021
enriquefynn
pushed a commit
to enriquefynn/go-ethereum
that referenced
this pull request
Mar 10, 2021
…20391) * cmd, core, eth, light, trie: dump clean cache periodically * eth: update config * trie: minor fix * core, trie: address comments * eth: remove useless * trie: print clean cache dump start too Co-authored-by: Péter Szilágyi <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR enables the cache dump and reload from a disk journal.
Accumulate more entries in memory will help us to speed up. However
warm-up the cache can take a few minutes. So disk based journal can help
a lot.
Besides all entries we maintain in the cache is contract code and trie nodes.
The key of entries is essentially the hash of value. So we can blindly reload the
entries from journal and don't need to take care of the validity of data.