-
Notifications
You must be signed in to change notification settings - Fork 3.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Tendermint Block Pruning #7265
Tendermint Block Pruning #7265
Conversation
Codecov Report
@@ Coverage Diff @@
## master #7265 +/- ##
==========================================
+ Coverage 55.60% 56.25% +0.65%
==========================================
Files 457 584 +127
Lines 27440 40563 +13123
==========================================
+ Hits 15257 22820 +7563
- Misses 11083 15823 +4740
- Partials 1100 1920 +820 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM pending comments for maintainers
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Unblocking
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
lgtm
baseapp/abci.go
Outdated
|
||
// Define the state pruning interval, i.e. the block interval at which the | ||
// underlying logical database is persisted to disk. | ||
statePruningInterval := int64(app.cms.GetPruning().Interval) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think this should use KeepEvery
, not Interval
. Consider the case of KeepEvery: 1000
and Interval: 10
. At height 10890, this will allow block pruning up to 10880, even though the last state height persisted to disk is 10000 - if the node restarts, it will try to replay from 10000, but can't since the blocks are gone.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You're absolutely right! Can't believe I missed this. Interval
is when the heights are actually pruned from disk -- nomenclature always confuses me...
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
However, in this case H=10890
, and X=H-KeepEvery=9890
which isn't totally correct either. I think we need to do X=H-(H%KeepEvery)
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think we need to do
X=H-(H%KeepEvery)
?
Yes, that's more accurate. H-KeepEvery
would work, but has an excessive safety margin. If you go for minNonZero
then take care when H<KeepEvery
, since this expression results in 0
and thus would be ignored.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
take care when
H<KeepEvery
, since this expression results in0
and thus would be ignored.
Correct, in this case, we must be sure to return 0
as to not prune anything because that would be bad!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Updated. @erikgrinaker could you take another look?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good!
It occurred to me that another way to handle this would be to simply return a configuration error if minRetainBlocks
was set lower than all these other parameters - instead of magically extending it. Not really sure which UX is better.
Co-authored-by: Erik Grinaker <[email protected]>
Co-authored-by: Erik Grinaker <[email protected]>
Co-authored-by: Erik Grinaker <[email protected]>
Description
Implement Tendermint block pruning support via
ResponseCommit.RetainHeight
:min-retain-blocks
in theapp.toml
configuration with an equivalent CLI flagGetBlockRentionHeight
during ABCI Commit which follows https://github.com/tendermint/spec/blob/master/rfc/001-block-retention.mdcloses: #6164
/cc @marbar3778 @erikgrinaker
Before we can merge this PR, please make sure that all the following items have been
checked off. If any of the checklist items are not applicable, please leave them but
write a little note why.
docs/
) or specification (x/<module>/spec/
)godoc
comments.Unreleased
section inCHANGELOG.md
Files changed
in the Github PR explorerCodecov Report
in the comment section below once CI passes