Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Independent roothash pruning policy #1920

Closed
kostko opened this issue Jul 15, 2019 · 1 comment · Fixed by #2463
Closed

Independent roothash pruning policy #1920

kostko opened this issue Jul 15, 2019 · 1 comment · Fixed by #2463
Assignees
Labels
c:consensus/cometbft Category: CometBFT c:roothash Category: root hash service c:runtime/indexer Category: runtime block and/or tag indexer c:storage Category: storage

Comments

@kostko
Copy link
Member

kostko commented Jul 15, 2019

Currently the Ekiden runtime pruning policy is coupled with the Tendermint ABCI state pruning policy. This makes it hard to control how runtime history is kept as there is no simple correlation between Tendermint blocks and roothash rounds.

This should be changed so that the Ekiden runtime pruning policy becomes independent. In order to support this even while Tendermint ABCI state may be pruned, we need to make sure to keep a copy of all non-pruned Ekiden runtime blocks. This can be achieved either by:

  • Keeping them in a separate index (e.g., the existing roothash block index which can be moved from the tendermint roothash backend and repurposed).
  • Turning the pruning policy around so that any Tendermint blocks which are still required by the Ekiden runtime pruning policy are not pruned (e.g., the new Ekiden runtime pruning policy directs what is pruned).

The benefit of the first approach is that we can independently prune consensus and the downside is that we would duplicate some amount of state (e.g., Ekiden runtime blocks that would exist in both ABCI state until pruned and in this new index). Note that the separate index is needed even in the second case in order to map round numbers to consensus heights (this is the current implementation).

The refactored pruning policy support must also take into account that storage needs to use it in order to know what to sync and where checkpoints (e.g., GetCheckpoint) can be used to speed up syncing.

Estimated cost: 1 sprint

@kostko kostko added c:roothash Category: root hash service c:storage Category: storage c:consensus/cometbft Category: CometBFT c:runtime/indexer Category: runtime block and/or tag indexer labels Jul 15, 2019
@kostko kostko changed the title Checkpointing pruning policy Independent roothash pruning policy Oct 7, 2019
@kostko
Copy link
Member Author

kostko commented Dec 2, 2019

The pruning policy should be per-runtime.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
c:consensus/cometbft Category: CometBFT c:roothash Category: root hash service c:runtime/indexer Category: runtime block and/or tag indexer c:storage Category: storage
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant