Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Figure out good value for snapEvery parameter for LedgerDB #1026

Closed
edsko opened this issue Sep 13, 2019 · 4 comments
Closed

Figure out good value for snapEvery parameter for LedgerDB #1026

edsko opened this issue Sep 13, 2019 · 4 comments
Assignees
Labels
consensus issues related to ouroboros-consensus praos

Comments

@edsko
Copy link
Contributor

edsko commented Sep 13, 2019

The ledger DB stores a snapshot in-memory after every snapEvery blocks. A high value means faster rollback, but also more memory usage: we might lose opportunities for garbage collection, not only due to the UTxO (or other data structures) shrinking but also from the recreated spines of these data structures when updating or extending them. It is however not at all obvious how this parameter affects performance and memory usage in absolute values; we should measure this and set snapEvery accordingly.

This is not too relevant for Byron, as rollbacks there are basically absent, but will be important for Praos.

(If it turns out that we do want a relatively high value for snapEvery, we might want to consider treating the tip-1 ledger state special also, to have specific support for the common case of rollbacks of 1.)

@edsko edsko added the consensus issues related to ouroboros-consensus label Sep 13, 2019
@edsko edsko added this to the Consensus_Praos milestone Sep 13, 2019
@mrBliss mrBliss added the praos label Nov 7, 2019
@mrBliss mrBliss removed this from the Consensus_Praos milestone Nov 7, 2019
@mrBliss mrBliss added the shelley label Nov 7, 2019
@edsko
Copy link
Contributor Author

edsko commented Dec 18, 2019

This is not essential for the first mainnet release.

@edsko
Copy link
Contributor Author

edsko commented Jan 10, 2020

Note that this ticket is about the in-memory snapshots; see #1264 for the on-disk snapshots.

@dcoutts
Copy link
Contributor

dcoutts commented Jan 10, 2020

If it turns out that we do want a relatively high value for snapEvery, we might want to consider treating the tip-1 ledger state special also, to have specific support for the common case of rollbacks of 1.

The special case of 1 looks like it may be useful for:

  • Praos short forks due to slot collisions
  • Forging blocks when a race condition means our chain is extended in our slot (again with a Praos slot collision)
  • Clients like wallets wanting the very recent ledger state (via the local state query protocol) since they're slightly behind (processing the last block while the new one has just arrived).

@mrBliss
Copy link
Contributor

mrBliss commented Jul 23, 2020

@kderme's investigation in #1936 has shown that the overhead of snapEvery = 1 is acceptable and around 5% compared to snapEvery = 100. We'll go ahead with snapEvery = 1 and simplify the LedgerDB.

@mrBliss mrBliss closed this as completed Jul 23, 2020
@mrBliss mrBliss added this to the S18 2020-07-30 milestone Jul 23, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
consensus issues related to ouroboros-consensus praos
Projects
None yet
Development

No branches or pull requests

4 participants