Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

LedgerDB spine optimisation #959

Closed
mrBliss opened this issue Aug 27, 2019 · 2 comments
Closed

LedgerDB spine optimisation #959

mrBliss opened this issue Aug 27, 2019 · 2 comments
Assignees
Labels
consensus issues related to ouroboros-consensus ledger db optimisation Performance optimisation

Comments

@mrBliss
Copy link
Contributor

mrBliss commented Aug 27, 2019

When profiling the following: adding all blocks from disk as fast as possible with block validation disabled, the LedgerDB appeared at the top of the heap/performance profile output.

I suspect it is due to the structure of the LedgerDB: it is essentially a list with a spine that is k elements long, that is traversed and reallocated each time a block is added. Instead of storing k Skip elements in the list, one Skip k element could be stored, bringing down the cost significantly.

@mrBliss mrBliss added consensus issues related to ouroboros-consensus optimisation Performance optimisation labels Aug 27, 2019
@mrBliss mrBliss added this to the Consensus_ChainDB milestone Aug 27, 2019
@edsko
Copy link
Contributor

edsko commented Sep 2, 2019

It's not quite so easy, the Skip nodes do record information (they store references to the blocks that were applied there). We could avoid storing that information, at the cost of introducing more dependency between the LedgerDB and the volatile DB.

However, we can address this in quite a different way. Will open a separate ticket.

@edsko
Copy link
Contributor

edsko commented Sep 2, 2019

Obsoleted by #986.

@edsko edsko closed this as completed Sep 2, 2019
@mrBliss mrBliss removed this from the Consensus_ChainDB milestone Nov 7, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
consensus issues related to ouroboros-consensus ledger db optimisation Performance optimisation
Projects
None yet
Development

No branches or pull requests

2 participants