Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

leveldb/memdb: avoid repeated comparisons #384

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

holiman
Copy link

@holiman holiman commented Dec 8, 2021

When the memory database does a search through the skiplist, it descends one level whenver it either finds that the list is ended, or that the comparison fails (we've passed the item).
In the latter case, we descend a level, and a non-insignificant amount of times, the search on that level will also end on the same exact item -- since if the item was at the upper level, it's guaranteed to be present on the lower level too.

By adding one local int, to keep track of, basically, "an item we found on the upper level, which is known to be past the item we're searching for", we can avoid doing that comparison again.

Using this fix, plus a global comparison counter, I made these charts. It imports 30M items, each with a 32 byte key and a 1 byte value. At 100 points during this import, I dump out the average comparison ops performed in that range.

This is master
memdb-12-4-31457280

It executed it in 175.91 seconds, doing 1416736538 comparison calls in total.

This is this PR:

memdb-12-4-31457280 pr

It executed it in 171.51 seconds, doing 1333783947 comparison calls in total.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant