go tendermint badger: don't scan whole keyspace #2664
Conversation
pro-wh
force-pushed
the
pro-wh/bugfix/badger
branch
from
6b06653 to
573f04a
Compare
Codecov Report
@@ Coverage Diff @@
## master #2664 +/- ##
=========================================
- Coverage 63.21% 63.1% -0.12%
=========================================
Files 365 365
Lines 34592 34577 -15
=========================================
- Hits 21868 21820 -48
- Misses 10014 10045 +31
- Partials 2710 2712 +2
Continue to review full report at Codecov.
|
| @@ -338,22 +338,22 @@ func (it *badgerDBIterator) Next() { | |||
| panic("Next with invalid iterator") | |||
| } | |||
|
|
|||
| for { | |||
There was a problem hiding this comment.
Wow, why did we keep scanning before? This looks like it kept scanning until reaching the end of the database because once the key was no longer inside the domain, we just kept going until the underlying iterator was valid.
Looking at the old bolt implementation it seems like the same was done there. Where did we get those semantics from, looking at other TM-DB backends, this doesn't seem to be the case? cc @Yawning
There was a problem hiding this comment.
key suspect is this https://github.com/oasislabs/oasis-core/blob/v20.1.2/go/consensus/tendermint/db/badger/badger.go#L302
I don't yet understand the case where we would need that, and I want to try to write an implementation that doesn't rely on that. Our tests don't cover it either. But if it's true that we need to scan into the beginning of the range, that's why we'd need this loop.
|
Why do you want to merge this into |
Just wanted to say the same, so this should go into master first. We can backport it later. |
|
I'll rebase on master |
pro-wh
force-pushed
the
pro-wh/bugfix/badger
branch
from
573f04a to
6292b79
Compare
pro-wh
force-pushed
the
pro-wh/bugfix/badger
branch
2 times, most recently
from
cdc4b82 to
5c5d95c
Compare
|
Ah, coverage of the changed routines is pretty good too |
- Responsibility for correct seeking is moved down to BadgerDB. - Responsibility for caching keys and values is moved up or down. - Bounds checking is moved to Valid. - Methods now don't call Close internally. Remember to call it yourself and don't abuse Valid.
pro-wh
force-pushed
the
pro-wh/bugfix/badger
branch
from
5c5d95c to
db9a494
Compare
when we run off the end of a range iteration
but now I need to try rewriting the whole badger iterator to make sure I understand it