Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

db: avoid unnecessary key comparisons during iteration #1840

Merged
merged 1 commit into from
Aug 2, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions internal/base/internal.go
Original file line number Diff line number Diff line change
Expand Up @@ -71,6 +71,10 @@ const (
// seqNum.
InternalKeyKindMax InternalKeyKind = 21

// InternalKeyZeroSeqnumMaxTrailer is the largest trailer with a
// zero sequence number.
InternalKeyZeroSeqnumMaxTrailer = uint64(InternalKeyKindInvalid)

// A marker for an invalid key.
InternalKeyKindInvalid InternalKeyKind = 255

Expand Down
17 changes: 14 additions & 3 deletions iterator.go
Original file line number Diff line number Diff line change
Expand Up @@ -548,21 +548,32 @@ func (i *Iterator) nextUserKey() {
if i.iterKey == nil {
return
}
done := i.iterKey.SeqNum() == 0
trailer := i.iterKey.Trailer
done := i.iterKey.Trailer <= base.InternalKeyZeroSeqnumMaxTrailer
if i.iterValidityState != IterValid {
i.keyBuf = append(i.keyBuf[:0], i.iterKey.UserKey...)
i.key = i.keyBuf
}
for {
i.iterKey, i.iterValue = i.iter.Next()
i.stats.ForwardStepCount[InternalIterCall]++
if done || i.iterKey == nil {
// NB: We're guaranteed to be on the next user key if the previous key
// had a zero sequence number (`done`), or the new key has a trailer
// greater or equal to the previous key's trailer. This is true because
// internal keys with the same user key are sorted by Trailer in
// strictly monotonically descending order. We expect the trailer
// optimization to trigger around 50% of the time with randomly
// distributed writes. We expect it to trigger very frequently when
// iterating through ingested sstables, which contain keys that all have
// the same sequence number.
if done || i.iterKey == nil || i.iterKey.Trailer >= trailer {
break
}
if !i.equal(i.key, i.iterKey.UserKey) {
break
}
done = i.iterKey.SeqNum() == 0
done = i.iterKey.Trailer <= base.InternalKeyZeroSeqnumMaxTrailer
trailer = i.iterKey.Trailer
}
}

Expand Down