Skip to content

Commit

Permalink
*: add simpleLevelIterator, reduce merging levels in external iter
Browse files Browse the repository at this point in the history
Currently, we create a new merging iterator level for each file
passed into NewExternalIter. This is unnecessary for most use-cases
of creating ExternalIters around lots of sstables, as we can externally
guarantee that many of those sstables won't have overlapping points
with each other. We can have the caller pass this knowledge
by specifying a [][]sstable.ReadableFile where each sub-slice
obeys level invariants for files within it, and is also already
sorted by user keys.

This change makes the interface change to allow for the above
optimization, and also adds a `simpleLevelIter` that implements
forward iteration within a single "level". For files that don't
contain range deletes, we shove all the point iters into one
`simpleLevelIter`, greatly reducing merging iterator levels
and speeding up its operations by a lot.

Fixes cockroachdb/cockroach#83051.
  • Loading branch information
itsbilal committed Aug 18, 2022
1 parent c135b6d commit 0984569
Show file tree
Hide file tree
Showing 5 changed files with 475 additions and 45 deletions.
Loading

0 comments on commit 0984569

Please sign in to comment.