Skip to content

Commit

Permalink
compaction: elision-only compactions for tables with only range keys
Browse files Browse the repository at this point in the history
Currently, a table is marked as eligible for elision-only compaction by
the `elisionOnlyAnnotator` under the following circumstances:

- the table's range deletion estimate is greater than or equal to 10% of
  the total table size, OR
- the number of deletions is greater than or equal to 10% of the table's
  total point key entries.

If a table contains only range keys, the second predicate is true (given
that `0 >= 0`), scheduling an elision-only compaction. Howeve, if the
table contains only range key-sets, such keys cannot be elided, and the
compaction picker will continue to schedule the table for elision,
without effect. This can tie up compaction slots.

While it is _technically_ possible that a table with containing
exclusively range keys, but no range key sets _could_ be eligible for an
elision-only compaction (i.e. if there are no spans underneath, or
snapshots preventing the elision, etc.), the utility of such a
compaction is minimal, given that a compaction into a table containing a
few range keys would be inexpensive.

Tweak the elision-only compaction heuristics to skip elision-only
compactions of tables that contain exclusively range keys.
  • Loading branch information
nicktrav committed Jun 13, 2022
1 parent ae99f4f commit 1022ff8
Show file tree
Hide file tree
Showing 4 changed files with 55 additions and 1 deletion.
7 changes: 6 additions & 1 deletion compaction_picker.go
Original file line number Diff line number Diff line change
Expand Up @@ -1239,8 +1239,13 @@ func (a elisionOnlyAnnotator) Accumulate(f *fileMetadata, dst interface{}) (inte
// which may be collapsed. Ideally, we would have 'obsolete keys'
// statistics that would include tombstones, the keys that are
// dropped by tombstones and duplicated user keys. See #847.
//
// Note that tables that contain exclusively range keys (i.e. no point keys,
// `NumEntries` and `RangeDeletionsBytesEstimate` are both zero) are excluded
// from elision-only compactions.
// TODO(travers): Consider an alternative heuristic for elision of range-keys.
if f.Stats.RangeDeletionsBytesEstimate*10 < f.Size &&
f.Stats.NumDeletions*10 < f.Stats.NumEntries {
f.Stats.NumDeletions*10 <= f.Stats.NumEntries {
return dst, true
}
if dst == nil {
Expand Down
1 change: 1 addition & 0 deletions compaction_picker_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -1176,6 +1176,7 @@ func TestCompactionOutputFileSize(t *testing.T) {
}
m.Stats.Valid = true
m.Stats.RangeDeletionsBytesEstimate = uint64(v)
m.Stats.NumDeletions = 1 // At least one range del responsible for the deletion bytes.
}
}
m.SmallestSeqNum = m.Smallest.SeqNum()
Expand Down
1 change: 1 addition & 0 deletions compaction_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -2097,6 +2097,7 @@ func TestCompactionTombstones(t *testing.T) {
compactInfo = &info
},
},
FormatMajorVersion: FormatNewest,
}
var err error
d, err = runDBDefineCmd(td, opts)
Expand Down
47 changes: 47 additions & 0 deletions testdata/compaction_tombstones
Original file line number Diff line number Diff line change
Expand Up @@ -244,3 +244,50 @@ range-deletions-bytes-estimate: 0
maybe-compact
----
[JOB 100] compacted(default) L5 [000004] (794 B) + L6 [000006] (13 K) -> L6 [] (0 B), in 1.0s (2.0s total), output rate 0 B/s

# A table containing only range keys is not eligible for elision.
# RANGEKEYDEL or RANGEKEYUNSET.

define
L6
rangekey:a-b:{(#1,RANGEKEYDEL)}
L6
rangekey:b-c:{(#2,RANGEKEYUNSET,@1)}
L6
rangekey:c-d:{(#3,RANGEKEYSET,@1)}
----
6:
000004:[a#1,RANGEKEYDEL-b#72057594037927935,RANGEKEYDEL]
000005:[b#2,RANGEKEYUNSET-c#72057594037927935,RANGEKEYUNSET]
000006:[c#3,RANGEKEYSET-d#72057594037927935,RANGEKEYSET]

wait-pending-table-stats
000004
----
num-entries: 0
num-deletions: 0
num-range-keys: 1
point-deletions-bytes-estimate: 0
range-deletions-bytes-estimate: 0

wait-pending-table-stats
000005
----
num-entries: 0
num-deletions: 0
num-range-keys: 1
point-deletions-bytes-estimate: 0
range-deletions-bytes-estimate: 0

wait-pending-table-stats
000006
----
num-entries: 0
num-deletions: 0
num-range-keys: 1
point-deletions-bytes-estimate: 0
range-deletions-bytes-estimate: 0

maybe-compact
----
(none)

0 comments on commit 1022ff8

Please sign in to comment.