You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
jbowens opened this issue
Mar 27, 2023
· 1 comment
Labels
A-storageRelating to our storage engine (Pebble) on-disk storage.C-cleanupTech debt, refactors, loose ends, etc. Solution not expected to significantly change behavior.T-storageStorage Team
The current implementation of the storage.CheckSSTConflicts function is very complicated, and it's very difficult to reason about its correctness. We've uncovered many bugs (eg, #99566) in its implementation as a result, and today largely rely on randomized testing (from KV nemesis and the unit test in #98408) to ensure correctness. We should rewrite and simplify it.
Some ideas:
@itsbilal suggested a four-iterator approach. We'd use 1 range key iterator and 1 point iterator for both the engine and the sstable.
We could implement a "tandem" iterator that keeps the engine iterator positioned appropriately as the sstable iterator moves across the keyspace. There are some subtleties in what "positioned appropriately" means (eg, the engine iterator may need to read before the sstable iterator's position in order to observe a range key). Separating out the mechanics of keeping the two iterators in sync should make the actual conflict checking more tractable.
blathers-crlbot
added
the
C-bug
Code not up to spec/doc, specs & docs deemed correct. Solution expected to change code/behavior.
label
Mar 27, 2023
jbowens
added
C-cleanup
Tech debt, refactors, loose ends, etc. Solution not expected to significantly change behavior.
and removed
C-bug
Code not up to spec/doc, specs & docs deemed correct. Solution expected to change code/behavior.
labels
Mar 27, 2023
A-storageRelating to our storage engine (Pebble) on-disk storage.C-cleanupTech debt, refactors, loose ends, etc. Solution not expected to significantly change behavior.T-storageStorage Team
The current implementation of the
storage.CheckSSTConflicts
function is very complicated, and it's very difficult to reason about its correctness. We've uncovered many bugs (eg, #99566) in its implementation as a result, and today largely rely on randomized testing (from KV nemesis and the unit test in #98408) to ensure correctness. We should rewrite and simplify it.Some ideas:
Jira issue: CRDB-26007
The text was updated successfully, but these errors were encountered: