DeleteRange regression tests using public API #4476
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
There was a problem hiding this comment.
ajkr has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.
abhimadan
approved these changes
Oct 10, 2018
| ASSERT_EQ(0, NumTableFilesAtLevel(0)); | ||
| // Roughly the left half of L1 files should have overlapping boundary keys, | ||
| // while the right half should not. | ||
| ASSERT_GE(NumTableFilesAtLevel(1), kNumFiles); |
There was a problem hiding this comment.
Does this check mean that it's possible for there to be more files after compaction than before?
There was a problem hiding this comment.
Yeah, because the decision to flush an L0 file is different from the decision to cut over to a new output file during L0->L1:
- For L0 we flush after we have exactly
kFileBytes / kValueByteskeys in the memtable. - For L0->L1, we cut over to a new file once we have
kFileBytesin the SST file. But there may be less thankFileBytes / kValueByteskeys at that point since things other than value size count towards the size limit (e.g., keys and metadata).
petermattis
pushed a commit
to petermattis/rocksdb
that referenced
this pull request
Oct 17, 2018
Summary: I wrote a couple tests using the public API to expose/prevent the bugs we talked. In particular, - When files have overlapping endpoints and a range tombstone spans them, ensure the largest key does not reappear to readers. This was happening due to a bug that skipped writing range tombstones to an output file when their begin key exactly matched the file's largest key. - When a tombstone spans multiple atomic compaction units, ensure newer keys do not disappear by being compacted beneath it. This happened due to a range tombstone appearing untruncated to readers when it spanned files with overlapping endpoints, even if it extended into files without overlapping endpoints (i.e., different atomic compaction units). Pull Request resolved: facebook#4476 Differential Revision: D10286001 Pulled By: ajkr fbshipit-source-id: bb5ca51d0f90812fb37bfe1d01aec93f7eda55aa
41 tasks
petermattis
pushed a commit
to cockroachdb/rocksdb
that referenced
this pull request
Dec 5, 2018
Summary: I wrote a couple tests using the public API to expose/prevent the bugs we talked. In particular, - When files have overlapping endpoints and a range tombstone spans them, ensure the largest key does not reappear to readers. This was happening due to a bug that skipped writing range tombstones to an output file when their begin key exactly matched the file's largest key. - When a tombstone spans multiple atomic compaction units, ensure newer keys do not disappear by being compacted beneath it. This happened due to a range tombstone appearing untruncated to readers when it spanned files with overlapping endpoints, even if it extended into files without overlapping endpoints (i.e., different atomic compaction units). Pull Request resolved: facebook#4476 Differential Revision: D10286001 Pulled By: ajkr fbshipit-source-id: bb5ca51d0f90812fb37bfe1d01aec93f7eda55aa
32 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
I wrote a couple tests using the public API to expose/prevent the bugs we talked. In particular,
Test Plan: both of these tests fail before #4432 and pass afterwards