ingest: Add invariant-gated checks for accurate use of virtual sstable bounds #2593
Comments
itsbilal
changed the title
raggar
added a commit
to raggar/pebble
that referenced
this issue
Jun 8, 2023
This pr adds bounds checking (global and block) for singleLevelIterator when returning a value. Fixes: cockroachdb#2593 Release note: None
raggar
added a commit
to raggar/pebble
that referenced
this issue
Jun 9, 2023
This pr adds bounds checking (global and block) for singleLevelIterator when returning a value. Fixes: cockroachdb#2593 Release note: None
raggar
added a commit
to raggar/pebble
that referenced
this issue
Jun 9, 2023
This pr adds bounds checking (global and block) for singleLevelIterator when returning a value. Fixes: cockroachdb#2593 Release note: None
raggar
added a commit
to raggar/pebble
that referenced
this issue
Jun 12, 2023
…or when returning a value. Fixes: cockroachdb#2593 Release note: None
raggar
added a commit
to raggar/pebble
that referenced
this issue
Jun 14, 2023
…or when returning a value. Fixes: cockroachdb#2593 Release note: None
raggar
added a commit
to raggar/pebble
that referenced
this issue
Jun 14, 2023
…or when returning a value. Fixes: cockroachdb#2593 Release note: None
raggar
added a commit
to raggar/pebble
that referenced
this issue
Jun 15, 2023
…or when returning a value. Fixes: cockroachdb#2593 Release note: None
raggar
added a commit
to raggar/pebble
that referenced
this issue
Jun 15, 2023
…or when returning a value. Fixes: cockroachdb#2593 Release note: None
raggar
added a commit
to raggar/pebble
that referenced
this issue
Jun 15, 2023
…or when returning a value. Fixes: cockroachdb#2593 Release note: None
raggar
added a commit
that referenced
this issue
Jun 15, 2023
…or when returning a value. Fixes: #2593 Release note: None
|
Reopening this issue to refer to just the tight calculation of bounds; there is one user key right at the Smallest/Largest bounds. We can confirm this in |
raggar
added a commit
to raggar/pebble
that referenced
this issue
Jul 19, 2023
Added a check inside of `FileMetadata.ValidateVirtual()` to ensure that the keys returned from a virtual sstable are within the bounds of the Smallest and Largest metadata keys. Informs: cockroachdb#2593
raggar
added a commit
to raggar/pebble
that referenced
this issue
Jul 21, 2023
Added a check inside of `FileMetadata.ValidateVirtual()` to ensure that the keys returned from a virtual sstable are within the bounds of the Smallest and Largest metadata keys. Informs: cockroachdb#2593
raggar
added a commit
to raggar/pebble
that referenced
this issue
Jul 21, 2023
Added a check inside of `FileMetadata.ValidateVirtual()` to ensure that the keys returned from a virtual sstable are within the bounds of the Smallest and Largest metadata keys. Informs: cockroachdb#2593
raggar
added a commit
to raggar/pebble
that referenced
this issue
Jul 21, 2023
Added a check inside of `FileMetadata.ValidateVirtual()` to ensure that the keys returned from a virtual sstable are within the bounds of the Smallest and Largest metadata keys. Informs: cockroachdb#2593
raggar
added a commit
to raggar/pebble
that referenced
this issue
Jul 21, 2023
Added a check inside of `FileMetadata.ValidateVirtual()` to ensure that the keys returned from a virtual sstable are within the bounds of the Smallest and Largest metadata keys. Informs: cockroachdb#2593
raggar
added a commit
to raggar/pebble
that referenced
this issue
Jul 21, 2023
Added a check inside of `FileMetadata.ValidateVirtual()` to ensure that the keys returned from a virtual sstable are within the bounds of the Smallest and Largest metadata keys. Informs: cockroachdb#2593
raggar
added a commit
to raggar/pebble
that referenced
this issue
Jul 21, 2023
Added a check inside of `FileMetadata.ValidateVirtual()` to ensure that the keys returned from a virtual sstable are within the bounds of the Smallest and Largest metadata keys. Informs: cockroachdb#2593
raggar
added a commit
to raggar/pebble
that referenced
this issue
Jul 27, 2023
Added a check inside of `FileMetadata.ValidateVirtual()` to ensure that the keys returned from a virtual sstable are within the bounds of the Smallest and Largest metadata keys. Informs: cockroachdb#2593
raggar
added a commit
to raggar/pebble
that referenced
this issue
Jul 27, 2023
Added a check inside of `FileMetadata.ValidateVirtual()` to ensure that the keys returned from a virtual sstable are within the bounds of the Smallest and Largest metadata keys. Informs: cockroachdb#2593
raggar
added a commit
to raggar/pebble
that referenced
this issue
Jul 28, 2023
Added a check inside of `FileMetadata.ValidateVirtual()` to ensure that the keys returned from a virtual sstable are within the bounds of the Smallest and Largest metadata keys. Informs: cockroachdb#2593
raggar
added a commit
to raggar/pebble
that referenced
this issue
Jul 28, 2023
Added a check inside of `FileMetadata.ValidateVirtual()` to ensure that the keys returned from a virtual sstable are within the bounds of the Smallest and Largest metadata keys. Informs: cockroachdb#2593
raggar
added a commit
to raggar/pebble
that referenced
this issue
Aug 1, 2023
Added a check inside of `FileMetadata.ValidateVirtual()` to ensure that the keys returned from a virtual sstable are within the bounds of the Smallest and Largest metadata keys. Informs: cockroachdb#2593
raggar
added a commit
to raggar/pebble
that referenced
this issue
Aug 1, 2023
Added a check inside of `FileMetadata.ValidateVirtual()` to ensure that the keys returned from a virtual sstable are within the bounds of the Smallest and Largest metadata keys. Informs: cockroachdb#2593
raggar
added a commit
to raggar/pebble
that referenced
this issue
Aug 1, 2023
Added a check inside of `FileMetadata.ValidateVirtual()` to ensure that the keys returned from a virtual sstable are within the bounds of the Smallest and Largest metadata keys. Informs: cockroachdb#2593
raggar
added a commit
to raggar/pebble
that referenced
this issue
Aug 1, 2023
Added a check inside of `FileMetadata.ValidateVirtual()` to ensure that the keys returned from a virtual sstable are within the bounds of the Smallest and Largest metadata keys. Informs: cockroachdb#2593
raggar
added a commit
to raggar/pebble
that referenced
this issue
Aug 1, 2023
Added a check inside of `FileMetadata.ValidateVirtual()` to ensure that the keys returned from a virtual sstable are within the bounds of the Smallest and Largest metadata keys. Informs: cockroachdb#2593
raggar
added a commit
to raggar/pebble
that referenced
this issue
Aug 1, 2023
Added a check inside of `FileMetadata.ValidateVirtual()` to ensure that the keys returned from a virtual sstable are within the bounds of the Smallest and Largest metadata keys. Informs: cockroachdb#2593
raggar
added a commit
to raggar/pebble
that referenced
this issue
Aug 1, 2023
Added a check inside of `FileMetadata.ValidateVirtual()` to ensure that the keys returned from a virtual sstable are within the bounds of the Smallest and Largest metadata keys. Informs: cockroachdb#2593
raggar
added a commit
to raggar/pebble
that referenced
this issue
Aug 1, 2023
Added a check inside of `FileMetadata.ValidateVirtual()` to ensure that the keys returned from a virtual sstable are within the bounds of the Smallest and Largest metadata keys. Informs: cockroachdb#2593
raggar
added a commit
to raggar/pebble
that referenced
this issue
Aug 3, 2023
Added a check inside of `FileMetadata.ValidateVirtual()` to ensure that the keys returned from a virtual sstable are within the bounds of the Smallest and Largest metadata keys. Informs: cockroachdb#2593
raggar
added a commit
that referenced
this issue
Aug 3, 2023
Added a check inside of `FileMetadata.ValidateVirtual()` to ensure that the keys returned from a virtual sstable are within the bounds of the Smallest and Largest metadata keys. Informs: #2593
|
Fixed with #2698 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
When #2538 merges, we will be synthesizing file bounds for virtual sstables that got keys masked through an Excise. And when #2582 merges, we'll be doing the same for sstables split by virtualizations We should add invariants.Enabled-gated checks to ensure that iterators created on top of virtual sstables do not return excised/masked keys, and that every key returned is within {Smallest,Largest}{PointKey,RangeKey}. We should also confirm if the bounds were calculated correctly to begin with.
After #2616 is complete, this issue only refers to verifying the tight calculation of sstable bounds, i.e. that there is a user key at each of the Smallest, Largest bounds. This check can happen in
fileManifest.ValidateVirtual, however it would require creating iterators and is hence best to be invariants-gated as well.The text was updated successfully, but these errors were encountered: