kvserver: iterator panic during split #104007
Comments
erikgrinaker
added
C-bug
|
Hi @erikgrinaker, please add branch-* labels to identify which branch(es) this release-blocker affects. 🦉 Hoot! I am a Blathers, a bot for CockroachDB. My owner is dev-inf. |
|
Saw this in in a manual |
|
We aren't sanitizing local keys, like what we do for the other split path: Lines 5834 to 5836 in 2a7427c This is unrelated*. The problem occurs because the iterator is setup with the range's endKey, which is global, whilst we are passing the range's start key again to The start key |
|
Did repro with: startKey: keys.MinKey,
endKey: keys.Meta1KeyMax,The range shouldn't be split regardless, since we don't split cockroach/pkg/kv/kvserver/replica_command.go Lines 391 to 393 in 2a7427c The split key is irrelevant. Lines 5834 to 5836 in 2a7427c
|
104082: storage: prevent iter panic on meta1 split key r=itsbilal a=kvoli It was possible that a load based split was suggested for `meta1`, which would call `MVCCFirstSplitKey` and panic as the `meta1` start key `\Min` is local, whilst the `meta1` end key is global `0x02 0xff 0xff`. Add a check that the start key is greater than the `meta1` end key before processing in `MVCCFirstSplitKey` to prevent the panic. Note `meta1` would never be split regardless, as `storage.IsValidSplitKey` would fail after finding a split key. Also note that if the desired split key is a local key, the same problem doesn't exist as the minimum split key would be used to seek the first split key instead. Fixes: #104007 Release note: None Co-authored-by: Austen McClernon <[email protected]>
It was possible that a load based split was suggested for `meta1`, which would call `MVCCFirstSplitKey` and panic as the `meta1` start key `\Min` is local, whilst the `meta1` end key is global `0x02 0xff 0xff`. Add a check that the start key is greater than the `meta1` end key before processing in `MVCCFirstSplitKey` to prevent the panic. Note `meta1` would never be split regardless, as `storage.IsValidSplitKey` would fail after finding a split key. Also note that if the desired split key is a local key, the same problem doesn't exist as the minimum split key would be used to seek the first split key instead. Fixes: #104007 Release note: None
It was possible that a load based split was suggested for `meta1`, which would call `MVCCFirstSplitKey` and panic as the `meta1` start key `\Min` is local, whilst the `meta1` end key is global `0x02 0xff 0xff`. Add a check that the start key is greater than the `meta1` end key before processing in `MVCCFirstSplitKey` to prevent the panic. Note `meta1` would never be split regardless, as `storage.IsValidSplitKey` would fail after finding a split key. Also note that if the desired split key is a local key, the same problem doesn't exist as the minimum split key would be used to seek the first split key instead. Fixes: #104007 Release note: None
It was possible that a load based split was suggested for `meta1`, which would call `MVCCFirstSplitKey` and panic as the `meta1` start key `\Min` is local, whilst the `meta1` end key is global `0x02 0xff 0xff`. Add a check that the start key is greater than the `meta1` end key before processing in `MVCCFirstSplitKey` to prevent the panic. Note `meta1` would never be split regardless, as `storage.IsValidSplitKey` would fail after finding a split key. Also note that if the desired split key is a local key, the same problem doesn't exist as the minimum split key would be used to seek the first split key instead. Fixes: cockroachdb#104007 Release note: None
Seen when running
cockroach workload init kv --splits 10000on a new roachprod cluster. Seems like fallout from #103690.Jira issue: CRDB-28305
The text was updated successfully, but these errors were encountered: