kv: guardrails for range size #96549
Labels
C-enhancement
Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception)
T-kv
KV Team
Comments
shralex
added
C-enhancement
shralex
added a commit
to shralex/cockroach
that referenced
this issue
Feb 7, 2023
We've seen that small range sizes can be detrimental to various components. This PR makes it so users can't lower max_range_size below 64MiB (half of the default min_range_size), instead of 64KiB previously. Epic: https://cockroachlabs.atlassian.net/browse/CRDB-24182 Fixes: cockroachdb#96549
shralex
added a commit
to shralex/cockroach
that referenced
this issue
Feb 10, 2023
We've seen that small range sizes can be detrimental to various components. This PR makes it so users can't lower max_range_size below 64MiB (half of the default min_range_size), instead of 64KiB previously. Epic: https://cockroachlabs.atlassian.net/browse/CRDB-24182 Fixes: cockroachdb#96549
shralex
added a commit
to shralex/cockroach
that referenced
this issue
Feb 11, 2023
We've seen that small range sizes can be detrimental to various components. This PR makes it so users can't lower max_range_size below 64MiB (half of the default min_range_size), instead of 64KiB previously. Release Note: Small ranges have been known to cause problems in various CRDB subsystems. This PR prevents setting max_range_size below COCKROACH_MIN_RANGE_MAX_BYTES, an environment variable which defaults to 64MiB (half of the default minimum range size). Epic: https://cockroachlabs.atlassian.net/browse/CRDB-24182 Fixes: cockroachdb#96549
shralex
added a commit
to shralex/cockroach
that referenced
this issue
Feb 21, 2023
We've seen that small range sizes can be detrimental to various components. This PR makes it so users can't lower max_range_size below 64MiB (half of the default min_range_size), instead of 64KiB previously. Release Note: Small ranges have been known to cause problems in various CRDB subsystems. This PR prevents setting max_range_size below COCKROACH_MIN_RANGE_MAX_BYTES, an environment variable which defaults to 64MiB (half of the default minimum range size). Epic: https://cockroachlabs.atlassian.net/browse/CRDB-24182 Fixes: cockroachdb#96549
shralex
added a commit
to shralex/cockroach
that referenced
this issue
Feb 23, 2023
We've seen that small range sizes can be detrimental to various components. This PR makes it so users can't lower max_range_size below 64MiB (half of the default min_range_size), instead of 64KiB previously. Release Note: Small ranges have been known to cause problems in various CRDB subsystems. This PR prevents setting max_range_size below COCKROACH_MIN_RANGE_MAX_BYTES, an environment variable which defaults to 64MiB (half of the default minimum range size). Epic: https://cockroachlabs.atlassian.net/browse/CRDB-24182 Fixes: cockroachdb#96549
craig bot
pushed a commit
that referenced
this issue
Mar 25, 2023
96725: sql: increase minimal max_range_size to 64MiB r=shralex a=shralex We've seen that small range sizes can be detrimental to various components. This PR makes it so users can't lower max_range_size below 64MiB (half of the default min_range_size), instead of 64KiB previously. Release note: Small ranges have been known to cause problems in various CRDB subsystems. This PR prevents setting max_range_size below COCKROACH_MIN_RANGE_MAX_BYTES, an environment variable which defaults to 64MiB (half of the default minimum range size). Epic: https://cockroachlabs.atlassian.net/browse/CRDB-24182 Fixes: #96549 98759: backupccl: create stripped crdb_internal.fingerprint overload r=stevendanna a=msbutler This patch adds a variant of crdb_internal.fingerprint(), which creates a "stripped" fingerprint of the target span. Namely, `crdb_internal.fingerping(span,true)` will return a fingerprint that is agnostic to the mvcc timestamps and the index prefix of the key, and considers only the latest mvcc history of the key span. For example, suppose the user fingerprinted a table at some system time, then backed up and restored it to that same system time. The restored table should have the same fingerprint! This fingerprint variant is signicantly more scalable than SHOW EXPERIMENTAL FINGERPRINT, as it uses export requests compared to a simple scan. Fixes #98570 Release note: None 99371: roachtest: lower default timeout r=srosenberg a=renatolabs The previous default of 10 hours if unnecessarily long. As a consequence, if a test has a bug that causes it to hang, it will take 10h's worth of cluster usage, even if it's a test that generally succeeds in half an hour. This should hopefully help with the timeouts we have been seeing in roachtest nightly runs. Epic: none Release note: None Co-authored-by: shralex <[email protected]> Co-authored-by: Michael Butler <[email protected]> Co-authored-by: Renato Costa <[email protected]>
blathers-crl bot
pushed a commit
that referenced
this issue
Mar 25, 2023
We've seen that small range sizes can be detrimental to various components. This PR makes it so users can't lower max_range_size below 64MiB (half of the default min_range_size), instead of 64KiB previously. Release Note: Small ranges have been known to cause problems in various CRDB subsystems. This PR prevents setting max_range_size below COCKROACH_MIN_RANGE_MAX_BYTES, an environment variable which defaults to 64MiB (half of the default minimum range size). Epic: https://cockroachlabs.atlassian.net/browse/CRDB-24182 Fixes: #96549
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
We've seen that changing max_range_size to a low value can break multiple sub-systems. We should implement guardrails for setting min/max range size (for user and/or system ranges).
Jira issue: CRDB-24182
The text was updated successfully, but these errors were encountered: