Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

backupccl: add ignoreTimestamps flag to crdb_internal.fingerprint #98570

Closed
msbutler opened this issue Mar 14, 2023 · 2 comments · Fixed by #98759
Closed

backupccl: add ignoreTimestamps flag to crdb_internal.fingerprint #98570

msbutler opened this issue Mar 14, 2023 · 2 comments · Fixed by #98759
Assignees
@msbutler
Labels
A-disaster-recovery branch-release-23.1 Used to mark GA and release blockers, technical advisories, and bugs for 23.1 C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception) GA-blocker stability-period Intended to be worked on during a stability period. Use with the Milestone field to specify version. T-disaster-recovery

Comments

@msbutler
Copy link
Collaborator

msbutler commented Mar 14, 2023 •
edited by cockroach-jira-scripts
Loading

Currently, key timestamps affect the fingerprint computed by crdb_internal.fingerprint(). Suppose the cmd had an ignoreTimestamps flag which ignored key timestamps during fingerprint computation, the user could then call crdb_internal.fingerprint() to verify correctness, at scale, for more use cases. Some examples:

  • Regressions due to job restarts: Ensure a restore that was paused and resumed restores the same logical key-values as a restore without a pause-resume. Same application for Import.
  • Roundtrip regressions: Ensure a backup-restore roundtrip restores the same keys.
  • Regressions over time: Ensure that a restore cmd using the same large backup fixture restores the same keys every time.

At smaller scale (a few GBs), the user can already accomplish this via SHOW EXPERIEMNTAL_FINGERPRINTS FROM TABLE..., but building this new crdb_internal.fingerprint() flag, would allows us to verify correctness at arbitrary scale (in theory) and much more quickly.

Jira issue: CRDB-25339
@msbutler msbutler added C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception) GA-blocker stability-period Intended to be worked on during a stability period. Use with the Milestone field to specify version. labels Mar 14, 2023
@blathers-crl
Copy link

blathers-crl bot commented Mar 14, 2023

Hi @msbutler, please add branch-* labels to identify which branch(es) this release-blocker affects.

🦉 Hoot! I am a Blathers, a bot for CockroachDB. My owner is dev-inf.

@msbutler msbutler added branch-release-23.1 Used to mark GA and release blockers, technical advisories, and bugs for 23.1 T-disaster-recovery labels Mar 14, 2023
@blathers-crl
Copy link

blathers-crl bot commented Mar 14, 2023

cc @cockroachdb/disaster-recovery

@msbutler msbutler self-assigned this Mar 14, 2023
msbutler added a commit to msbutler/cockroach that referenced this issue Mar 16, 2023
@msbutler
backupccl: create stripped crdb_internal.fingerprint overload
b62a95f 
This patch adds a variant of crdb_internal.fingerprint(), which creates a
"stripped" fingerprint of the target span. Namely,
`crdb_internal.fingerping(span,true)` will return a fingerprint that is
agnostic to the mvcc timestamps and the index prefix of the key, and considers
only the latest mvcc history of the key span.

For example, suppose the user fingerprinted a table at some system time, then
backed up and restored it to that same system time. The restored table should
have the same fingerprint!

This fingerprint variant is signicantly more scalable than SHOW EXPERIMENTAL
FINGERPRINT, as it uses export requests compared to a simple scan.

Fixes cockroachdb#98570

Release note: None
@msbutler msbutler mentioned this issue Mar 16, 2023
Merged
msbutler added a commit to msbutler/cockroach that referenced this issue Mar 21, 2023
@msbutler
backupccl: create stripped crdb_internal.fingerprint overload
dedb24d 
This patch adds a variant of crdb_internal.fingerprint(), which creates a
"stripped" fingerprint of the target span. Namely,
`crdb_internal.fingerping(span,true)` will return a fingerprint that is
agnostic to the mvcc timestamps and the index prefix of the key, and considers
only the latest mvcc history of the key span.

For example, suppose the user fingerprinted a table at some system time, then
backed up and restored it to that same system time. The restored table should
have the same fingerprint!

This fingerprint variant is signicantly more scalable than SHOW EXPERIMENTAL
FINGERPRINT, as it uses export requests compared to a simple scan.

Fixes cockroachdb#98570

Release note: None
msbutler added a commit to msbutler/cockroach that referenced this issue Mar 23, 2023
@msbutler
backupccl: create stripped crdb_internal.fingerprint overload
5fc09b4 
This patch adds a variant of crdb_internal.fingerprint(), which creates a
"stripped" fingerprint of the target span. Namely,
`crdb_internal.fingerping(span,true)` will return a fingerprint that is
agnostic to the mvcc timestamps and the index prefix of the key, and considers
only the latest mvcc history of the key span.

For example, suppose the user fingerprinted a table at some system time, then
backed up and restored it to that same system time. The restored table should
have the same fingerprint!

This fingerprint variant is signicantly more scalable than SHOW EXPERIMENTAL
FINGERPRINT, as it uses export requests compared to a simple scan.

Fixes cockroachdb#98570

Release note: None
msbutler added a commit to msbutler/cockroach that referenced this issue Mar 24, 2023
@msbutler
backupccl: create stripped crdb_internal.fingerprint overload
fe22d22 
This patch adds a variant of crdb_internal.fingerprint(), which creates a
"stripped" fingerprint of the target span. Namely,
`crdb_internal.fingerping(span,true)` will return a fingerprint that is
agnostic to the mvcc timestamps and the index prefix of the key, and considers
only the latest mvcc history of the key span.

For example, suppose the user fingerprinted a table at some system time, then
backed up and restored it to that same system time. The restored table should
have the same fingerprint!

This fingerprint variant is signicantly more scalable than SHOW EXPERIMENTAL
FINGERPRINT, as it uses export requests compared to a simple scan.

Fixes cockroachdb#98570

Release note: None
msbutler added a commit to msbutler/cockroach that referenced this issue Mar 24, 2023
@msbutler
sql/keys introduce key decoders that do not rely on a codec
b7d6f75 
The codec normally used to decode and encode crdb keys requires a tenant prefix
upon creation. For callers that want to manipulate keys across tenants, the per
tenant codec provides a poor experience. This patch provides a few helper
functions to strip key prefixes without a codec.

Informs cockroachdb#98570
Release note: None
craig bot pushed a commit that referenced this issue Mar 25, 2023
@shralex @msbutler @renatolabs
Merge #96725 #98759 #99371
86ad731 
96725: sql: increase minimal max_range_size to 64MiB r=shralex a=shralex

We've seen that small range sizes can be detrimental to various components. This PR makes it so users can't lower max_range_size below 64MiB (half of the default min_range_size), instead of 64KiB previously.

Release note: Small ranges have been known to cause problems in various CRDB subsystems. This PR prevents setting max_range_size below COCKROACH_MIN_RANGE_MAX_BYTES, an environment variable which defaults to 64MiB (half of the default minimum range size). 

Epic: https://cockroachlabs.atlassian.net/browse/CRDB-24182
Fixes: #96549

98759: backupccl: create stripped crdb_internal.fingerprint overload r=stevendanna a=msbutler

This patch adds a variant of crdb_internal.fingerprint(), which creates a "stripped" fingerprint of the target span. Namely, `crdb_internal.fingerping(span,true)` will return a fingerprint that is agnostic to the mvcc timestamps and the index prefix of the key, and considers only the latest mvcc history of the key span.

For example, suppose the user fingerprinted a table at some system time, then backed up and restored it to that same system time. The restored table should have the same fingerprint!

This fingerprint variant is signicantly more scalable than SHOW EXPERIMENTAL FINGERPRINT, as it uses export requests compared to a simple scan.

Fixes #98570

Release note: None

99371: roachtest: lower default timeout r=srosenberg a=renatolabs

The previous default of 10 hours if unnecessarily long. As a consequence, if a test has a bug that causes it to hang, it will take 10h's worth of cluster usage, even if it's a test that generally succeeds in half an hour.

This should hopefully help with the timeouts we have been seeing in roachtest nightly runs.

Epic: none

Release note: None

Co-authored-by: shralex <[email protected]>
Co-authored-by: Michael Butler <[email protected]>
Co-authored-by: Renato Costa <[email protected]>
@craig craig bot closed this as completed in cf91202 Mar 25, 2023
blathers-crl bot pushed a commit that referenced this issue Mar 25, 2023
@msbutler
sql/keys introduce key decoders that do not rely on a codec
0aa5388 
The codec normally used to decode and encode crdb keys requires a tenant prefix
upon creation. For callers that want to manipulate keys across tenants, the per
tenant codec provides a poor experience. This patch provides a few helper
functions to strip key prefixes without a codec.

Informs #98570
Release note: None
@blathers-crl blathers-crl bot mentioned this issue Mar 25, 2023
Merged
blathers-crl bot pushed a commit that referenced this issue Mar 25, 2023
@msbutler
backupccl: create stripped crdb_internal.fingerprint overload
ebb96e4 
This patch adds a variant of crdb_internal.fingerprint(), which creates a
"stripped" fingerprint of the target span. Namely,
`crdb_internal.fingerpint(span,true)` will return a fingerprint that is
agnostic to the mvcc timestamps and the index prefix of the key, and considers
only the latest mvcc history of the key span. This cmd should only get used
over user table space, i.e. on keys with an index prefix.

For example, suppose the user fingerprinted a table at some system time, then
backed up and restored it to that same system time. The restored table should
have the same fingerprint!

crdb_internal.fingerpint(span,false) is equivalent to
crdb_internal.fingerpint(span,NULL,LATEST).

This fingerprint variant is signicantly more scalable than SHOW EXPERIMENTAL
FINGERPRINT, as it uses export requests compared to a simple scan.

Fixes #98570

Release note: None
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@msbutler msbutler

Labels
A-disaster-recovery branch-release-23.1 Used to mark GA and release blockers, technical advisories, and bugs for 23.1 C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception) GA-blocker stability-period Intended to be worked on during a stability period. Use with the Milestone field to specify version. T-disaster-recovery
Projects
No open projects
Disaster Recovery Backlog
Archived in project
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

backupccl: create stripped crdb_internal.fingerprint overload msbutler/cockroach
1 participant
@msbutler