Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

roachtest: add backup/mvcc-range-tombstones #89761

Merged

Conversation

erikgrinaker
Copy link
Contributor

@erikgrinaker erikgrinaker commented Oct 11, 2022

This patch adds a roachtest to ensure backup and restore works as expected in the presence of MVCC range tombstones. It uses data from TPCH's order table, 16 GB across 8 CSV files.

  1. Import half of the tpch.orders table (odd-numbered files).
  2. Take fingerprint, time 'initial'.
  3. Take a full database backup.
  4. Import the other half (even-numbered files), but cancel the import and roll the data back using MVCC range tombstones. Done twice.
  5. Take fingerprint, time 'canceled'.
  6. Successfully import the other half.
  7. Take fingerprint, time 'completed'.
  8. Drop the table and wait for deletion with MVCC range tombstone.
  9. Take an incremental database backup with revision history.

It then does point-in-time restores of the database at times 'initial', 'canceled', 'completed', and the latest time, and compares the fingerprints to the original data.

Resolves #89644.

Release note: None

@erikgrinaker erikgrinaker requested review from msbutler and a team October 11, 2022 17:10
@erikgrinaker erikgrinaker self-assigned this Oct 11, 2022
@cockroach-teamcity
Copy link
Member

This change is Reviewable

@erikgrinaker erikgrinaker force-pushed the roachtest-backup-range-tombstones branch 4 times, most recently from a6728c8 to 7378421 Compare October 11, 2022 18:42
Copy link
Collaborator

@msbutler msbutler left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good overall! left a few questions. thanks for doing this!

pkg/cmd/roachtest/tests/backup.go Outdated Show resolved Hide resolved
pkg/cmd/roachtest/tests/backup.go Outdated Show resolved Hide resolved
pkg/cmd/roachtest/tests/backup.go Outdated Show resolved Hide resolved
@erikgrinaker erikgrinaker force-pushed the roachtest-backup-range-tombstones branch 3 times, most recently from 610946e to 8434265 Compare October 12, 2022 07:38
This patch adds a roachtest to ensure backup and restore works as
expected in the presence of MVCC range tombstones. It uses data from
TPCH's order table, 16 GB across 8 CSV files.

1. Import half of the `tpch.orders` table (odd-numbered files).
2. Take fingerprint, time 'initial'.
3. Take a full database backup.
4. Import the other half (even-numbered files), but cancel the import
   and roll the data back using MVCC range tombstones. Done twice.
5. Take fingerprint, time 'canceled'.
6. Successfully import the other half.
7. Take fingerprint, time 'completed'.
8. Drop the table and wait for deletion with MVCC range tombstone.
9. Take an incremental database backup with revision history.

It then does point-in-time restores of the database at times 'initial',
'canceled', 'completed', and the latest time, and compares the
fingerprints to the original data.

Release note: None
Copy link
Collaborator

@msbutler msbutler left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great! Thanks for writing this!

@erikgrinaker
Copy link
Contributor Author

TFTR!

bors r+

@craig
Copy link
Contributor

craig bot commented Oct 12, 2022

Build succeeded:

@craig craig bot merged commit 350f992 into cockroachdb:master Oct 12, 2022
@erikgrinaker erikgrinaker deleted the roachtest-backup-range-tombstones branch October 30, 2022 13:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

roachtest: backup/restore with MVCC range tombstones
3 participants