backupccl: update restore/with-pause test #97800
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
msbutler
requested review from
smg260 and
renatolabs
and removed request for
a team
|
This change is |
|
Pre frontier checkpointing metrics: this 80 GB Restore with three pauses + ~3 minutes of sleep takes around 40 minutes, a throughput around 18 mb/s/node. |
This patch replaces the restore2TB/nodes=10/with-pause test with the new restore/pause/tpce/80GB/aws/nodes=4/cpus=8 test. The test now uses the new 80GB tpce fixture, and always pauses the restore job after around 25%, 50%, 75% completion. This patch also increases the portability of the new restore roachtest framework, allowing the new test to use its facilities. Going forward, this test will make it easier to benchmark and test future changes to restore checkpointing. Fixes cockroachdb#94093 Informs cockroachdb#87843 Release note: None
msbutler
force-pushed
the
butler-restore-pause-test
branch
from
92a31b4 to
493a875
Compare
lidorcarmel
approved these changes
Mar 3, 2023
| PrometheusNameSpace, Subsystem: "restore", Name: "duration"}, []string{"test_name"}) | ||
| // TODO(msbutler): to test the correctness of checkpointing, we should | ||
| // restore the same fixture without pausing it and fingerprint both restored | ||
| // databases. |
There was a problem hiding this comment.
we can hardcode the fingerprint.. (we still not get it from the unpaused run but it can be done manually).
There was a problem hiding this comment.
yeah, i plan to do that once this fingerprint bug is addressed #97916
| return sp.c.RunE(ctx, sp.c.Node(1), sp.restoreCmd(target, "")) | ||
| } | ||
|
|
||
| func (sp *restoreSpecs) runDetached(ctx context.Context, target string) (jobspb.JobID, error) { |
There was a problem hiding this comment.
I think you can now delete the old runRestoreDetached?
There was a problem hiding this comment.
yup, in a follow up PR, i plan delete all the old infra and bank tests :D
|
TFTR! bors r=lidorcarmel |
|
Build succeeded: |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This patch replaces the restore2TB/nodes=10/with-pause test with the new restore/pause/tpce/80GB/aws/nodes=4/cpus=8 test. The test now uses the new 80GB tpce fixture, and always pauses the restore job after around 25%, 50%, 75% completion.
This patch also increases the portability of the new restore roachtest framework, allowing the new test to use its facilities.
Going forward, this test will make it easier to benchmark and test future changes to restore checkpointing.
Fixes #94093
Informs #87843
Release note: None