Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Allow ExportRequest to resume exports on timestamp boundary #68231

Closed
aliher1911 opened this issue Jul 29, 2021 · 0 comments · Fixed by #68102
Closed

Allow ExportRequest to resume exports on timestamp boundary #68231

aliher1911 opened this issue Jul 29, 2021 · 0 comments · Fixed by #68102
Assignees
@aliher1911
Labels
A-kv Anything in KV that doesn't belong in a more specific category. C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception) T-kv KV Team

Comments

@aliher1911
Copy link
Contributor

Is your feature request related to a problem? Please describe.
I'm always frustrated when export request can not return back an SST because size of all versions for the key in backup exceeds kv.bulk_sst.max_allowed_overage.

Describe the solution you'd like
Export request should return next key timestamp together with resume span and accept it as an optional argument so that export can have finer grained control over export boundaries. This is also needed to make export batches smaller and provide more efficient export throttling control.

Additional context
Part of #66486
@aliher1911 aliher1911 added C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception) A-kv Anything in KV that doesn't belong in a more specific category. T-kv KV Team labels Jul 29, 2021
@aliher1911 aliher1911 self-assigned this Jul 29, 2021
@aliher1911 aliher1911 mentioned this issue Aug 5, 2021
Merged
@craig craig bot closed this as completed in ab15e5f Aug 12, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@aliher1911 aliher1911

Labels
A-kv Anything in KV that doesn't belong in a more specific category. C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception) T-kv KV Team
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

storage/storageccl: Add an option to break export mid key aliher1911/cockroach
1 participant
@aliher1911