Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

kvserver: reads during replica removal can return empty results #64325

Closed
erikgrinaker opened this issue Apr 28, 2021 · 0 comments · Fixed by #64324
Closed

kvserver: reads during replica removal can return empty results #64325

erikgrinaker opened this issue Apr 28, 2021 · 0 comments · Fixed by #64324
Assignees
@erikgrinaker
Labels
A-kv-replication Relating to Raft, consensus, and coordination. branch-release-20.1 Used to mark GA and release blockers, technical advisories, and bugs for 20.1 branch-release-20.2 Used to mark GA and release blockers, technical advisories, and bugs for 20.2 C-bug Code not up to spec/doc, specs & docs deemed correct. Solution expected to change code/behavior. C-technical-advisory Caused a technical advisory S-0-visible-logical-error Database stores inconsistent data in some cases, or queries return invalid results silently.

Comments

@erikgrinaker
Copy link
Contributor

erikgrinaker commented Apr 28, 2021 •
edited
Loading

Read requests submitted during replica removal may be evaluated on the removed replica, returning empty results. This was discovered when restoring a backup did not recover all of the data -- an entire range was missing. This was caused by the export request racing with a range merge and replica removal, where the export request was evaluated on the removed (empty) replica, returning an empty result and omitting the range from the backup.
@erikgrinaker erikgrinaker added C-bug Code not up to spec/doc, specs & docs deemed correct. Solution expected to change code/behavior. A-kv-replication Relating to Raft, consensus, and coordination. labels Apr 28, 2021
@erikgrinaker erikgrinaker self-assigned this Apr 28, 2021
@erikgrinaker erikgrinaker mentioned this issue Apr 28, 2021
Merged
@craig craig bot closed this as completed in 9190550 Apr 28, 2021
This was referenced Apr 29, 2021
Merged
Merged
Merged
@knz knz added the S-0-visible-logical-error Database stores inconsistent data in some cases, or queries return invalid results silently. label Apr 30, 2021
@shermanCRL shermanCRL mentioned this issue Apr 30, 2021
Merged
@rytaft rytaft added C-technical-advisory Caused a technical advisory branch-release-20.2 Used to mark GA and release blockers, technical advisories, and bugs for 20.2 branch-release-20.1 Used to mark GA and release blockers, technical advisories, and bugs for 20.1 labels Dec 6, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@erikgrinaker erikgrinaker

Labels
A-kv-replication Relating to Raft, consensus, and coordination. branch-release-20.1 Used to mark GA and release blockers, technical advisories, and bugs for 20.1 branch-release-20.2 Used to mark GA and release blockers, technical advisories, and bugs for 20.2 C-bug Code not up to spec/doc, specs & docs deemed correct. Solution expected to change code/behavior. C-technical-advisory Caused a technical advisory S-0-visible-logical-error Database stores inconsistent data in some cases, or queries return invalid results silently.
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

kvserver: synchronize replica removal with read-only requests erikgrinaker/cockroach
3 participants
@knz @erikgrinaker @rytaft