Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve error handling when reading request and Region meta change in concurrency (#1101) #1109

Conversation

JaySon-Huang
Copy link
Contributor

@JaySon-Huang JaySon-Huang commented Sep 21, 2020

manually cherry-pick of #1101


What problem does this PR solve?

Issue Number: close #1095

Problem Summary:
If region meta changed between learner read and get streams from storage, we can not ensure the correctness of read data. We should retry those key ranges.
Before this PR, if the super batch is enabled and happens to this error, an error will directly be thrown to users. Users need to retry their queries. We should handle those retry inside TiFlash.

What is changed and how it works?

  • If we happen to RegionException after read from storage, and super batch is enabled, then
    • clear streams read from local
    • push those regions into region_retry and read from remote storage later
  • Refine some codes for FailPoints

Related changes

  • Need to cherry-pick to the release branch 4.0

Check List

Tests

  • Integration test

Side effects

  • Performance regression
    • If we happen to this situation, it takes more time to read those key-ranges from remote. And we don't apply filters on remote storage now

Release note

  • Improve error handling when reading request and Region meta change in concurrency

Signed-off-by: JaySon-Huang <[email protected]>
@JaySon-Huang JaySon-Huang added type/enhancement The issue or PR belongs to an enhancement. CHERRY-PICK cherry pick labels Sep 21, 2020
@JaySon-Huang JaySon-Huang added this to the v4.0.7 milestone Sep 21, 2020
@JaySon-Huang JaySon-Huang self-assigned this Sep 21, 2020
@JaySon-Huang
Copy link
Contributor Author

/run-all-tests

1 similar comment
@JaySon-Huang
Copy link
Contributor Author

/run-all-tests

Copy link
Contributor

@windtalker windtalker left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ti-srebot ti-srebot added the status/LGT1 Indicates that a PR has LGTM 1. label Sep 21, 2020
@JaySon-Huang JaySon-Huang merged commit d49635e into pingcap:release-4.0 Sep 21, 2020
@JaySon-Huang JaySon-Huang deleted the refine_retry_when_validation_fail_4.0 branch September 21, 2020 09:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CHERRY-PICK cherry pick status/LGT1 Indicates that a PR has LGTM 1. type/enhancement The issue or PR belongs to an enhancement.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants