-
Notifications
You must be signed in to change notification settings - Fork 412
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
storage: GC segments contained by the pack #6010
Conversation
Signed-off-by: Wish <[email protected]>
Signed-off-by: Wish <[email protected]>
Signed-off-by: Wish <[email protected]>
Signed-off-by: Wish <[email protected]>
Signed-off-by: Wish <[email protected]>
Signed-off-by: Wish <[email protected]>
Signed-off-by: Wish <[email protected]>
Signed-off-by: Wish <[email protected]>
Signed-off-by: Wish <[email protected]>
Signed-off-by: Wish <[email protected]>
[REVIEW NOTIFICATION] This pull request has been approved by:
To complete the pull request process, please ask the reviewers in the list to review by filling The full list of commands accepted by this bot can be found here. Reviewer can indicate their review by submitting an approval review. |
Signed-off-by: Wish <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
Signed-off-by: Wish <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
lgtm
Signed-off-by: Wish <[email protected]>
/run-all-tests |
Signed-off-by: Wish <[email protected]>
Coverage for changed files
Coverage summary
full coverage report (for internal network access only) |
/merge |
@breezewish: It seems you want to merge this PR, I will help you trigger all the tests: /run-all-tests You only need to trigger If you have any questions about the PR merge process, please refer to pr process. Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository. |
This pull request has been accepted and is ready to merge. Commit hash: ca9529d
|
close #5975 Signed-off-by: Wish <[email protected]>
/run-unit-test |
Coverage for changed files
Coverage summary
full coverage report (for internal network access only) |
In response to a cherrypick label: failed to apply #6010 on top of branch "release-6.3":
|
What problem does this PR solve?
Issue Number: close #5975
What is changed and how it works?
Previously, we count segment rows by using the sum rows of overlapped packs. When the segment rows are < 0.7 rows of DTFiles, the DTFile will be regarded as "reclaimable" and the stable will be re-generated.
In this PR, we count segment rows by using the sum rows of fully contained packs. This means, the calculated segment rows will be smaller than before, and segments will more likely being GCed.
Here is an illustration of the new contained rule:
Consider the case where there is only 1 pack in the DTFile. When segment is fully or partially contained by this pack, the segment will be GCed, no matter how many actual rows are there in the segment. This leads to extra GCs in some cases, but the negative impact is small, as the cost of rewriting 1 pack should be small.
Check List
Tests
Side effects
Documentation
Release note