Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

schedulers: unify the use and GC of hot-region's pendings and regionPendings #3921

Merged
merged 12 commits into from
Aug 3, 2021

Conversation

HunDunDM
Copy link
Member

@HunDunDM HunDunDM commented Jul 28, 2021

Signed-off-by: HunDunDM [email protected]

What problem does this PR solve?

  • The current pendings and regionPendings records duplicate data, and they are each useful.
  • The current weight attenuation strategy is unreasonable.
  • Misuse of CreateTime by the current GC strategy may lead to premature GC.

What is changed and how it works?

  • unify the use and GC of hot-region's pendings and regionPendings, no longer related to CreateTime.
  • weight is constant at 1.

Check List

Tests

  • Unit test

Code changes

Side effects

Related changes

Release note

None

@ti-chi-bot
Copy link
Member

ti-chi-bot commented Jul 28, 2021

[REVIEW NOTIFICATION]

This pull request has been approved by:

  • lhy1024
  • nolouch

To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment.
After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.

The full list of commands accepted by this bot can be found here.

Reviewer can indicate their review by submitting an approval review.
Reviewer can cancel approval by submitting a request changes review.

@ti-chi-bot ti-chi-bot added the release-note Denotes a PR that will be considered when it comes time to generate release notes. label Jul 28, 2021
@ti-chi-bot ti-chi-bot requested review from lhy1024 and rleungx July 28, 2021 12:53
@codecov
Copy link

codecov bot commented Jul 28, 2021

Codecov Report

Merging #3921 (319524f) into master (386b044) will increase coverage by 0.16%.
The diff coverage is 91.89%.

❗ Current head 319524f differs from pull request most recent head 286220b. Consider uploading reports for the commit 286220b to get more accurate results
Impacted file tree graph

@@            Coverage Diff             @@
##           master    #3921      +/-   ##
==========================================
+ Coverage   74.51%   74.68%   +0.16%     
==========================================
  Files         246      246              
  Lines       25068    25068              
==========================================
+ Hits        18680    18721      +41     
+ Misses       4723     4696      -27     
+ Partials     1665     1651      -14     
Flag Coverage Δ
unittests 74.68% <91.89%> (+0.16%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
server/schedulers/utils.go 92.85% <ø> (-0.38%) ⬇️
server/schedulers/hot_region.go 86.83% <91.89%> (+1.05%) ⬆️
server/kv/etcd_kv.go 69.56% <0.00%> (-10.15%) ⬇️
server/tso/local_allocator.go 71.62% <0.00%> (-6.76%) ⬇️
server/statistics/hot_cache_task.go 75.00% <0.00%> (-4.17%) ⬇️
server/member/member.go 64.70% <0.00%> (-2.14%) ⬇️
server/tso/tso.go 60.77% <0.00%> (-0.56%) ⬇️
client/base_client.go 84.37% <0.00%> (-0.53%) ⬇️
server/tso/allocator_manager.go 66.77% <0.00%> (-0.34%) ⬇️
server/cluster/cluster.go 83.01% <0.00%> (+0.47%) ⬆️
... and 11 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 386b044...286220b. Read the comment docs.

@HunDunDM HunDunDM added the component/scheduler Scheduler logic. label Jul 28, 2021
Copy link
Contributor

@nolouch nolouch left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@ti-chi-bot ti-chi-bot added the status/LGT1 Indicates that a PR has LGTM 1. label Jul 28, 2021
@nolouch nolouch requested review from bufferflies and removed request for rleungx July 29, 2021 03:34
@nolouch
Copy link
Contributor

nolouch commented Jul 29, 2021

PTAL @bufferflies

server/schedulers/hot_region.go Outdated Show resolved Hide resolved
server/schedulers/hot_region.go Show resolved Hide resolved
@rleungx
Copy link
Member

rleungx commented Jul 29, 2021

BTW, please update the PR description.

@ti-chi-bot ti-chi-bot added release-note-none Denotes a PR that doesn't merit a release note. and removed release-note Denotes a PR that will be considered when it comes time to generate release notes. labels Jul 29, 2021
@nolouch
Copy link
Contributor

nolouch commented Jul 29, 2021

PTAL @rleungx

@nolouch
Copy link
Contributor

nolouch commented Aug 2, 2021

@HunDunDM conflict.

@ti-chi-bot ti-chi-bot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Aug 2, 2021
# Conflicts:
#	server/schedulers/utils.go
@ti-chi-bot ti-chi-bot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Aug 2, 2021
Signed-off-by: HunDunDM <[email protected]>
Signed-off-by: HunDunDM <[email protected]>
Signed-off-by: HunDunDM <[email protected]>
@ti-chi-bot ti-chi-bot added status/LGT2 Indicates that a PR has LGTM 2. and removed status/LGT1 Indicates that a PR has LGTM 1. labels Aug 3, 2021
@rleungx
Copy link
Member

rleungx commented Aug 3, 2021

/hold

@ti-chi-bot ti-chi-bot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Aug 3, 2021
@HunDunDM
Copy link
Member Author

HunDunDM commented Aug 3, 2021

/hold cancel

@ti-chi-bot ti-chi-bot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Aug 3, 2021
@HunDunDM
Copy link
Member Author

HunDunDM commented Aug 3, 2021

/merge

@ti-chi-bot
Copy link
Member

@HunDunDM: It seems you want to merge this PR, I will help you trigger all the tests:

/run-all-tests

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

@ti-chi-bot
Copy link
Member

This pull request has been accepted and is ready to merge.

Commit hash: 286220b

@ti-chi-bot ti-chi-bot added the status/can-merge Indicates a PR has been approved by a committer. label Aug 3, 2021
@ti-chi-bot ti-chi-bot merged commit c06414f into tikv:master Aug 3, 2021
@HunDunDM HunDunDM deleted the hot-sche-pending branch August 3, 2021 10:24
@HunDunDM HunDunDM added needs-cherry-pick-release-5.0 The PR needs to cherry pick to release-5.0 branch. needs-cherry-pick-release-5.1 Type: Need cherry pick to release-5.1 labels Aug 29, 2021
ti-chi-bot pushed a commit to ti-chi-bot/pd that referenced this pull request Aug 29, 2021
@ti-chi-bot
Copy link
Member

In response to a cherrypick label: new pull request created: #4050.

@ti-chi-bot
Copy link
Member

In response to a cherrypick label: new pull request created: #4051.

HunDunDM added a commit to ti-chi-bot/pd that referenced this pull request Sep 1, 2021
ti-chi-bot added a commit that referenced this pull request Sep 1, 2021
* scheduler: simplify pending influence (#3658)

Signed-off-by: HunDunDM <[email protected]>

* schedulers: unify the use and GC of hot-region's pendings and regionPendings (#3921)

Signed-off-by: HunDunDM <[email protected]>

* scheduler: remove the useless code in hot-region-scheduler (#3833)

Signed-off-by: HunDunDM <[email protected]>

* scheduler: fix inaccurate statistics and config (#3949)

Signed-off-by: HunDunDM <[email protected]>

* tiny fix

Signed-off-by: HunDunDM <[email protected]>

* tiny fix

Signed-off-by: HunDunDM <[email protected]>

Co-authored-by: HunDunDM <[email protected]>
HunDunDM added a commit to ti-chi-bot/pd that referenced this pull request Sep 8, 2021
ti-chi-bot added a commit that referenced this pull request Sep 9, 2021
* scheduler: simplify pending influence (#3658)

Signed-off-by: HunDunDM <[email protected]>

* schedulers: unify the use and GC of hot-region's pendings and regionPendings (#3921)

Signed-off-by: HunDunDM <[email protected]>

* scheduler: remove the useless code in hot-region-scheduler (#3833)

Signed-off-by: HunDunDM <[email protected]>

* scheduler: fix inaccurate statistics and config (#3949)

Signed-off-by: HunDunDM <[email protected]>

Co-authored-by: HunDunDM <[email protected]>
HunDunDM added a commit to ti-chi-bot/pd that referenced this pull request Sep 14, 2021
ti-chi-bot added a commit that referenced this pull request Sep 14, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
component/scheduler Scheduler logic. needs-cherry-pick-release-5.0 The PR needs to cherry pick to release-5.0 branch. needs-cherry-pick-release-5.1 Type: Need cherry pick to release-5.1 release-note-none Denotes a PR that doesn't merit a release note. status/can-merge Indicates a PR has been approved by a committer. status/LGT2 Indicates that a PR has LGTM 2.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants