Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Supports some small regions(table) can not be merge. #2171

Closed
nolouch opened this issue Mar 2, 2020 · 7 comments
Closed

Supports some small regions(table) can not be merge. #2171

nolouch opened this issue Mar 2, 2020 · 7 comments
Labels
difficulty/medium Medium task. help wanted Denotes an issue that needs help from a contributor. Must meet "help wanted" guidelines. type/enhancement The issue or PR belongs to an enhancement.

Comments

@nolouch
Copy link
Contributor

nolouch commented Mar 2, 2020

Description

Sometimes, some regions are hot but not large, do not merge them can improve performance. we want to specify some regions(table) can not be merge.

Score

  • 900

Mentor

Recommended Skills

  • Go

Learning Materials

@nolouch nolouch added help wanted Denotes an issue that needs help from a contributor. Must meet "help wanted" guidelines. difficulty/medium Medium task. labels Mar 2, 2020
@nolouch nolouch changed the title UCP: Supports some little regions(table) can not be merge. UCP: Supports some small regions(table) can not be merge. Mar 2, 2020
@nolouch nolouch added the type/enhancement The issue or PR belongs to an enhancement. label Mar 2, 2020
@sourcelliu
Copy link
Contributor

/pick-up-challenge

@sre-bot
Copy link
Contributor

sre-bot commented Mar 14, 2020

@mantuliu pick up issue success

@sourcelliu
Copy link
Contributor

sourcelliu commented Mar 17, 2020

@nolouch My idea is to add a configuration parameter: MergeHotRegionInterval, the default value is 24 hours.
Plan A : At each heartbeat of the region, a judgment logic is added. If the current region is hot, the last hot time of this reigion is stored in the memory. Maintain a map of hot regions or other data structures in memory. The merge scheduler adds a judgment condition to the region that needs to be merged, and determines whether the time when the region became hot last time is within the time range of MergeHotRegionInterval. If it is within this time range, this region will not be merged.

Plan B: Or add a field to the RegionInfo of core / region.go, indicating the time when the region was last marked as hot, and store the value of this new field to disk with the RegionInfo information.

What do you think of this ideas? Plan A does not store related hot region information to disk. Some hot regions may be merged by mistake during the process of pd master-slave switch. Plan B can store the last hot time to disk.

@sourcelliu
Copy link
Contributor

/give-up-challenge

@sre-bot
Copy link
Contributor

sre-bot commented Mar 26, 2020

@mantuliu give up issue success

@nolouch nolouch changed the title UCP: Supports some small regions(table) can not be merge. Supports some small regions(table) can not be merge. Sep 30, 2020
@nolouch
Copy link
Contributor Author

nolouch commented Sep 30, 2020

Hi @mantuliu , are you still interested in solving this problem? We have encountered a situation where there is a small table with hot spots. After the split, we must prevent it from being merged.

@nolouch
Copy link
Contributor Author

nolouch commented Jul 19, 2021

ongoing #3839 closed this one.

@nolouch nolouch closed this as completed Jul 19, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
difficulty/medium Medium task. help wanted Denotes an issue that needs help from a contributor. Must meet "help wanted" guidelines. type/enhancement The issue or PR belongs to an enhancement.
Projects
None yet
Development

No branches or pull requests

3 participants