hot region schedule supports store weight

[sourcelliu]

What problem does this PR solve?

UCP #2168

What is changed and how it works?

Add a method for calculate the storeWeight

func calcStoreWeight(regionWeight, leaderWeight float64) float64 {
	storeWeight := (regionWeight + leaderWeight) / 2
	return math.Max(storeWeight, minWeight)
}

byteRate and KeyRate are recalculated using:

ByteRate: byteRate / storeWeight,
KeyRate:  keyRate / storeWeight,

Check List

Tests

• Unit test

• Manual test
  Command: sysbench oltp_read_only (table size=10000000, threads=300)
  before
  
  

after set store 1 hot-weight=1 store 2 hot-weight=1 store 7 hot-weight =2

Side effects

• Increased code complexity

[sre-bot]

Thanks for your contribution. If your PR get merged, you will be rewarded 1294 points.

[sourcelliu]

/run-all-tests

[nolouch]

PTAL @rleungx @disksing

[nolouch]

Hi @mantuliu, thanks for your contribution.
Could you do more tests or test it in a real cluster?

[nolouch]

This weight may need to be separated from the cluster weight, because this is related to load, and the previous weight is more related to size. you can wait for #2239.

[sourcelliu]

OK.I will do more test and wait for #2239.

[nolouch]

maybe do it in stepRank? compareSrcStore and compareDstStore also need it.

[sourcelliu]

I didn't implement this in the stepRank method, because stepRank might become something like the following:

func stepRank(rk0 float64, step float64) func(float64) int64 {
   return func(rate storeWeight float64) int64 {
      return int64((rate/storeWeight - rk0) / step)
   }
}

This makes all the methods related to stepRank need to be changed,such as stLdByteRate,stLdKeyRate,sliceLoadCmp,sliceLPCmp. I personally think that such changes will increase the complexity of these functions. But I still tried to optimize this logic and added a getRateWithStoreWeight function.

what do you think?^_^

[nolouch]

Thanks for your contribution, I will take the time to take a look and test.

[codecov-io]

Codecov Report

Merging #2240 into master will increase coverage by 0.14%.
The diff coverage is 80.00%.

@@            Coverage Diff             @@
##           master    #2240      +/-   ##
==========================================
+ Coverage   76.25%   76.39%   +0.14%     
==========================================
  Files         199      199              
  Lines       21591    21722     +131     
==========================================
+ Hits        16464    16595     +131     
+ Misses       3891     3888       -3     
- Partials     1236     1239       +3     

Impacted Files	Coverage Δ
server/api/store.go	65.01% <39.13%> (-2.30%)	⬇️
tools/pd-ctl/pdctl/command/component_command.go	63.54% <58.33%> (-0.51%)	⬇️
server/cluster/cluster.go	81.12% <66.66%> (+0.25%)	⬆️
tools/pd-ctl/pdctl/command/store_command.go	65.75% <66.66%> (+0.05%)	⬆️
server/core/storage.go	73.78% <77.77%> (+0.04%)	⬆️
server/schedulers/hot_region.go	82.55% <94.28%> (+0.58%)	⬆️
pkg/mock/mockcluster/mockcluster.go	91.25% <100.00%> (+0.34%)	⬆️
server/api/router.go	91.28% <100.00%> (+0.09%)	⬆️
server/config_manager/config_manager.go	81.97% <100.00%> (+0.37%)	⬆️
server/core/store.go	78.91% <100.00%> (+0.29%)	⬆️
... and 16 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c57c39d...e93f9b2. Read the comment docs.

[sourcelliu]

Thanks for your contribution, I will take the time to take a look and test.

Thank you,I added a new parameter(hot-weight) to measure the ability of the store to handle hot regions. I will test it in an actual cluster.

[nolouch]

Hi, @mantuliu thanks for your contribution.
We invite you to join our sig-scheduling, you can communicate with us more conveniently in slack channel.

[sourcelliu]

@nolouch Thank you. I have joined.

[rleungx]

cool!

[nolouch]

Hi @mantuliu .
Recently, we have adjusted some hotspot scheduling strategies, which may be a little different from your previous version. Would you resolve the conflict firstly？thanks :)

[sourcelliu]

Hi @mantuliu .
Recently, we have adjusted some hotspot scheduling strategies, which may be a little different from your previous version. Would you resolve the conflict firstly？thanks :)

OK,I will resolve the conflict and make a test.