Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(hotspot): new algorithm of hotspot detection #479

Merged
merged 226 commits into from
Mar 5, 2020
Merged
Changes from 1 commit
Commits
Show all changes
226 commits
Select commit Hold shift + click to select a range
813164d
get_app_partition_stat
Nov 7, 2019
c66556b
hotspot
Nov 7, 2019
42aa1c4
modify get_app_partition_stat
Nov 7, 2019
6f08952
Merge branch 'command_helper' into hotspot
Nov 7, 2019
8aee24d
hotspot
Nov 8, 2019
a3a5758
hotspot
Nov 8, 2019
cc1c49f
hot spot
Nov 8, 2019
ddc52bc
hot spot
Nov 8, 2019
89e6f5f
hot spot
Nov 8, 2019
7750438
hot spot
Nov 8, 2019
98ee1fa
hotspot
Nov 8, 2019
30a1fd4
hot spot
Nov 8, 2019
db88ea3
hot spot
Nov 8, 2019
d012ff9
Merge remote-tracking branch 'upstream/master' into hotspot
Nov 19, 2019
7fa659e
merge master
Nov 19, 2019
f7e29ef
hotspot
Nov 19, 2019
7f34a93
format
Nov 19, 2019
b18dede
rdsn
Nov 19, 2019
2b01847
update rdsn
Nov 20, 2019
bd02151
Merge branch 'master' into hotspot
Nov 20, 2019
e2dfa84
hotspot
Nov 27, 2019
c0ca02d
1
Smityz Dec 9, 2019
766783f
1
Smityz Dec 9, 2019
09d6862
1
Smityz Dec 9, 2019
5821113
change name
Smityz Dec 9, 2019
0ae00a4
new
Smityz Dec 12, 2019
41fccbb
12.16
Smityz Dec 16, 2019
e7064be
format
Smityz Dec 16, 2019
2741811
add
Smityz Dec 16, 2019
94742e0
1
Smityz Dec 17, 2019
9f33305
1
Smityz Dec 17, 2019
90786e8
!
Smityz Dec 17, 2019
eb8d4bf
1
Smityz Dec 18, 2019
36e1bc9
Merge branch 'master' into hotspot
Smityz Jan 13, 2020
ffeb5b4
new
Smityz Jan 19, 2020
709585b
Merge branch 'master' into hotspot
Smityz Jan 19, 2020
36621fb
format
Smityz Jan 19, 2020
a007925
algo_1
Smityz Jan 31, 2020
338a7f4
1
Smityz Feb 5, 2020
0afe33e
1
Smityz Feb 5, 2020
7a59f97
1
Smityz Feb 5, 2020
f240e70
1
Smityz Feb 5, 2020
092e588
1
Smityz Feb 6, 2020
090f307
1
Smityz Feb 6, 2020
9ddd5a9
Merge branch 'temp' into hotspot
Smityz Feb 6, 2020
ecbf85f
1
Smityz Feb 6, 2020
85b45b8
1
Smityz Feb 6, 2020
a837ae5
1
Smityz Feb 6, 2020
b936822
1
Smityz Feb 6, 2020
4f68c39
1
Smityz Feb 6, 2020
0b750ec
fix travis bug
Smityz Feb 6, 2020
53adfb1
1
Smityz Feb 6, 2020
251f8c6
1
Smityz Feb 6, 2020
e976612
class->struct
Smityz Feb 7, 2020
56c9de7
change app_name_out
Smityz Feb 7, 2020
bdacfa9
tableHotspotPolicy -> table_hotspot_policy
Smityz Feb 7, 2020
e8cb72c
desc data_store
Smityz Feb 7, 2020
f19413f
//
Smityz Feb 7, 2020
81559bb
auto const
Smityz Feb 7, 2020
0f9294c
init perf_counter
Smityz Feb 7, 2020
b0cd27b
qps skew
Smityz Feb 7, 2020
72ec2d4
get_store_handler
Smityz Feb 7, 2020
0717e92
license
Smityz Feb 7, 2020
5c7db02
cpp -> h
Smityz Feb 7, 2020
33622f3
assert
Smityz Feb 7, 2020
09ce517
at -> []
Smityz Feb 7, 2020
5404323
at -> []
Smityz Feb 7, 2020
6f881fc
dassert
Smityz Feb 7, 2020
0341e60
sth
Smityz Feb 7, 2020
e2fa54d
static const
Smityz Feb 7, 2020
aea568f
add desc
Smityz Feb 7, 2020
0e149ae
AppStatCounters -> app_stat_counters
Smityz Feb 7, 2020
7a0df6f
1
Smityz Feb 7, 2020
b6cc144
format
Smityz Feb 7, 2020
b3a72af
format
Smityz Feb 7, 2020
96f69e1
newline
Smityz Feb 7, 2020
db81d78
newline
Smityz Feb 7, 2020
244fd99
data_store -> hotspot_partition_data
Smityz Feb 10, 2020
0646b82
reconsitution
Smityz Feb 10, 2020
2b32635
format
Smityz Feb 11, 2020
e85b233
1
Smityz Feb 11, 2020
f19a853
Merge branch 'master' of github.com:XiaoMi/pegasus
Smityz Feb 11, 2020
6bf39c4
Merge branch 'master' into hotspot
Smityz Feb 11, 2020
8392b09
1
Smityz Feb 11, 2020
b758a56
1
Smityz Feb 11, 2020
c9855ad
1
Smityz Feb 11, 2020
9079515
format
Smityz Feb 11, 2020
e874e24
1
Smityz Feb 11, 2020
2819158
1
Smityz Feb 11, 2020
9a46d99
1
Smityz Feb 11, 2020
ac7066a
1
Smityz Feb 11, 2020
bd9a798
1
Smityz Feb 11, 2020
76ed9b9
1
Smityz Feb 11, 2020
68980bd
1
Smityz Feb 11, 2020
f7d1b3d
Merge branch 'hotspot' of https://github.com/Smityz/pegasus into hotspot
Smityz Feb 11, 2020
da87922
1
Smityz Feb 11, 2020
48bff52
Merge branch 'hotspot' of https://github.com/Smityz/pegasus into hotspot
Smityz Feb 11, 2020
9706a8c
Merge branch 'master' into hotspot
acelyc111 Feb 13, 2020
eef408a
Merge branch 'hotspot' of https://github.com/Smityz/pegasus into hotspot
Smityz Feb 13, 2020
1095cf4
1
Smityz Feb 14, 2020
8f96b55
Merge branch 'hotspot' of git.n.xiaomi.com:tangyanzhao/pegasus into h…
Smityz Feb 14, 2020
6908dcc
1
Smityz Feb 14, 2020
bba55e6
Merge branch 'hotspot' of git.n.xiaomi.com:tangyanzhao/pegasus into h…
Smityz Feb 14, 2020
058f7cf
1
Smityz Feb 14, 2020
2c11a8b
Merge branch 'hotspot' of git.n.xiaomi.com:tangyanzhao/pegasus into h…
Smityz Feb 14, 2020
9c988cc
rebase
Smityz Feb 14, 2020
242a828
1
Smityz Feb 14, 2020
901fa5c
smart point
Smityz Feb 14, 2020
391d15f
smart point
Smityz Feb 14, 2020
913eba3
smart point
Smityz Feb 14, 2020
56c2af7
smart point
Smityz Feb 14, 2020
1f74869
smart point
Smityz Feb 14, 2020
da89c6d
smart point
Smityz Feb 14, 2020
9399e4b
smart point
Smityz Feb 14, 2020
208a3c8
smart point
Smityz Feb 14, 2020
9dac882
smart point
Smityz Feb 14, 2020
4b3c800
smart point
Smityz Feb 14, 2020
e77a2d6
1
Smityz Feb 14, 2020
82f7cb0
format
Smityz Feb 14, 2020
397fa99
Merge branch 'hotspot' of github.com:Smityz/pegasus into hotspot
Smityz Feb 14, 2020
c4baebc
int
Smityz Feb 15, 2020
1f1eabc
int
Smityz Feb 15, 2020
6157635
int
Smityz Feb 15, 2020
5e78295
auto & algo
Smityz Feb 15, 2020
c6ede4b
auto & algo
Smityz Feb 15, 2020
4559087
debug
Smityz Feb 15, 2020
c8a95e8
debug
Smityz Feb 15, 2020
7771582
debug
Smityz Feb 15, 2020
be05227
wrong
Smityz Feb 15, 2020
19af795
Merge branch 'hotspot' of git.n.xiaomi.com:tangyanzhao/pegasus into h…
Smityz Feb 15, 2020
6626673
debug
Smityz Feb 15, 2020
5a40abd
Merge branch 'hotspot' of git.n.xiaomi.com:tangyanzhao/pegasus into h…
Smityz Feb 15, 2020
3a581c6
wrong
Smityz Feb 15, 2020
419d2f9
debug
Smityz Feb 15, 2020
e214126
debug
Smityz Feb 15, 2020
8502074
debug
Smityz Feb 15, 2020
caef4e8
debug
Smityz Feb 15, 2020
6680488
debug finish
Smityz Feb 15, 2020
a31108b
12
Smityz Feb 15, 2020
795052b
1
Smityz Feb 15, 2020
9b9b66b
format
Smityz Feb 15, 2020
97c73a2
1
Smityz Feb 16, 2020
ffc5b9f
1
Smityz Feb 16, 2020
6bb308b
1
Smityz Feb 16, 2020
993a5ee
rebase
Smityz Feb 16, 2020
72a65ff
1
Smityz Feb 16, 2020
14b7b25
1
Smityz Feb 16, 2020
9610546
1
Smityz Feb 16, 2020
ff381a8
1
Smityz Feb 16, 2020
2dbd891
1
Smityz Feb 16, 2020
80b1710
1
Smityz Feb 17, 2020
765dc0e
1
Smityz Feb 17, 2020
08ad3f9
Update table_hotspot_policy.h
Smityz Feb 17, 2020
81c6d21
Update table_hotspot_policy.h
Smityz Feb 17, 2020
6bbd882
1
Smityz Feb 17, 2020
efdf402
a
Smityz Feb 17, 2020
6db4f01
Merge branch 'hotspot' of git.n.xiaomi.com:tangyanzhao/pegasus into h…
Smityz Feb 17, 2020
8fa33a5
1
Smityz Feb 17, 2020
78a683e
Merge branch 'hotspot' of github.com:Smityz/pegasus into hotspot
Smityz Feb 17, 2020
789c79b
1
Smityz Feb 17, 2020
49ed06e
add rdb_estimate_num_keys
Smityz Feb 17, 2020
0e826e3
add rdb_estimate_num_keys
Smityz Feb 17, 2020
01ac850
add rdb_estimate_num_keys
Smityz Feb 17, 2020
c754ff0
change paritition_desc of perf_counter
Smityz Feb 17, 2020
2959431
change paritition_desc of perf_counter
Smityz Feb 17, 2020
6f06ec1
hotspot_algo_qps_variance
Smityz Feb 18, 2020
5f24098
merge
Smityz Feb 18, 2020
1492438
new alg
Smityz Feb 18, 2020
e501b81
modify
Smityz Feb 18, 2020
ad7b4c2
new alg
Smityz Feb 18, 2020
28f7b6b
new alg
Smityz Feb 18, 2020
83023d6
new alg
Smityz Feb 18, 2020
937f8ef
1
Smityz Feb 18, 2020
738d1ab
new alg
Smityz Feb 18, 2020
38626c6
1
Smityz Feb 18, 2020
162c5dc
1
Smityz Feb 18, 2020
5632c8b
new alg
Smityz Feb 18, 2020
8b6526c
new alg
Smityz Feb 18, 2020
a85072b
new alg
Smityz Feb 18, 2020
4b92787
new alg
Smityz Feb 18, 2020
962ad7a
new alg
Smityz Feb 18, 2020
700700d
new alg
Smityz Feb 18, 2020
691a722
new alg
Smityz Feb 18, 2020
ccfd7ce
new alg
Smityz Feb 18, 2020
3a90d4b
format
Smityz Feb 21, 2020
e9bd7f7
1
Smityz Feb 21, 2020
caf71b9
1.00
Smityz Feb 21, 2020
751f0fa
Merge branch 'master' into hotspot
Smityz Feb 21, 2020
a11e763
1
Smityz Mar 2, 2020
5e3e01d
1
Smityz Mar 2, 2020
62491ec
1
Smityz Mar 2, 2020
3dd5528
1
Smityz Mar 2, 2020
2874fea
1
Smityz Mar 2, 2020
0eab0ab
Merge branch 'hotspot' of git.n.xiaomi.com:tangyanzhao/pegasus into h…
Smityz Mar 2, 2020
928b036
1
Smityz Mar 2, 2020
34dfc99
Merge branch 'hotspot' of git.n.xiaomi.com:tangyanzhao/pegasus into h…
Smityz Mar 2, 2020
c30caa5
1
Smityz Mar 2, 2020
c1a9abc
Merge branch 'hotspot' of git.n.xiaomi.com:tangyanzhao/pegasus into h…
Smityz Mar 2, 2020
2bfaf48
1
Smityz Mar 2, 2020
57a0dd6
1
Smityz Mar 2, 2020
0b9f078
1
Smityz Mar 2, 2020
63a851b
1
Smityz Mar 2, 2020
9d83db8
1
Smityz Mar 2, 2020
2347ddf
1
Smityz Mar 2, 2020
c311d7c
Merge branch 'hotspot' of git.n.xiaomi.com:tangyanzhao/pegasus into h…
Smityz Mar 2, 2020
5875ccc
1
Smityz Mar 2, 2020
ca391bd
1
Smityz Mar 2, 2020
519f7a9
1
Smityz Mar 2, 2020
cdfd18c
Merge branch 'hotspot' of github.com:Smityz/pegasus into hotspot
Smityz Mar 2, 2020
697c043
Merge branch 'master' into hotspot
acelyc111 Mar 2, 2020
2fff9f7
1
Smityz Mar 2, 2020
dd15287
Merge branch 'hotspot' of github.com:Smityz/pegasus into hotspot
Smityz Mar 2, 2020
64fb9b1
1
Smityz Mar 2, 2020
796922e
Update config.ini
Smityz Mar 3, 2020
5c60972
Update config.ini
Smityz Mar 3, 2020
3aaa19a
1
Smityz Mar 5, 2020
097523d
1
Smityz Mar 5, 2020
e39ce48
Merge branch 'master' into hotspot
acelyc111 Mar 5, 2020
d855f6a
1
Smityz Mar 5, 2020
1fd9bdf
Merge branch 'hotspot-dev' into hotspot
Smityz Mar 5, 2020
af44c92
Merge branch 'master' into hotspot
Smityz Mar 5, 2020
dcbbcfe
Update src/server/info_collector.cpp
acelyc111 Mar 5, 2020
7c6a468
Update src/server/info_collector.cpp
acelyc111 Mar 5, 2020
e51d478
Update src/server/info_collector.cpp
acelyc111 Mar 5, 2020
1d9e6c2
Update src/server/info_collector.cpp
Smityz Mar 5, 2020
eceb124
Update table_hotspot_policy.h
Smityz Mar 5, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
add rdb_estimate_num_keys
Smityz committed Feb 17, 2020

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
commit 49ed06e2a3043d76b2a65dad93de5984177826e4
1 change: 1 addition & 0 deletions src/server/info_collector.h
Original file line number Diff line number Diff line change
@@ -62,6 +62,7 @@ class info_collector
rdb_index_and_filter_blocks_mem_usage->set(
row_stats.total_rdb_index_and_filter_blocks_mem_usage);
rdb_memtable_mem_usage->set(row_stats.total_rdb_memtable_mem_usage);
rdb_estimate_num_keys->set(row_stats.total_rdb_estimate_num_keys);
read_qps->set(row_stats.get_total_read_qps());
write_qps->set(row_stats.get_total_write_qps());
}
2 changes: 2 additions & 0 deletions src/server/table_stats.h
Original file line number Diff line number Diff line change
@@ -44,6 +44,7 @@ struct table_stats
total_rdb_block_cache_total_count += row.rdb_block_cache_total_count;
total_rdb_index_and_filter_blocks_mem_usage += row.rdb_index_and_filter_blocks_mem_usage;
total_rdb_memtable_mem_usage += row.rdb_memtable_mem_usage;
total_rdb_estimate_num_keys += row.rdb_estimate_num_keys;
}

void merge(const table_stats &row_stats)
@@ -100,6 +101,7 @@ struct table_stats
double total_rdb_block_cache_total_count = 0;
double total_rdb_index_and_filter_blocks_mem_usage = 0;
double total_rdb_memtable_mem_usage = 0;
double total_rdb_estimate_num_keys = 0;
double max_total_qps = 0;
double min_total_qps = INT_MAX;
double max_total_cu = 0;