Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

scripts: add pegasus_replica_thread.sh #115

Merged
merged 2 commits into from
Jul 11, 2018
Merged

scripts: add pegasus_replica_thread.sh #115

merged 2 commits into from
Jul 11, 2018

Conversation

qinzuoyan
Copy link
Contributor

@qinzuoyan qinzuoyan commented Jul 11, 2018

这个脚本主要是统计一个table的replica在各个节点上THREAD_POOL_REPLICATION的线程分布情况。

脚本执行的结果是获得类似这样的一个表格:
(注:每行一个节点,每列是分派到这个thread上的replica个数)

$ ./scripts/pegasus_replica_thread.sh 10.118.48.29:41601,10.118.48.28:41601 wifi_position 24
UID: 10000
PID: 42104
cluster: 10.118.48.29:41601,10.118.48.28:41601
app_name: wifi_position
thread_num: 24
Start time: Wed Jul 11 17:41:09 CST 2018

node  \  thread_id      0  1  2  3  4  5  6  7  8  9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 #replica  max #max
10.142.10.53:41801      3  3  2  7  4  2  3  7  2  4  2  3  3  6  4  2  1  3  2  4  0  4  1  5       24    7    2
10.142.10.54:41801      3  3  2  4  1  3  1  6  4  7  3  2  4  3  2  5  4  7  2  2  1  2  3  3       24    7    2
10.142.5.23:41801       4  3  4  1  5  2  3  2  3  2  2  5  6  3  5  3  5  3  3  3  4  0  3  3       24    6    1
10.142.5.24:41801       3  1  6  0  2  4  5  1  5  3  3  3  3  4  4  1  4  5  3  5  5  1  3  3       24    6    1
10.142.5.25:41801       2  4  5  4  4  3  2  3  3  3  1  4  3  2  3  5  4  4  4  2  2  5  1  4       24    5    3
10.142.5.26:41801       4  6  1  7  5  3  1  3  1  3  3  5  1  3  1  5  1  2  1  5  3  4  4  4       24    7    1
10.142.5.27:41801       5  4  3  1  6  3  4  2  3  2  5  2  3  3  2  1  3  1  5  2  5  5  5  1       24    6    1
10.142.5.28:41801       0  4  1  5  0  5  2  2  1  5  6  3  1  4  3  6  2  4  5  4  3  4  2  5       24    6    2
10.142.5.29:41801       3  2  2  4  3  3  6  3  5  2  4  2  5  2  5  4  6  2  4  1  2  4  2  1       24    6    2
10.142.5.30:41801       3  0  4  0  3  5  6  4  6  2  4  4  4  3  4  1  3  2  4  2  5  1  6  1       24    6    3
                                                                                                           7     
Notes:
  max   : maximum replica count dispatching on single thread
  #max  : thread count dispatching maximum replicas

Finish time: Wed Jul 11 17:41:09 CST 2018
Statistics done, elasped time is 0 seconds.

可以看到,在10.142.10.53:41801上,thread_id=3需要处理7个replica的写请求,但是thread_id=20处理的replica个数为0。

我们希望这样的分派尽量均衡,避免有的thread上需要处理的replica太多。因为单个thread上的task都是串行执行的,如果某个thread上的replica个数太多,就会成为瓶颈。

在load balance的时候需要考虑以上因素,参见 XiaoMi/rdsn#124

@qinzuoyan qinzuoyan merged commit 2bab725 into master Jul 11, 2018
@qinzuoyan qinzuoyan deleted the qinzuoyan branch July 11, 2018 11:19
neverchanje pushed a commit to neverchanje/pegasus that referenced this pull request Jul 13, 2019
Former-commit-id: 3cc66ef6838ed68624d6b2f0ec079fafd2575036 [formerly 2bab725]
Former-commit-id: 28363be536802dce0d12079d2aa052b7b16ee3a2
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants