Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fix: fix bug that benchmark can't exit #834

Merged
merged 2 commits into from
Nov 5, 2021
Merged

fix: fix bug that benchmark can't exit #834

merged 2 commits into from
Nov 5, 2021

Conversation

levy5307
Copy link
Contributor

@levy5307 levy5307 commented Nov 4, 2021

What problem does this PR solve?

when I use pegasus benchmark, I found that it can't stop. Just like this

➜  pegasus git:(master) ✗ ./run.sh bench
W2021-11-04 19:01:22.598 (1636023682598965061 2940) : overwrite default thread pool for task RPC_CM_QUERY_PARTITION_CONFIG_BY_INDEX from THREAD_POOL_META_SERVER to THREAD_POOL_DEFAULT
W2021-11-04 19:01:22.598 (1636023682598998924 2940) : overwrite default thread pool for task RPC_CM_QUERY_PARTITION_CONFIG_BY_INDEX_ACK from THREAD_POOL_META_SERVER to THREAD_POOL_DEFAULT
Init pegasus succeed
Hashkeys:       16 bytes each
Sortkeys:       16 bytes each
Values:         100 bytes each
Entries:        10000
FileSize:       1 MB (estimated)
WARNING: Assertions are enabled; benchmarks unnecessarily slow
------------------------------------------------
Statistics for write:  
258.285 micros/op; 3871 ops/sec; 0.387169 MB/s 
Count: 10000 Average: 258.2799  StdDev: 237.58
Min: 173  Median: 229.4177  Max: 9768
Percentiles: P50: 229.42 P75: 283.04 P99: 546.86 P99.9: 4800.00 P99.99: 6600.00
------------------------------------------------------
(     170,     250 ]     6732  67.320%  67.320% #############
(     250,     380 ]     3022  30.220%  97.540% ######
(     380,     580 ]      175   1.750%  99.290% 
(     580,     870 ]       27   0.270%  99.560% 
(     870,    1300 ]        7   0.070%  99.630% 
(    1300,    1900 ]        4   0.040%  99.670% 
(    1900,    2900 ]        9   0.090%  99.760% 
(    2900,    4400 ]       12   0.120%  99.880% 
(    4400,    6600 ]       11   0.110%  99.990% 
(    6600,    9900 ]        1   0.010% 100.000% 

Statistics for read:  
63.4951 micros/op; 15749 ops/sec; 1.57492 MB/s (10000 of 10000 found)
Count: 10000 Average: 63.4918  StdDev: 70.54
Min: 41  Median: 63.5275  Max: 6017
Percentiles: P50: 63.53 P75: 70.76 P99: 108.30 P99.9: 168.03 P99.99: 2900.00
------------------------------------------------------
(      34,      51 ]      669   6.690%   6.690% #
(      51,      76 ]     8643  86.430%  93.120% #################
(      76,     110 ]      619   6.190%  99.310% #
(     110,     170 ]       61   0.610%  99.920% 
(     170,     250 ]        2   0.020%  99.940% 
(     250,     380 ]        1   0.010%  99.950% 
(     380,     580 ]        1   0.010%  99.960% 
(     580,     870 ]        1   0.010%  99.970% 
(    1900,    2900 ]        2   0.020%  99.990% 
(    4400,    6600 ]        1   0.010% 100.000% 

Statistics for delete:  
282.163 micros/op; 3544 ops/sec; 
Count: 10000 Average: 282.1580  StdDev: 379.25
Min: 171  Median: 250.9728  Max: 17680
Percentiles: P50: 250.97 P75: 320.46 P99: 576.95 P99.9: 6600.00 P99.99: 14000.00
------------------------------------------------------
(     170,     250 ]     4965  49.650%  49.650% ##########
(     250,     380 ]     4677  46.770%  96.420% #########
(     380,     580 ]      262   2.620%  99.040% #
(     580,     870 ]       35   0.350%  99.390% 
(     870,    1300 ]       14   0.140%  99.530% 
(    1300,    1900 ]        8   0.080%  99.610% 
(    1900,    2900 ]        8   0.080%  99.690% 
(    2900,    4400 ]       12   0.120%  99.810% 
(    4400,    6600 ]        9   0.090%  99.900% 
(    6600,    9900 ]        7   0.070%  99.970% 
(    9900,   14000 ]        2   0.020%  99.990% 
(   14000,   22000 ]        1   0.010% 100.000% 

^C

It blocked forever, until I press the "Ctrl + C".
That is because we didn't call dsn_exit() at the end. So I fixed it in this pull request.
After fixed, it runs like this:

➜  pegasus git:(master) ✗ ./run.sh bench
W2021-11-04 19:13:23.980 (1636024403980086950 14784) : overwrite default thread pool for task RPC_CM_QUERY_PARTITION_CONFIG_BY_INDEX from THREAD_POOL_META_SERVER to THREAD_POOL_DEFAULT
W2021-11-04 19:13:23.980 (1636024403980128620 14784) : overwrite default thread pool for task RPC_CM_QUERY_PARTITION_CONFIG_BY_INDEX_ACK from THREAD_POOL_META_SERVER to THREAD_POOL_DEFAULT
Init pegasus succeed
Hashkeys:       16 bytes each
Sortkeys:       16 bytes each
Values:         100 bytes each
Entries:        10000
FileSize:       1 MB (estimated)
WARNING: Assertions are enabled; benchmarks unnecessarily slow
------------------------------------------------
Statistics for write:  
235.781 micros/op; 4241 ops/sec; 0.424123 MB/s 
Count: 10000 Average: 235.7727  StdDev: 86.66
Min: 167  Median: 220.0389  Max: 4003
Percentiles: P50: 220.04 P75: 245.12 P99: 540.41 P99.9: 923.75 P99.99: 3900.00
------------------------------------------------------
(     110,     170 ]       13   0.130%   0.130% 
(     170,     250 ]     7973  79.730%  79.860% ################
(     250,     380 ]     1679  16.790%  96.650% ###
(     380,     580 ]      293   2.930%  99.580% #
(     580,     870 ]       31   0.310%  99.890% 
(     870,    1300 ]        8   0.080%  99.970% 
(    2900,    4400 ]        3   0.030% 100.000% 

Statistics for read:  
106.863 micros/op; 9357 ops/sec; 0.935778 MB/s (10000 of 10000 found)
Count: 10000 Average: 106.8576  StdDev: 239.36
Min: 46  Median: 84.5540  Max: 8651
Percentiles: P50: 84.55 P75: 103.18 P99: 446.67 P99.9: 4400.00 P99.99: 8651.00
------------------------------------------------------
(      34,      51 ]       36   0.360%   0.360% 
(      51,      76 ]     3816  38.160%  38.520% ########
(      76,     110 ]     4563  45.630%  84.150% #########
(     110,     170 ]     1108  11.080%  95.230% ##
(     170,     250 ]      251   2.510%  97.740% #
(     250,     380 ]      111   1.110%  98.850% 
(     380,     580 ]       45   0.450%  99.300% 
(     580,     870 ]       20   0.200%  99.500% 
(     870,    1300 ]       13   0.130%  99.630% 
(    1300,    1900 ]        9   0.090%  99.720% 
(    1900,    2900 ]       11   0.110%  99.830% 
(    2900,    4400 ]        7   0.070%  99.900% 
(    4400,    6600 ]        6   0.060%  99.960% 
(    6600,    9900 ]        4   0.040% 100.000% 

Statistics for delete:  
476.705 micros/op; 2097 ops/sec; 
Count: 10000 Average: 476.6974  StdDev: 774.65
Min: 175  Median: 360.7282  Max: 33948
Percentiles: P50: 360.73 P75: 479.83 P99: 3841.86 P99.9: 9664.29 P99.99: 22000.00
------------------------------------------------------
(     170,     250 ]      754   7.540%   7.540% ##
(     250,     380 ]     4985  49.850%  57.390% ##########
(     380,     580 ]     3528  35.280%  92.670% #######
(     580,     870 ]      352   3.520%  96.190% #
(     870,    1300 ]      133   1.330%  97.520% 
(    1300,    1900 ]       65   0.650%  98.170% 
(    1900,    2900 ]       56   0.560%  98.730% 
(    2900,    4400 ]       43   0.430%  99.160% 
(    4400,    6600 ]       48   0.480%  99.640% 
(    6600,    9900 ]       28   0.280%  99.920% 
(    9900,   14000 ]        5   0.050%  99.970% 
(   14000,   22000 ]        2   0.020%  99.990% 
(   33000,   50000 ]        1   0.010% 100.000% 

dsn exit with code 0

@levy5307 levy5307 changed the title fix: fix bug that benchmark can't stop fix: fix bug that benchmark can't exit Nov 4, 2021
@levy5307 levy5307 merged commit f98f16e into apache:master Nov 5, 2021
@levy5307 levy5307 deleted the fix-benchmark branch November 5, 2021 06:55
acelyc111 pushed a commit that referenced this pull request Jun 23, 2022
@hycdong
feat(disk): add broken disk check while initialize (#834)
f1827ec
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Smityz Smityz Smityz approved these changes

@hycdong hycdong hycdong approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@levy5307 @hycdong @Smityz