Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

perf(core) do not register event handlers on control planes #8452

Merged
merged 1 commit into from
Mar 22, 2022
Merged

perf(core) do not register event handlers on control planes #8452

merged 1 commit into from
Mar 22, 2022

Conversation

bungle
Copy link
Member

@bungle bungle commented Feb 23, 2022

Summary

Control planes only enable Admin API and Admin API does not use cache,
nor does it need to build router or plugins iterator based on events.

@bungle bungle force-pushed the perf/no-event-handlers-on-cp branch from 5d17f7c to f67c6c5 Compare February 23, 2022 21:02
@dndx
Copy link
Member

dndx commented Feb 24, 2022

Just one question. If we skip the worker events registration in CP, will it cause the necessary cluster events to not be generated between CPs running in a cluster?

e.g. if there are 2 CPs running inside the same cluster, will this cause the other CP (the one not receiving the Admin API call) to miss the event and not push out new configs to it's DP?

@bungle bungle force-pushed the perf/no-event-handlers-on-cp branch from f67c6c5 to 1c061e8 Compare February 24, 2022 10:20
@bungle
Copy link
Member Author

bungle commented Feb 24, 2022 •
edited
Loading

Just one question. If we skip the worker events registration in CP, will it cause the necessary cluster events to not be generated between CPs running in a cluster?

e.g. if there are 2 CPs running inside the same cluster, will this cause the other CP (the one not receiving the Admin API call) to miss the event and not push out new configs to it's DP?

It does not. That's the test I adjusted. You see we have test for this.

@bungle
perf(core) do not register event handlers on control planes
09204a8 
### Summary

Control planes only enable Admin API and Admin API does not use cache,
nor does it need to build router or plugins iterator based on events.
@bungle bungle force-pushed the perf/no-event-handlers-on-cp branch from 1c061e8 to 09204a8 Compare March 4, 2022 13:02
@github-actions
Copy link
Contributor

github-actions bot commented Mar 4, 2022

🚀 Performance test result

Test Suite: 01-rps 02-flamegraph (baseline,single_route,simple)

Click to expand 
Please see Github Actions artifacts for flamegraphs.

  ### Result for upstream directly (run 1):
Running 30s test @ http://10.103.96.3:8088/test
  5 threads and 100 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency   566.15us    1.25ms  12.12ms   87.12%
    Req/Sec   173.06k    11.42k  240.52k    69.71%
  25862472 requests in 30.10s, 3.37GB read
Requests/sec: 859231.96
Transfer/sec:    114.72MB
### Result for upstream directly (run 2):
Running 30s test @ http://10.103.96.3:8088/test
  5 threads and 100 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency   573.59us    1.27ms  12.13ms   87.34%
    Req/Sec   157.51k     9.15k  217.08k    76.55%
  23524445 requests in 30.10s, 3.07GB read
Requests/sec: 781544.79
Transfer/sec:    104.35MB
### Result for upstream directly (run 3):
Running 30s test @ http://10.103.96.3:8088/test
  5 threads and 100 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency   568.68us    1.26ms  16.07ms   87.38%
    Req/Sec   158.48k     9.06k  206.70k    77.55%
  23669096 requests in 30.10s, 3.09GB read
Requests/sec: 786353.93
Transfer/sec:    104.99MB
### Combined result for upstream directly:
RPS     Avg: 809043.56
Latency Avg: nanms    Max: nanms
  
### Test Suite: git:a49ebf4444cbcf80e5f18edb3b585d5765cfef38 #simple #no_plugins #single_route
### Result for Kong git:a49ebf4444cbcf80e5f18edb3b585d5765cfef38 (run 1):
Running 30s test @ http://10.103.96.1:8000/s1-r1
  5 threads and 100 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     1.46ms    4.68ms 209.95ms   95.83%
    Req/Sec    26.73k     5.92k   46.32k    63.15%
  3986523 requests in 30.06s, 855.23MB read
Requests/sec: 132607.71
Transfer/sec:     28.45MB
### Result for Kong git:a49ebf4444cbcf80e5f18edb3b585d5765cfef38 (run 2):
Running 30s test @ http://10.103.96.1:8000/s1-r1
  5 threads and 100 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     1.38ms    2.58ms  59.54ms   91.82%
    Req/Sec    26.95k     6.25k   44.84k    59.39%
  4027357 requests in 30.10s, 864.00MB read
Requests/sec: 133801.70
Transfer/sec:     28.70MB
### Result for Kong git:a49ebf4444cbcf80e5f18edb3b585d5765cfef38 (run 3):
Running 30s test @ http://10.103.96.1:8000/s1-r1
  5 threads and 100 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     1.38ms    2.54ms  49.62ms   91.76%
    Req/Sec    26.84k     6.15k   46.61k    59.40%
  4006741 requests in 30.04s, 859.57MB read
Requests/sec: 133377.95
Transfer/sec:     28.61MB
### Combined result for Kong git:a49ebf4444cbcf80e5f18edb3b585d5765cfef38:
RPS     Avg: 133262.45
Latency Avg: 1.41ms    Max: 209.95ms
  
### Test Suite: git:a49ebf4444cbcf80e5f18edb3b585d5765cfef38 #simple #no_plugins 10 services each has 10 routes
### Result for Kong git:a49ebf4444cbcf80e5f18edb3b585d5765cfef38 (run 1):
Running 30s test @ http://10.103.96.1:8000/
  5 threads and 100 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     1.36ms    2.47ms  49.28ms   92.14%
    Req/Sec    25.98k     5.79k   40.82k    58.98%
  3888274 requests in 30.10s, 834.16MB read
Requests/sec: 129180.61
Transfer/sec:     27.71MB
### Result for Kong git:a49ebf4444cbcf80e5f18edb3b585d5765cfef38 (run 2):
Running 30s test @ http://10.103.96.1:8000/
  5 threads and 100 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     1.36ms    2.51ms  55.89ms   92.12%
    Req/Sec    26.16k     5.84k   43.99k    59.02%
  3911863 requests in 30.10s, 839.22MB read
Requests/sec: 129964.58
Transfer/sec:     27.88MB
### Result for Kong git:a49ebf4444cbcf80e5f18edb3b585d5765cfef38 (run 3):
Running 30s test @ http://10.103.96.1:8000/
  5 threads and 100 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     1.39ms    2.69ms  57.81ms   92.58%
    Req/Sec    25.90k     5.79k   44.56k    59.91%
  3876740 requests in 30.10s, 831.68MB read
Requests/sec: 128796.86
Transfer/sec:     27.63MB
### Combined result for Kong git:a49ebf4444cbcf80e5f18edb3b585d5765cfef38:
RPS     Avg: 129314.02
Latency Avg: 1.37ms    Max: 57.81ms
  
### Test Suite: git:a49ebf4444cbcf80e5f18edb3b585d5765cfef38 #simple #key-auth 10 services each has 10 routes with key-auth, 100 consumers
### Result for Kong git:a49ebf4444cbcf80e5f18edb3b585d5765cfef38 (run 1):
Running 30s test @ http://10.103.96.1:8000/
  5 threads and 100 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     1.34ms    2.34ms  78.80ms   93.89%
    Req/Sec    22.74k     4.48k   35.99k    64.58%
  3398290 requests in 30.07s, 729.04MB read
Requests/sec: 113025.88
Transfer/sec:     24.25MB
### Result for Kong git:a49ebf4444cbcf80e5f18edb3b585d5765cfef38 (run 2):
Running 30s test @ http://10.103.96.1:8000/
  5 threads and 100 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     1.34ms    2.19ms  53.24ms   93.12%
    Req/Sec    22.57k     4.59k   36.62k    63.38%
  3372629 requests in 30.06s, 723.53MB read
Requests/sec: 112198.84
Transfer/sec:     24.07MB
### Result for Kong git:a49ebf4444cbcf80e5f18edb3b585d5765cfef38 (run 3):
Running 30s test @ http://10.103.96.1:8000/
  5 threads and 100 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     1.33ms    2.19ms  54.75ms   93.38%
    Req/Sec    22.64k     4.52k   37.39k    63.87%
  3378894 requests in 30.04s, 724.88MB read
Requests/sec: 112484.22
Transfer/sec:     24.13MB
### Combined result for Kong git:a49ebf4444cbcf80e5f18edb3b585d5765cfef38:
RPS     Avg: 112569.65
Latency Avg: 1.34ms    Max: 78.80ms
  
### Test Suite: git:master #simple #no_plugins #single_route
### Result for Kong git:master (run 1):
Running 30s test @ http://10.103.96.1:8000/s1-r1
  5 threads and 100 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     1.38ms    2.51ms  50.04ms   91.84%
    Req/Sec    26.72k     5.95k   43.86k    59.83%
  3990357 requests in 30.10s, 856.06MB read
Requests/sec: 132569.93
Transfer/sec:     28.44MB
### Result for Kong git:master (run 2):
Running 30s test @ http://10.103.96.1:8000/s1-r1
  5 threads and 100 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     1.42ms    2.74ms  53.94ms   92.02%
    Req/Sec    26.91k     6.29k   42.56k    59.00%
  4016246 requests in 30.03s, 861.61MB read
Requests/sec: 133754.27
Transfer/sec:     28.69MB
### Result for Kong git:master (run 3):
Running 30s test @ http://10.103.96.1:8000/s1-r1
  5 threads and 100 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     1.43ms    2.85ms  68.22ms   92.23%
    Req/Sec    27.05k     6.26k   45.51k    60.29%
  4039583 requests in 30.10s, 866.62MB read
Requests/sec: 134206.49
Transfer/sec:     28.79MB
### Combined result for Kong git:master:
RPS     Avg: 133510.23
Latency Avg: 1.41ms    Max: 68.22ms
  
### Test Suite: git:master #simple #no_plugins 10 services each has 10 routes
### Result for Kong git:master (run 1):
Running 30s test @ http://10.103.96.1:8000/
  5 threads and 100 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     1.35ms    2.45ms  55.53ms   92.22%
    Req/Sec    25.81k     5.60k   39.62k    58.95%
  3859988 requests in 30.10s, 828.09MB read
Requests/sec: 128241.19
Transfer/sec:     27.51MB
### Result for Kong git:master (run 2):
Running 30s test @ http://10.103.96.1:8000/
  5 threads and 100 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     1.35ms    2.50ms  59.59ms   92.31%
    Req/Sec    25.78k     5.61k   38.30k    58.85%
  3852535 requests in 30.09s, 826.49MB read
Requests/sec: 128048.78
Transfer/sec:     27.47MB
### Result for Kong git:master (run 3):
Running 30s test @ http://10.103.96.1:8000/
  5 threads and 100 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     1.37ms    2.59ms  72.77ms   92.45%
    Req/Sec    25.77k     5.62k   40.86k    59.28%
  3853517 requests in 30.10s, 826.70MB read
Requests/sec: 128025.96
Transfer/sec:     27.47MB
### Combined result for Kong git:master:
RPS     Avg: 128105.31
Latency Avg: 1.36ms    Max: 72.77ms
  
### Test Suite: git:master #simple #key-auth 10 services each has 10 routes with key-auth, 100 consumers
### Result for Kong git:master (run 1):
Running 30s test @ http://10.103.96.1:8000/
  5 threads and 100 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     1.37ms    2.37ms 132.11ms   93.70%
    Req/Sec    22.22k     4.23k   33.36k    65.85%
  3321010 requests in 30.05s, 712.46MB read
Requests/sec: 110498.78
Transfer/sec:     23.71MB
### Result for Kong git:master (run 2):
Running 30s test @ http://10.103.96.1:8000/
  5 threads and 100 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     1.39ms    2.42ms  66.86ms   93.79%
    Req/Sec    21.99k     4.54k   34.66k    65.14%
  3288906 requests in 30.10s, 705.57MB read
Requests/sec: 109266.08
Transfer/sec:     23.44MB
### Result for Kong git:master (run 3):
Running 30s test @ http://10.103.96.1:8000/
  5 threads and 100 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     1.32ms    2.12ms  60.04ms   93.15%
    Req/Sec    22.23k     4.38k   36.46k    64.38%
  3322255 requests in 30.06s, 712.73MB read
Requests/sec: 110511.53
Transfer/sec:     23.71MB
### Combined result for Kong git:master:
RPS     Avg: 110092.13
Latency Avg: 1.36ms    Max: 132.11ms
  

Kong error logs are also available in Github Actions artifacts.

Download Artifacts

@dndx dndx merged commit cdf7990 into master Mar 22, 2022
@dndx dndx deleted the perf/no-event-handlers-on-cp branch March 22, 2022 15:16
dndx added a commit that referenced this pull request Mar 22, 2022
@dndx
docs(CHANGELOG) add change log entry for #8452
d4a6f39 
Do not reigister unnecessary event handlers on Hybrid mode Control Plane nodes
dndx added a commit that referenced this pull request Mar 25, 2022
@dndx @kikito
docs(CHANGELOG) add change log entry for #8452
6983215 
Do not register unnecessary event handlers on Hybrid mode Control Plane nodes

Co-authored-by: Enrique García Cota <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@dndx dndx dndx approved these changes

Assignees
No one assigned
Labels
core/proxy size/S
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@bungle @dndx