Single-threaded benchmarks beat multi-threaded version

[emschwartz]

I tried adjusting the number of Tokio worker threads (using the TOKIO_WORKER_THREADS env var) while running the Hyper benchmarks and was surprised to find that the performance seems nearly the same if not better with only 1 worker thread.

Does anyone understand why that might be the case?

For context, both of these are running on an 8-core, dedicated CPU Linode machine with 16 GB of RAM.

Multi thread

     Running benches/end_to_end.rs (target/release/deps/end_to_end-dca2380bf04e2a3b)

running 19 tests
test http1_consecutive_x1_both_100kb                        ... bench:      66,477.23 ns/iter (+/- 14,140.13) = 3080 MB/s
test http1_consecutive_x1_both_10mb                         ... bench:   5,927,614.60 ns/iter (+/- 141,589.99) = 3537 MB/s
test http1_consecutive_x1_empty                             ... bench:      12,406.49 ns/iter (+/- 315.80)
test http1_consecutive_x1_req_10b                           ... bench:      13,836.26 ns/iter (+/- 334.06)
test http1_parallel_x10_empty                               ... ignored
test http1_parallel_x10_req_10kb_100_chunks                 ... ignored
test http1_parallel_x10_req_10mb                            ... ignored
test http1_parallel_x10_res_10mb                            ... ignored
test http1_parallel_x10_res_1mb                             ... ignored
test http2_consecutive_x1_empty                             ... bench:      20,369.33 ns/iter (+/- 387.84)
test http2_consecutive_x1_req_100kb                         ... bench:      81,246.68 ns/iter (+/- 6,177.12) = 1260 MB/s
test http2_consecutive_x1_req_10b                           ... bench:      23,318.18 ns/iter (+/- 570.06)
test http2_parallel_x10_empty                               ... bench:      82,584.22 ns/iter (+/- 1,593.23)
test http2_parallel_x10_req_10kb_100_chunks                 ... bench:  23,587,531.40 ns/iter (+/- 33,778,212.74) = 434 MB/s
test http2_parallel_x10_req_10kb_100_chunks_adaptive_window ... bench:   6,192,914.50 ns/iter (+/- 26,899,904.87) = 1653 MB/s
test http2_parallel_x10_req_10kb_100_chunks_max_window      ... bench:   5,947,339.00 ns/iter (+/- 319,046.43) = 1721 MB/s
test http2_parallel_x10_req_10mb                            ... bench:  43,917,411.10 ns/iter (+/- 1,516,811.77) = 2387 MB/s
test http2_parallel_x10_res_10mb                            ... bench:  78,406,853.40 ns/iter (+/- 9,386,097.86) = 1337 MB/s
test http2_parallel_x10_res_1mb                             ... bench:   4,298,477.30 ns/iter (+/- 102,934.25) = 2439 MB/s

test result: ok. 0 passed; 0 failed; 5 ignored; 14 measured; 0 filtered out; finished in 63.27s

     Running benches/pipeline.rs (target/release/deps/pipeline-aed9e7f648c9c37a)

running 1 test
test hello_world_16 ... bench:      44,624.71 ns/iter (+/- 454.68) = 44 MB/s

test result: ok. 0 passed; 0 failed; 0 ignored; 1 measured; 0 filtered out; finished in 0.40s

     Running benches/server.rs (target/release/deps/server-9dd3df1af679d909)

running 8 tests
test raw_tcp_throughput_large_payload   ... bench:     329,928.74 ns/iter (+/- 11,328.66) = 3031 MB/s
test raw_tcp_throughput_small_payload   ... bench:      33,615.90 ns/iter (+/- 746.27) = 4 MB/s
test throughput_chunked_large_payload   ... bench:     309,575.21 ns/iter (+/- 5,995.23) = 3230 MB/s
test throughput_chunked_many_chunks     ... bench:  35,281,819.70 ns/iter (+/- 26,217,654.57) = 28 MB/s
test throughput_chunked_small_payload   ... bench:      23,675.69 ns/iter (+/- 2,307.99) = 7 MB/s
test throughput_fixedsize_large_payload ... bench:     263,313.46 ns/iter (+/- 7,234.05) = 3798 MB/s
test throughput_fixedsize_many_chunks   ... bench:  26,283,718.70 ns/iter (+/- 26,250,872.75) = 38 MB/s
test throughput_fixedsize_small_payload ... bench:      36,870.51 ns/iter (+/- 2,014.28) = 4 MB/s

test result: ok. 0 passed; 0 failed; 0 ignored; 8 measured; 0 filtered out; finished in 32.71s

Single-thread (TOKIO_WORKER_THREADS=1)

     Running benches/end_to_end.rs (target/release/deps/end_to_end-dca2380bf04e2a3b)

running 19 tests
test http1_consecutive_x1_both_100kb                        ... bench:      69,299.77 ns/iter (+/- 6,787.54) = 2955 MB/s
test http1_consecutive_x1_both_10mb                         ... bench:   6,151,905.75 ns/iter (+/- 39,428.37) = 3408 MB/s
test http1_consecutive_x1_empty                             ... bench:      12,288.68 ns/iter (+/- 166.90)
test http1_consecutive_x1_req_10b                           ... bench:      13,582.33 ns/iter (+/- 400.78)
test http1_parallel_x10_empty                               ... ignored
test http1_parallel_x10_req_10kb_100_chunks                 ... ignored
test http1_parallel_x10_req_10mb                            ... ignored
test http1_parallel_x10_res_10mb                            ... ignored
test http1_parallel_x10_res_1mb                             ... ignored
test http2_consecutive_x1_empty                             ... bench:      20,559.21 ns/iter (+/- 392.98)
test http2_consecutive_x1_req_100kb                         ... bench:      74,367.19 ns/iter (+/- 427.62) = 1376 MB/s
test http2_consecutive_x1_req_10b                           ... bench:      24,329.70 ns/iter (+/- 1,806.88)
test http2_parallel_x10_empty                               ... bench:      82,368.02 ns/iter (+/- 1,500.82)
test http2_parallel_x10_req_10kb_100_chunks                 ... bench:  23,613,831.70 ns/iter (+/- 33,805,225.21) = 433 MB/s
test http2_parallel_x10_req_10kb_100_chunks_adaptive_window ... bench:   6,069,434.75 ns/iter (+/- 138,405.62) = 1687 MB/s
test http2_parallel_x10_req_10kb_100_chunks_max_window      ... bench:   5,957,309.20 ns/iter (+/- 57,630.82) = 1718 MB/s
test http2_parallel_x10_req_10mb                            ... bench:  43,533,722.20 ns/iter (+/- 3,087,991.40) = 2408 MB/s
test http2_parallel_x10_res_10mb                            ... bench:  80,000,328.60 ns/iter (+/- 9,427,560.55) = 1310 MB/s
test http2_parallel_x10_res_1mb                             ... bench:   4,587,693.60 ns/iter (+/- 428,340.20) = 2285 MB/s

test result: ok. 0 passed; 0 failed; 5 ignored; 14 measured; 0 filtered out; finished in 66.46s

     Running benches/pipeline.rs (target/release/deps/pipeline-aed9e7f648c9c37a)

running 1 test
test hello_world_16 ... bench:      31,647.09 ns/iter (+/- 3,712.64) = 62 MB/s

test result: ok. 0 passed; 0 failed; 0 ignored; 1 measured; 0 filtered out; finished in 1.21s

     Running benches/server.rs (target/release/deps/server-9dd3df1af679d909)

running 8 tests
test raw_tcp_throughput_large_payload   ... bench:     268,912.27 ns/iter (+/- 5,042.34) = 3719 MB/s
test raw_tcp_throughput_small_payload   ... bench:      20,706.44 ns/iter (+/- 287.95) = 7 MB/s
test throughput_chunked_large_payload   ... bench:     252,944.14 ns/iter (+/- 19,078.46) = 3954 MB/s
test throughput_chunked_many_chunks     ... bench:  43,998,876.80 ns/iter (+/- 17,601,228.52) = 22 MB/s
test throughput_chunked_small_payload   ... bench:      23,813.97 ns/iter (+/- 567.32) = 7 MB/s
test throughput_fixedsize_large_payload ... bench:     249,587.60 ns/iter (+/- 6,923.92) = 4007 MB/s
test throughput_fixedsize_many_chunks   ... bench:  26,395,926.35 ns/iter (+/- 19,977,792.35) = 37 MB/s
test throughput_fixedsize_small_payload ... bench:      23,513.24 ns/iter (+/- 4,154.19) = 6 MB/s

test result: ok. 0 passed; 0 failed; 0 ignored; 8 measured; 0 filtered out; finished in 42.24s

[emschwartz]

Ah, actually just noticed that all the benches use the single threaded runtime 🤦