Runtime flavours

[RPallas92]

The runtime used by Pingora is based on Tokio. There are two flavours:

1. Tokio standard multi-thread runtime that has work stealing.
2. A custom multi-thread runtime without work stealing.

According to the comments in the code the second runtime is as efficient as the single-threaded runtime while allows the async program to use multiple cores. However, I took a look at the benchmarks and the second flavour has higher latency and handles less reqs/sec.

In which real cases a multi-thread runtime without work stealing could be more efficient?

[eaufavor]

Could you share your test methods and results? Thanks.

This https://github.com/cloudflare/pingora/blob/main/pingora-runtime/benches/hello.rs bench shows a case where the non-stealing runtime could be better. The benchmark is just to show that there is a potential trade-off. The numbers and results could vary based on your exact workload and hardware.