Do tokio reuse spawn_blocking threads?

[sylvain101010]

Hi,

I was wondering what is best to execute blocking tasks for a web service. So I was asking myself: Do tokio reuse threads from the blocking threadpool launched with spawn_blocking?

from my basic tests it seems so. When calling 100 times a function which launch 2 spawn_blocking:

pub async fn route_blocking() -> HttpResponse {
    let _ = tokio::task::spawn_blocking(|| {
        let duration = std::time::Duration::from_secs(2);
        std::thread::sleep(duration);
    }).await;
    let res = tokio::task::spawn_blocking(|| {
        let duration = std::time::Duration::from_secs(2);
        std::thread::sleep(duration);
        let mut data = HashMap::new();
    data.insert("hello", "from blocking2");
    Response::data(data)
    }).await.unwrap();


    HttpResponse::Ok().json(res)
}

# 100 concurrent calls
$ ab -c 100 -n 500 http://127.0.0.1:8888/api/blocking

my process creates only ~120 threads which may indicate that the threads a reused.

Or should I use a third-party, fixed-size threadpool like rayon?

[hawkw]

Tokio uses a dedicated thread pool of blocking worker threads for running blocking tasks. When a blocking worker completes its current blocking task, it returns to the blocking pool and will execute new blocking work when new blocking tasks are spawned.

When all blocking workers in the pool are busy, new blocking threads will be spawned, up to a limit configured by the max_threads runtime builder setting. Skimming the docs, it seems like this might not be stated clearly enough in the API documentation, we could definitely add something to clarify this.

[sylvain101010]

Crystal clear, thank you again @hawkw !

I also noticed that after a 'short' amount of time without requests, the threads count of my application go down from ~120 to ~6. As if tokio killed threads of my blocking threadpool.

Blocking threads are killed after KEEP_ALIVE time right?

tokio/tokio/src/runtime/blocking/pool.rs

Line 76 in 52cd240

const KEEP_ALIVE: Duration = Duration::from_secs(10);

If it's true, may I try a small PR to make it more clear in the docs tomorrow?

[Darksonn]

PRs are always welcome!

[Darksonn]

You can configure the keep-alive with https://docs.rs/tokio/0.3/tokio/runtime/struct.Builder.html#method.thread_keep_alive

[sylvain101010]

#3253