Eliminate blocking on the event loop when possible

[toddaaro]

In the current design the scheduler takes one scheduling action, registers a callback to wake up in the run_sched_once function, and then blocks on the event loop. This "block" is implemented as just finishing the execution associated with the run_sched_once callback. As there is a callback registered it immediately wakes up and repeats the process.

This should be unnecessary. The drop back to the event loop is a very long process relative to a task context swap, so this is likely very bad for performance. A better solution would be to keep the scheduler active until it runs out of stuff to do, and then block on the event loop.

To do this we should have a "live" flag for the scheduler. If the flag is set the scheduler is guarenteed to check for messages and likely work before going to sleep. While active the scheduler algorithm becomes:

1. Call run on the event loop with the NO_WAIT flag set. This will make it so that if there are no events, the function returns immediately instead of blocking.
2. Find a task to run, and run it. Repeat a small number of times to get better task throughput.
3. If the live flag is false, call a blocking run.
4. If no work has been done this round, set the live flag to false.
5. Goto (1)

The libuv "uv_run" function has three modes. UV_RUN_DEFAULT is what we have been using, a blocking "do stuff until the program is done" use of the event loop. UV_RUN_NOWAIT polls for new events and then does not block if there are no events ready. This is the style that we want for the non-blocking run case. UV_RUN_ONCE is how we would perform blocking runs. It will block until it handles a single event, and then return how many pending events there are.

To wake up the scheduler while it is "sleeping", the old idle-callback is used. Except now the function provided simply modifies the live flag to "awake", and the scheduler loop logic takes over. In any case where an idle callback would be registered and the live flag is set the callback can be skipped.

[toddaaro]

This builds on #8340 somewhat as it also changes how idle callbacks work in the scheduler.

[brson]

As far as I can tell the structure you suggest isn't about eliminating 'blocking' per se - if there are events to be run then the loop doesn't block - but is a proposal to improve performance by using polling instead of events.

From the description I believe that the performance win you expect to get here is from not using the idle callback. I'm skeptical that calling the idle callback is a significant amount of the work that the event loop is doing though. The work imposed by an idle callback on uv_run is the following (from loop-watcher.c):

  void uv__run_idle(uv_loop_t* loop) {                                      \
    uv_idle_t* h;                                                         \
    ngx_queue_t* q;                                                           \
    ngx_queue_foreach(q, &loop->idle_handles) {                             \
      h = ngx_queue_data(q, uv_idle_t, queue);                            \
      h->idle_cb(h, 0);                                                     \
    }                                                                         \
  } 

That is, traversing an element of a linked list and calling a function pointer.

This structure has a downside that it would impose more requirements on our generic event loop, and that it is more complicated.

I'm not inclined to make this change without evidence that the current callback-based structure is a problem.

[thestinger]

no longer relevant to the standard libraries (#17325)