Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Don't always wake up sleeping schedulers #11325

Closed
wants to merge 1 commit into from
Closed

Don't always wake up sleeping schedulers #11325

wants to merge 1 commit into from

Commits on Jan 5, 2014

  1. Don't always wake up sleeping schedulers 

    I created a benchmark recently of incrementing a global variable inside of a
`extra::sync::Mutex`, and it turned out to be horribly slow. For 100K
increments (per thread), the timings I got were:

    1 green thread: 10.73ms
    8 green threads: 10916.14ms
    1 native thread: 9.19ms
    8 native threads: 4610.18ms

Upon profiling the test, most of the time is spent in `kevent()` (I'm on OSX)
and `write()`. I thought that this was because we were falling into epoll too
much, but after implementing the scheduler only falling back to epoll() if there
is no work or active I/O handles, it didn't fix the problem.

The problem actually turned out to be that the schedulers were in high
contention over the tasks being run. With RUST_TASKS=1, this test is blazingly
fast (78ms), and with RUST_TASKS=2, its incredibly slow (3824ms). The reason
that I found for this is that the tasks being enqueued are constantly stolen by
other schedulers, meaning that tasks are just getting ping-ponged back and forth
around schedulers while the schedulers spend *a lot* of time in `kevent` and
`write` waking each other up.

This optimization only wakes up a sleeping scheduler on every 8th task that is
enqueued. I have found this number to be the "low sweet spot" for maximizing
performance. The numbers after I made this change are:

    1 green thread: 13.96ms
    8 green threads: 80.86ms
    1 native thread: 13.59ms
    8 native threads: 4239.25ms

Which indicates that the 8-thread performance is up to the same level of
RUST_TASKS=1, and the other numbers essentiallyt stayed the same.

In other words, this is a 136x improvement in highly contentious green programs.
    @alexcrichton
    alexcrichton committed Jan 5, 2014
    Configuration menu
    Copy the full SHA
    ccea27c View commit details
    Browse the repository at this point in the history