Speed up green task spawning

[alexcrichton]

These commits pick off some low-hanging fruit which were slowing down spawning green threads. The major speedup comes from fixing a bug in stack caching where we never used any cached stacks!

The program I used to benchmark is at the end. It was compiled with rustc --opt-level=3 bench.rs --test and run as RUST_THREADS=1 ./bench --bench. I chose to use RUST_THREADS=1 due to #11730 as the profiles I was getting interfered too much when all the schedulers were in play (and shouldn't be after #11730 is fixed). All of the units below are in ns/iter as reported by --bench (lower is better).

	green	native	raw
osx before	12699	24030	19734
linux before	10223	125983	122647
osx after	3847	25771	20835
linux after	2631	135398	122765

Note that this is not a benchmark of spawning green tasks vs native tasks. I put in the native numbers just to get a ballpark of where green tasks are. This is benchmark is clearly benefiting from stack caching. Also, OSX is clearly not 5x faster than linux, I think my VM is just much slower.

All in all, this ended up being a nice 4x speedup for spawning a green task when you're using a cached stack.

extern mod extra;
extern mod native;
use std::rt::thread::Thread;

#[bench]
fn green(bh: &mut extra::test::BenchHarness) {
    let (p, c) = SharedChan::new();
    bh.iter(|| {
        let c = c.clone();
        spawn(proc() {
            c.send(());
        });
        p.recv();
    });
}

#[bench]
fn native(bh: &mut extra::test::BenchHarness) {
    let (p, c) = SharedChan::new();
    bh.iter(|| {
        let c = c.clone();
        native::task::spawn(proc() {
            c.send(());
        });
        p.recv();
    });
}

#[bench]
fn raw(bh: &mut extra::test::BenchHarness) {
    bh.iter(|| {
        Thread::start(proc() {}).join()
    });
}

[alexcrichton]

I also think that there are only 4 allocations remaining:

• proc() for task::spawn - I think this is required no matter what
• ~Task - we may be able to put the Task at the top of the stack rather than allocating on the heap. This would be difficult.
• ~GreenTask - I think this could follow the same strategy as Task by allocating it at the top of the stack, but again, very difficult and dubiously worth it.
• ~Registers - apparently this is needed to align registers to 16-bytes. This should not be necessary at all to have an allocation here.

[brson]

Very promising results.

[pcwalton]

🤘 I'd love to benchmark this against raw pthread stack spawning.