eRFC: Experimentally add coroutines to Rust

[alexcrichton]

This is an experimental RFC for adding a new feature to the language,
coroutines (also commonly referred to as generators). This RFC is intended to be
relatively lightweight and bikeshed free as it will be followed by a separate
RFC in the future for stabilization of this language feature. The intention here
is to make sure everyone's on board with the general idea of
coroutines/generators being added to the Rust compiler and available for use on
the nightly channel.

Rendered

What's a experimental RFC?

[alexcrichton]

cc @rust-lang/compiler, @Zoxc, @vadimcn

[eddyb]

The only thing I can see that I don't immediately agree with is the "coroutine" terminology 😆 (I've gotten used to the ES "generator" vs "coroutine" dichotomy which is stackless vs stackful), all the non-fundamental constructs I had doubts about, from other RFCs, seem to be gone.

We have to be careful to avoid stabilizing a type like CoroutineStatus in a state that doesn't allow representing yield x, but I'm too not worried about that. There was also that discussion about having a variant for yield ("suspend") and another for yield x ("yield a value"), and this RFC sidesteps the problem altogether by not including the latter, but it might be relevant later.

It'd be interesting to see how far people can get with emulating actual generators on top of plain yield - with C++'s "resumable functions", that's what they do, but it relies on shared mutable state.
A direct port of the C++ method appears impossible, with Rc<(Ref)Cell<T>> being the closest option.

[eternaleye]

One nit I have, in the cleanliness/ergonomics domain - the example you give is an #[async] function returning an io::Result<()>. Should this:

• Return an impl Future<Item=io::Result<()>>? If so, what should the Future's Error be?
• Return an impl Future<Item=(), Error=io::Error>? If so, what should be done for non-Result-returning #[async] functions? Should they return impl Future<Item=T, Error=!>? Does Option<T> become impl Future<Item=T, Error=()>? Arbitrary types implementing Try?

Connected with this, what conversions should await!() apply? For the first option, clearly something makes it out to the ? operator - but what happens if the Future is capable of failing and the written return value is a Result?

EDIT: Ah, I missed that you do specify the latter desugaring, though I think a clarification for #[async] functions returning non-Result values would be beneficial.

EDIT 2 (Electric Boogaloo): One thing I do feel should be brought up is a specific section of this post by Joe Duffy about using asynchrony in Midori, and specifically using coroutines with async/await syntax sugar:

An interesting consequence was a new difference between a method that awaits before returning a T, and one that returns an Async<T> directly. This difference didn’t exist in the type system previously. This, quite frankly, annoyed the hell out of me and still does. For example:

async int Bar()  { return await Foo(); }
Async<int> Bar() { return async Foo(); }

We would like to claim the performance between these two is identical. But alas, it isn’t. The former blocks and keeps a stack frame alive, whereas the latter does not. Some compiler cleverness can help address common patterns – this is really the moral equivalent to an asynchronous tail call – however it’s not always so cut and dry.

On its own, this wasn’t a killer. It caused some anti-patterns in important areas like streams, however. Developers were prone to awaiting in areas they used to just pass around Async<T>s, leading to an accumulation of paused stack frames that really didn’t need to be there.

It may not be reasonable to insist on having a story for this right away, but I think there should at least be a story on having a story for it 😛

[arielb1]

It'd be interesting to see how far people can get with emulating actual generators on top of plain yield - with C++'s "resumable functions", that's what they do, but it relies on shared mutable state.
A direct port of the C++ method appears impossible, with Rc<(Ref)Cell> being the closest option.

Sharing an Rc<Cell<Option<T>>> between the generator is semantically equivalent to returning values from yield.

[SimonSapin]

Coroutines are not, and will not become a stable language feature as a result of this RFC.

(Emphasis added.)

It’s easy to overlook the latter part of the quoted sentence. I’d like the RFC to say that a path exists to eventually stabilize corountines (subject to future discussion and RFCs) for use cases other than async/await, for example for returning -> impl Iterator<Item=Foo>. That corountines are not intended to always be a hidden implementation detail of async/await (even if they are at first).

[eddyb]

@arielb1 Ah, indeed, with the ability to put in a value and later take it out, Cell<Option<T>> works.

[est31]

s/future /for /

[est31]

So the RFC proposes to add stackless coroutines as experimental feature, but doesn't really describe what stackless coroutines would look like, and what the advantages of stackless coroutines are over stackfull ones. I'd like more on that, especially as I'm not certain how this RFC manages to avoid the issue of stackless coroutines that yielding is only possible from the top level coroutine. Is it the state machines?

[Diggsey]

I'm definitely in favour of this feature, although I'm still hoping that a generic "execution context" parameter is added to futures, similar to rust-lang/futures-rs#129, to replace the scoped thread-locals which are currently used.

This would require the Coroutine trait to allow passing a parameter to resume.

At the risk of encouraging further bike-shedding, would it be possible to add the details of the new types and traits to the RFC?

[ihrwein]

Do we really need the await macro? Can't we apply ? to all Futures, so ? first awaits the result, then does the early return based on the value?

[eternaleye]

@ihrwein The problem there is "What if you want to await on a future, but not early-return on error?" Catching ? is still not available, IIUC, and so this would rule out valid code.

On the other hand, I would be curious about making await!() specially handle cases where Error=! and directly produce the value. It has ergonomic benefits, though it may need balanced against added complexity and magic.

[ihrwein]

@eternaleye That makes it clear, thanks. I believe if this pattern will be very common (await + ?), then ? could be extended to work with futures (although I see that they are somewhat orthogonal).

[ahicks92]

I would like to see iterators made more of a focus here. I don't necessarily see making them a priority as more burdensome than making async/await work, since we essentially already have to. Our story for writing complex iterators from scratch is abysmal; I'd prefer us to not have to wait for another RFC/implementation/nightly cycle after this one.

@est31
If I recall, the problem with stackful coroutines is that they require an implicit allocation for the stack. They also have more overhead, and you can obtain something similar (all be it not identical and with a cost) by boxing recursive calls to yourself. Any coroutine which doesn't recursively call itself or contain a coroutine that does can be flattened, with a compile-time size.

[nikomatsakis]

I just wanted to make a meta-comment about this RFC and the role that I see it playing. In discussions on internals and elsehwere, we've been going back and forth for some time about the best way to handle complex language extensions like coroutines, syntax extensions, embedded Rust support, and so forth. These are basically all cases where there seems to be a clear high-level need, but there is also a very large "design space" to explore. In such cases, it's not clear that writing up a detailed RFC is the best place to start. Often, a lot of the interesting questions only arise when you start putting the design into practice. Furthermore, even if we could write a big RFC, everybody is busy, and it's not clear that the various teams will have the time to really vet and approve something that complex.

We've handled this in the past in a number of ways. Sometimes we can write up a kind of "conservative" RFC that describes a limited form of the feature (e.g., for impl Trait) and then work on follow-up RFCs. Other times, e.g. with "naked functions", we opted to approve an RFC, but made it clear in comments that we weren't really sure if we wanted the feature. Still other times, e.g. with platform-specific ABIs and syntax extensions, we've wound up landing PRs without any formal RFC at all, but with the intention of writing one later.

To better handle these cases, I'm hoping we can kind of formalize an "experimental RFC" process. The idea is that we'll start with an "experimental RFC" like this one, that is primarily aimed at explaining the motivation and also a high-level sketch of what we plan to do. So, in this case, the motivation is that async-await is needed, and the high-level plan is that we hope to build it outside of the language, using two more fundamental features: procedural macros and some form of coroutine/generator syntax to do the state-machine transformation. The goal of this RFC is then to evaluate that high-level strategy and affirm that we think it's a promising path to explore. As such, I don't imagine that this RFC will require as much deliberation as other RFCs, since it's not making any kind of commitments or firm decisions.

It's good for the RFC to also identify some of the questions that we intend to answer as we go, but we shouldn't expect it to have answers. Examples might be: what precisely should the async-await macros look like? How exactly will the coroutines be surfaced in Rust syntax, and what traits etc will they use? Those conversations can take place later, and as we come to conclusions, we should track the answers and be working on follow-up RFCs which will address them in particular.

The key point then is that the experimental RFC is not enough to stabilize something. For that, we need a proper RFC that lays out the plan. This RFC can be informed by the experience and working implementation, so we should have a lot of data to use.

Now, in a sort of meta-meta twist, I don't know how well this process will work. I view this experimental RFC as itself an experiment in process. The main thing I want to avoid is that we wind up with a lot of features that "work" but don't have much for a formal spec. We have enough of that grandfathered in already. ;) This is why I wanted to ensure we have a follow-up RFC, and I think we should be careful that when we land patches on "experimental code", we insist that it comes with good, well-organized test cases that document the behavior and the edge cases. In other words, just because something is experimental doesn't mean it has an excuse to be sloppy. (Ideally, we'd be trying to maintain some informal documentation in the nightly book, as well -- but I think this applies to all the nightly features.)

The other concern of course is that we're going to be maintaining this code in master. It will interact with the rest of the system. It will cause regressions. In a way, this is a real strength: one of the things we want to be evaluating about a new feature is how it interacts with the rest of our system! But it's also a reason that we should all decide whether to take on that maintaince burden.

[est31]

Do we really need the await macro? Can't we apply ? to all Futures, so ? first awaits the result, then does the early return based on the value?

I'd prefer if you could distinguish between "this might return in an error condition" and "this is an async break point". If the await macro is seen as too long for such an use case we should introduce another operator IMO. But let's not bikeshed this at the "experimental" stage of this RFC.

[ivandardi]

@eternaleye Hm... How about this? Calling ? on a future will await it and return early on error, as per expected behavior of the ? operator. If you want to match on the error instead of returning, you could call an .await() method on the future, which will await it and return a Result without early return. That would mean that doing future()? and future().await()? will be the same thing.

[nikomatsakis]

Now, in a non-meta comment, I'm very much in favor of this RFC and this general approach. It seems to me that "coroutines/generators" are the right fundamental primitive to start with; they can support so many different applications (iterators, as @camlorn points out, but also other "future-like" traits, as well as thinks like parsers and lexers). Having async-await work via syntax extension feels good to me since it puts all of these other kinds of applications on equal footing; they may want their own "wrapper" macros or types. I like that as a side-benefit of this work we will be improving syntax extensions, and in particular I imagine there will be a good focus on their error messages and so forth, since we'll want to make #[async] and await! feel "as native as possible".

(One other observation: I think that our experience with try! eventually migrating to ? has been quite good so far. It seems to me that there is a viable path for things to start out as syntax extensions and then move into the language proper if that seems advantageous.)

A few responses to some comments I saw:

---

@eternaleye

One nit I have, in the cleanliness/ergonomics domain [...]

I do think these are good questions to raise, though I think that we don't have to settle them by any means to accept the RFC. In other words, figuring out the most ergonomic way to encode async-await feels like the kind of thing that we can do during the implementation period, and I wouldn't want this RFC to try and do it. But maybe it'd be good to start an internals thread or some other place to hammer out these details.

(For example, another thing that occurs to me is that, since #[async] functions just get a token tree from their input, they really could push harder on syntax within the function body. e.g., we could make await a "pseudo-keyword", or support async for instead of #[async] for.)

---

@est31

So the RFC proposes to add stackless coroutines as experimental feature, but doesn't really describe what stackless coroutines would look like, and what the advantages of stackless coroutines are over stackfull ones.

This does seem like something that might be good to talk about, since choosing to build on stackless coroutines is a key building block here. I think one of the obvious advantages of stackless coroutines is that they don't really require any sort of "runtime", or at least they push that decision outward. If you use something where we allocate a stack ahead of time, then you have to decide who is allocating the stack and manage it -- you also run into some hard questions about what to do if the stack is not big enough, and so forth. I think @vadimcn's stackless coroutines RFC has a bit more here, although it doesn't talk as much about it.

---

@Diggsey

At the risk of encouraging further bike-shedding, would it be possible to add the details of the new types and traits to the RFC?

I'm not sure if I think that should be in the RFC -- at minimum, I'd want to clearly label them as non-normative. But it's probably a good idea to try and write-up that description, perhaps posting it into the internals thread on the topic for more discussion. I also think that the we ought to document these things, at least in some form, in the unstable book in @Zoxc's branch.

---

@camlorn

I would like to see iterators made more of a focus here. I don't necessarily see making them a priority as more burdensome than making async/await work, since we essentially already have to.

I do think that's a great area to dig into, though I'd rather do it in a specific thread on the topic than here, since it's really a question of working out the fine details -- it's clear it will work in some form or another. For example, @vadimcn's RFC #1823 proposed blanket impls of Iterator for all FnMut() -> CoResult<..> generators, which would mean that one could do things like:

fn process<T: Iterator<Item = ...>>(...) {  }

// The `|| .. yield ...` syntax creates a generator, which
// implicitly implements `Iterator`:
process(|| {
    for x in elem {
        yield ...
    }
})

That's really slick, but this precise approach will likely run afoul of the existing coherence rules (although I just remembered that the Fn traits are fundamental, which may help here). If so, there are various things we could do about it. The simplest would to have a wrapper type (process(iterator(|| ...yield..))), but we may also be able to introduce some kind of "fundamental" trait that the compiler's generators implement which then permits the blanket impl. Or, perhaps we could not have generators use the FnMut trait but rather a different one (as @Zoxc prefers), which might help (depends on the details...).

(By the way, I think this is another good example of where the experimental process would help. That is, once we have the basic system implemented, we can make branches that experiment with various approaches around iterators and see what happens, in terms of its impact on existing impls etc.)

[seanmonstar]

If you want to match on the error instead of returning, you could call an .await() method on the future, which will await it and return a Result without early return.

This sort of thing also appeared in the Try RFC, when wondering about how ? should work for Poll<T, E>. Sometimes you want try!, sometimes try_ready!, etc, and could be done poll? and poll.ready()?.

[cramertj]

First of all, I want to express how excited I am to see all the progress that has been made on this. It seems like it's being done in a methodical and careful way. I'm interested to see what we learn from having experience using this feature on nightly.

One nit: I don't see any explicit mention in the RFC of why the CoroutineToFuture wrapper type is needed, rather than having coroutines implement futures directly. I remember some discussion of coherence issues around overlapping blanket implementations of Future-- for example, an implementation of Future for T: Coroutine<Poll<T, E>> would overlap with an implementation for T: OtherTrait. These problems would be avoided by implementing Future for CoroutineToFuture<T> where T: Coroutine<Poll<T, E>>. Was that the goal here?

[contactomorph]

it's intended that no new keywords are added to Rust yet to support async/await. This is done for a number of reasons, but one of the most important is flexibility. It allows us to stabilize features more quickly and experiment more quickly as well.

@alexcrichton I have bad feelings about the use of procedural macros for features like asynchronicity/generators/coroutines where keywords may be expected. Isn't this starting to make Rust really over-complicated for a supposed flexibility? I mean it's somehow providing the illusionary impression that the number of keywords is constant whereas - as I see it - it is in fact splitting the keyword space into two syntactical classes, one of them (procedural macros) being easier to extend. Aren't we creating another language inside the language? Maybe I haven't realised before this was the point of procedural macros to begin with. Am I the only one worried about this kind of consideration?

[alexcrichton]

Ok thanks for all the initial comments everyone! I've pushed an update with two pieces:

• Some description in the "detailed design" about what a stackless coroutine would be from a high level. Brought up by @est31 I think it's a good point that there should be some description of what this feature is intended to be, even if all the specifics aren't hammered out.

• I've started adding sections of "open questions". This is along the lines of @nikomatsakis's "questions that we intend to answer as we go". We're not really positioned today to give solid answers to these questions (that adequately weigh all the alternatives). That's the motivation for experimentation though! We just want to make sure we don't lose track of any of these questions, as any RFC for stabilization would want to answer everything in those sections.

---

@cramertj

I don't see any explicit mention in the RFC of why the CoroutineToFuture wrapper type is needed, rather than having coroutines implement futures directly

I believe the alternative is:

impl<T: Coroutine> Future for T {
    // ...
}

(or something like that)

I'd personally be very wary of such an impl in the futures crate because it already conflicts with one impl and it also seems to unnecessarily constrain us for future additions. Additionally, for futures at least, the "wrapper type" should be 100% invisible and not actually be a problem in practice.

I added an "open question" about this though in how the traits interact. This question/problem I'd expect is far more relevant to iterators, for example, as the translation between coroutines an iterators is in theory much simpler.

---

@contactomorph

@alexcrichton I have bad feelings about the use of procedural macros for features like asynchronicity/generators/coroutines where keywords may be expected

A good point! I've made sure to register this as an open question to make sure we loop back around to. For now though my hope is to keep this RFC as minimal as possible to impact on the compiler and make it as easy as possible in terms of maintenance burden to land in the compiler. Along those lines the choice of new keywords isn't taken for now.

It's worth pointing out two imporant aspects, though. The historic try! macro has now been stabilized with special syntax (?) which gives us a concrete data point for "it works in practice to transition from macros to concrete syntax". In that sense moving to dedicated syntax I believe is always an option we'll have open to us.

The second point though is along the lines of what @nikomatsakis mentioned in his first response to @eternaleye. The #[async] attribute takes an arbitrary token stream (unparsed code in Rust) and we get to entirely define what that means for async/await. In that sense while this would still literally require #[async] syntactically but inside of the function we could choose to treat async and await as keywords. This'll require tweaking existing parsers but it's all possible to do!

Basically, I think the choice of syntax extensions early on provides us a maximal amount of flexibility moving forward. We can switch to keywords later, we can experiment with keywords right now. Lots of open possibilities!

[johansigfrids]

If the point of a experimental RFC is that it must be followed up by a proper RFC that lays out the details omitted by the eRFC would it make sense to have an Deferred Questions section, both to document what the the final RFC needs to cover and to be transparent about why the eRFC doesn't touch upon some topics?

[Ixrec]

@johansigfrids The second half of the "Detailed design" section already is that list of questions. Are you saying there are some other questions that need to be added to it or that the list needs to be moved into a separate section to draw even more attention to it or something?

[johansigfrids]

I'm suggesting it would be a separate section as a sibling to Unresolved Questions to make it more clear. That the open questions mentioned in design are deferred to final RFC is currently hidden inside the text.

[nirui]

Sorry to bother you guys, just here to ask whether or not async / await (Or promise / futures) will ever be come a language standard rather than just a external crate?

I'm currently trying to developing a website using Rust, but I found that some third-party external crates I needed does not support Async IO, which really hurts server performance a lot.

Consider it's really really hard to ask all crate authors to uniformly use something like Tokio or other fancy third-party Async IO crate that uses futures crate, maybe it's better to setup a standard in the standard library so everybody can sort of work on the same page (and reduce some of those struggle of life)?

[est31]

Just ask them to adopt tokio or something based on futures-rs. Those two are the standard. Optionally you can ask them to add an async mode so that people are not forced to pull in tokio or futures-rs.

[nirui]

Well, I must getting something wrong. tokio and futures both had different version policy that differ than the language itself, for me this means they can update / alter their API without consider sync the break policy of the Rust language (They may already did), thus it's hard a standard does it?

I did a little research on the context and found that you guys are more leaned towards reduce the content that can be put into the standard library for some reason, maybe that's alright, but some features can really work better in the core.

Some language already did that (to use standard library to introduce standards, not just put essential code), and they work wonderfully like a dream.

Actually, if you look real close, Rust sort if did it too, std::net for example, which is totally unrelated to the language + can be implemented by an external crate, and yet still in the standard library. So as std::fs, and std::os, maybe even std::io. std::thread should also not be in the standard library as it can also be implement in a external crate.

But you guys still put them in there, presumably to provide something standard so everybody can be on the same table and be little happier?

What would happen if those things does not existed in the std? Then somebody wants to work on somebody::net may needs to wait until somebody else finishes somebodyelse::os. And somebody working on somebody::webframework will has to wait until somebody::net, someanother::thread and someanother::os all been finished. Chaos itsn't?

Rust is a hard language already, please do something to make user's life easier.

Bonus question: Why do people in the PHP world tend to build their project on top of a full stack framework rather than compose a custom framework by themselves, even they knew a full stack framework is often slower than the composed one?

[Diggsey]

@Reinit tokio and futures have the same sem-ver versioning policy that the rest of the ecosystem does. The problem is that neither of them have reached version 1.0 yet: they're simply not ready.

I'm afraid if you're looking for a production-ready async system with strong backwards compatibility guarantees, that's just not available yet. Moving it into the standard library won't help in that regard.

[nirui]

@Diggsey OK, then looks I need to wait a little while.

Thank you.

[Celthi]

@Diggsey OK, then looks I need to wait a little while.

Thank you.

Hi, Is it now what you are waiting for? I'm a newbie for Rust, I'm browsing this thread and don't know if the current Rust world has reached an agreement on the concepts and implemtation. Tokio seems to reach to 1.0

[mehcode]

Hi, Is it now what you are waiting for? I'm a newbie for Rust, I'm browsing this thread and don't know if the current Rust world has reached an agreement on the concepts and implemtation. Tokio seems to reach to 1.0

It's a much different world in 2021 in Rust. Future is available in the std library. Tokio and async-std are both 1.x and production-ready for async network programming in Rust.

[lovely-error]

This feature is amazing! Is it possible to 'inline' yielding to an outer coroutine?
Like this

let k = || yield { 
       let d: u32 = a().coawait;
};

struct Thing;

fn a() -> impl Coroutine<Return = u32, Yield = Thing> {
    || {
        yield Thing;
        return 233;
    }
}

If yes, then this subsumes rust's default ugly async!!

Do you think that it is possible? @alexcrichton

[shepmaster]

@VVishion is that rust-lang/rust#68923?

[VVishion]

@shepmaster Oh, yes it is. Thanks!

[mocenigo]

Please make this a standard feature!