Async IO

[steveklabnik]

Rust currently only includes synchronous IO in the standard library. Should we also have async IO as well, or leave that to an external crate? What would the design look like?

Moved from rust-lang/rust#6842

Related: #388

[Valloric]

I don't see why this couldn't just remain in third-party external crates.

[steveklabnik]

It can. Hence the 'community library' and 'libs' tags both.

[Anachron]

Depends on who is actually maintaining the crate.

In my opinion it should be at least some from the core members as once async and sync are not on the same page anymore, it can lead to confusion of even worse, broken projects.

What I mean is this:
Once the community wrote async versions of the std, the community will make decisions on itself, whether something should be one or another way.

It will be hard to keep up-to-date and collaborateurs may not have digged into the core of Rust, though it will either go into a different direction or need someone to sync it with the syncronous version of the core.

[josephglanville]

The problem with "leaving it up to third party crates" is not having a blessed implementation that all libraries can interoperate with.

This problem has already happened a few times now, namely Ruby and Python both have many competing and incompatible asynchronous IO libraries (gevent/twisted, celluloid/eventmachine).

In and of itself that doesn't sound so bad until you realise the huge amount of stuff that gets built on top of said libraries. When you aren't then able to use powerful libraries with each other because they belong to different "async" camps then things get sad pretty quickly.

Contrast this to C#, a language with built in async primitives, it also ships most of the higher level async integrated code (HTTP client etc). There is a single blessed solution and every library builds on top of it, they all interoperate and everyone is happy.

I think it's super important to have async IO in core or blessed in some way to avoid fragmentation.

[m13253]

I don't see why this couldn't just remain in third-party external crates.

But I think there should be language-level support for some key features that are important to async programming. That includes:

• Coroutines (we need compiler support to ensure thread-safe)
• Python-styled yield statement (though it originally makes an iterator, in async programming it is used to save current execution point and get back later)
• Or C#-styled async/await statement instead
• Green threads (it is possible for 3rd-party libraries, but providing an "official" green threading library avoids different network libraries conflicting with each other because they used different green threading libraries)

Yes we can already build a Node.JS-styled callback based async library -- that makes no sense. We need language-level support to build a modern one.

[flaub]

I think it's important to distinguish between traditional async I/O from use cases requiring the standard C select() API. I think that the existing synchronous I/O API is incomplete without the ability to cancel blocking calls. This doesn't seem to be robustly implementable without using select().

I'd like to see the current synchronous I/O API extended to support cancellation without necessarily exposing the underlying select. The goal here is not to provide a highly scalable or particularly efficient way of scheduling I/O requests, but merely a way to cancel blocking calls.

[gotgenes]

• Python-styled yield statement (though it originally makes an iterator, in async programming it is used to save current execution point and get back later)
• Or C#-styled async/await statement instead

A minor correction to the "Python-styled" comment: it's technically yield from – syntax introduced in Python 3.3. (But yes, it is still based on generators).

Regarding async/await, it's worth noting that this syntax was proposed for Python 3, and has been provisionally accepted for Python 3.5. PEP 492 lists a fair number of languages that have adopted or have proposed the async/await keywords. PEP 492 also does a fair job of describing the weaknesses of the yield from approach, as well as the advantages of async/await.

Not that bandwagons are always the best reason to choose a direction, but async/await will become a very widespread idiom, and supporting it in Rust would provide great accessibility to those of us coming from other languages.

Yes we can already build a Node.JS-styled callback based async library -- that makes no sense. We need language-level support to build a modern one.

Hear, hear!

[ryanhiebert]

Being somebody familiar with Python and its async story, but not as familiar with compiled static languages, it would be helpful to me (and perhaps others) if somebody could comment on something that I'm familiar with from Python.

In Python 3.5, async and await will be based on yield and yield from coroutines based on generators under the hood. This seems like a pretty elegant design to me, but I'd love to hear if there are any problems with that kind of approach.

[gotgenes]

In Python 3.5, async and await will be based on yield and yield from coroutines based on generators under the hood. This seems like a pretty elegant design to me, but I'd love to hear if there are any problems with that kind of approach.

From @nathanaeljones's comment on rust-lang/rust#6842:

One of the earliest comments asked about .NET's async system. It's a callback system, like most, but there is a lot of syntactic sugar, and the 3 different systems that are exposed have different levels of overhead.

The most popular is C# async/await, which generates a closure and state machine to provide a continuation callback. Stack and thread context restoration is expensive, but you can opt out per-task.

Now, what is "expensive"? I'm not sure. (I'm in the same boat as @ryanhiebert. I program mostly in Python. I was made aware of Rust by @mitsuhiko's blog posts.)

[lilith]

@gotgenes The expense depends on how much thread-local storage you're using. I know that it's low enough now that new APIs are async-only. This really depends on the language runtime (and the operating system); I don't think much can be learned about the performance implications by looking at other languages.

[ghost]

From @flaub:

I think it's important to distinguish between traditional async I/O from use cases requiring the standard C select() API. I think that the existing synchronous I/O API is incomplete without the ability to cancel blocking calls. This doesn't seem to be robustly implementable without using select().

I'd like to see the current synchronous I/O API extended to support cancellation without necessarily exposing the underlying select. The goal here is not to provide a highly scalable or particularly efficient way of scheduling I/O requests, but merely a way to cancel blocking calls.

I agree with this. Even trivial programs might suffer from hacks due to the inability to interrupt synchronous calls. (See rust-lang/rust#26446.) I think the language should support asynchronous IO, personally, but if it were really too complex to add to the API, synchronous IO should be made programmatic.

[phaux]

Would be cool to have something like GJ in the core.

[jimrandomh]

Please first put in the straightforward wrapper around select/pselect. I understand you also want to build something better, but my first experience with Rust was hearing that it was 1.0, trying to do a project that involved using pseudoterminals, very close to something I'd already done in C, which involves waiting for input from the user and from a subprocess simultaneously. It immediately got bogged down in a rabbit hole of reverse-engineering macros from Linux system headers and corner cases of FFI, ending up much much more difficult than it had any right to be.

[retep998]

@jimrandomh Keep in mind that select on Windows only works on sockets, and you can't really wait on groups of files/terminals/pipes. If someone can create a crate with async that works well on both Windows and non-Windows, then I'm sure a lot more attention will be paid to getting async in Rust.

[gotgenes]

Keep in mind that select on Windows only works on sockets, and you can't really wait on groups of files/terminals/pipes.

Lack of complete support in Windows didn't stop Python from using it.

[michallepicki]

I guess this belongs here: https://medium.com/@paulcolomiets/asynchronous-io-in-rust-36b623e7b965

[boazsegev]

👍

I doubt we could standardize an Async IO API for all Rust applications without making it part of the Core library... and Async IO seems (to me) to be super important - even if it's just a fallback to a select call (i.e. returning a Result::Err("would block") instead of blocking).

I believe that a blessed / standard Async IO API is essential in order to promote Rust as a network programming alternative to Java, C, and other network languages (even, perhaps, Python or Ruby).

Also, considering Async IO would probably benefit the programming of a Browser, this would help us keep Mozilla invested.

...

Than again, I'm new to Rust, so I might have missed an existing solution (and no, mio isn't an existing solution, it a whole-sale IO framework).

[chpio]

we could standardize the interface in the core and let lib developers do the implementations. that way every one would use that one "blessed" interface but we could have multiple competing implementations (it may be a good idea, i dont know :)).

Also there could be multiple async-api-abstraction-levels just like in JS:

async-await:

async function myFunc() {
  const data = await loadStuffAsync();
  return data.a + data.b;
}

promises:

function myFunc() {
  return loadStuffAsync().then(data => data.a + data.b);
}

callbacks (i hate them ;)):

function myFunc(cb) {
  loadStuffAsync((err, data) => {
    if (err) {
      return cb(err);
    }

    cb(null, data.a + data.b);
  });
}

streams:

tcpSocket
  .pipe(decodeRequest) // byte stream ->[decodeRequest]-> Request object stream
  .pipe(handleRequest) // Request object stream ->[handleRequest]-> Response object stream
  .pipe(encodeResponse) // Response object stream ->[encodeResponse]-> byte stream
  .pipe(tcpSocket)
  .on('error', handleErrors);

or is there already a nice stream implementation in rust? capable of...

• highWaterMark with push back: pauses the previous handler when the queue of the current one is full
• multitasking: each handler is executed in its own coroutine/thread
• byte & object streams
• error handling

[boazsegev]

Looking over the code for the mio crate, I noticed that some unsafe code was required to implement the epoll/kqueue system calls that allow for evented IO (mio isn't purely Async IO, as it still uses blocking IO methods)...

It seems to me that unsafe code should be limited, as much as possible, to Rust's core and FFI implementations.

The "trust me" paradigm is different when developers are asked to trust Rust's core team vs. when they are asked to trust in third parties.

I doubt that competitive implementations, as suggested by @chpio , would do a better job at promoting a performant solution... although it could, possibly, be used to select a most performant solution for the underlying core library.

Ruby on Rails is a good example of how a less performant solution (although more comfortably designed) could win in a competitive environment.

[seanmonstar]

There's nothing wrong with unsafe code. A crate that uses it shouldn't be discouraged. Unsafe is required whenever memory safety is sufficiently complicated that the compiler cannot reason about it.

In this specific case though, unsafe is used because Rust demands all ffi code be marked unsafe. The compiler cannot reason about functions defined by other languages. You will never have code that uses epoll without unsafe (even if that unsafety were eventually tucked into a module in libstd).

[boazsegev]

@seanmonstar - On the main part, I agree with your assessment.

However... Rust's main selling point is safety and I do believe that forcing third parties to write unsafe code does hurt the sales pitch. Also, unsafe code written by third parties isn't perceived as trust-worthy as unsafe code within the core library.

I'm aware that it's impossible to use the epoll and kqueue API without using unsafe code and this is part of the reason I believe that writing an Async IO library would help utilize low level system calls while promoting Rust's main feature (safety).

Having said that, I'm just one voice. Both opinions have their cons and pros and both opinions are legitimate.

[ahicks92]

I'm not sure this is the place and maybe I need to open a separate issue somewhere, but since we don't seem to have it, I'd rate some sort of abstraction over at least socket select as super important. I got to this issue by looking for that and finding other issues that linked here indirectly from 2014; since I see at least one other comment here saying the same thing, I figure I'd add my two cents. Before I go on, I should admit that I'm still from the outside looking in; I really, really want to use Rust and plan to do so in the immediate future, but haven't yet. My primary is C++ and my secondary is Python.
While an async I/O library is really a very good idea and definitely gets a +1 from me if only because Python proves that not putting it in the language/standard library will cause epic-level fragmentation, the lack of a standard library select means that I have to essentially opt into some sort of third party crate or write my own abstraction over the calls. I'm considering Rust for the development of a network protocol as a learning project and can spend whatever time I choose, so I have some flexibility in this regard. But the inability to easily find a platform-neutral select in the standard library is leaving a bad taste in my mouth right now.
I'd say that getting select in at least for sockets as soon as possible would close a rather big and critical hole, as the only other alternatives that don't involve third-party libraries that I'm seeing seem to involve either a thread for every connection or fiddling around with timeouts. In the latter case, the documentation says the error depends on the platform--I get to write yet another abstraction layer! I'll probably opt into Mio for my current project, but I still consider this a shortcoming because all I really need is select.
Speaking more generally, having used both Twisted and Gevent some (though admittedly not enough to be called an expert), I like the look of Asyncio and think copying/borrowing from it might be a good starting point. Twisted always degenerated to inlineCallbacks and gevent always became all the difficulties of threads but with the "advantage" that it lies about this, offering mostly false hope. Since other languages seem to be converging on Asyncio, any solution that looks like it would get my admittedly far-from-expert vote. I'd go so far as to say that we should do it by implementing Python-style generators/coroutines, but that's probably a separate issue and I'm certainly nowhere near thinking about starting any RFCs at the moment.

[tailhook]

While an async I/O library is really a very good idea and definitely gets a +1 from me if only because Python proves that not putting it in the language/standard library will cause epic-level fragmentation

It turns out that python proves quite contrary. There was asyncore in python. But it was quickly obsoleted by twisted (and tornado/gevent... much later). And now there is an asyncio which may be obsoleted by curio, as the latter looks like much nicer by design (but still too early to reason about it's success).

At the end of the day, implementing yield-like construct looks promising. But it's too early to put any async IO library code into the stdlib. There are virtually no downsides of using external crates for the task.

[gotgenes]

At the end of the day, implementing yield-like construct looks promising.

As mentioned earlier, Python has already moved on from yield from to async/await as the syntax of choice for asynchronous operations, for reasons outlined in PEP-492.

I'd like to second pointing out curio as an interesting new model for async in Python.

[ahicks92]

This is interesting. I've never heard of asyncore before now, but it looks like a very complicated way to use a select call. I'm not surprised that it didn't become popular, especially given the 1999 release date (I found one source placing it at Python 1.5.2, but can't find official confirmation). I'm not very convinced that it's good evidence that I'm wrong about Python proving the fragmentation point. I'm not saying that I'm right, just that I need more convincing before dropping my viewpoint as incorrect. In my opinion, something okay with many protocols is better than 5 or 6 options, each more amazing than the last, but each supporting different protocols.
I still think my point about select stands and that it should be put into the standard library as soon as possible. It or something like it is the first step to an async framework. The disadvantage of opting into an external crate for async I/O when all you need is select is that everyone needs to learn the external crate; by contrast, select is a very simple call and can be explained in a few paragraphs. Even if a crate containing only select exists, though, I fail to see any disadvantage to adding it to std::net in some form.

[grigio]

• async-await in JavaScript ES2016 or ESNEXT https://github.com/tc39/ecmascript-asyncawait/blob/master/server.asyncawait.js#L44
• async-await in Rust https://github.com/rockneurotiko/async-await

It seems that async-await style is popular as non-blocking pattern.

[steveklabnik]

as a Python core developer I wanted to offer to answer any Python questions people may have.

Thank you so much @brettcannon !

[njsmith]

One thing that might be worth highlighting about Python's experience so far: we actually started out using async/await as a convenience layer on top of a traditional promise/futures-oriented API, like how async/await look in C# and JS. But in the mean time, people have started exploring other approaches and it's possible we'll actually move away from that approach. I wrote a long blog post making the case for this. I'm sure not all of it carries over the rust context, but it might be of interest in any case.

[brettcannon]

I just want to second reading the blog post by @njsmith as it explains why Python might be shifting how we implement event loops while not having to change async/await thanks to how it's just an API to Python.

[eddyb]

@njsmith The futures-rs effort (with tokio on top) is readiness-based (i.e. poll), not using callbacks.
It seems to me that Python moved from callbacks to something more akin a state machine?
In which case, I don't believe Rust ever had anything serious built on top of callbacks.

[njsmith]

@eddyb: futures-rs certainly seems to use callbacks, in the sense that I see lots of f: F arguments in the Future trait? But I'm definitely not enough of an expert on rust or futures-rs to say anything definitive about how the Python experience does or doesn't carry over.

[eddyb]

@njsmith Those are adapters, e.g. imagine map(self, f) returning a f(await self) async fn.
The fundamental API is the poll method, everything else deals with building state machines for the readiness model without language-level async fn sugar.

This is akin to how Iterator is "internal" (i.e. you call .next() to get a value instead of being called back) and we have Iterator adapters like map and filter, but we still haven't added generators.
In fact, I expect async fn to be built on top of some more general formulation of generators.

[gotgenes]

@njsmith I actually came here to provide a link to your async/await blog post to augment discussion, as it was incredibly thoughtful, but you beat me to it!

I'll add that @mitsuhiko also recently wrote a blog post on async in Python. Maybe particularly pertinent to this thread are his thoughts on the overloading iterators.

[brettcannon]

The post by @mitsuhiko is specifically about asyncio and targeted at library authors (to put it in proper context).

[eddyb]

You forgot loop { yield None; } at the end (or yield None; panic!();). Too much boilerplate IMO.

[c0b]

any update in 2017?

[chpio]

@c0b
#2033 + https://github.com/alexcrichton/futures-await

[steveklabnik]

And https://tokio.rs/ generally

[c0b]

can https://tokio.rs/ be used together with the above async await? I'm not seeing an example

[chpio]

can https://tokio.rs/ be used together with the above async await?

Yeap, Tokio is also just using futures. futures is the library at the heart of all async operations in rust.

I'm not seeing an example

https://github.com/alexcrichton/futures-await/blob/master/examples/echo.rs

[Ixrec]

@c0b https://internals.rust-lang.org/t/help-test-async-await-generators-coroutines/5835 might be the best place to start if you're trying to use the async/await experiment currently on nightly.

[leonerd]

If it's of any interest, I've been busy implementing this idea in Perl, and observing that Python, C#, JavaScript and Dart all also do basically the same thing.

Perl: https://metacpan.org/pod/Future::AsyncAwait
Python: https://docs.python.org/3/library/asyncio-task.html
C#: https://docs.microsoft.com/en-us/dotnet/csharp/async
JavaScript: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Statements/async_function
Dart: https://www.dartlang.org/articles/language/await-async

If you have something of an "official language overview" page similar to those above, I'd like to add it to my collection.

[Centril]

Since discussion has moved on from here to tokio.rs, futures-rs, and other RFCs, I'll go ahead and close this issue.

[janhohenheim]

@Centril I think it would be useful to link those issues for readers stumbling across this one. Do you happen to have them at hand?

[Centril]

Not so much about directly adding async IO directly to the standard library but rather enabling as crates:

• eRFC: Experimentally add coroutines to Rust #2033
• Standard library API for immovable types #2349

[m13253]

In reply to @Centril ,

Since discussion has moved on from here to tokio.rs, futures-rs, and other RFCs

This is what I (as well as some of us) are preventing from happening for years.
Some of us are actually preventing this move to third-party crates.

Lack of language-level async IO would cause incompatibility among third-party crates, as I have already stated before. Additionally, since Rust offers FFI, lack of language-level async IO would cause problems for C code to utilize concurrent programming.

In fact, nearly all languages have async IO. But only those who provide language-level concurrency (e.g. Go, Python 3, C#, etc.) may have more or less chance to win the "language for the cloud" battle. We all agree that Rust is a powerful language far more than the need to build a browser engine. But we don't want to see Rust losing the cloud battlefield, do we?

Really sorry if my language is offensive to you. But I disagree with your opinion that "we are relying on third-party crates". True concurrency requires implementations on compiler level that third-party crates cannot offer.

---

Update: Thank you for your response below. ❤️

[Centril]

@m13253

note: I'm only describing things as they are, not as they ought to be. =) Anyone is free to file full RFC proposals for libstd / language level async IO and we will judge those on their merits.

[Ixrec]

Note that "other RFCs" includes https://github.com/rust-lang/rfcs/blob/master/text/2033-experimental-coroutines.md, which basically is async/await syntax (albeit slightly less pretty since it's done with procedural macros for now). That seems like "language-level async IO" to me, even if it's not-so-secretly sugar over the futures library.

[leonerd]

That seems like "language-level async IO" to me, even if it's not-so-secretly sugar over the futures library.

That's OK. It's surface syntax sugar over futures in Perl as well. :) Probably true of many languages

[BatmanAoD]

@m13253 In case you hadn't heard, here are more up-to-date proposals:

• async/await notation for ergonomic asynchronous IO #2394
• RFC: add futures and task system to libcore #2418