async trait for runtime

[NobodyXu]

I propose to add a new trait for the runtime, which provides portable APIs for creating sockets, pipe, AsyncFd, timers and etc.

Motivation

As mentioned in this comment:

I don't think having async IO traits like Read/Write/BufRead is enough to unify the ecosystem.

Many crates, like reqwest, http server, openssh-mux-client need to create a network socket (tcp/udp/unix socket) and without a way to create it in a portable manner, they will resort back to use a specific runtime.

There is also crates like tokio-pipe which wraps the pipe for tokio users, I think we need to have some way to create AsyncFd in a portable manner.

portable runtime trait 0.2.0

Executor 0.3: #13 (comment)
Reactor 0.2: #13 (comment)

portable runtime trait 0.1.0

Here is a rough scratch of what I expected, many APIs are elided because there are too many APIs:

(NOTE that these APIs are heavily influenced by tokio because it is the most familiar async runtime API to me)

use std::io::Result;

trait Runtime {
    type Handle: Handle;

    async fn yield_now(&self);

    fn block_in_place<F: FnOnce() -> R, R>(&self, f: F) -> R;

    fn handle(&self) -> &Self::Handle;
}

trait Handle: Clone {
    type JoinHandle<T>: JoinHandle<T>;

    fn block_on<T>(&self, f: impl Future<Output = T>) -> T;

    fn spawn<T>(&self, f: impl Future<Output = T> + Send + 'static) -> Self::JoinHandle<T>;
    fn spawn_local<T>(&self, f: impl Future<Output = T> + 'static) -> Self::JoinHandle<T>;

    /// Provided function, automatically call spawn if f is Send, otherwise call spawn_local.
    /// 
    /// # Motivation
    /// 
    /// Suppose that a user want to write some data into TcpStream at background,
    /// they will likely to use spawn.
    /// 
    /// However, spawn requires Send, in order to use it, they would have to write
    /// `RuntimeNet<TcpStream: Send>`, which is cumbersome.
    /// 
    /// Another option is to just use spawn_local, but it only gives concurrency, not
    /// parallelism.
    /// 
    /// Thus, Runtime should have a provided method spawn_auto to automatically
    /// spawn it on local or spawn it on any thread.
    /// 
    /// This will make the portable Runtime trait much easier to use, and
    /// it would even improve the portability, since users do not have to worry
    /// about the "Send" bound anymore.
    fn spawn_auto<T>(&self, f: impl Future<Output = T> + 'static) -> Self::JoinHandle<T>;
}

trait HandleStd: Handle {
    fn spawn_blocking<F: FnOnce() -> R, R>(&self, f: F) -> Self::JoinHandle<R>;
}

enum JoinErrorKind {
    /// Out of memor when spawning
    Oom,
    /// The future panic
    Panic(Box<dyn Any + Send>),
    /// Cancelled
    Cancelled,
    /// Other
    Other(Box<dyn core::error::Error + Send>),
}
trait JoinError: core::error::Error {
    fn error_kind(self) -> JoinErrorKind;
}

trait JoinHandleBase {
    type Error: JoinError;
}
trait JoinHandle<T>: Future<Output = core::result::Result<T, Self::Error>> + JoinHandleBase {
    fn cancel(&self);
    fn is_done(&self) -> bool;
}

/// Provides AsyncHandle
trait RuntimeAsyncHandle: Runtime {
    type AsyncHandle: AsyncHandle;

    #[cfg(unix)]
    fn from_fd(&self, fd: std::os::unix::io::OwnedFd) -> Result<Self::AsyncHandle>;

    #[cfg(windows)]
    fn from_handle(&self, fd: std::os::windows::io::OwnedHandle) -> Result<Self::AsyncHandle>;
}

trait AsyncHandle: Read + Write + Seek {
    type ReadBuffered: AsyncHandle + BufRead;
    type WriteBuffered: AsyncHandle + BufWrite;
    type ReadWriteBuffered: AsyncHandle + BufRead + BufWrite;

    fn try_clone(&self) -> Result<Self>;

    async fn close(self) -> Result<()>;

    fn into_buffered_read(self, n: usize) -> Self::ReadBuffered;
    fn into_buffered_write(self, n: usize) -> Self::WriteBuffered;

    fn into_buffered(self, read_buffer_size: usize, write_buffer_size: usize) -> Self::ReadWriteBuffered;
}

/// Optional
trait AsyncHandleNet: AsyncHandle {
    type AcceptStream: Stream<Item = Result<AsyncHandle>>;
    type RecvMsgStream: Stream<Item = Result<(Buffer, Auxiliary)>>;

    async fn accept(&self) -> Result<AsyncHandle>;

    /// For multi-shot accept supported by io-uring
    /// Provides a default implementation.
    async fn into_accept_stream(self) -> Result<Self::AcceptStream>;

    async fn sendmsg(&self, buffer: IoSlice<'_>, aux: Auxiliary) -> Result<usize>;
    async fn recvmsg(&self, buffer: IoSliceMut<'_>, aux: Auxiliary) -> Result<usize>;

    /// For multi-shot recvmsg supported by io-uring
    /// Provides a default implementation.
    async fn into_recvmsg_stream(self) -> Result<Self::RecvMsgStream>;

    // Provides default implementation which fallback to read + write if the runtime/target does not support sendfile syscall.
    async fn sendfile(&self, ...) -> Result<usize>;
}

/// Optional, for io-uring
trait AsyncHandleNetExt: AsyncHandleNet {
    async fn connect(&self, ...) -> Result<()>;
}

/// Optional
trait AsyncHandleFile: AsyncHandle + Seek {
    async fn fsync(&self) -> Result<()>;

    /// If the runtime/os does not support this, then it can issue fsync instead.
    async fn fdatasync(&self) -> Result<()>;

    /// If the runtime/os does not support this, then it can issue fdatasync instead.
    async fn sync_file_range(&self, off: u64, n: u64) -> Result<()>;

    /// If the runtime/os does not support this, then it can use read + write to simulate this.
    async fn copy_file_range(&self, off_in: u64, other: &Self, off_out: u64, len: u64) -> Result<()>;

    async fn metadata(&self) -> Result<MetaData>;
    async fn set_metadata(&self, metadata: MetaData) -> Result<()>;
}

/// fs operations, like open, optional
trait RuntimeFs: Runtime {
    async fn open(&self) -> Result<Self::AsyncHandle>;
}

/// signal handling, optional
trait RuntimeSignal: Runtime {
    fn ctrl_c(&self) -> Result<()>;
}

/// spawn process, optional
trait RuntimeProcess: Runtime {
    type Child: ...;

    fn wait(&self, child: std::process::Child) -> Result<Self::Child>;
}

/// Perhaps the same set of methods as tokio::time::Interval?
trait Interval {...}

/// time handling, optional but most likely implemented
trait RuntimeTime: Runtime {
    type Interval: Interval;

    fn interval_at(&self, start: Option<Instant>, period: Duration) -> Self::Interval;
}

#[cfg(unix)]
mod unix {
    use super::*;

    enum Madvice { ... }
    trait RuntimeMem: Runtime {
        async unsafe fn madvice(&self, addr: *mut (), len: usize, advice: Madvice) -> Result<()>;
    }

    trait RuntimeAsyncHandleExt: RuntimeAsyncHandle {
        async fn openat2(&self, ...) -> Result<Self::AsyncHandle>;
    }

    enum SignalKind { ... }
    trait RuntimeSignalExt: RuntimeSignal {
        type Signal: Stream<Item = Result<SigInfo>>;

        fn signal(&self, kind: SinglaKind) -> Result<Self::Signal>;
    }
}

#[cfg(linux)]
mod linux {
    trait AsyncHandlePipe: AsyncHandle {
        async fn splice(&self, ...) -> Result<usize>;

        async fn tee(&self, ...) -> Result<usize>;
    }
}

Amendment Proposal: Archive zero-cost abstraction

After I written up this proposal, I realized that it is not zero-cost.
So I created the Amendment Proposal: Archive zero-cost abstraction.

[SabrinaJewson]

It would be simpler to not have separate process/net/file/stdio APIs but rather provide a single unified Socket type (actually, might be better to have Fd on Unix and Socket + Handle on Windows) similar to socket2’s API as well as a spawn_blocking function, and then TcpStream, File, Process etc can all build on that generically. So, more like:

pub trait Runtime {
    #[cfg(unix)]
    type Fd: Fd;

    // true for io_uring, false for epoll, used to determine whether file APIs should just use `spawn_blocking`
    fn supports_non_socket_fds(&self) -> bool;
}

// unix-only
pub trait Fd: From<OwnedFd> + Into<OwnedFd> + AsyncRead + AsyncWrite {}

and std can define a TcpStream<Inner>, where on Unix Inner: Fd and on Windows Inner: Socket.

[NobodyXu]

It would be simpler to not have separate process/net/file/stdio APIs

I doubt that will be a good idea, since the async executor for embedded environment or for some special use cases, e.g. runtime for the main loop of games, does not need to support process/net/file/stdio APIs.

Instead, they could simply just implement Runtime for spawning tasks and RuntimeTime for managing time.

[SabrinaJewson]

Ah, you misunderstood me. I meant that process/net/file/stdio should all be together, but time and task spawning should still be separate.

[NobodyXu]

Ah, you misunderstood me. I meant that process/net/file/stdio should all be together, but time and task spawning should still be separate.

Sorry for that, integrating these traits does sound reasonable, but I am still not completely sure this is a good idea though.

[NobodyXu]

Oops...

[NobodyXu]

Having this method:

    // true for io_uring, false for epoll, used to determine whether file APIs should just use `spawn_blocking`
    fn supports_non_socket_fds(&self) -> bool;

in Runtime will definitely be necessary.

[NobodyXu]

Also, we need to find a way to obtain the runtime.

I personally think the proposal of having a "context" is the best solution for this, since it is zero-cost (does not require thread-local variable) and the crate can easily put a bound on the "context".

[NobodyXu]

Ah, you misunderstood me. I meant that process/net/file/stdio should all be together, but time and task spawning should still be separate.

After a second thought, I think it will be better to keep it separated.

While it's true that process/net/file/stdio can be put together, the implementation of the runtime often split them into multiple features.

For example, tokio provides feature flags for each one of them to speedup compilation and reduce bloat.

Similarly, async-std also provides feature flags.

Thus, I think it will be better to leave them as separate traits so that the runtime can choose to provide feature flags for each of them.

[SabrinaJewson]

Under my design, types like TcpStream, process::Child, Stdin etc would still be equally feature flagged. All that wouldn't be feature flagged would be the underlying low-level system call wrappers which would be common between the high level net/process/io interfaces anyway.

[NobodyXu]

Under my design, types like TcpStream, process::Child, Stdin etc would still be equally feature flagged. All that wouldn't be feature flagged would be the underlying low-level system call wrappers which would be common between the high level net/process/io interfaces anyway.

I am a bit confused.

Do you mean that process/net/file/stdio would be in the same trait be feature gated?

[SabrinaJewson]

Each runtime would just provide an implementation of Socket/Handle/Fd, which provides all the functionality of a socket/handle/file descriptor registered with io_uring/IOCP/epoll/etc under an asynchronous interface. It does not specify or keep track of whether said FD refers to a process, or a network socket, or a file, since the underlying interface is the same for them all.

On top of it, the standard library can provide high-level typed wrappers like TcpStream which would hold an Fd and give an API that keeps track of stuff like FD type in the type system. So:

// generic, runtime-independent file API
pub struct File<F: Fd> {
    fd: F,
}
impl<F: Fd> File<F> {
    pub async fn open(runtime: &F::Runtime, path: &Path) -> io::Result<Self> {
        let fd = runtime.open(path).await?;
        Ok(Self { fd })
    }
}
impl<F: Fd> async Read for File<F> {
    async fn read(&mut self, buf: ReadBufRef<'_, '_>) -> io::Result<()> {
        self.fd.read(buf).await
    }
}
// etc

[NobodyXu]

This actually sounds good.

I think we need the runtime to implement AsyncHandle (can be constructed from OwnedFd or windows's OwnedHandle), Socket for async accept, (multishot) async recv/send implementation, process spawning, timer, fs operations.

[NobodyXu]

@SabrinaJewson I've adopted your suggestion and updated the proposal.

[NobodyXu]

@nrc I have written down a detailed proposal and it seems that we need at least type GAT for spawning tasks.

[NobodyXu]

Amendment Proposal: Archive zero-cost abstraction

Motivation

Currently, in order to implement the portable Runtime trait, the async runtime
must either use reference counting (Rc/Arc) or global static variables.

None of the solutions above is zero-cost.

Reference counting requires boxing and every time a new AsyncHandle and other types
or Future is created, it needs to increase the reference counting and it also
needs to decrease it on Drop.

Global static variable is also not zero-cost since it might makes the executable
larger or requires use of once_cell.

Thus, I decided to propose this to archive zero-cost abstraction.

Modification

In order to archive zero-cost, we must eliminate the need of reference counting
or global static variables and replace them with a reference to the runtime.

To do so, we must add lifetime to every type that implements
the portable runtime trait.

There is two way to accomplish this, one is to use lifetime GAT and the other one is
to have a new rust language feature that provides 'self lifetime.

Lifetime GAT

We would need to apply the following modifications:

trait Handle: Clone {
    type JoinHandle<'a, T>: JoinHandle<T>;
}
trait RuntimeAsyncHandle: Runtime {
    type AsyncHandle<'a>: AsyncHandle;
}
trait AsyncHandle: Read + Write + Seek {
    type ReadBuffered<'a>: AsyncHandle + BufRead;
    type WriteBuffered<'a>: AsyncHandle + BufWrite;
    type ReadWriteBuffered<'a>: AsyncHandle + BufRead + BufWrite;
}
trait AsyncHandleNet: AsyncHandle {
    type AcceptStream<'a>: Stream<Item = Result<AsyncHandle>>;
    type RecvMsgStream<'a>: Stream<Item = Result<(Buffer, Auxiliary)>>;
}
trait RuntimeProcess: Runtime {
    type Command<'a>: ...;
}
trait RuntimeTime: Runtime {
    type Interval<'a>: Interval;
}
#[cfg(unix)]
mod unix {
    trait RuntimeSignalExt: RuntimeSignal {
        type Signal<'a>: Stream<Item = Result<SigInfo>>;
    }
}

This requires each type in the traits to be modified and add <'a> bound
to them.

This also means that we would have to depend on GAT and it might further
complicate the portable async story, especially for new users.

'self lifetime

On the contrary, using 'self lifetime is trival.

No modification to triat, we just need to add 'self to the type specified
when implementing these traits.

spawn

In additional to these changes, we also need to relax the 'static lifetime bound
in spawn:

fn spawn<'this, 'future: 'this, F, T>(&'this self, f: F)
    where F: Future<Output = T> + Send + Sync + 'future -> Self::JoinHandle<T>;

This makes sure that the futures' lifetime is at least as long as the runtime.

Though the implementation of runtime will be more complicated, as they have to handle the order of dropping correctly,
manually create the vtable for the future since they cannot use dyn Future<...> anymore
and they also need to Pin their runtime to ensure that it cannot forgotten safely.

Expected implementation

I would expect the following implementation:

struct SampleRtInner {
    ...
}

struct SampleRt {
    /// Declare thread_pool first to ensure it will
    /// be dropped before inner.
    thread_pool: ScopedThreadPool,
    inner: SampleRtInner,
}

impl Runtime for Pin<&SampleRt> {
    ...
}

impl SampleRt {
    pub fn new() -> Self {
        todo!()
    }

    pub fn new_scoped<F, T>(f: F) -> T
        where F: FnOnce(Pin<&Self>) -> T
    {
        let this = Self::new();
        std::pin::pin!(this);

        f(this.deref())
    }
}

Call for review

I know this amendment proposal must be controversial as this requires either
GAT or a new feature 'self, but I think if we want truely portable and zero-cost
runtime traits, it is the way to go.

@nrc @SabrinaJewson Can you guys review this please?
Thanks in advance!

[nrc]

Hey @NobodyXu I haven't looked in detail at the amendment above, I'm still at the information and requirements gathering stage of work on executors. But two comments: I expect using lifetime GATs is fine in general, they seem on a solid path towards stabilisation. I don't think being zero-cost in the sense of having no runtime overhead at all is necessary, the cost of a few statics or an occasional reference count incr/decr is well worth it for the sake of improved ergonomics (if that is in fact the trade-off here).

[NobodyXu]

I expect using lifetime GATs is fine in general, they seem on a solid path towards stabilisation.

It's good to hear that!

the cost of a few statics or an occasional reference count incr/decr is well worth it for the sake of improved ergonomics (if that is in fact the trade-off here).

Well, I don't think using a few statics or reference counting improve ergonomics.

The problem here is zero-cost runtime abstraction requires lifetime GAT, but if that is fine, then I don't think it will have a lot impact on ergonomics.

In fact, I would argue the current API makes more sense.

Users of this portable runtime can clearly see how their AsyncHandle, Interval and futures are created and understand it instead of treating it like black magic and get confused when it fails.

For example, beginners might attempt to create AsyncFd or TcpSocket in non-async context, like a regular fn f() -> TcpSocket.

When using that function f, it will work fine most of the time, but it is easy for beginners to abuse and call that in non-async place where #[tokio::main]/#[tokio::test] has not been used or a tokio::runtime::Runtime has been created but Runtim::enter is not called.

In that case, the code will compile, but fail at runtime with a panic.

With this portable runtime proposal, this won't happen, because users need a Runtime trait (probably as context) to create AsyncHandle, TcpSocket or etc and it will fail at compile time if they forget to add that as a context.

[NobodyXu]

the cost of a few statics or an occasional reference count incr/decr is well worth it for the sake of improved ergonomics (if that is in fact the trade-off here).

In additional to the ergonomic improvement I mentioned in the last comment, the amendment proposal also improves ergonomic for spawn.

Currently, spawn has to be 'static, which is understandable but too strict.
With this amendment proposal, spawn can accept any future that lives as long as the runtime itself.

[NobodyXu]

@nrc The original proposal also uses specialisation:

// Provided function, automatically call spawn if f is  Send, otherwise call spawn_local.
fn spawn_auto<T>(&self, f: impl Future<Output = T> + 'static) -> Self::JoinHandle<T>;

Motivation

Suppose that a user want to write some data into TcpStream at background,
they will likely to use spawn.

However, spawn requires Send. In order to use it, they would have to write
RuntimeNet<TcpStream: Send>, which is cumbersome.

Another option is to just use spawn_local, but it only gives concurrency, not
parallelism.

Thus, Runtime should have a provided method spawn_auto to automatically
spawn it on local or spawn it on any thread.

This will make the portable Runtime trait much easier to use, and
it would even improve the portability, since users do not have to worry
about the Send bound anymore.

[NobodyXu]

This proposal definitely needs more feedbacks, espeically from whom maintains the async runtime.

@rust-lang/wg-async, @rust-lang/libs-api
@Darksonn @joshtriplett @ihciah @Noah-Kennedy @HippoBaro

[Darksonn]

Well, I looked it over and it seems rather complicated. But perhaps that's unavoidable. I don't think the current location of your spawn_local method is compatible with the spawn_local in Tokio due to the requirement of using a LocalSet, which does not have a handle type.

[NobodyXu]

Thank you @joshtriplett @Noah-Kennedy @ibraheemdev @SabrinaJewson @nrc @Darksonn for providing me with feedback!

async portable runtime 0.2.0

In portable runtime 0.2.0, I split the functionalities into executor and reactor to make the interface simpler, more compositable and easier to standardize.

This does not mean that all runtimes have to support plugable executor.
Runtime like tokio can still implement both executor and reactor for efficiency.

Portable runtime 0.2.0 also avoids use of async and generics in the reactor APIs so that the users of this crate can simply take a &dyn Reactor via context.

Executor 0.2

Checkout #13 (comment) for executor 0.3, which provides better APIs.

Reactor

Below is the basic interface for the reactor:

use std::io;
use std::marker;
use std::task::{Context, Poll};

trait ReactorPoller {
    fn register(&self, pollable: Pollable<'_>, interest: Interest) -> io::Result<Registration<'_>>;
    fn deregister(&self, pollable: Pollable<'_>, registration: Registration<'_>) -> io::Result<()>;

    fn poll_read_ready(
        &self,
        cx: &mut Context,
        registration: &mut Registration<'_>,
    ) -> Poll<io::Result<()>>;
    fn poll_write_ready(
        &self,
        cx: &mut Context,
        registration: &mut Registration<'_>,
    ) -> Poll<io::Result<()>>;

    fn clear_read_ready(&self, registration: &mut Registration<'_>);
    fn clear_write_ready(&self, registration: &mut Registration<'_>);
}

/// An opaque type holding platform-dependent fd/handle
struct Pollable<'a>;

impl<'a> Pollable<'a> {
    #[cfg(unix)]
    pub fn from_fd(fd: std::os::unix::io::BorrowedFd<'a>) -> Self;

    #[cfg(windows)]
    pub fn from_handle(handle: std::os::windows::io::BorrowedHandle<'a>) -> Self;

    #[cfg(windows)]
    pub fn from_socket(socket: std::os::windows::io::BorrowedSocket<'a>) -> Self;
}

/// An opaque type.
struct Registration<'reactor>([usize; 2], marker::PhantomData<&'reactor ()>);

/// A bitflag that contains READABLE and WRITEABLE.
struct Interest;

ReactorTime:

use std::time;

trait ReactorTimer {
    fn poll_sleep(&self, duration: time::Duration, cx: &mut Context) -> Poll<()>;
}

ReactorCompletion for completion based runtime like tokio-uring:

trait ReactorCompletion {
    fn poll_for_response(
        &self,
        pollable: &RequestDesc<'_>,
        cx: &mut Context,
    ) -> Poll<io::Result<()>>;
    fn get_response(&self, pollable: RequestDesc<'_>) -> Option<RawResponse<'_>>;
    fn consume_response(&self, raw_response: RawResponse<'_>) -> Poll<io::Result<()>>;

    fn register_pollable(
        &self,
        pollable: Pollable<'_>,
        cx: &mut Context,
    ) -> Poll<io::Result<Request<'_, PollableDesc<'_>>>>;
    fn unregister_pollable(
        &self,
        pollable_desc: PollableDesc<'_>,
        cx: &mut Context,
    ) -> Poll<io::Result<Request<'_, ()>>>;

    fn register_buffer(
        &self,
        buffer: OwnedBuffer,
        cx: &mut Context,
    ) -> Poll<io::Result<Request<'_, BufDesc<'_>>>>;
    fn unregister_buffer(
        &self,
        buf_desc: BufDesc<'_>,
        cx: &mut Context,
    ) -> Poll<io::Result<Request<'_, ()>>>;

    fn accept(
        &self,
        pollable_desc: &PollableDesc<'_>,
        cx: &mut Context,
    ) -> Poll<io::Result<Request<'_, OwnedPollable>>>;
    fn register_multishot_accept(
        &self,
        pollable_desc: &PollableDesc<'_>,
        cx: &mut Context,
    ) -> Poll<io::Result<MultishotRequest<'_, OwnedPollable>>>;

    unsafe fn read(
        &self,
        pollable_desc: &PollableDesc<'_>,
        buf_desc: &mut BufDesc<'_>,
        cx: &mut Context,
    ) -> Poll<io::Result<Request<'_, usize>>>;
    unsafe fn write(
        &self,
        pollable_desc: &PollableDesc<'_>,
        buf_desc: &BufDesc<'_>,
        cx: &mut Context,
    ) -> Poll<io::Result<Request<'_, usize>>>;
}

/// An opaque type
struct RequestDesc<'reactor>(u64, marker::PhantomData<&'reactor ()>);

struct Request<'reactor, T>(
    &'reactor dyn ReactorCompletion,
    Option<RequestDesc<'reactor>>,
    PhantomData<T>,
);

impl<'reactor, T> Request<'reactor, T> {
    pub unsafe fn new<R: ReactorCompletion>(
        reactor: &'reactor R,
        request_desc: RequestDesc,
    ) -> Self;
}
impl<T> Future for Request<'_, T>
where
    T: FromRawResponse,
{
    // poll_for_response => get_response => FromRawResponse::from => consume_response
}

struct MultishotRequest<'reactor, T>(RequestDesc<'reactor>, marker::PhantomData<T>);
impl<'reactor, T> MultishotRequest<'reactor, T> {
    pub fn request(&self) -> Request<'reactor, T>;
}

struct RawResponse<'reactor> {
    cqe_addr: *const (),
    index: usize,
    phantom: marker::PhantomData<&'reactor ()>,
}

trait FromRawResponse {
    unsafe fn from(raw_response: &RawResponse<'reactor>) -> Self;
}

/// Similar to Pollable but owned.
struct OwnedPollable;

/// An opaque type
struct PollableDesc<'reactor>(u64, marker::PhantomData<&'reactor ()>);

struct OwnedBuffer(*mut u8, usize);
impl OwnedBuffer {
    unsafe fn new(ptr: *mut u8, len: usize) -> Self {
        Self(ptr, len)
    }
}

/// An opaque type
struct BufDesc<'reactor>(u64, marker::PhantomData<&'reactor ()>);

[NobodyXu]

async executor 0.3

After posting about async executor 0.2, I realized it is possible to design the APIs of executor without any generics, async or associated item.

And I also realized that the ExecutorExt API is a bit too complex so I simplified them a bit and also removed the use of generics.

Executor

use std::error;
use std::task::{Context, Poll};

trait Executor {
    /// Always return `Poll::Pending` then wake up the task.
    fn yield_now(&self, cx: &mut Context<'_>) -> Poll<!>;

    /// If there are local tasks, then the executor can refuse to block.
    fn try_enter_blocking_mode(&self) -> Result<(), ()>;

    /// Fail if not in blocking mode.
    fn try_exit_blocking_mode(&self) -> Result<(), ()>;

    /// This new API leaves how to allocate the future, how to implement
    /// the JoinHandle and the managment of lifetime to the users of this
    /// API.
    ///
    /// # Safety
    ///
    /// future must live until it is done or Executor get dropped.
    unsafe fn spawn(&self, future: SpawnableFuture) -> Result<(), SpawnError>;

    /// # Safety
    ///
    /// future must live until it is done or Executor get dropped.
    unsafe fn spawn_local(&self, future: SpawnableLocalFuture) -> Result<(), SpawnError>;

    /// # Safety
    ///
    /// future must live until it is done or Executor get dropped.
    unsafe fn spawn_blocking(&self, fn_once: SpawnableFnOnce) -> Result<(), SpawnError>;
}

enum SpawnError {
    /// Out of memor when spawning
    Oom,
    /// Other
    Other(Box<dyn error::Error>),
}

struct FutureVtable {
    /// Must not panic
    poll: fn(*mut (), cx: &mut Context<'_>) -> Poll<()>,
    /// Must not panic
    drop: fn(*mut ()),
}

struct SpawnableFuture {
    /// Must be `Send`able
    data: *mut (),
    vtable: FutureVtable,
}
unsafe impl Send for SpawnableFuture {}

impl Unpin for SpawnableFuture {}
impl Future for SpawnableFuture {
    type Output = ();

    // Call vtable.poll
}

impl Drop for SpawnableFuture {
    // Call vtable.drop
}

struct SpawnableLocalFuture {
    data: *mut (),
    vtable: FutureVtable,
}

impl Unpin for SpawnableLocalFuture {}
impl Future for SpawnableLocalFuture {
    type Output = ();

    // Call vtable.poll
}

impl Drop for SpawnableLocalFuture {
    // Call vtable.drop
}

struct FnOnceVtable {
    /// Must not panic
    call: fn(*mut ()),
    /// Must not panic
    drop: fn(*mut ()),
}

struct SpawnableFnOnce {
    /// Must be `Send`able
    data: *mut (),
    vtable: FnOnceVtable,
}
unsafe impl Send for SpawnableFnOnce {}

impl FnOnce for SpawnableFnOnce {
    // Call vtable.call
}

impl Drop for SpawnableFnOnce {
    // Call vtable.drop
}

ExecutorExt

use std::io;
use std::marker;
use std::num;

/// Extension trait for executor/reactor co-op, optional and experimental.
trait ExecutorExt: Executor {
    /// Can register multiple hanlder generator.
    ///
    /// On every thread, executor will call the generator to create
    /// a per-thread event handler.
    ///
    /// When there are multiple per-thread handlers, they will be
    /// called in arbitary order.
    ///
    /// # Safety
    ///
    /// event_handler_generator must live until it is unregistered or this
    /// executor is dropped.
    ///
    /// The event handler returned by the event_handler_generator must also
    /// live as long as event_handler_generator.
    unsafe fn register_event_handler(
        &self,
        event_handler_generator: EventHandlerGenerator,
    ) -> EventHandlerId<'_>;

    /// This will unregister the event handler generator and all event handler
    /// it generated.
    fn unregister_event_handler(
        &self,
        event_handler_id: EventHandlerId<'_>,
    ) -> EventHandlerGenerator;
}

struct EventHandlerGeneratorVtable {
    /// Must not panic
    generate_new_handle:
        fn(*const (), thread_id: std::thread::ThreadId) -> io::Result<EventHandler<'_>>,
    /// Must not panic
    drop: fn(*mut ()),
}
struct EventHandlerGenerator {
    /// Must implement `Send` and `Sync`
    data: *mut (),
    vtable: EventHandlerGeneratorVtable,
}
unsafe impl Send for EventHandlerGenerator {}
unsafe impl Sync for EventHandlerGenerator {}

impl Fn for EventHandlerGenerator {
    // Call vtable.generate_new_handle
}
impl Drop for EventHandlerGenerator {
    // Call vtable.drop
}

struct EventHandlerVtable {
    /// Must not panic
    handle: fn(*mut (), event: Event<'_>) -> io::Result<Action>,
    /// Must not panic
    drop: fn(*mut ()),
}
struct EventHandler<'event_handler_generator> {
    data: *mut (),
    vtable: EventHandlerVtable,
    phantom: PhantomData<&'event_handler_generator EventHandlerGenerator>,
}

impl FnMut for EventHandler {
    // Call vtable.handle
}
impl Drop for EventHandler {
    // Call vtable.drop
}

/// Opaque type
struct EventHandlerId<'executor>(u64, marker::PhantomData<&'a ()>);

enum Event<'a> {
    /// Future is executed and return `Poll::Pending`.
    FuturePending(&'a FutureStatistics),
    /// Future is executed and completed.
    FutureDone(FutureId),
    /// Tasks are migrated to current thread.
    TaskMigrated(num::NonZeroUsize),
    /// Tasks are taken away from current thread.
    TasksTakenAway(num::NonZeroUsize),
}

enum Action {
    /// Steal tasks from other threads.
    StealTask,
    /// Gift tasks to other threads.
    GiftTask,
    /// Continue without any action.
    Noop,
}

/// Must uniquely identify every future
#[derive(Copy, Clone, Debug)]
struct FutureId(u64);

/// Can obtain statistics like how long a future has been running,
/// how many times it yeilds with `Poll::Pending`.
struct FutureStatistics {
    future_id: FutureId,
    // ...
}

[Noah-Kennedy]

So I've taken some time to look through this again.

Regarding this approach, I don't think this gets us much.

First, starting with the spawn methods, having them be unsafe like this makes the whole abstraction useless. If the purpose is for library authors to be able to not care what runtime they are on, then it is self-defeating to make these methods unsafe and tied to whatever lifetimes the runtime supports, as the library authors don't know this information. These methods should simply require 'static to fit the base case that all runtimes support.

Second, I think we are taking the wrong approach here. Currently, we seem to be trying to define the behavior that runtimes should adopt in a very tight-fitting manner, and I don't think we actually solve any problems here. Library users don't care if there is work stealing, and they don't really benefit from an API that stubs out a reactor like this, as this is an implementation detail of the runtimes that should be encapsulated from them.

I think what we really need here is to have common traits for different types, and to have a factory object of some sort which allows us to construct types matching those traits via it's associated types. This is more similar to the initial proposal. This allows library authors to write code independent of the runtime they are on, so long as that runtime offers a factory that they can use. It would be quite simple actually from an authors perspective, especially as compared to the other options discussed.

This factory object could also provide an API for task spawning, but it should be as simple as spawn/spawn_local/spawn_blocking, and I think for those we should try and map what tokio and async-std do because that seems to be the common behavior across runtimes here.

[NobodyXu]

First, starting with the spawn methods, having them be unsafe like this makes the whole abstraction useless. If the purpose is for library authors to be able to not care what runtime they are on, then it is self-defeating to make these methods unsafe and tied to whatever lifetimes the runtime supports, as the library authors don't know this information. These methods should simply require 'static to fit the base case that all runtimes support.

The std library would provide wrappers over these unsafe primitives like std::task::yield_now, std::task::spawn, std::task::spawn_local, std::task::spawn_auto, std::task::spawn_blocking, std::task::try_block_in_place.

The std library would implement the JoinHandle that is portable to any runtime and the executor only needs to execute these tasks in any possible manner.

With the existing unsafe API, it is also possible to implement scoped spawning in std, though I don't mind making the exsiting spawning API safe and then add a new set of scoped spawning APIs to Executor.

Second, I think we are taking the wrong approach here. Currently, we seem to be trying to define the behavior that runtimes should adopt in a very tight-fitting manner, and I don't think we actually solve any problems here. Library users don't care if there is work stealing, and they don't really benefit from an API that stubs out a reactor like this, as this is an implementation detail of the runtimes that should be encapsulated from them.

I presume you are talking about ExecutorExt here.

It is not a trait exposed to ordinary users, but rather the reactor implementation so that they can register their own poller to the executor and co-op with any executor, inspired by @joshtriplett when discussing on zulip.

Of course, this is kind of my wet dream since its APIs are indeed too tight, so I will probably remove it for now.

I think what we really need here is to have common traits for different types, and to have a factory object of some sort which allows us to construct types matching those traits via it's associated types. This is more similar to the initial proposal. This allows library authors to write code independent of the runtime they are on, so long as that runtime offers a factory that they can use. It would be quite simple actually from an authors perspective, especially as compared to the other options discussed.

I actually changed that to this because of concerns raised by @joshtriplett .

The previous one uses generics and associated items (directly or through use of async in trait), so it is impossible to use it as a trait object and requires monomorphization in every async crate which might be too slow and prevents sharing the code if multiple reactors are used in the same binary somehow.

@joshtriplett also mentioned that I have the completion APIs packaged into the existing polling APIs, making it harder to change the completion APIs and add more APIs in the future.

With these concerns in mind, I decided to make the new reactor a low-level API that does not use generics or associated items at all, and split the completion and polling APIs into ReactorCompletion and ReactorPoller.

Then we can have a high-level API wrapping these lowlevel ReactorPoller/ReactorCompletion and it is actually simpler to implement.

For example, ReactorPoller only has 6 methods, which corresponds to tokio::io::unix::AsyncFd's methods.
The std library can use it to implement its own AsyncHandle, then uses that to implement TcpSocket, UnixSocket and etc using existing facilities in the std library.

That will remove a lot of duplicate code to deal with TcpSocket, UnixSocket and etc in tokio and async-std.

For ReactorCompletion, it models closely after io-uring's actual interface so it is also pretty straight-forward to implement for tokio-uring and then we will again provide high-level APIs similar to it.

The high-level APIs can use specialisation to detect presence of ReactorCompletion like this:

trait Reactor: ReactorPoller + Sealed {
    fn to_reactor_completion(&self) -> Option<&dyn ReactorCompletion> {
        None
    }
}
impl<T: ReactorPoller + ReactorCompletion + Sealed> Reactor for T {
    fn to_reactor_completion(&self) -> Option<&dyn ReactorCompletion> {
        Some(self)
    }
}

which can be used in generic functions or as a trait object, then we would have highlevel APIs like this:

struct TcpStream {
    inner: std::net::TcpStream,
    /// Registration for ReactorPoller
    registration: Registration,
}
impl TcpStream {
    pub async fn connect<A: ToSocketAddrs>(addr: A) -> Result<TcpStream>
        with reactor: &Reactor;
}

[uazu]

I guess you realize that there is not just tokio and async-std to consider. I want to implement this interface for Stakker, which is one-runtime-per-thread never-blocking actor-model runtime, with typically mio in the background taking care of waiting for lots of FDs/handles and waiting for the next timer expiry (although it could run on top of any other event loop). So "block_on" is problematic, and "spawn_blocking" too. Can I opt out of those interfaces? It will likely cause deadlocks.

If a crate that Stakker is hosting via this interface wants to spawn threads of its own, no problem, but apart from a cross-thread waker of some kind, the runtime can't have anything to do with what it's doing in other threads, since the entire runtime is non-Send/Sync. Okay, maybe not all crates will be able to exist within those limitations (i.e. maybe I can't host everything), but I think the vast majority of protocol stuff should be able to work at least.

Right now, the only stuff that's portable to Stakker is stuff that's very low-level, like embedded-websocket, i.e. something that does no I/O itself. Really a lot more crates could be written to be portable. So if some interface can be standardized, I will support it as far as is possible in a single-threaded runtime.

[NobodyXu]

So "block_on" is problematic

block_on is now implemented as try_enter_block_on and try_exit_block_on #13 (comment) .
You can just always return Err for these functions.

and "spawn_blocking" too

Why would spawn_blocking cause deadlock?
It is usually implemented using a blocking thread-pool that is separated from the thread-pool for executing futures.

the runtime can't have anything to do with what it's doing in other threads, since the entire runtime is non-Send/Sync

You can simply implement spawn using spawn_local.

And there will be an interface called spawn_auto that automatically spawns locally if the future does not implement Send so that users of the portable runtime doesn't have to implement these themselves.

[uazu]

Why would spawn_blocking cause deadlock? It is usually implemented using a blocking thread-pool that is separated from the thread-pool for executing futures.

Maybe I misunderstood the purpose of spawn_blocking. So it doesn't actually block. Yes, I can implement a thread-pool if that is what is required. I guess there are some additional complexities to handle, e.g. non-blocking I/O through this interface from these tasks needs a reactor in those threads (potentially?), also they may spawn normal tasks (back in the runtime thread?). The whole thing needs thinking through.

You can simply implement spawn using spawn_local. And there will be an interface called spawn_auto that automatically spawns locally if the future does not implement Send so that users of the portable runtime doesn't have to implement these themselves.

Okay. I don't think Stakker will be the only single-threaded runtime for Rust (I think there are others already), so ideally things should be arranged so that the base API supports single-threaded, and people coding against this interface will understand that they are limiting their portability if they use certain additional methods. If everyone uses 'spawn' by default then straight away a big chunk of the ecosystem is cut off. It's not just this kind of shard-per-thread runtime approach, but also embedded/etc where maybe there aren't even threads.

Whenever things have settled down a bit I will try to implement the interface.

[uazu]

You can simply implement spawn using spawn_local

Okay, got it -- so 'spawn' won't be a problem.

[NobodyXu]

Maybe I misunderstood the purpose of spawn_blocking.

It is for blocking io, e.g. doing file io on a polling runtime.

I guess there are some additional complexities to handle, e.g. non-blocking I/O through this interface from these tasks needs a reactor in those threads (potentially?), also they may spawn normal tasks (back in the runtime thread?). The whole thing needs thinking through.

If your reactor does not implement Send, then the users will not be able to use the reactor in these blocking threads.
And I agree that the whole things still need to be worked on.

If everyone uses 'spawn' by default then straight away a big chunk of the ecosystem is cut off.

I plan to let them use spawn_auto, which spawns on local if they do not implement Send.

Okay, got it -- so 'spawn' won't be a problem.

You got it, spawn only enables the executor to run the tasks on any thread and move it between them, but you can still run them on the current thread.

[Noah-Kennedy]

I think the whole idea of spawn_auto is problematic. On many runtimes, you cannot spawn a local task from any context, so implicitly falling back to local spawning if the future is !Send is a significant footgun, and prevents this method from being used in the manner it seems intended, which is as a default usage pattern for libraries. Ultimately, we should try to make it so that libraries don't need to be aware of runtime details any more than absolutely necessary. There might be ways around this, and I have a few ideas here.

We do need a way of spawning that works for both runtimes where tasks are not sendable (as there are advantages to "isolated" thread-per-core runtimes), as well as work-stealing runtimes. @uazu brought up a really great point here, and we need to try and figure out a solution.

Give me some time, and I can get a proof-of-concept of my ideas hacked out later today.

[Noah-Kennedy]

Also, thinking on this more, we do not need block_on. This is an implementation detail that doesn't really serve a use case. We really just need a system for constructing the basic IO types, and doing spawning.

Also, another note since this ticket is getting heated at times:

Folks, let's please keep in mind that the problem we need to address is that library authors and others seeking to write systems that can work on multiple runtimes are unable to do so easily, due to the coupling to a runtime's spawn method primarily, and secondarily due to the need to couple to a runtime's specific IO types. We don't need to solve this by crafting a framework that tries to abstract over every implementation detail of every runtime. We just need to look at the minimum set of abstractions capable of making this easier, and make sure that these abstractions are broadly capable of being met by different classes of runtimes.

[joshtriplett]

@Noah-Kennedy We do need a std::task::block_on though.

[Noah-Kennedy]

@joshtriplett why? What use case does it help with?

[joshtriplett]

@Noah-Kennedy "I'm in synchronous code, and I need to await a future; I need some way to switch from sync mode to async mode." (The reverse of the spawn_blocking use case.)

[Noah-Kennedy]

How common is this for a library to need to do though? I cannot think of an actual time to use this. I am trying to weigh this against the fact that not all runtimes can do this (a lot of thread-per-core designs can't).

[joshtriplett]

@Noah-Kennedy Also, sorry, I responded quickly to that one point without acknowledging and appreciating the rest of your message:

Folks, let's please keep in mind that the problem we need to address is that library authors and others seeking to write systems that can work on multiple runtimes are unable to do so easily, due to the coupling to a runtime's spawn method primarily
...
We don't need to solve this by crafting a framework that tries to abstract over every implementation detail of every runtime. We just need to look at the minimum set of abstractions capable of making this easier

I really appreciate and agree with this point, and thank you for expressing it.

[joshtriplett]

How common is this for a library to need to do though? I cannot think of an actual time to use this.

I've found it quite common in application code, extremely common in documentation/example/test code, and moderately common in library code (notably in library code that isn't 100% inside the async world).

I don't think either spawn_blocking or block_on can always-return-error, because that will break quite a bit of extant code. I would much rather have an interface that guarantees support for both of those, even if implementing them is easier on some runtimes than others.

And I am trying to weigh this against the fact that not all runtimes can do this (a lot of thread-per-core designs can't).

As far as I can tell, any design can, but it may have to go through more steps to do so. I would like to understand better how a runtime couldn't, rather than just that it doesn't naturally fit into its model and requires an extra layer. Just as spawn_blocking likely requires a blocking thread pool, I would expect block_on to require some machinery that may be trivial for some runtimes but may be more complex for others.

I am hoping that the executor trait can serve as a backend for std::task::block_on, std:task::spawn, and std::task::spawn_blocking, the trio of which would serve as the primary interface to an executor. (I would also expect, based on our other conversations, to need some minimal reactor interface for saying "here are my waitable objects in case you want to wait on them in some way other than dedicated threads", though I personally think it might be appropriate to have a default implementation for those methods.)

[uazu]

As far as I can tell, any design can, but it may have to go through more steps to do so. I would like to understand better how a runtime couldn't

In Stakker there are two contexts: outside the main loop, which is synchronous, and stuff called from the main loop (actor methods and future polling), which can never block. So how can I implement block_on if it is called from code called by the main loop? I cannot block at this point. Maybe if it was block_on(...).await then that would be non-blocking, but then that's just the same as .await.

Outside the main loop, I guess it could be used to start a standard main loop. Yes, that makes sense if that's going to be the standard way to start a new runtime. (I'll have to select what kind of main loop and what's pulled in with cargo features I guess, but that's my problem.)

Maybe the key is to have one interface for "running outside the runtime" which includes block_on, and another interface for "running inside the runtime" which doesn't.

[Noah-Kennedy]

@joshtriplett There are a lot of different designs for what I am going to refer to as "isolated worker" thread-per-core runtimes, so I'm going to generalize a bit.

A lot of "isolated worker" thread-per-core designs try to basically model all of the worker threads as parallel systems, allowing for a lot less synchronization overall. This can be a very effective paradigm for server applications in conjunction with things like SO_REUSEPORT, and this is a very common pattern for custom runtimes written for a specific performance use cases or categories of workloads as a result.

Generally the setup process for these runtimes involves first spawning a bunch of threads pinned to different CPU cores, so block_on doesn't really work here; you can't enter the runtime context from outside the runtime to drive a particular future in the current thread, as the current thread you launch the runtime from isn't a part of the runtime context. Many of these designs also involve a factory pattern for initialization, as they tend to be custom-built for contexts where you are running a server on every worker, using SO_REUSEPORT to balance incoming connections between them. Spawning a task from within the runtime context involves allocating a new task on the worker and then inserting the task into the workers local queue (no synchronization required).

With these designs you can support spawning from outside the runtime context quite easily via a global queue or other means (not going into this, there are lots of weird designs though). Generally, these designs are optimized under the assumption that spawning from outside the runtime is less common than spawning from inside the runtime, so the latter path is much faster.

These runtimes may seem a bit odd for those of us coming from async-std or tokio, but this is actually a well-established pattern that has considerable benefits for many applications, and which I suspect may become more common in the rust space due to the fact that it works really well with io_uring. I'm trying to keep these designs in mind wrt our proposals here.

[joshtriplett]

@Noah-Kennedy If you can support spawning from outside the runtime context, in theory you could implement block_on as "spawn this future, make it signal a channel on completion, then synchronously wait on that channel". And block_on doesn't have to be the most optimized path, it just needs to work.

@uazu Generally speaking, some runtimes do have restrictions that you can't call block_on recursively from inside a future running on the runtime, only from code running outside any future or code in a spawn_blocking closure. We could document that as a restriction that applies to some runtime implementations.

[NobodyXu]

@uazu Generally speaking, some runtimes do have restrictions that you can't call block_on recursively from inside a future running on the runtime, only from code running any future or code in a spawn_blocking closure. We could document that as a restriction that applies to some runtime implementations.

Maybe we can make this more explicit, by stating that spawning/block_on can only be used when any type that implements the Executor trait is available as a context parameter?

[joshtriplett]

@NobodyXu I don't think that would work; I would expect spawning (and the executor context) to be available everywhere, while block_on may be limited to contexts that aren't in a future (but the executor should still be available for spawning).

[NobodyXu]

I don't think that would work; I would expect spawning (and the executor context) to be available everywhere

Yeah, I also expect the executor context to available on any future and inside the spawn_blocking closure.

For reactor, this is more complex as it might not implement Send if the rutime wants to have a reactor per thread design.

while block_on may be limited to contexts that aren't in a future (but the executor should still be available for spawning).

Not sure how to ensure we are not in a future though, I wish there is such mechanism in the language to express this because now several functions in tokio panic at runtime, which is IMO a footgun.

[joshtriplett]

I agree that it's a footgun, and it's nice when implementations avoid that footgun, but despite that, block_on is still useful and important.

[uazu]

For reactor, this is more complex as it might not implement Send if the rutime wants to have a reactor per thread design.

From Stakker's point of view, if I implement block_on, and it is running inside the runtime, I will panic. The only other alternative would be to spawn the future, but that probably also breaks the user's expectations.

If I implement spawn_blocking, it is probably going to have to be under a completely different executor and reactor in that other thread. So I can follow whatever rules typically apply, like "allow block_on" or whatever.

while block_on may be limited to contexts that aren't in a future (but the executor should still be available for spawning).

Not sure how to ensure we are not in a future though, I wish there is such mechanism in the language to express this because now several functions in tokio panic at runtime, which is IMO a footgun.

There is such a mechanism, but you have to buy into it totally. This is how Stakker works. There are borrows that are passed down the stack, and if you don't have that borrow, then you don't have access to a whole bunch of stuff. This is all tied into QCell and friends (similar to GhostCell). It also means that Stakker only ever runs a (relatively) shallow stack before returning to the main loop, because if you get any deeper you lose the important borrows. I wrote this all up HERE. Unfortunately for futures, there is no way to pass the borrow through the poll call, because nobody thought of that. Or maybe it could be put into the Context but it needs someone high up to think it all through. Generators I think will eventually have support for passing borrows into a resume, which become unavailable again at the next yield -- some day anyway.

Edit: Specifically, it's THIS PAGE that explains how the borrowing works in Stakker.

[uazu]

@uazu Generally speaking, some runtimes do have restrictions that you can't call block_on recursively from inside a future running on the runtime, only from code running outside any future or code in a spawn_blocking closure. We could document that as a restriction that applies to some runtime implementations.

Yes, that sounds ideal. Perhaps we can have a way to express levels of support, e.g. Stakker might support level 2, and tokio/async_std level 4, or whatever. Then if a crate claims to only require level 1, then it's clear which runtimes it can run on. (Or maybe expressed as a required-feature list.)

[NobodyXu]

Generators I think will eventually have support for passing borrows into a resume, which become unavailable again at the next yield -- some day anyway.

That will be an effect system, I'm not sure if rust wants to support that.

Perhaps we could work with the keyword generics wg to see if they have any idea to solve this?

P.S. @uazu Thanks for the links, using QhostCell sounds interesting and I will have a read at the links you provided.

[NobodyXu]

Or maybe it could be put into the Context but it needs someone high up to think it all through.

Note that this will not work, since users can come up with their own context.

For example, the crate futures has its own context type so passing the token in context just wouldn't work.

@uazu Perhaps you can also use the "with context" which is planned to be used to pass around executor/reactor to pass around your token?

[NobodyXu]

I think this blog post has proposed a more elegant solution:

Instead of creating Reactor traits, it proposes extending Context to allow blocking callback to be added if not already set.

This would avoid the global variables, the Reactor traits and need of context parameters.

But if some library crates need to abstract over functionalities provided by async runtime reactors, they would still need the reactor traits.

[Noah-Kennedy]

I like the idea of extending context