anyio integration

[uSpike]

Fixes: #811

This adds support for asyncio and trio by leveraging anyio and a little more structured concurrency in spots.

Things to do yet:

• 100% tests passing (there's one failing test at the moment)
• TestClient rewritten for anyio, probably with portals.
• anyio 3.0 release

This adds a hard dependency for starlette, which didn't exist before, and was a selling point. In my opinion, the benefit is worth it. It would be great to use starlette with either uvicorn or hypercorn using Trio.

This also removes the need for aiofiles which is gained for free with anyio.

Some drawbacks are that the other integrations such as graphql are only currently compatible with asyncio. But, this does not make those integrations incompatible. It does give users the ability to use trio without them, and maybe they can be ported in the future as well.

I created a branch for framework benchmarks here: https://github.com/uSpike/FrameworkBenchmarks/tree/starlette-anyio

See results: https://www.techempower.com/benchmarks/#section=test&shareid=c92a1338-0058-486c-9e47-c92ec4710b6a&hw=ph&test=query&a=2

There is a performance penalty to using anyio. The weighted composite score is 173 vs 165, so a 4.5% drop in performance.

[uSpike]

There's a weird bit of code in starlette/requests.py:

            async with anyio.move_on_after(
                0.001
            ):  # XXX: too small of a deadline and this blocks
                message = await self._receive()

The existing code used asyncio.wait_for(..., 0.00000001) or some really small number. I found that with anyio.move_on_after I needed a larger number for the deadline or else it would block. I'm not sure of a better way to handle that.

EDIT: refactored to an event in 4dd8c5d

[agronholm]

Is there/does there need to be an anyio equivalent to asyncio.iscoroutinefunction()?

Not really, no. Trio does not do any tricks in that regard. As I understand, starting with Python 3.10 even asyncio doesn't need that.

[uSpike]

03e312e added a chunk of work on testclient. On my machine, the tests finish but hang with a number of threads blocking. Not sure what's up there...

EDIT: not a problem anymore

[agronholm]

My recommendation is to base this work on AnyIO 3.0, of which the first RC release is imminent. Since this is a new feature, I don't think backwards compatibility is necessary.

[agronholm]

On that note, the 3.0.0rc1 pre-release is out now.

[agronholm]

These should get you going on AnyIO 3. I had at least one question among the suggestions too.

[uSpike]

I've pushed changes to use anyio 3.0.0rc1, thanks @agronholm !

There's some failing tests left:

• test_state_data_across_multiple_middlewares
• test_wsgi_get
• test_wsgi_post
• test_wsgi_exc_info

Those tests seem to not be getting data in response.text from Testclient.get()

There's another test I've had to skip because it blocks forever: test_graphql_async_old_style_executor, need to investigate that one a bit.

[agronholm]

Those test failures don't look too bad. Let me know if you need more help. You can catch me on Gitter.

[uSpike]

starlette/graphql.py:GraphQLApp.__init__ states:

        else:
            # Old style. Use 'executor'.
            # We should remove this in the next median/major version bump.
            self.executor = executor
            self.executor_class = None
            self.is_async = isinstance(executor, AsyncioExecutor)

The test for this is blocking forever. I'd rather not spend too much effort debugging that, since it's slated to be removed. I will remove it assuming that this PR would bring about a median/major version bump.

[JayH5]

I think it's time 🚀

[uSpike]

Cheers @JayH5 and thank you everyone for your help/work on this. I think this is an important step in bringing structured concurrency to the mainstream and I'm grateful for the opportunity to contribute. ❤️

[agronholm]

Just one question: how soon can we expect to see this in a release?

[graingert]

@agronholm I think it's all but released now: #1202