Synchronous connection interface

[tomchristie]

For working from sync test cases, or for working from the console, it'd be useful to be able to acquire a synchronous interface onto a connection.

Eg.

database = Database(DATABASE_URL)

connection = database.sync_connection()
connection.execute(...)

Or...

@pytest.fixture
def connection():
    return database.sync_connection()

def test_something(client, database):
    client.post("/users", json=...)
    assert database.fetch_all(...) = ...

[woile]

I'm having some kind of issue related to this, I don't know if it fits as a bug, but I'm doing this:

def test_app():
    query = users.insert()
    values = {
        "username": "woile",
    }
    database.execute(query, values)

    client = TestClient(application.app)
    response = client.get("/users")
    assert ..something...

But, because database is async, it's expecting an await, so a warning is raised when running the tests. And it doesn't execute the query.

On the other hand if I do this:

@pytest.mark.asyncio
async def test_app():
    query = users.insert()
    values = {
        "username": "woile",
    }
    await database.execute(query, values)

    client = TestClient(application.app)
    response = client.get("/users")
    assert ...something...

fails because TestClient is already running an event loop.

RuntimeError: This event loop is already running

What is the best approach to follow in this case? Note: My asyncio knowledge is small.

[woile]

Update

I found a way to execute database synchronous for now:

import asyncio

def sync(coroutine):
    event_loop = None
    try:
        event_loop = asyncio.get_event_loop()
    except RuntimeError:
        event_loop = asyncio.new_event_loop()
        asyncio.set_event_loop(event_loop)
    return event_loop.run_until_complete(coroutine)

def test_app():
    query = users.insert()
    values = {
        "username": "woile",
    }
    sync(database.execute(query, values))

    client = TestClient(application.app)
    response = client.get("/users")
    assert ..something...

source

[gvbgduh]

asgiref provides some base implementations like https://github.com/django/asgiref/blob/master/asgiref/sync.py
Shall we consider re-using this implementation for providing sync interfaces?

[tomchristie]

The other way around for us to approach this would be to provide an async test client.

Then we can just use pytest.mark.asyncio and use async test cases all the way through, with async requests and async database queries.

This https://github.com/encode/requests-async gets us a lot closer to doing that, since we've got a package that can either be used for making outgoing async HTTP requests, or for an async test client, or plugging into a stub service to mock out network requests. (eg. when you're running the test suite, you probably want to stub out any external network requests that your service makes.)

[gvbgduh]

Yes, async test client sounds just right! I think I can make an initial draft for this based on the requests_async.ASGISession, as it seems to be working. Probably within starlette as it has to replicate the sync test client.

[jordic]

we solved all of these issues with our async test client. We have a fixture that yields the client and the app. Also we register the database as a zca IDatabase utility. and this makes so easy to swap, mock or whatever a singleton throught all the app.
ZCA: zope component architecture. (the zca is old but still powerful, pyramid is based on it.
Will try to provide a gist with our app factory

[taybin]

@jordic Any progress on publishing it? I've run into the same thing.

[taybin]

@gvbgduh I'm using your draft for AsyncTestClient and so far, so good. I'd love to see it get merged into Starlette.

[tomplex]

@taybin I found this which helped me get all of my tests to pass with relatively minimal pain.

[tomchristie]

@taybin - See httpx for an async test client, potentially also with the LifespanManager from https://github.com/florimondmanca/asgi-lifespan. See encode/httpx#350 (comment) for an example.

Will need to update Starlette to start using this at some point.