Using uvloop instead of asyncio leads to grouping of responses to multiple requests/messages

[SerodioJ]

Initial Checks

• I confirm this was discussed, and the maintainers suggest I open an issue.
• I'm aware that if I created this issue without a discussion, it may be closed without a response.

Discussion Link

I am opening the issue because another user commented on the opened discussion thread, and no maintainer interacted with the thread.

https://github.com/encode/uvicorn/discussions/1367

Description

When running a FastAPI app with uvicorn default settings (--loop auto, which uses uvloop when installed), I started to notice that sometimes when handling multiple requests for the same path the response took longer than expected to arrive at the client. This issue became more noticeable when I tried using WebSockets to improve performance by returning the result in parts over the same connection instead of making multiple HTTP requests.

See discussion for more details.

Is this behavior expected when using the default event loop?

Example Code

FastAPI App Code

import time

from fastapi import FastAPI, WebSocket

app = FastAPI()

@app.websocket("/ws")
async def websocket_hadler(websocket: WebSocket):
    await websocket.accept()
    title = await websocket.receive_text()
    start = time.time()
    await websocket.send_json(
        {"id": title, "status": "start", "time": time.time() - start}
    )
    for i in range(3):
        time.sleep(2)
        await websocket.send_json(
            {
                "id": title,
                "part": i,
                "time": time.time() - start,
            }
        )
    await websocket.send_json({"id": title, "status": "end", "time": time.time() - start})
    await websocket.close()

@app.get("/slow")
async def slow_endpoint():
    start = time.time()
    time.sleep(3)
    return {"time": time.time() - start}

HTTP Client Code

import requests
import time
from threading import Thread
from multiprocessing import Process

def req(exec_id):
   start = time.time()
   response = requests.get(f"http://localhost:8000/slow")
   end = time.time()
   print({
       "id": exec_id,
       "client": end-start,
       "server": response.json()
   })

print("Threads")
for i in range(4):
   t = Thread(target=req, args=(i,))
   t.start()

# print("Process")
# for i in range(4):
#     p = Process(target=req, args=(i,))
#     p.start()

WebSocket Client Code

import time
import json
from websockets.sync.client import connect

with connect("ws://localhost:8000/ws") as websocket:
   websocket.send("uvloop") # or "asyncio"
   start = time.time()
   t = {}
   while t.get("status") != "end":
       t = json.loads(websocket.recv())
       print(t)
       print("----------------------------")
       print(f"Client Time: {time.time() - start}")
       print("#############################")

Python, Uvicorn & OS Version

Running uvicorn 0.30.6 with CPython 3.12.5 on Linux

[Kludex]

The issue is with uvloop itself, not with uvicorn. I don't know why, but it looks like uvloop is not switching tasks properly... If you do await anyio.sleep(0) it will force the event loop to switch tasks, and then you can have the behavior you want.

Please open an issue with a MRE on uvloop. This MRE was too big for uvicorn itself.

[Kludex]

@graingert do you know something about this?

[graingert]

I'll look into it

[graingert]

I've repeated this with plain websockets (mre incoming) will look at repeating with streams

[Kludex]

Thanks Thomas. 🙏❤️