Using uvloop instead of asyncio leads to weird behavior #1367
-
|
When running a FastAPI app with uvicorn default settings (--loop auto, which uses uvloop when installed), I started to notice that sometimes when handling multiple requests for the same path the response took longer than expected to arrive at the client. This issue became more noticeable when I tried using WebSockets to improve performance by returning the result in parts over the same connection instead of making multiple HTTP requests. I made this mini FastAPI app that represents the situation of my app and used a Jupyter Notebook to run both the WebSocket client and HTTP Client. import time
from fastapi import FastAPI, WebSocket
app = FastAPI()
@app.websocket("/ws")
async def websocket_hadler(websocket: WebSocket):
await websocket.accept()
title = await websocket.receive_text()
start = time.time()
await websocket.send_json(
{"id": title, "status": "start", "time": time.time() - start}
)
for i in range(3):
time.sleep(2)
await websocket.send_json(
{
"id": title,
"part": i,
"time": time.time() - start,
}
)
await websocket.send_json({"id": title, "status": "end", "time": time.time() - start})
await websocket.close()
@app.get("/slow")
async def slow_endpoint():
start = time.time()
time.sleep(3)
return {"time": time.time() - start}The client code and the results obtained using uvicorn app:app --port 8000 (defaults to --loop auto, that selects uvloop) and uvicorn app:app --port 8000 --loop asyncio to run the app are listed below: WebSocketfrom websocket import create_connection
import time
import json
ws = create_connection("ws://localhost:8000/ws")
ws.send("uvloop") # or "asyncio"
start = time.time()
t = {}
while t.get("status") != "end":
t = json.loads(ws.recv())
print(t)
print("----------------------------")
print(f"Client Time: {time.time() - start}")
print("#############################")
ws.close()uvloopasyncioRest APIUsing multiprocessing and threading presented similar results. import requests
import time
from threading import Thread
from multiprocessing import Process
def req(exec_id):
start = time.time()
response = requests.get(f"http://localhost:8000/slow")
end = time.time()
print({
"id": exec_id,
"client": end-start,
"server": response.json()
})
print("Threads")
for i in range(4):
t = Thread(target=req, args=(i,))
t.start()
# print("Process")
# for i in range(4):
# p = Process(target=req, args=(i,))
# p.start()uvloopasyncioEnvironmentI am using Ubuntu 20.04, and a Python 3.7.11 environment created via conda (4.10.3). Questions
|
Beta Was this translation helpful? Give feedback.
Replies: 5 comments 4 replies
-
|
I'm not sure I understand the issue here seems like uvloop behaves the same as asyncio in both case, 6s client time in /ws and 3s for /slow endpoint. What am I missing you want to tell ? |
Beta Was this translation helpful? Give feedback.
-
|
For /ws when running with uvloop all messages arrive in 6s, even though some are already "ready" in 2s, 4s (server side). With asyncio, these messages arrive in 2s, 4s and 6s (client side), that would be the desired behavior. |
Beta Was this translation helpful? Give feedback.
-
|
note that you're blocking with time.sleep, is that normal ? |
Beta Was this translation helpful? Give feedback.
-
|
Yes, the places that I use time.sleep correspond to places I make blocking function calls that take a while to complete. I replaced them with time.sleep for simplification, but the time results obtained are similar. |
Beta Was this translation helpful? Give feedback.
-
|
Nobody seems to have answered this, does anyone know if this is still happening and if so, why? |
Beta Was this translation helpful? Give feedback.
-
|
I can test with a fresh install (FastAPI + dependencies) to see if it's still happening. |
Beta Was this translation helpful? Give feedback.
-
|
@Doch88 I tested with a fresh installation using a Python 3.12 environment and this behavior still persists. CodeFastAPI App Code import time
from fastapi import FastAPI, WebSocket
app = FastAPI()
@app.websocket("/ws")
async def websocket_hadler(websocket: WebSocket):
await websocket.accept()
title = await websocket.receive_text()
start = time.time()
await websocket.send_json(
{"id": title, "status": "start", "time": time.time() - start}
)
for i in range(3):
time.sleep(2)
await websocket.send_json(
{
"id": title,
"part": i,
"time": time.time() - start,
}
)
await websocket.send_json({"id": title, "status": "end", "time": time.time() - start})
await websocket.close()
@app.get("/slow")
async def slow_endpoint():
start = time.time()
time.sleep(3)
return {"time": time.time() - start}HTTP Client Code import requests
import time
from threading import Thread
from multiprocessing import Process
def req(exec_id):
start = time.time()
response = requests.get(f"http://localhost:8000/slow")
end = time.time()
print({
"id": exec_id,
"client": end-start,
"server": response.json()
})
print("Threads")
for i in range(4):
t = Thread(target=req, args=(i,))
t.start()
# print("Process")
# for i in range(4):
# p = Process(target=req, args=(i,))
# p.start()WebSocket Client Code import time
import json
from websockets.sync.client import connect
with connect("ws://localhost:8000/ws") as websocket:
websocket.send("uvloop") # or "asyncio"
start = time.time()
t = {}
while t.get("status") != "end":
t = json.loads(websocket.recv())
print(t)
print("----------------------------")
print(f"Client Time: {time.time() - start}")
print("#############################")ResultsuvloopHTTP (some times the times are OK -> 3, 6, 9 and 12) WebSocket asyncioHTTP (the grouping behavior was not observed in any attempt) WebSocket EnvironmentI am using Ubuntu 22.04 and a Python 3.12.5 environment created via conda (4.10.3). annotated-types==0.7.0
anyio==4.5.0
certifi==2024.8.30
charset-normalizer==3.3.2
click==8.1.7
dnspython==2.6.1
email_validator==2.2.0
fastapi==0.115.0
fastapi-cli==0.0.5
gevent==24.2.1
greenlet==3.1.0
h11==0.14.0
httpcore==1.0.5
httptools==0.6.1
httpx==0.27.2
idna==3.10
Jinja2==3.1.4
markdown-it-py==3.0.0
MarkupSafe==2.1.5
mdurl==0.1.2
pydantic==2.9.2
pydantic_core==2.23.4
Pygments==2.18.0
python-dotenv==1.0.1
python-multipart==0.0.9
PyYAML==6.0.2
requests==2.32.3
rich==13.8.1
setuptools==75.1.0
shellingham==1.5.4
sniffio==1.3.1
starlette==0.38.5
typer==0.12.5
typing_extensions==4.12.2
urllib3==2.2.3
uvicorn==0.30.6
uvloop==0.20.0
watchfiles==0.24.0
websocket==0.2.1
websockets==13.0.1
wheel==0.44.0
zope.event==5.0
zope.interface==7.0.3 |
Beta Was this translation helpful? Give feedback.
-
|
I just opened an Issue #2464 about it. |
Beta Was this translation helpful? Give feedback.
-
|
Replied on #2464 (comment). |
Beta Was this translation helpful? Give feedback.
Replied on #2464 (comment).