Idea for an implementation of delayed shutdown for e.g applications running in Kubernetes

[Jakobhenningjensen]

I have seen several issues/question on e.g StackOverflow regarding "gracefull shutdown" in FastAPI/uvicorn, and I had the same issue. While we have the timeout_graceful_shutdown there is a a problem in e.g Kubernetes. When we are scaling number of pods/instances down, we are sending a SIGTERM to the pod, but the load-balancer keep sending requests until the health/ready does not respond status 200. Say the load-balancer pings health/ready every 5 seconds until it gets a non-200-status, then the app could still get requests for 5 seconds after the SIGTERM has been received meaning that all requests in that 5 sec interval would fail.

By adding a delayed shutdown (example of a PR here Jakobhenningjensen#1) we simply delay the shutdown of the app by a given number of seconds, while setting a flag in the serving app (uvicorn_shutdown_triggered). The app can then use this property to check, if a health-point should return 200 or something else to notify the load-balancer that the app is currently under shutdown, while still accepting requests. After the delayed_shutdown time has passed the app shuts down as it normally would.

Example:

from fastapi import FastAPI
import asyncio

app = FastAPI()
app.uvicorn_shutdown_triggered = False

@app.get("/health/ready", response_class=PlainTextResponse)
def health_ready():
    if app.uvicorn_shutdown_triggered:
        return PlainTextResponse("Under shutdown", 425)
    return PlainTextResponse("OK", 200)


@app.post("/do_some_stuff", response_class=PlainTextResponse)
def do_some_stuff(body):
    await asyncio.sleep(20) #simulate a long request


if __name__=="__main__":
    import uvicorn
    uvicorn.run(app=app, host=host, port=port, log_config=None, timeout_graceful_shutdown=30, shutdown_delay=5)

Now running the app and send a SIGTERM we still have 5 seconds where the app can accept requests, but the health/ready would now respond 425 instead of 200 telling a load-balancer that "we are currently under shutdown".

Setting the uvicorn_shutdown_triggered automatically could (in theory) overwrite an attribute with that name in the app which is used for something else.
One could add another argument to Config e.g shutdown_flag_attr_name="uvicorn_shutdown_triggered" where this is the attribute that would be set inapp, giving some of the control to the app-user, but I don't know if that is a concern.