Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add delayed shutdown #1

Merged
merged 1 commit into from
Jun 28, 2024
Merged

Add delayed shutdown #1

merged 1 commit into from
Jun 28, 2024

Conversation

Jakobhenningjensen
Copy link
Owner

@Jakobhenningjensen Jakobhenningjensen commented Jun 28, 2024

I have seen several issues/question on e.g StackOverflow regarding "gracefull shutdown" in FastAPI/uvicorn, and I had the same issue. While we have the timeout_graceful_shutdown there is a a problem in e.g Kubernetes. When we are scaling number of pods/instances down, we are sending a SIGTERM to the pod, but the load-balancer keep sending requests until the health/ready does not respond status 200. Say the load-balancer pings health/ready every 5 seconds until it gets a non-200-status, then the app could still get requests for 5 seconds after the SIGTERM has been received meaning that all requests in that 5 sec interval would fail.

By adding a delayed shutdown we simply delay the shutdown of the app by a given number of seconds, while setting a flag in the serving app (uvicorn_shutdown_triggered). The app can then use this feature to check if a health-point should return 200 or something else to notify the load-balancer that the app is currently under shutdown, while still accepting requests. After the delayed_shutdown time has passed the app shuts down as it normally would.

Example:

from fastapi import FastAPI
import asyncio

app = App()
app.uvicorn_shutdown_triggered = False

@app.get("/health/ready", response_class=PlainTextResponse)
def health_ready():
    if app.uvicorn_shutdown_triggered:
        return PlainTextResponse("Under shutdown", 425)
    return PlainTextResponse("OK", 200)


@app.post("/do_some_stuff", response_class=PlainTextResponse)
def do_some_stuff(body):
    await asyncio.sleep(20) #simulate a long request


if __name__=="__main__":
    import uvicorn
    uvicorn.run(app=app, host=host, port=port, log_config=None, timeout_graceful_shutdown=30, shutdown_delay=5)

Now running the app and send a SIGTERM we still have 5 seconds where the app can accept requests, but the health/ready would now respond 425 instead of 200 telling a load-balancer that "we are currently under shutdown".

Setting the uvicorn_shutdown_triggered automatically could (in theory) overwrite an attribute with that name in the app which is used for something else.
One could add another argument to Config e.g shutdow_flag_attr_name="uvicorn_shutdown_triggered" where this is the attribute that would be set inapp, giving some of the control to the app-user, but I don't know if that is a concern.

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
In e.g Kubernetes we might want to delay the shutdown when pods are scaling down.
@Jakobhenningjensen Jakobhenningjensen merged commit c2a3448 into master Jun 28, 2024
@Jakobhenningjensen Jakobhenningjensen deleted the add_shutdown_delay branch July 2, 2024 11:13
@Jakobhenningjensen Jakobhenningjensen restored the add_shutdown_delay branch November 11, 2024 15:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants