Streaming Response suppresses errors

[RamiAwar]

When using Streaming response, there is an underlying assumption that the generator (for the stream) will not encounter any errors.

In case it does, nothing gets sent back to the client, and an unhandled Runtime error occurs server-side.

From starlette/responses.py - line 221:

    async def stream_response(self, send: Send) -> None:
        await send(
            {
                "type": "http.response.start",
                "status": self.status_code,
                "headers": self.raw_headers,
            }
        )
        async for chunk in self.body_iterator:
            if not isinstance(chunk, bytes):
                chunk = chunk.encode(self.charset)
            await send({"type": "http.response.body", "body": chunk, "more_body": True})

        await send({"type": "http.response.body", "body": b"", "more_body": False})

Notice how we're simply iterating over body_iterator, not handling errors anywhere.

I've been able to kind of fix this by changing the code to the following:

   async def stream_response(self, send: Send) -> None:
        await send(
            {
                "type": "http.response.start",
                "status": self.status_code,
                "headers": self.raw_headers,
            }
        )
        try:
            async for chunk in self.body_iterator:
                if not isinstance(chunk, bytes):
                    chunk = chunk.encode(self.charset)
                await send({"type": "http.response.body", "body": chunk, "more_body": True})
        except Exception as e:
            logging.getLogger("main").error(e)
            await send({"type": "http.response.body", "body": b"", "more_body": False})
        else:
            await send({"type": "http.response.body", "body": b"", "more_body": False})

Now at least I see the error being logged.

Should I submit a PR for this?
If so, there are a few questions I have:

1- How do you log errors in Starlette normally?

2- How can we communicate this error back to the client?
I found http Trailer headers to be an option, but couldn't get them to work since the python requests library ignores them as part of r.iter_content which is a very common way of consuming streams.

Does it make sense to populate the body with the error? Given that the client is expecting specific chunk formats, seems annoying to throw the responsibility of checking body for errors on them maybe?

[jhominal]

1. If you are using the Starlette class as an ASGI app, then the middlewares ExceptionMiddleware and ServerErrorMiddleware would handle the error:
   • ExceptionMiddleware would usually transform the response into a 500 response - however, as the response has already started, ExceptionMiddleware then raises a RuntimeError instead;
   • ServerErrorMiddleware would also usually transform the response into a 500 response - but again, as the response has already started, that response is not sent. Then, ServerErrorMiddleware raises the error again, where it can be logged by the server;
   • When instantiating Starlette, you can pass exception_handlers that would be tasked with handling errors raised by ServerErrorMiddleware;
2. While the idea of using trailers could seem like a reasonable idea, they are not viable for this purpose:
   • the HTTP client must advertise support for trailers before they can be used - as far as I know, very few HTTP clients actually support that feature;
   • trailers in HTTP/1.1 can only be used if sending back a chunked response - that is, without Content-Length - but even a StreamingResponse may have an explicit Content-Length;
   • there is no technical support for sending trailers in the ASGI specification AFAIK (a new message type would be needed) - at best, it would be an extension to the current specification;
   • there is no semantic support in HTTP trailers to tell a HTTP client "actually, this response I began sending you is actually in error. Sorry!"

It does not make sense to populate the body with an error - the client would not know how to check for an error, any already sent Content-Length header would become invalid, etc. Basically, I do not know that there is anything better to do, from the server side, than to cut the connection when an exception is raised and uncaught, other than cutting the connection - the client will see that its response download was interrupted, and will know to deal with that issue (unless the exception happens just as the full content has been sent, in which case, the client will not care that there was an error on the server)

[RamiAwar]

I haven't looked into it yet, I'm using FastAPI, so there's probably a way to access it.

[RamiAwar]

There has to be something that's swallowing it right now, cause I only get a RuntimeError.

[jhominal]

Is there a usage of the fastapi middleware decorator?

[RamiAwar]

Just one for Sentry and one for Pydantic validation errors.

Will try commenting out the Sentry handler, cause that could be swallowing the error.

app.add_middleware(SentryAsgiMiddleware)


@app.exception_handler(ValidationError)
async def validation_exception_handler(request, exc):
    """Handles pydantic validation errors"""
    return PlainTextResponse(str(exc), status_code=400)

[RamiAwar]

I do think that's the case now that I test it. The Sentry middleware was preventing it from bubbling up!

Thanks for the help, turns out it wasn't a bug after all, the error is indeed handled at some level.