Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Upgrading from 1.9.5 to 1.9.8 triggers OOM error #1631

Closed
euri10 opened this issue Sep 19, 2022 · 3 comments · Fixed by #1649
Closed

Upgrading from 1.9.5 to 1.9.8 triggers OOM error #1631

euri10 opened this issue Sep 19, 2022 · 3 comments · Fixed by #1649

Comments

@euri10
Copy link

euri10 commented Sep 19, 2022

How do you use Sentry?

Sentry Saas (sentry.io)

Version

1.9.8

Steps to Reproduce

hello, fastapi app here with an upload endpoint, files uploaded are usually 15Mo in size,

reverting to 1.9.5 fixes the issue, on 1.9.8 it's exhausting the container memory quite fast on an upload of several files (about 8 generally).

I'm using sentry this way, just reading the notes I see a deprecation, maybe the reason of the crash

    if app_settings_loaded.sentry_dsn is not None:
        # sentry middleware
        sentry_sdk.init(dsn=app_settings_loaded.sentry_dsn)
        app.add_middleware(SentryAsgiMiddleware)

that said I dont see the deprecation warning in logs.

the app crash and I dont get sentry logs, I think it's normal considering it appears to be an issue with it

Expected Result

well to be able to upload my files :)

Actual Result

backend_1          | INFO:     172.18.0.3:54200 - 2022-09-19 13:57:21,244 -'POST /uploads/x1 HTTP/1.1' 500 Internal Server Error
backend_1          | ERROR:    Exception in ASGI application
backend_1          | Traceback (most recent call last):
backend_1          |   File "/opt/venv/lib/python3.10/site-packages/uvicorn/protocols/http/httptools_impl.py", line 404, in run_asgi
backend_1          |     result = await app(  # type: ignore[func-returns-value]
backend_1          |   File "/opt/venv/lib/python3.10/site-packages/uvicorn/middleware/proxy_headers.py", line 78, in __call__
backend_1          |     return await self.app(scope, receive, send)
backend_1          |   File "/opt/venv/lib/python3.10/site-packages/fastapi/applications.py", line 269, in __call__
backend_1          |     await super().__call__(scope, receive, send)
backend_1          |   File "/opt/venv/lib/python3.10/site-packages/sentry_sdk/integrations/starlette.py", line 293, in _sentry_patched_asgi_app
backend_1          |     return await middleware(scope, receive, send)
backend_1          |   File "/opt/venv/lib/python3.10/site-packages/sentry_sdk/integrations/asgi.py", line 138, in _run_asgi3
backend_1          |     return await self._run_app(scope, lambda: self.app(scope, receive, send))
backend_1          |   File "/opt/venv/lib/python3.10/site-packages/sentry_sdk/integrations/asgi.py", line 187, in _run_app
backend_1          |     raise exc from None
backend_1          |   File "/opt/venv/lib/python3.10/site-packages/sentry_sdk/integrations/asgi.py", line 182, in _run_app
backend_1          |     return await callback()
backend_1          |   File "/opt/venv/lib/python3.10/site-packages/starlette/applications.py", line 124, in __call__
backend_1          |     await self.middleware_stack(scope, receive, send)
backend_1          |   File "/opt/venv/lib/python3.10/site-packages/sentry_sdk/integrations/starlette.py", line 98, in _create_span_call
backend_1          |     await old_call(*args, **kwargs)
backend_1          |   File "/opt/venv/lib/python3.10/site-packages/starlette/middleware/errors.py", line 184, in __call__
backend_1          |     raise exc
backend_1          |   File "/opt/venv/lib/python3.10/site-packages/starlette/middleware/errors.py", line 162, in __call__
backend_1          |     await self.app(scope, receive, _send)
backend_1          |   File "/opt/venv/lib/python3.10/site-packages/sentry_sdk/integrations/starlette.py", line 98, in _create_span_call
backend_1          |     await old_call(*args, **kwargs)
backend_1          |   File "/opt/venv/lib/python3.10/site-packages/opentelemetry/instrumentation/asgi/__init__.py", line 346, in __call__
backend_1          |     await self.app(scope, wrapped_receive, wrapped_send)
backend_1          |   File "/opt/venv/lib/python3.10/site-packages/sentry_sdk/integrations/asgi.py", line 138, in _run_asgi3
backend_1          |     return await self._run_app(scope, lambda: self.app(scope, receive, send))
backend_1          |   File "/opt/venv/lib/python3.10/site-packages/sentry_sdk/integrations/asgi.py", line 148, in _run_app
backend_1          |     raise exc from None
backend_1          |   File "/opt/venv/lib/python3.10/site-packages/sentry_sdk/integrations/asgi.py", line 145, in _run_app
backend_1          |     return await callback()
backend_1          |   File "/opt/venv/lib/python3.10/site-packages/starlette/middleware/cors.py", line 92, in __call__
backend_1          |     await self.simple_response(scope, receive, send, request_headers=headers)
backend_1          |   File "/opt/venv/lib/python3.10/site-packages/starlette/middleware/cors.py", line 147, in simple_response
backend_1          |     await self.app(scope, receive, send)
backend_1          |   File "/opt/venv/lib/python3.10/site-packages/ratelimit/core.py", line 82, in __call__
backend_1          |     return await self.app(scope, receive, send)
backend_1          |   File "/opt/venv/lib/python3.10/site-packages/sentry_sdk/integrations/starlette.py", line 191, in _sentry_exceptionmiddleware_call
backend_1          |     await old_call(self, scope, receive, send)
backend_1          |   File "/opt/venv/lib/python3.10/site-packages/sentry_sdk/integrations/starlette.py", line 191, in _sentry_exceptionmiddleware_call
backend_1          |     await old_call(self, scope, receive, send)
backend_1          |   File "/opt/venv/lib/python3.10/site-packages/sentry_sdk/integrations/starlette.py", line 98, in _create_span_call
backend_1          |     await old_call(*args, **kwargs)
backend_1          |   File "/opt/venv/lib/python3.10/site-packages/starlette/exceptions.py", line 93, in __call__
backend_1          |     raise exc
backend_1          |   File "/opt/venv/lib/python3.10/site-packages/starlette/exceptions.py", line 82, in __call__
backend_1          |     await self.app(scope, receive, sender)
backend_1          |   File "/opt/venv/lib/python3.10/site-packages/sentry_sdk/integrations/starlette.py", line 98, in _create_span_call
backend_1          |     await old_call(*args, **kwargs)
backend_1          |   File "/opt/venv/lib/python3.10/site-packages/fastapi/middleware/asyncexitstack.py", line 21, in __call__
backend_1          |     raise e
backend_1          |   File "/opt/venv/lib/python3.10/site-packages/fastapi/middleware/asyncexitstack.py", line 18, in __call__
backend_1          |     await self.app(scope, receive, send)
backend_1          |   File "/opt/venv/lib/python3.10/site-packages/starlette/routing.py", line 670, in __call__
backend_1          |     await route.handle(scope, receive, send)
backend_1          |   File "/opt/venv/lib/python3.10/site-packages/starlette/routing.py", line 266, in handle
backend_1          |     await self.app(scope, receive, send)
backend_1          |   File "/opt/venv/lib/python3.10/site-packages/starlette/routing.py", line 65, in app
backend_1          |     response = await func(request)
backend_1          |   File "/opt/venv/lib/python3.10/site-packages/sentry_sdk/integrations/fastapi.py", line 77, in _sentry_app
backend_1          |     info = await extractor.extract_request_info()
backend_1          |   File "/opt/venv/lib/python3.10/site-packages/sentry_sdk/integrations/starlette.py", line 443, in extract_request_info
backend_1          |     content_length = await self.content_length()
backend_1          |   File "/opt/venv/lib/python3.10/site-packages/sentry_sdk/integrations/starlette.py", line 480, in content_length
backend_1          |     raw_data = await self.raw_data()
backend_1          |   File "/opt/venv/lib/python3.10/site-packages/sentry_sdk/integrations/starlette.py", line 491, in raw_data
backend_1          |     return await self.request.body()
backend_1          |   File "/opt/venv/lib/python3.10/site-packages/starlette/requests.py", line 236, in body
backend_1          |     self._body = b"".join(chunks)
backend_1          | MemoryError
@euri10 euri10 changed the title Upgrading from 1.9.5 to 1.9.5 triggers OOM error Upgrading from 1.9.5 to 1.9.8 triggers OOM error Sep 19, 2022
@smeubank smeubank added bug Component: SDK Core Dealing with the core of the SDK and removed Status: Untriaged labels Sep 28, 2022
@antonpirker antonpirker added this to the FastAPI/Starlette Support milestone Sep 28, 2022
@antonpirker
Copy link
Member

This is related to this: #1595

@Kludex
Copy link

Kludex commented Oct 3, 2022

Why the SentryAsgiMiddleware uses StarletteIntegration?

@antonpirker
Copy link
Member

I do not understand your question @Kludex

Using SentryAsgiMiddleware directly is deprecated. Now one just need to do a sentry_sdk.init() without any integrations, because the init will detect starlette (and other modules) automatically and enable the respective integrations.

antonpirker added a commit that referenced this issue Oct 3, 2022
, #1631, #1595, #1573)

* Do not read request body to determine content length.
* Made AnnotatedValue understandable
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants