Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

instrumentation: fastapi prometheus middleware #1007

Open
wants to merge 1 commit into
base: jjaakola-aiven-fastapi
Choose a base branch
from

Conversation

nosahama
Copy link
Contributor

@nosahama nosahama commented Dec 11, 2024

About this change - What it does

Added a middleware to instrument the fastapi requests.

Screenshot 2024-12-11 at 14 13 37

@nosahama nosahama requested review from a team as code owners December 11, 2024 13:10
Copy link

Coverage report

Click to see where and how coverage changed

FileStatementsMissingCoverageCoverage
(new stmts)
Lines missing
  src/schema_registry
  __main__.py 10
  src/schema_registry/middlewares
  __init__.py 37
  prometheus.py
Project Total  

This report was generated by python-coverage-comment-action

src/schema_registry/middlewares/prometheus.py Outdated Show resolved Hide resolved
setattr(request.state, prometheus.START_TIME_REQUEST_KEY, time.time())

# Extract request labels
path = request.url.path
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This can cause quite high cardinality, can we align this with the path template. E.g. /subjects/<subject>/versions?

Copy link
Contributor Author

@nosahama nosahama Dec 17, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes I can think around it, it was similar thought I had on the tracing PR #1009, where I extract the resource from the path as I didn't find a way to get the template, as this was the traces, it looked fine to trace the resource, as further spans contain relevant information and this reduces the trace cardinality.

In prometheus tho, I do not think this should be a problem, major cardinality issues arise with too many labels. To instrument, might be necessary to see exactly the path that's being checked, but I also get your point. Let me reason around it and get back.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking at this discussion - fastapi/fastapi#12975, the route template is only accessed after a successful response, I do need the path already to instrument the requests in progress.
Unless we want to fallback and extract the base resource, i.e. resource = request.url.path.split("/")[1]. From a template like /subjects/{subject}/versions and a request for /subjects/test-topic-value/versions, this will yield subjects.

src/schema_registry/middlewares/prometheus.py Outdated Show resolved Hide resolved
Added a middleware to instrument the fastapi requests
@nosahama nosahama force-pushed the nosahama/EC-627-fastapi-prometheus-metrics branch from d95a95c to dac8fd2 Compare December 17, 2024 13:23
@nosahama
Copy link
Contributor Author

I will close this in favour of the OTel metrics adding in #1011

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants