-
Notifications
You must be signed in to change notification settings - Fork 2.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix ExternalLabels() for Prometheus v3.0 #7893
Fix ExternalLabels() for Prometheus v3.0 #7893
Conversation
Prometheus v3.0.0-rc.0 introduces a new scrape protocol (`PrometheusText1.0.0`) which is present by default in the global configuration. It breaks the Thanos sidecar when it wants to retrieve the external labels. This change replaces the use of the Prometheus `GlobalConfig` struct by a minimal struct which unmarshals only the `external_labels` key. See also prometheus-operator/prometheus-operator#7078 Signed-off-by: Simon Pasquier <[email protected]>
dcb1c2e
to
cdbbf1a
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM! Thanks for fixing!
I wonder, if bumping prometheus version in our e2e tests also makes sense (probably good to do once v3 is fully released)?
What we've done for Prometheus operator (and this is how we found out the issue) is to configure a periodic workflow running all our e2e tests with the latest v3.0.0 release (currently rc.0). It gives an early signal on potentially breaking changes. |
Yup, I think we might need similar! In any case, merging this fix |
Heads up - Prometheus v3.0.0 is now out, so folks using the latest Thanos release v0.36.1 will start to see Sidecar breaking. A workaround is to remove the new global:
...
scrape_protocols:
- OpenMetricsText1.0.0
- OpenMetricsText0.0.1
- PrometheusText0.0.4 |
Changes
Prometheus v3.0.0-rc.0 introduces a new scrape protocol (
PrometheusText1.0.0
) which is present by default in the global configuration. It breaks the Thanos sidecar when it wants to retrieve the external labels.This change replaces the use of the Prometheus
GlobalConfig
struct by a minimal struct which unmarshals only theexternal_labels
key.See also prometheus-operator/prometheus-operator#7078
Verification
Unit test added.