-
Notifications
You must be signed in to change notification settings - Fork 2.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Upgraded Prometheus and Cortex dependencies #3382
Conversation
Signed-off-by: Marco Pracucci <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks! That looks like ok approach to me 👍
Some CI failures though.
Signed-off-by: Marco Pracucci <[email protected]>
Signed-off-by: Marco Pracucci <[email protected]>
Let me think about priv type problem 🤔 |
Signed-off-by: Marco Pracucci <[email protected]>
Signed-off-by: Marco Pracucci <[email protected]>
Double checked and this looks fair to me. I would use Prometheus tsdb multi error though everywhere, except where we assert error type. We should probably use local type for this to explicitness. We can fix later this readability suggestion 👍 LGTM |
Signed-off-by: Marco Pracucci <[email protected]>
Signed-off-by: Marco Pracucci <[email protected]>
Signed-off-by: Marco Pracucci <[email protected]>
Signed-off-by: Marco Pracucci <[email protected]>
* Upgraded Prometheus and Cortex dependencies Signed-off-by: Marco Pracucci <[email protected]> * Updated mod replace Signed-off-by: Marco Pracucci <[email protected]> * Fixed linter Signed-off-by: Marco Pracucci <[email protected]> * Use errutil.MultiError everywhere Signed-off-by: Marco Pracucci <[email protected]> * Small fixes Signed-off-by: Marco Pracucci <[email protected]> * Do not use Prometheus MultiError in cmd too Signed-off-by: Marco Pracucci <[email protected]> * Fixed linter Signed-off-by: Marco Pracucci <[email protected]> * Fixed dot in a comment Signed-off-by: Marco Pracucci <[email protected]> * Added copyight Signed-off-by: Marco Pracucci <[email protected]> Signed-off-by: Oghenebrume50 <[email protected]>
Changes
We fixed a TSDB WAL corruption which we urgently need to backport to Cortex and deploy (we already got 2 incidents this week because of this).
Prometheus has recently done a refactoring of
MultiError
, which introduced a circular breaking change in Cortex and Thanos. To solve this, I've internedMultiError
into Cortex first (cortexproject/cortex#3425) and in this PR I'm upgrading Cortex and Prometheus. Thanos does an use theMultiError
data type assertion in compactor and receiver, but the data type is not exposed anymore, so to unblock this situation I've interned the previous version ofMultiError
in Thanos, in order to gain some time to discuss how to better address it, while being able to rollout the WAL corruption fix to Cortex.Thoughts?
Verification
Existing tests.