Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Delete Prometheus labels for deleted queues #305

Merged
merged 2 commits into from
Sep 10, 2024
Merged

Conversation

DrJosh9000
Copy link
Contributor

What

Track which queues are present in the most recent Collect for each cluster, and delete them from the metrics when they are no longer there.

Possible fix for #304

Why

#296 removed some code that was well-intentioned, but heavy-handed, and didn't make sense in a setup monitoring more than one cluster (or a cluster and unclustered...) But then there was nothing to remove old queues, which is a potential memory issue (minor) and causes metrics for deleted queues to get stuck on the last value (confusing).

@DrJosh9000 DrJosh9000 requested a review from wolfeidau September 9, 2024 08:08
@DrJosh9000 DrJosh9000 force-pushed the remove-deleted-queues branch from 4ca0c98 to 0cd0210 Compare September 9, 2024 08:09
@DrJosh9000 DrJosh9000 changed the title Delete labels for deleted queues Delete Prometheus labels for deleted queues Sep 10, 2024
@wolfeidau
Copy link
Contributor

@DrJosh9000 Another good pickup, I was interested in the note on order in https://pkg.go.dev/github.com/prometheus/client_golang/prometheus#MetricVec.DeleteLabelValues and recommendation to use https://pkg.go.dev/github.com/prometheus/client_golang/prometheus#MetricVec.Delete instead.

@DrJosh9000 DrJosh9000 force-pushed the remove-deleted-queues branch from 0cd0210 to 152e57a Compare September 10, 2024 00:27
@DrJosh9000
Copy link
Contributor Author

Good call @wolfeidau! I've changed a couple of other label-related things to be similar while here.

Copy link
Contributor

@wolfeidau wolfeidau left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍🏻 🚀

@DrJosh9000 DrJosh9000 merged commit 9ad2cd1 into master Sep 10, 2024
1 check passed
@DrJosh9000 DrJosh9000 deleted the remove-deleted-queues branch September 10, 2024 00:45
This was referenced Sep 11, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants