You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
We've had issues with stale metrics hanging around (#129) and wanted to start using TTLs to get rid of them (#164).
My question is about metrics that are rarely updated. Based on my understanding of #164, if a metric passes its TTL without being pushed into the statsd exporter it will be marked as stale and will no longer be reported to Prometheus. So what happens if the metric is reported again sometime later? Will it be resurrected and reported to Prometheus again or is it permanently quieted after hitting its TTL? This is all on a per "label values set" basis, of course.
It looks like before #246 any TTL'd metrics would have stayed dead forever, but that PR seems to have fixed that. I just want to confirm my understanding before using the option.
Thanks!
The text was updated successfully, but these errors were encountered:
Metrics that have expired are forgotten and removed; if you start sending them again later they will reappear. Counters, summaries, gauges will be reset in the process. This means you will have gaps in the rarely sent metrics, and counters may not work the way you expect them to at all (you cannot rate() over a single sample).
If possible, the best way to avoid stale metric issues is to tie the exporter lifecycle to the application, for example by deploying it as a sidecar.
I'm going to close the issue since there is nothing for us to do here, if you do encounter weirdness around TTL handling please do file a new issue!
We've had issues with stale metrics hanging around (#129) and wanted to start using TTLs to get rid of them (#164).
My question is about metrics that are rarely updated. Based on my understanding of #164, if a metric passes its TTL without being pushed into the statsd exporter it will be marked as stale and will no longer be reported to Prometheus. So what happens if the metric is reported again sometime later? Will it be resurrected and reported to Prometheus again or is it permanently quieted after hitting its TTL? This is all on a per "label values set" basis, of course.
It looks like before #246 any TTL'd metrics would have stayed dead forever, but that PR seems to have fixed that. I just want to confirm my understanding before using the option.
Thanks!
The text was updated successfully, but these errors were encountered: