Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Promtail metric vector improvement. #1667

Closed
cyriltovena opened this issue Feb 10, 2020 · 0 comments · Fixed by #1683
Closed

Promtail metric vector improvement. #1667

cyriltovena opened this issue Feb 10, 2020 · 0 comments · Fixed by #1683
Labels
component/agent type/enhancement Something existing could be improved

Comments

@cyriltovena
Copy link
Contributor

As of today we've by-passed a lot of the prometheus golang client to build our own metric engine. This allowed us to build metric on the fly for each stream created by promtail.

However in the long term, streams are volatile (new stream are created and disappear every time a file is rolling) which could cause some memory issues if we were to keep too many metrics for some streams.

Since those metrics are created in one place https://github.com/grafana/loki/blob/267dbe0085a7dbff8062b850f2e5776268acf4d2/pkg/logentry/metric/metricvec.go we should register them into a global map and run some sort of stale metric manager that removes metrics that haven't been updated for a while (1~5min, could be configurable)

However it should still work if the metrics comes back suddenly.

/cc @slim-bean @rfratto

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
component/agent type/enhancement Something existing could be improved
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant