[Prometheus][Collector] Support Customizing condition

[BenB196]

Today the Prometheus Collector integration's condition is hardcoded to a bool for leader election:

integrations/packages/prometheus/data_stream/collector/agent/stream/stream.yml.hbs

Line 30 in 0c96463

condition: ${kubernetes_leaderelection.leader} == true

The issue with this is it makes it so that the Prometheus collector integration can't be used for auto-discovery. Similar to other integrations

Example, Istio:

integrations/packages/istio/data_stream/access_logs/agent/stream/stream.yml.hbs

Line 5 in 0c96463

condition: {{ condition }}

Given that Prometheus is one of the more common methods for exposing metrics within things like Kubernetes, it would be a great value add to be able to define conditions

[tetianakravchenko]

Hi @BenB196.

The issue with this is it makes it so that the Prometheus collector integration can't be used for auto-discovery. Similar to other integrations

In case of prometheus collector you need to provide the list of hosts, that will be scraped by this integration - https://github.com/elastic/integrations/blob/main/packages/prometheus/data_stream/collector/manifest.yml#L6-L13

Could you please explain what issue you are trying to solve with condition field here?

[BenB196]

Hi @tetianakravchenko the issue currently with the Prometheus collector is that you can only provide a static list of hosts. If you want to monitor a deployment on Kubernetes that has multiple (dynamically named) pods, you'll want to scrape all those pods individually. With the current setup, you are unable to achieve this.

However, this does become achievable once you add the ability to set the condition.

The Istio Proxy Metrics is a working example of using the Prometheus Collector, in conjunction with condition to be able to achieve this outcome.