Custom metrics and why Loki and not Mimir

[Memfisrain]

Hi,

We're currently in the process of validating grafana-faro as a replacement to our custom FE observability solution. Overall it seems pretty exciting and fits our needs, however we have some questions which is not widely described in the documentation or anywhere else, but still very interesting for us.

Metrics are not covered very well in sdk and documentation and as I understand we have just custom measurement which are stored in Loki and it's not very clear why that solution was chosen. Maybe you can elaborate on these questions to make it clearer?

1. Why Loki was selected instead of Mimir and prometheus like metric types (histogram, counter, gauge)?
2. In case we store many FE business metrics in prometheus and apply different kind of aggregations, can we easily migrate to Loki storage or we need to check every aggregation parity between PromQL and LogQL?
3. What can we do when we need to calculate a heave metric like aggregation with some percentile calculation for 1 month period, for example?
4. Do you have any plans to introduce a choice for customers on where to store metrics: Loki or Prometheus | Mimir?

Thanks for the SDK and all your efforts towards OSS

[Memfisrain]

@codecapitano hey, maybe you can elaborate on this? Would be very helpful

[codecapitano]

Hi @Memfisrain thanks for the questions!

1. Why Loki was selected instead of Mimir and prometheus like metric types (histogram, counter, gauge)?

This is because of high cardinality data. The amount of labels each metric would require, would cause an explosion of active series.

2. In case we store many FE business metrics in prometheus and apply different kind of aggregations, can we easily migrate to Loki storage or we need to check every aggregation parity between PromQL and LogQL?

Unfortunately can't answer this yet. Needs to do some research first.

3. What can we do when we need to calculate a heave metric like aggregation with some percentile calculation for 1 month period, for example?

Yes this can be done with Loki, but performance won't be great.

4. Do you have any plans to introduce a choice for customers on where to store metrics: Loki or Prometheus | Mimir?

No we don't have plans for Faro to store metrics (called measurements in Faro) in Prometheus or Mimir.

/cc @cedricziel

[Memfisrain]

@codecapitano thanks for some clarifications, maybe you have had an experience with https://grafana.com/docs/loki/latest/rules/#recording-rules and we can leverage it for heavy aggregations?

[codecapitano]

Unfortunately I don't have experience with the recording-rules feature.
I think the Loki community channels may be a good place to ask.