How to Filter label_values?

[pcbl]

Hi Folks,

I am a bit Newbie to Loki + Grafana and I believe I am doing something wrong.... But my proble is quite simple:

I am building on grafana a query variable which presents the values of a given label(ServiceName on my sample). Simple as that:
label_values(ServiceName)

This query would return something like:

• Loki
• Grafana
• Promtail
• LogService

So far so good. My issue starts when I try to actually filter these results... let´s say i would like to see only services which starts with L... Loki and LogService from my sample... Based on some samples I saw during some search, it should be set as something like this:

label_values({ServiceName=~"L.*"}, ServiceName)

The issue is that the {ServiceName=~"L.*"} bit seems to be completelly ignored... as it has absolutelly no effect and all values mentioned above, including the ones which DOESN`T start with L....

What am I doing wrong? How could I get this working? Can anyone share some advise?

Many thanks in advance!

BR
Pedro

[cyriltovena]

In Grafana you can use regex below the query to filter out what you don't want like this

Now this feature seems to use the labels API and so it won't answer to regexp like you're showing.

/cc @marefr @aocenas Should this use the series API instead of the labels API ?

[pcbl]

I noticed the Regex on the query options, but on my case I want to use the filter on the Value group/tags... I would then filter the label_values with the $tag value... On that case the regex will not help much... Or am i missing something?

I was wondering because I already use a similar syntax for a prometheus datasource. But on this specific case I need to use Loki as datasource. I also investigated a little bit the API such as "http://localhost:12345/loki/api/v1/label/ServiceName/values" but I did not saw a way where I could add some filter.... But I must say I am a newbie on Loki + Grafana... Have a lot to learn!

[cyriltovena]

Yes I think this is a Grafana issue.It should use the series API which support your use case and not the labels API. But the long time ago only the labels api was existing.

They will come back to you, let's wait their input.

[pcbl]

OK. Just as additional Information, I am using Grafana 7.0.4.

[stale]

This issue has been automatically marked as stale because it has not had any activity in the past 30 days. It will be closed in 7 days if no further activity occurs. Thank you for your contributions.

[pgassmann]

@pcbl in this blogpost they use a prometheus metric as filter: https://grafana.com/blog/2020/04/08/loki-quick-tip-how-to-create-a-grafana-dashboard-for-searching-logs-using-loki-and-prometheus/

The following is $pod, which contains: label_values(container_network_receive_bytes_total{namespace=~"$namespace"},pod). This will select the actual pod and narrows the log search space. (Note that we’re using container_network_receive_bytes_total as a metric name to find the available pods, but you can use any metric that will represent all the pods in your environment.)

I don't collect any metrics yet. so I cannot use that.

[matiasba]

Has anyone found a work around for this limitation? I'm not using Prometheus but y really need to make a templated logs dashboard.

[jorgelbg]

Just leaving this here, because a Google search led me to this issue before I could find the currently open one:

grafana/grafana#25205 (comment)