-
Notifications
You must be signed in to change notification settings - Fork 329
[questing] Empty grafana dashboard #219
Comments
@mstaalesen this isnt exclusively something you can help with, but do you
have some time to help with this question?
…On Mon, Aug 5, 2019, 7:32 PM Igor Likhomanov ***@***.***> wrote:
Problem: Dashboard 1598 is empty, data from zipkin is in Prometheus
Setup: k8s cluster with zipkin, prometheus (helm) and grafana (helm)
I am using zipkin-transport-http to send data to zipkin
Dashboard: https://grafana.com/grafana/dashboards/1598
gnetId: 1598
revision: 15
Zipkin deployment config on k8s:
annotations:
prometheus.io/scrape: 'true'
prometheus.io/port: '9411'
prometheus.io/path: '/actuator/prometheus'
spec:
containers:
- name: zipkin
image: 'openzipkin/zipkin:2.15.0'
Example response produces by zipkin:9411/actuator/prometheus
zipkin_collector_messages_total{transport="http",} 223.0
Dashboard is empty:
[image: Screenshot 2019-08-05 at 11 20 42]
<https://user-images.githubusercontent.com/6287367/62453715-2fa56000-b773-11e9-9bbe-38a2584bf60a.png>
Data is in prometheus:
[image: Screenshot 2019-08-05 at 11 20 58]
<https://user-images.githubusercontent.com/6287367/62453736-39c75e80-b773-11e9-8fe8-e579ea2548d7.png>
Example: Messages received per transport
Example query from dashboard:
sum by(transport)(rate(zipkin_collector_messages_total{job="zipkin",instance=~"$instances"}[$__interval]))
And seems i do not have job="zipkin",instance=~"$instances"
So this would work:
sum by(transport)(rate(zipkin_collector_messages_total{}[5m]))
Seems somehow job and instance are missed 🤔 Maybe I am missing something?
Thanks for the help!
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
<#219>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AAAPVV7TZGMH3EYQEXH66ZDQC7XSFANCNFSM4IJJIK4Q>
.
|
Sorry for the late response (vacation). Some of these parameters are based on what the labels are set for the scrape targets. So to solve this in your case @Igor-lkm, you need to change the query from |
@mstaalesen yep, thanks :) So I did it:
The point was that it's not working out-of-the-box and if there is a way to make it work out-of-the-box 🤔 |
Sorry, I missed that part! One approach could be to just remove |
@mstaalesen If we remove
So seems to remove If we would be able to remove @mstaalesen Is there a github repo with this board, so i can contribute if I would have a good idea? :) |
there is no github repo. you can download and change it until we figure out
what normal is wrt change management of grafana dashboards
https://grafana.com/dashboards/1598
cc @abesto
…On Fri, Aug 23, 2019 at 12:19 AM Igor Likhomanov ***@***.***> wrote:
@mstaalesen <https://github.com/mstaalesen> If we remove job="zipkin", we
would miss instances in the filter on top, as they depend on a job:
"label": "Zipkin instances",
"multi": true,
...
"query": "label_values(http_server_requests_seconds_bucket{job=\"zipkin\"}, instance)",
So seems to remove job="zipkin" is not an option 🤔
If we would be able to remove job="zipkin" and still get an instances
list, that would solve the problem 🤔
@mstaalesen <https://github.com/mstaalesen> Is there a github repo with
this board, so i can contribute if I would have a good idea? :)
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#219>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AAAPVVY6HYQKMSP5Z5WZPO3QF24B3ANCNFSM4IJJIK4Q>
.
|
That's correct, I don't know of a standard way to properly version-control dashboards hosted on grafana.com. A quick search turns up https://git.abolivier.bzh/babolivier/grafana-dashboards-manager, but it's anybodies guess whether that actually adds value over "use a Git repo, and an admin sometimes syncs to grafana.com". |
/me chuckles
|
I think the "workaround" on that would be to just use a metric that we know is unique to zipkin, or if that doesn't work - add a regexp to filter out only zipkin instances. With regards to how to version control it. There really is no good way of doing it as of right now. I think there is some work being done where you save your dashboard as code. I've seen some talks on the topic, but I don't think its mature enough to be used by "the masses" yet. |
Problem: Dashboard 1598 is empty, data from zipkin is in Prometheus
Setup: k8s cluster with zipkin, prometheus (helm) and grafana (helm)
I am using
zipkin-transport-http
to send data to zipkinDashboard: https://grafana.com/grafana/dashboards/1598
Zipkin deployment config on k8s with Prometheus scraping configuration:
Example response produces by
zipkin:9411/actuator/prometheus
Dashboard is empty:
Data is in prometheus:
Example: Messages received per transport
Example query from dashboard:
And seems i do not have
job="zipkin",instance=~"$instances"
So this would work ok:
In table view looks like this:
Job:
kubernetes-pods
Seems somehow
job
andinstance
are missed 🤔 Maybe I am missing something?Thanks for the help!
Added:
Works ok with different labels.
Looks like it is a broad question at the end.
For example RabbitMQ has metrics like
rabbitmq_node_mem_used
, prefixed withrabbitmq
(dashboard 4279)The text was updated successfully, but these errors were encountered: