Remote write to Prometheus Server => No metrics #9259
Comments
|
thanks for reporting this. we will troubleshoot the issue. |
|
I checked with the current master and prometheus 2.54.0. For Linux and other Unix like systems, we can use node_uname_info in Prometheus 2.54.0 with prometheus_remote_write output plugin. |
|
@cosmo0920 Thanks for having a look, the machines showing in the above screenshot are running Prom 2.54.0 in agent mode with Node-Exporter goversion="go1.22.3" version="1.8.1" Maybe a difference in the node-exporter version / implementation since metrics exported are not all here ? If I install node-exporter in same VM as fluent-bit, I can see it's exporting node_os_info and node_os_version but missing on fluent-bit, same thing for node_pressure... Grafana Dashboard for node-exporter seems to be using these metrics (node_pressure_cpu_waiting_seconds_total, node_pressure_memory_waiting_seconds_total, node_pressure_io_waiting_seconds_total...) I exported fluent-bit's node-exporter to see which metrics were being exposed and was indeed able to find those in my central prometheus so remote-write is 100% confirmed to be properly working but some metrics are missing This bring another question to the table to make, how / would it be possible for fluent-bit to append similar "default" labels as Prometheus like:
Grafana's node-exporter dashboard uses "Instance" and "Job" so for fluent-bit to be/become a proper drop-in replacement to Prometheus Agent + Node-Exporter, it would be awesome to add those, what you think ? 
|
How about using [OUTPUT]
name prometheus_remote_write
match *
host xxxxxxx.yyyyy.zzzz
port 9090
uri /api/v1/write
log_response_payload true
tls off
tls.verify off
add_label instance ${HOSTNAME}
add_label job fluent-bit |
|
Yep that does the trick ! I can live with that :) For the missing metrics VS node-exporter, anything you thinking of ? |
|
This issue is stale because it has been open 90 days with no activity. Remove stale label or comment or this will be closed in 5 days. Maintainers can add the |
|
This issue was closed because it has been stalled for 5 days with no activity. |
Bug Report
Describe the bug
Using node_exporter_metrics input with prometheus_remote_write output doesn't push metrics to Prometheus.
To Reproduce
Expected behavior
Expect to see node-exporter metrics on the Prometheus Server as the following: (none of those below come from fluentbit, as the instance matches my other clients pushing with remote_write through a Prometheus AGENT)
Your Environment
Additional context
I'm looking to replace my current setup which has clients pushing node-exporter's metrics to a Prometheus Server through a Prometheus Agent.
Using Fluentbit with remote write would allow me to have an all-in-one tool that I could install on my clients to not only push metrics to Prometheus but logs to ES as well.
The text was updated successfully, but these errors were encountered: