textfile: the same metric occurring with different labels in two different prom files trigger error due to inconsistent help message in processFile

[cbryant42]

There's an issue that was fixed in node-exporter, but I seem to be running in to with Windows Exporter. Basically, I have a script writing to 2 different files with the same metric names but different label sets. The exporter is throwing an error because the help message reads 2 different file names. Has this been fixed and I am just unaware of a configuration flag for this, or is this something that needs to be addressed here as well?

prometheus/node_exporter#1885
prometheus/node_exporter#2475

[JDA88]

Can you please provide the 2 exact files that trigger the error (or a subset)? And also the exact error message?
What version of windows exporter are you using?

I currently have multiple servers where multiple files are generated (with same metrics but different labels and I have no issues)
# HELP and # TYPE are present and exactly the same on all files

[jezhiggins]

I've have the same issue

If I have one file

fruit{type="lemon"} 15
fruit{type="apple"} 12

everything runs as it should.

However, if those two metrics are in different files

fruit{type="lemon"} 15

fruit{type="apple"} 12

then all then no metrics are produced - not just from text files, we get nothing.

The error message on /metrics is

An error has occurred while serving metrics:

* collected metric fruit label:<name:"type" value:"lemon" > untyped:<value:15 >  has help "Metric read from C:\\Program Files\\windows_exporter\\textfile_inputs\\test.prom" but should have "Metric read from C:\\Program Files\\windows_exporter\\textfile_inputs\\breaks.prom"

If we have more labelled metrics in other text files, we get an extra error for each additional file.

This is using version 0.22.0, but I've seen the same behaviour (without having identified the error) in earlier versions.

[jezhiggins]

To be clear, having reread JDA88's comment, these are text files on the same machine, being collected by a single instance of windows_exporter. Aggregating labelled metrics across machines works fine, aggregated labelled metrics with the same exporter fails.

prometheus/node_exporter#1885 does appear to describe the exact same problem. I'll try and look at prometheus/node_exporter#2475 and see if it can be applied here too.

[jezhiggins]

Workaround

If you're able to add your own # HELP metric description to the prom files, then the labelled metrics with be merged together properly, and things will work.

[jezhiggins]

I think the fix in prometheus/node_exporter#2475 will work pretty much as is. I will try and raise a PR, but should caveat by noting I'm not a regular user of go and I'm in the middle of moving house.

It's clear the node_exporter and windows_exporter textfile collectors started from the same place, but have diverged over time. There's some minor incident differences (!present vs present == false) which it would nice to but aren't significant. However, there's a whole load of error checking, particularly around duplicate detection that windows_exporter is doing that perhaps should be carried back into node_exporter.

[JDA88]

From my experience if you two files look like this it will work. # HELP and # TYPE must be on every metrics in every files

# HELP fruit Fruit inventory
# TYPE fruit gauge
fruit{type="lemon"} 15

# HELP fruit Fruit inventory
# TYPE fruit gauge
fruit{type="apple"} 12

[github-actions]

This issue has been marked as stale because it has been open for 90 days with no activity. This thread will be automatically closed in 30 days if no further activity occurs.