node_exporter with Textfile directory metrics collection hangs

[harikum]

Host operating system: output of uname -a

Linux hostname 3.10.0-514.16.1.el7.x86_64 #1 SMP Fri Mar 10 13:12:32 EST 2017 x86_64 x86_64 x86_64 GNU/Linux

node_exporter version: output of node_exporter --version

/tmp/node_exporter --version

node_exporter, version 0.15.0 (branch: master, revision: a8d7d11)
build user: root@hostname
build date: 20171118-20:08:56
go version: go1.9.2

node_exporter command line flags

/tmp/node_exporter --collector.textfile.directory="/var/tmp/thin" &

Are you running node_exporter in Docker?

NO, running locally on the base host, not in a container

What did you do that produced an error?

let the collector run for over an hour and the process hangs...

What did you expect to see?

collector should not hang...

What did you see instead?

collector hangs, consistently across multiple systems.

Summary

/tmp/node_exporter --collector.textfile.directory="/var/tmp/thin" &

"/var/tmp/thin" folders contains *.prom files for parsing...

cat /var/tmp/thin/lvm.prom

node_lvm_thinpool_size{lvname="thinpool",vgname="docker"} 64.91
node_lvm_thinpool_used{lvname="thinpool",vgname="docker"} 19.52

after initial startup, exporter works fine, prometheus scrapes this end point sucessfull including the metrics from the *.prom file.

after about an hour or so, the end point becomes unresponsive.....

node_exporter PID on the host is still up but unresponsive..

root     22284  8947 20 12:19 pts/0    00:44:49 /tmp/node_exporter --collector.textfile.directory=/var/tmp/thin

tracing the collector pid does not return anything...

# strace -p 22284
Process 22284 attached
futex(0x1064530, FUTEX_WAIT, 0, NULL
^CProcess 22284 detached
 <detached ...>

the session on which the node_exporter process is dumps barrage of messages as shown as below....

<..snip..>
Gatherer #1] [from Gatherer #1] [from Gatherer #1] [from Gatherer #1] [from Gatherer #1] [from Gatherer #1] [from Gatherer #1] [from Gatherer #1] [from Gatherer #1] [from Gatherer #1]
[from Gatherer #1] [from Gatherer #1] [from Gatherer #1] [from Gatherer #1] [from Gatherer #1] [from Gatherer #1] [from Gatherer #1] [from Gatherer #1] [from Gatherer #1]
[from Gatherer #1] [from Gatherer #1] [from Gatherer #1] [from Gatherer #1] [from Gatherer #1] [from Gatherer #1] [from Gatherer #1] [from Gatherer #1] [from Gatherer #1]
[from Gatherer #1] [from Gatherer #1] [from Gatherer #1] [from Gatherer #1] [from Gatherer #1] [from Gatherer #1] [from Gatherer #1] [from Gatherer #1] [from Gatherer #1]
[from Gatherer #1] [from Gatherer #1] [from Gatherer #1] [from Gatherer #1] [from Gatherer #1] [from Gatherer #1] [from Gatherer #1] [from Gatherer #1] [from Gatherer #1]
[from Gatherer #1] [from Gatherer #1] [from Gatherer #1] [from Gatherer #1] [from Gatherer #1] [from Gatherer #1] [from Gatherer #1] [from Gatherer #1] [from Gatherer #1]
[from Gatherer #1] [from Gatherer #1] [from Gatherer #1] [from Gatherer #1] [from Gatherer #1] [from Gatherer #1] [from Gatherer #1] [from Gatherer #1] [from Gatherer #1]
[from Gatherer #1] [from Gatherer #1] [from Gatherer #1] [from Gatherer #1] [from Gatherer #1] [from Gatherer #1] [from Gatherer #1] [from Gatherer #1] [from Gatherer #1]
[from Gatherer #1] [from Gatherer #1] [from Gatherer #1] [from Gatherer #1] [from Gatherer #1] [from Gatherer #1] [from Gatherer #1] [from Gatherer #1] [from Gatherer #1]
[from Gatherer #1] [from Gatherer #1] [from Gatherer #1] [from Gatherer #1] [from Gatherer #1] [from Gatherer #1] [from Gatherer #1] [from Gatherer #1] [from Gatherer #1]
[from Gatherer #1] [from Gatherer #1] colle[1]+ Killed /tmp/node_exporter --collector.textfile.directory="/var/tmp/thin"

eventually the process needs to be killed
# kill -9 22284

[SuperQ]

This is a duplicate of #704.