gather elasticsearch indices and shard stats

[MatthewOHaraTR]

Required for all PRs:

• [x ] Signed CLA.
• [x ] Associated README.md updated.
• [x ] Has appropriate unit tests.

[MatthewOHaraTR]

@danielnelson Much easier with a new fork. I'd accumulated too much while the old one was open over the last 8 months. Thanks for your patience.

[MatthewOHaraTR]

@danielnelson The circle test failed, but unless I'm reading it incorrectly, its unrelated to any changes of mine. Specifically, the inputs/elasticsearch test did complete OK. The only error I noted was:

--- FAIL: TestMemcachedGeneratesMetrics (0.00s)
Error Trace: memcached_test.go:25
Error: Received unexpected error:
dial tcp 127.0.0.1:11211: getsockopt: connection refused
FAIL
FAIL github.com/influxdata/telegraf/plugins/inputs/memcached 0.022s

[lpic10]

If I understand correctly this is adding a new field named cluster_name after having changed the previous tag cluster_name to name with the exact same information. Is there any reason for this?

[MatthewOHaraTR]

Presumably better to simply contain this information in the cluster_name tag.

[lpic10]

any reason to have index_name as field and as a tag at same time?

[MatthewOHaraTR]

I can get rid of the redundant (non-tag) index_name.

[lpic10]

@mhohara I will try to test this soon, you need to update the README after the latest changes

[MatthewOHaraTR]

@lpic10 Readme updated. And thanks for looking at this.

[eran-totango]

@mhohara @lpic10 any ETA for the indicestats feature?

[MatthewOHaraTR]

@lpic10 Any progress on your review?

[florck]

👍

[MatthewOHaraTR]

@danielnelson @lpic10 Any progress here? I don't see much in the way of other updates to the ES plugin as of yet.

[danielnelson]

Sorry no progress to report, we still need to do #2650 #2711 first.

[MatthewOHaraTR]

Are you waiting for something from me on those two?

[danielnelson]

No, but you are certainly welcome to give fixing #2650 a shot.

[MatthewOHaraTR]

@danielnelson I'm not sure I understand the issue in #2650 . Could the e.masterNodeId from the catmaster be compared to a cat/nodeattrs call to localhost:9200 to get the NodeId of the node its on, to see whether 'it' is the master?

[danielnelson]

The problem is that there is a race condition, each server is collected concurrently via the anonymous goroutine in Gather, so the order in which e.isMaster is assigned too here is not defined. The isMaster value should probably not be a field on the plugin struct.

[MatthewOHaraTR]

In this PR's version of the code, the isMaster flag is only used in the Gather routine. Perhaps the necessary ES API calls to determine whether the node that telegraf is running on is the cluster master need to be put into the Gather routine?

[danielnelson]

I remember the PR code did not fix the problem, the routine that determines master status is less important than not storing it on the struct where it will be available/overwritten by other routines.

[MatthewOHaraTR]

The flag was put in the struct because it is the only output of the gatherNodeStats routine which did the ES API calls to get the node information and hence could decide whether a node was master. So it seemed the best/only way to return the isMaster information back to the calling Gather routine. Moving
the call needed to decide whether the node is master to the Gather routine seems a reasonable way to fix the issue (if I understand the issue correctly).

[hgfischer]

Is this abandoned? Do you need help? I am interested in this feature.

[danielnelson]

@hgfischer We need to address #2650 and #2711 before completing this issue.

[danielnelson]

Closed in #6060