feat(inputs.logstash): Record number of failures

[bartigor]

• Updated associated README.md.
• Wrote appropriate unit tests.

added a metric for counting errors by plugins on logstash input plugin

resolves #

I added the collection of metrics for the number of failures in the Logstash Input Plugin

[telegraf-tiger]

Thanks so much for the pull request!
🤝 ✒️ Just a reminder that the CLA has not yet been signed, and we'll need it before merging. Please sign the CLA when you get a chance, then post a comment here saying !signed-cla

[telegraf-tiger]

🤝 ✒️ Just a reminder that the CLA has not yet been signed, and we'll need it before merging. Please sign the CLA when you get a chance, then post a comment here saying !signed-cla

[bartigor]

!signed-cla

[telegraf-tiger]

Looks like new artifacts were built from this PR. Get them here!

Artifact URLs

• aarch64.rpm

• armel.rpm

• armv6hl.rpm

• i386.rpm

• ppc64le.rpm

• s390x.rpm

• x86_64.rpm

• darwin_amd64.tar.gz

• freebsd_amd64.tar.gz

• freebsd_i386.tar.gz

• linux_amd64.tar.gz

• linux_arm64.tar.gz

• linux_armel.tar.gz

• linux_armhf.tar.gz

• linux_i386.tar.gz

• linux_mips.tar.gz

• linux_mipsel.tar.gz

• linux_ppc64le.tar.gz

• linux_s390x.tar.gz

• static_linux_amd64.tar.gz

• windows_amd64.zip

• windows_i386.zip

• amd64.deb

• arm64.deb

• armel.deb

• armhf.deb

• i386.deb

• mips.deb

• mipsel.deb

• ppc64el.deb

• s390x.deb

[ivorybilled]

Makes sense, just curious about adding it as a new metric with the same name vs adding a field to the existing metric, is that necessary?

[bartigor]

About adding a new field to the existing metric: where do you sugest to add it?
I made a new metric becaus 'failues' are not on the same level in Logstash stats api https://www.elastic.co/guide/en/logstash/current/node-stats-api.html#pipeline-stats

[sspaink]

@bartigor sorry you haven't received a response in a while, are you still interested in working on this pull request? I think having it in a separate metric will probably be fine. I was wondering how you were planning on using the number of failures, wouldn't you need more context around those failures to make them a useful metric?

[bartigor]

I use this metric in my production to detect microservices that have changed the log format. So far, just the number of failures is enough for me? and if they have grown sharply, then it means that it is correctly parsed. You can probably expand the context, but so far I have not thought about how to do it better.

[sspaink]

Thank you for the context, if you could rebase your pull request against the latest master branch to have CI run again that would be great.

[bartigor]

It didn’t work to automatically merge master, a long time passed, and the master went ahead. I'll fix it when the time comes.

[sspaink]

@bartigor looks like the tests are failing due to some syntax issues, do you have time to resolve them?

[bartigor]

I fixed it, everything seems to compile and pass

[sspaink]

@bartigor sorry seems the unit tests didn't get triggered, I triggered them manually and they are failing with:

[ERROR] gofmt has found errors in the following files:
plugins/inputs/logstash/logstash.go

Run make fmt to fix them.
make: *** [Makefile:152: fmtcheck] Error 1

[bartigor]

fixed