Nats Jetstream consumer

[b3rtram]

Feature Request

Nats.io Jetstream consumer

Proposal:

The Nats server has a new streaming technology called jetstream. I work with nats using jetstream and it would be very helpful when Telegraf could consume messages from that technology https://docs.nats.io/nats-concepts/jetstream

Current behavior:

Nats consumer plugin consumes data but not from a jetstream consumer

Desired behavior:

Telegraf can be used to consume data from a nats jetstream consumer

Use case:

Nats jetstream persists the messages to file. It makes our system more resilient and easier to design when our services could restart without losing data

[reimda]

Hi @b3rtram, thanks for the feature request. It looks like jetstream support could be a good telegraf feature. Are you able to put together a PR to add support to the nats_consumer input?

[b3rtram]

Hi I am a developer and could try it, but I don't have so much experience in open source contribution.

[b3rtram]

Hi, i tried at weekend to implement this feature. My first approach of configuration is to add a new array for Jetstream Subjects.

JsSubjects []string toml:"js_subjects"

So a user could add normal subjects as usual and if needed js_subjects.
What do you think?

[reimda]

I think you're on the right track with "js_subjects". I'm not a nats/jetsream expert but from a quick read through the jetstream docs, it sounds like jetstream consumers subscribe by subject name.

Telegraf's kafka consumer input plugin does something similar. It has a setting "topics" which lets you choose which to subscribe to. Topics and jetstream subjects seem roughly analogous.

[b3rtram]

Ok thank you, have added a pull request.