Support sending metrics to Kafka

[pauldix]

We should be able to send metrics to a variety of places other than InfluxDB. This means that we'll need to have some sort of framework for defining new output sinks. This also means that we should be able to disable sending metrics to any of these output sinks, including InfluxDB.

What I'm thinking is to pull the InfluxDB settings into another area like outputs. Then, like plugins, define a number of different outputs where metrics can be sent to.

For Kafka, we should send metrics using the line protocol. This should be fairly simply using the InfluxDB client to convert the metrics to their line protocol equivalents.

The work to add support for Riemann (#34) will also need this output sink implementation.

[vic3lord]

👍

[jipperinbham]

👍

Has work started on this or are you looking for some help/ideas?

In another project similar to this, we use In/Out channels to wire up the message communication and defined a standard Msg struct to be passed along. It seems to work well for that project so didn't know if something similar would be good here as well.

[pauldix]

We haven't started work on this and are definitely looking for help. We want to be able to to output to one or more output sinks like Kafka or InfluxDB or Riemann

On Jul 8, 2015, at 3:11 PM, JP Phillips [email protected] wrote:

Has work started on this or are you looking for some help/ideas?

In another project similar to this, we use In/Out channels to wire up the message communication and defined a standard Msg struct to be passed along. It seems to work well for that project so didn't know if something similar would be good here as well.

—
Reply to this email directly or view it on GitHub.

[steverweber]

some help perhaps:
https://github.com/mozilla-services/heka/search?utf8=%E2%9C%93&q=Kafka