in_statsd: implement the support for "statsd" protocol

[fujimotos]

This is the first cut at adding statsd support to Fluent Bit.
You can use the in_statsd plugins as follows:

$ fluent-bit -i statsd -o stdout

... now you can input metrics like:

$ echo "click:10|c|@0.1" > /dev/udp/127.0.0.1/8125
$ echo "active:+10|g" > /dev/udp/127.0.0.1/8125

This plugin will parse the incoming messages and produce well-
formatted records like below:

{"type"=>"counter", "bucket"=>"click", "value"=>10.000000, "sample_rate"=>0.100000}
{"type"=>"gauge", "bucket"=>"active", "value"=>10.000000, "incremental"=>1}

With this, we can easily collect performance logs from services
with statsd ouput support (like cadvisor).

Signed-off-by: Fujimoto Seiji [email protected]

[fujimotos]

For reference, the original issue is #1546.

[edsiper]

thanks @fujimotos ! .

The overall plugin looks good, I've just asked for minor adjustments.

[fujimotos]

The overall plugin looks good, I've just asked for minor adjustments.

@edsiper Thank you! I posted a updated version of this patch at
e13df89 that should fix all the issues you pointed out.

I also posted the doc for the new plugin at fluent/fluent-bit-docs/pull/239.
Merge that too when you pull this PR into the main.

[fujimotos]

Rebased to v1.3.0.

[PettitWesley]

@fujimotos @edsiper I'm curious is in_statsd is planned for v1.4 or not?

AWS is very interested in this feature.

[fujimotos]

@eduardo @PettitWesley I took some time this morning to test this patch
(integrating with the latest master HEAD) and could confirm it worked.

So I believe this is safely mergeable into master.

Bash

$ echo 'foo:10|c' > /dev/udp/127.0.0.1/8125
$ echo 'foo:100|g' > /dev/udp/127.0.0.1/8125
$ echo 'foo:10|ms|@0.1' > /dev/udp/127.0.0.1/8125
$ echo 'foo:baa|s' > /dev/udp/127.0.0.1/8125

Fluent Bit (master HEAD + 1756)

$ fluent-bit -i statsd -o stdout
...
[0] statsd.0: [1579139088.542499259, {"type"=>"counter", "bucket"=>"foo", "value"=>10.000000, "sample_rate"=>1.000000}]
[0] statsd.0: [1579139092.781984318, {"type"=>"gauge", "bucket"=>"foo", "value"=>100.000000, "incremental"=>0}]
[0] statsd.0: [1579139111.553117959, {"type"=>"timer", "bucket"=>"foo", "value"=>10.000000, "sample_rate"=>0.100000}]
[0] statsd.0: [1579139129.373909083, {"type"=>"set", "bucket"=>"foo", "value"=>"baa"}]

[edsiper]

thanks!