Added allow_measurement_override and measurement_from_field flags

[ultramathman]

to allow setting measurement name from a field in the string
processed prior to output. For example, when translating graphite
input to influxdb output, it's more useful to set measurement from
the the metric name than as "logstash".

[contentfree]

Thanks. Can you add tests for this?

[ultramathman]

This fell off my stack. Sure. I'm also going to be adding one or two additional config options to prefer sending as tags instead of fields. I'll try to have those and the tests for them and this change by the end of the week (EST). I'll also read through and sign the CLA so those changes can be merged upstream.

[ultramathman]

I couldn't get the JRuby development stuff setup correctly to verify the tests run and succeed. After installing JRuby 9.0.4.0 and installing the rpec and bundler gems, running bundle install failed on the manticore gem (no such file to load fileutils). Attempting to install the fileutils gem, I kept getting "ERROR: Error installing fileutils: ERROR: Failed to build gem native extension."

So, since the code worked in my environment and the test syntax looked straightforward, I figured the additional tests would probably pass. If there's a problem, let me know and I'll keep working on getting the correct dev environment setup to run the tests.

Also, I signed the elastic.co CLA so these changes can be merged upstream.

[timgriffiths]

I not exactly sure why we need the extra config options send_as_fields and prefer_tags, can't the same functionality be achieved using a combination of send_as_tags and exclude_fields?

I can understand measurement_from_field

[ultramathman]

I found myself only sending "value" as a field. The prefer_tags option let me enumerate logstash fields I wanted to send as Influx fields rather than having to enumerate the logstash fields I wanted to send as Influx tags. For example, I have the following grok filter:

match => ["message", "ceph.cluster.%{GWORD:cluster}.%{GWORD:type}.%{GWORD:host}.%{GWORD:resource}.%{GWORD:metric} %{NUMBER:value} %{NUMBER:time}$"]

I want to send value as an Influx field and everything else as tags. Without prefer_tags and send_as_fields I need to do:

exclude_fields => ["@timestamp", "message", "@Version", "tags"]
send_as_tags => ["cluster", "type", "host", "resource", "metric"]

If I add another tag in a different grok filter, I then need to update send_as_tags. Turns out, I forgot to do that a lot. However, if I have prefer_tags and send_as fields it just looks like:

exclude_fields => ["@timestamp", "message", "@Version", "tags"]
prefer_tags => true

So in short, yes you can get the same outcome with send_as_tags and exclude_fields. However, prefer_tags and send_as_fields means that if you're primarily sending logstash fields as tags, you don't need to constantly update the output section as you write your grok rules.

[timgriffiths]

Ok that makes a lot more sense now, hope it gets merged 👍