Feature request: support graphite templates for plugin httpjson

[p3r7]

The statsd plugin support graphite templates (cf statsd plugin documentation and more general template documentation).

It would be nice for plugin httpjson to support this feature as well.

Indeed, it flattens the retrieved JSON and some subelements could be considered as tag names.

[sparrc]

statsd is a more natural fit for that, I have no idea how we could support something like this in httpjson, since the httpjson plugin constructs measurement names out of json trees rather than dot buckets

[p3r7]

Mhm ok, but, I fail to see the difference between a dot bucket and a flattened json path, with '.' as node separators.

Statsd and graphite work nice for push mode, and httpjson seemed like a good alternative for pull mode.

[sparrc]

I'd probably recommend prometheus, you can also push tags into the json directly so I'm not sure I understand the point of templating anyways

[p3r7]

Setting the tag into the json sets it globally, for the whole json document.
The idea here would have had been to have different tags for different fields inside a same document.

Here is a minimalist example.

The following JSON document

{
    "responseType": {
        "methodName1": 123,
        "methodName2": 134,
        "methodName3": 105
    }
}

Would translate after flattening into the fields:

responseType.methodName1 123
responseType.methodName2 134
responseType.methodName3 105

Having a template like so

templates = [
    "measurement.method"
]

Would allow to store our 3 fields into a single measurement, but with different tags:

responseType,method=methodName1 123
responseType,method=methodName2 134
responseType,method=methodName3 105

Maybe you see a better way to achieve this in pull mode ?

Concerning prometheus, I would rather not multiply the number of components deployed, and rely only on the telegraf / influxDB combo.

[sparrc]

There has been a request for an input plugin similar to httpjson that can scrape all data formats, see #813. note that there is already a workaround for that. You can use the exec plugin to call curl <url>.

The main reason I'm resistant to this is because once every 2-4 weeks someone opens a feature request for a different way of parsing their particular JSON metrics. It's impossible to entertain every JSON parsing feature because there are an infinite number of ways that generic JSON could be parsed into influx line-protocol. The only truly generic solution would be to have a JSON-path parser, see #1363

[sparrc]

what's more, you could also do what you want when #1965 gets merged, as you will be able to separate multiple JSON documents by using an array at the root.

[p3r7]

Well, sir, thank you sir.
Indeed, a JSON-path parser would be the most flexible solution.