Enable histogram panel to plot multiple fields

[bfiorini]

Closes #199

This is a first prototype to enable histogram panel to plot multiple fields. It also enables the pairing of a "plot" (a line in the graph) to a set of queries.
Thus it introduces a new concept to the panel: a plot, which is represented by one (or more) lines on the graph. It is associated with: a mode (same as before), a value field (when the mode is different than 'count'), and a set of query ('all','pinned','unpinned','selected'). The default color of a line is taken from the query it is paired to, but it can be changed for each plot. A plot can also have an alias, cause a pair (queryA, value_fieldA) may represents something different than the pair (queryA, value_fieldB).

Worth noting that the time_field is shared across all plots.

How the panel looks like: (left panel=>first query, right panel=>second query)

How the editor looks like: (editor for the second panel)

[rashidkpc]

I like the idea, but I think it needs some refinement before its ready to merge. For example, if you change the color for a "plot" it desyncs them visually from the query panel, so now there's no way to tell what query is paired to which line. Even without changing the color you can't tell what the line represents (count/min/mean/max/total)

[olej-a]

One query can be source for more histogram lines (multiple fields in message) and one field can be displayed as multiple lines, fe. min/max/mean. There may be hard to synchronize query panel and histogram panel. I prefer allow setup own histogram legend for each line/data series.

[miguelcoelhosantos]

Having all the searches in one place is in principle a good idea but in practice it's kind of awkward a lot of the times. I've seen dashboards with 15+ searches, and if you have 15+ searches colors start to overlap (depending on eye color resolution). So my current thinking is that the "master search panel" is a good idea for pages with 5 or 6 searches max.
If people want/need to have different histograms with different searches on the same dashboard (being min/max/mean/value or simply counts) than it might be a better idea to have more than one "search" panel, with a panel name and the searches+label+color. Than place these panels next to the histograms and reference them using panelname.label perhaps?
my 2 cents

P.S. the filtering panel(s) seem to be going in a really nice direction and are pretty useful to drill down on your searches.

[olej-a]

Real world example> I have data source in format timestamp rowtype,field1=value1,field2=value2,field3=value3

I want to display values in histogram (mean/max/min) for selected fields (field1, field2). I can query rowtype and timerange. Multiquery+histogram panel works just for count or single selected field expecting data in different format>

timestamp record_type_field1,generic_field=value1
timestamp record_type_field2,generic_field=value2
timestamp record_type_field3,generic_field=value3

It can be displayed in histogram with multiquery. Each query request record_type_fieldX and histogram will be setup to field generic_field.

For now there may be solution transform data before indexing in Elasticsearch or use tvvmb solution.

[acasademont]

How is this going? It would be really useful!

[slopes]

Do you have some news about this feature ? It would really help.

[gposton]

I'd like to see this accepted as well.

[yenery]

This feature would be very useful.

[Esya]

+1 for this feature, bump

[chnrxn]

+1 for me as well. This is exactly what I am looking for.

[latenightsurfer]

+1 also, this feature would be really a good benefit and exactly what is missing for me

[olej-a]

+1

[fbarbeira]

+1

[biancalana]

+1

[Seldaek]

I just ran into this limitation because I have one "sysstats" message type that has many fields like memory free, memory used, cpu idle, cpu used, etc. I wanted to use these many fields to show graphs in two colors with used/free, but right now it's not possible unless I create one message per field and then do a query for free memory and one for used memory where messages would both have a "value" field that the graph can display. I would love to see this merged.

[centos-bz]

+1 the fealtrue is the bigest limitation I think,It would be force me to use tvvmb's

[miguelcoelhosantos]

Will this be fixed with the new aggregation framework?
Will it be possible to pick different aggregations of fields in the same query and then pick which of these get plotted (e.g. stacked) on the histogram?

[electramax]

Honestly, this is the ONLY missing feature that steers us clear of Kibana, which is otherwise perfect. Does anyone know if tvvmb's solutuion works with the latest Kibana release? This looks like a version back, but I could be wrong.

[bfiorini]

Yeah it's quite old you're right, it does not work with the latest version for sure.

[electramax]

Thanks tvvmb. Any chance you're working on an update?

[bfiorini]

Not at the moment :)

[electramax]

Dang! I really hope they sort this out for future releases. We have been using straight Graphite for cross-metric comparison and always assumed it would be a standard feature for any data-visualization software. You know what they say about assuming... thanks mate!

[antongorshkov]

+1 would love to this see

[cjchand]

Dang! I really hope they sort this out for future releases. We have been using straight Graphite for cross-metric comparison and always assumed it would be a standard feature for any data-visualization software. You know what they say about assuming... thanks mate!

+1 on this. I'm currently working on getting metrics and logging for OpenStack into one place - preferably Kibana :). When you combine data visualization with the ability to facet and drill down to a given time range, that would make for a great "cockpit" in order to keep tabs on all those moving parts.

As it stands, it looks like I will have to view some data in Graphite because of the histogram panel's inability to graph multiple fields :( That means having two tools to do the job.

Sure, I can wrap both in an iframe to put them on the same page, but not having the two locked into the same timeframe makes is less valuable.

[plankthom]

+1 me too, but with a suggestion:

organize these selections of fields and aggregation methods in a seperate 'metrics' section (as done with 'query' and 'filtering').
While queries get there visual clues from color, metrics apply other style variants (e.g. line style, symbols, area fills, shifts in hue or lightness).

In the configuration of e.g. a histogram panel, one combines prefined queries with predefined metrics.

A general metrics section might also be the place to

• indicate the dimension of a field ( money, bits, counts, ...)
• define custom labels (e.g. 'mean request duration' ) that is prepended in the legend to the query label ('mean request duration of clientX')
• group metrics (as with the 'pinned' and 'unpinned' queries): allows histogram panels to select a consistent group of metrics. One can envision that groups get there own visual clues ( high-low plot, percentile or histogram buckets ..)

[miguelcoelhosantos]

Is this in the roadmap (particularly now that ES 1.0 is out)?

[elvarb]

+1

[KlavsKlavsen]

+1

[mcallaway]

+1

[mirath]

+1. This would be really useful, the community seems to agree on this point. What's keeping this from getting merged? Can we do something to have this in the upstream soon?

[martinseener]

+1

[MMore]

+1

[radaczynski]

+1

[bbradbury]

Jumping on the bandwagon-- this would be killer.

[wols]

+1

[ymkim92]

+1 I think this should be a mandatory function of next kibana.

[mickdekkers]

+1

[chrisapl]

+1

[ghost]

+1

[tisba]

I'd love to see this feature. I'm wondering though why this is laying around for almost a year now :(

[bergamoyat]

+1

[kelf]

+1

[rvera]

+1 this is highly desirable.

[dchichkov]

Looks like an alternative implementation is available here: #1296

[meng-fu]

+1

[Suraj2496242]

+1

[sbbird]

+1

[diegoasth]

+1

[felixbouleau]

+1

[pdf]

I sincerely wish people would stop leaving +1 comments here, they're not going to help get this merged any faster at this point, and you spam everyone who's monitoring this issue for progress, which just creates noise for everyone. I'm sorry for adding to the noise, but at least if this stops two people from leaving pointless comments, we'll be ahead.

[yuj]

i agreed. oops! :)