[Lens] [discuss] Improve data panel UX

[flash1293]

The data panel is a very central UI component of Lens and should make it as easy as possible to pick the right field. Especially for big index patterns with up to thousands of fields with very long names this can be very hard to do.

This issue is meant to collect ideas around this problem.

Be smart about ECS fields

If an index pattern has ECS compatible fields, we know things about them (e.g. approximate cardinality and whether it contains an ID or a metric). We can use this to show info icons / popovers with documentation about this field (this is an IP address which receives packages). As we know about the current data table, we can prioritize fields which make sense for the current chart and show them at the top of the list (e.g. if the user already has a terms agg on myfancyservice.http.response.status_code, then it might make sense to use the myfancyservice.http.response.body.bytes - lots of possibilities here.

Respect "collapsed field names" settings

There is the shortDots:enable setting which we should respect in Lens

Tree view for fields with same prefixes

We could collapse fields which share a prefix (maybe one or two levels) to shorten the list and encourage a drill down selection of the right field

Pick view based on size of index pattern

We shouldn't use tree view for every index pattern, but only for patterns which have a lot of common prefixes

(for completeness sake)

Highlight high-density and/or high-cardinality fields

Fields are most interesting if they have high density (few documents without this field) and high cardinality (a lot of different values). If we have this information somehow, we can use it to prioritize "interesting" fields. elastic/elasticsearch#43704 (comment)

Getting this data is difficult though, there is some discussion going on around this.

ping @AlonaNadler @cchaos @wylieconlon @chrisdavies @timroes

[cchaos]

Just a couple quick thoughts

Respect "collapsed field names" settings

This could also be a toggle-able setting within the fields panel itself. Meaning, if they don't have the setting turned on in advanced settings, they can still see the collapsed behavior by switching it on in Lens. Then you could think of the advanced setting more of a universal default not as the only way to turn this feature on.

Tree view for fields with same prefixes

We'd have to really explore the UI for this. It can get tricky when you start filtering by keyword or type using a tree view. Maybe a couple examples of other applications doing it well would help.

---

The idea of long field names being truncated or not fully shown is not unique to Lens. Obviously the current "solution" to that is using the collapsed dots. However, if we are investigating ways to make the UI present these better, we should consider how it could be used universally so that we're not creating unique-one-off experiences and that other apps might benefit from this solution.

[elasticmachine]

Pinging @elastic/kibana-app

[kertal]

I'm also interested in this discussion (intersection with the "field chooser" of discover)

[flash1293]

Another feature we should support is separating used fields and unused fields like discover is doing it:

[flash1293]

This issue is pretty old and there hasn't been much progress lately. Let's close it and reboot the effort in a better setting (I still think there's room for improvement here)