[Lens] Ignore global filters in data layers

[stratoula]

Describe the feature:
There are users that are asking to add the Ignore global filters functionality (per data layer) in Lens. This is useful in some cases, for example I want to filter some visualizations in a dashboard but exclude others from filtering.

This functionality also exists in TSVB so we should consider enabling it for the TSVB to Lens transition.
Moreover it already exists in the annotations layer.

[elasticmachine]

Pinging @elastic/kibana-vis-editors @elastic/kibana-vis-editors-external (Team:VisEditors)

[dej611]

Should it be per layer or per visualization?
Probably the former is easier to implement given the current state, but the latter makes more sense to me in your use case.

[stratoula]

Good question, TSVB allows both (per layer and per chart)
I was thinking that the per layer is closer to what we are doing with annotations and maybe gives more flexibility to the users but I see your point! @ninoslavmiskovic wdyt?

[flash1293]

+1 for the per layer switch, no need to make it more complex with another switch in another place IMHO

[ninoslavmiskovic]

Do we know if users are using the per chart on TSVB?

[flash1293]

@ninoslavmiskovic we don't track that

[ninoslavmiskovic]

In that case let's go for the layer to ensure consistency within Lens.

[stratoula]

I was thinking about this feature. I wonder if it makes sense to also have an indicator on the dashboard level that this panel is ignoring the dashboard filters. (?)

[flash1293]

Cc’ing the @elastic/kibana-presentation for the comment above - has there been thoughts along those lines? It seems to fit the “Panel filters” badge well

[Heenawter]

We have an issue defined for having a visual indicator for when global filters don't apply to a visualization (for example, when multiple data views are used in a single dashboard). I think this could tie nicely into that! 👀 cc @cqliu1

[nreese]

Another thing to consider is that part of the panel could be ignoring global filters while another part of the panel could be using global filters.

In Maps, individual layers can ignore global search and global time. This information is conveyed to users via a tooltip on each layer that specifies whether the results are narrowed by global time and global search.

[ThomThomson]

Great call about adding this info alongside the filter ignored system @Heenawter. IMO, we should maybe go further and combine all of the filtering information for a panel into one universal filtering info badge. The badge would serve the purpose of showing the user that something is up with the filtering of this panel, and clicking on it would surface relevant information like:

• Are there any panel-level filters saved into this panel?
• Are there global filters that are ignored by this panel due to a data view conflict etc?
• Are the individual layers of this panel configured with their own filters? Are they configured to ignore all or some of the global filters?

Since we already rely on the Embeddable implementations to surface the filters, we could modify our code to also allow them to surface some custom filtering info UI. That way, embeddable types like Lens and Maps which have multiple layers could build and display their own layer filtering info in this modal / flyout.

@nreese, even though Maps has the advantage of showing the layers in context, I think it could be beneficial to surface the panel filtering info via this badge as well. Do you think that might work?

[stratoula]

@ThomThomson ++ on your proposal. It will be super useful for our users to have this information on the dashboard level.

[jack-morrison]

Are the newer Controls implemented as global filters? (IIRC the now deprecated Input Controls appeared to work by adding/removing filters.) If this proposal would allow for a data layer to ignore a dashboard's Control selections, I'd be very excited about this feature.

My use case would be to essentially to "fix" a baseline series on a chart that remains stable through an end-user making Controls selections which would otherwise filter out the baseline's data.

[Control] Job #: 456
---------------------
[Chart] 
Series showing Job 123 (fixed baseline; ignores Control/global filters)
Series showing Job 456 (variable through Control selections)

To my knowledge, this isn't possible today in Lens using non-timeseries data.

[ThomThomson]

@jack-morrison I can confirm from the Dashboard side that it does pass down Control selections as filters to its children. It does that without creating new filter pills, but from the Lens POV, all passed down filters should be treated the same.

[jack-morrison]

@ThomThomson thanks for the quick response and info. If I understand correctly, this proposal should enable the use case I've been hoping for!

[code4mankind]

++ for the per layer switch - we are hitting this same limitation. The dashboard controls (filters) are passed thru to the "children" lens view, however, the annotation layer on the lens view does not respect the controls (filters).

In it's current form, it makes the annotations layer unusable. Our dashboard controls typically represent environments/services and require the selection of one option only in each control (providing a narrow filter for the dashboard). However, the annotations layer will overlay with data from all environments/services which can be misleading.

[stratoula]

@code4mankind annotations layer has this setting, not sure if you have seen it.