Creates ES|QL where filters for ordinal charts

[stratoula]

Summary

Closes #183425

This PR enables the creation of where clause filters by clicking a chart in Discover and wherever the ES|QL editor exists. This means that this is not available in dashboards.

This is possible only for ordinal charts. For date fields is quite difficult to know the interval so I don't allow it for now. We already support brushing so time filtering is already available.

Checklist

• Unit or functional tests were updated or added to match the most common scenarios
• This was checked for cross-browser compatibility

[elasticmachine]

Pinging @elastic/kibana-esql (Team:ESQL)

[afharo]

kibana.jsonc changes LGTM

[dej611]

Did a first review round.

Noticed some possible improvement over the appendWhereClauseToESQLQuery logic:

• forced casting fro ip field types

  from kibana_sample_data_logs | stats count() by ip | where ip == "55.237.229.59"
  

  that is a valid ES|QL, but currently what it gets configured is this version:

  from kibana_sample_data_logs | stats count() by ip | where `ip`::string=="55.237.229.59"
  

• why is it always quoting variable/fields? There's an AST function that can help here, identifying when a variable/field needs to be wrapped.

• where the filter is triggered by the multi-series tooltip is does not work

• is it a bit annoying that when the where clause is added to the query there's the probability that the visualization changes. I see how that would make "more sense" for some data changes, but perhaps the process should include a prompt to "suggest" to the user the new chart type while keeping the previous type?

When in dashboard I've noticed that legend items have the filter popup active. Of course clicking on them doesn't do anything, as the filtering I understand applies only to Discover, right?
This is present also in main, so it's ok to not specifically fix this here.

[stratoula]

@dej611 thanx. My answers to your questions:

forced casting fro ip field types ...

When we created the function ip fields needed casting. I want to revisit this function but I am waiting some more cast improvements from the ES team. I would do this on a follow up PR, it is irrelevant with the current, as this functionality already exists in main

why is it always quoting variable/fields? There's an AST function that can help here, identifying when a variable/field needs to be wrapped.

Same as above!

is it a bit annoying that when the where clause is added to the query there's the probability that the visualization changes. I see how that would make "more sense" for some data changes, but perhaps the process should include a prompt to "suggest" to the user the new chart type while keeping the previous type?

We have already an issue for this #184631

When in dashboard I've noticed that legend items have the filter popup active. Of course clicking on them doesn't do anything, as the filtering I understand applies only to Discover, right?

True I will create an issue for that

where the filter is triggered by the multi-series tooltip is does not work

Thanx I will take a look in this PR

[stratoula]

• appendWhereClauseToESQLQuery enhancements [ES|QL] Improvements on appendWhereClauseToESQLQuery #184726 (this function is already used in main)

• the popover issue [Lens] [ES|QL] Disable the legend filter popover when a Lens chart is rendered in a dashboard #184727

[stratoula]

@dej611 good catch with the tooltip actions, I disabled it for this PR (I had already done it for XY charts but not for heatmap and partition charts)

[drewdaemon]

Looking good! Just a couple of questions on some async code

[lukasolson]

I think I'm seeing another bug. I'm using this query:

from kibana_sample_data_logs
| stats total_bytes = sum(bytes) by agent

And in the Lens configuration, I'm using agent as the Breakdown field, and when I click on a bar in the chart, it doesn't filter and I get this error:

create_filters_from_value_click.ts:124 Uncaught (in promise) TypeError: Cannot read properties of undefined (reading 'meta')
    at create_filters_from_value_click.ts:124:1
    at Array.filter (<anonymous>)
    at appendFilterToESQLQueryFromValueClickAction (create_filters_from_value_click.ts:122:1)

Screen recording:

Screen.Recording.2024-06-06.at.1.53.03.PM.mov

[stratoula]

It is due to @dej611 's recommendation. It worked before. I will look into it, thanx

[stratoula]

@lukasolson fixed

[lukasolson]

Okay, I found another issue, but I think it's an existing bug that just shows up in this PR because clicking on a bar before this PR does nothing.

1. Load sample web logs
2. Switch to ES|QL and use the following query:

from kibana_sample_data_logs
| stats total_bytes = sum(bytes) by extension, bytes % 5

3. Edit the Lens visualization and drag bytes % 5 to the horizontal axis (notice the x-axis title still says "extension" even after you changed it
4. Click on a bar in the chart

Instead of filtering on bytes % 5, it filters on extension and you get the following error:

[esql] > Unexpected error from Elasticsearch: verification_exception - Found 1 problem
line 3:9: first argument of [`extension`==2] is [text] so second argument must also be [text] but was [integer]

Screen recording:

Untitled.mov

[lukasolson]

This is really awesome! It's great to see this all coming together. Just added a couple of minor nits below. Regarding the other bug, I'm not sure it's something we need to fix in this PR since it's probably an existing bug.

[lukasolson]

Nit: Can we use a constant (and export it) for 'UPDATE_ESQL_QUERY_TRIGGER'?

[stratoula]

The reason I didnt do it is because if I export it from the text-based languages plugin (where the trigger is being registered) it will create circular dependencies with the data plugin. I could export it from the esql-utils package but it seems a bit unrelated to me. So I will leave it as it is for now and I will think about it more when another usage will be required.

[stratoula]

@lukasolson thanx, I created a separate issue for the bug here #185636 because the root cause is different. If you remove the columns and add it by clicking the dimension button and filter you get the filter correctly (and the axis(, the drag and drop does something weird here.

[stratoula]

/ci

[kibana-ci]

💚 Build Succeeded

• Buildkite Build
• Commit: c1ba747

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id	before	after	diff
data	499	512	+13
textBasedLanguages	54	58	+4
total			+17

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
expressionHeatmap	26.5KB	26.6KB	+100.0B
expressionPartitionVis	36.4KB	36.5KB	+87.0B
expressionXY	128.7KB	128.7KB	-5.0B
textBasedLanguages	166.8KB	167.3KB	+523.0B
total			+705.0B

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id	before	after	diff
data	418.5KB	420.1KB	+1.6KB
textBasedLanguages	5.6KB	6.7KB	+1.1KB
total			+2.7KB

Unknown metric groups

async chunk count

id	before	after	diff
textBasedLanguages	2	3	+1

History

• 💚 Build #214607 succeeded a61f351
• 💛 Build #214430 was flaky 577c9ae
• 💚 Build #214429 succeeded 577c9ae
• 💚 Build #214386 succeeded ec35954
• 💚 Build #214350 succeeded d8836a9
• 💔 Build #214208 failed 1b5a3ae

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

[drewdaemon]

LGTM!