Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Elasticsearch query rule doesn't work with "KQL or Lucene" query #151609

Closed
ersin-erdal opened this issue Feb 20, 2023 · 3 comments · Fixed by #151632
Closed

Elasticsearch query rule doesn't work with "KQL or Lucene" query #151609

ersin-erdal opened this issue Feb 20, 2023 · 3 comments · Fixed by #151632
Assignees
@jughosta
Labels
bug Fixes for quality problems that affect the customer experience feedback_needed Team:DataDiscovery Discover, search (e.g. data plugin and KQL), data views, saved searches. For ES|QL, use Team:ES|QL.

Comments

@ersin-erdal
Copy link
Contributor

When i run Elasticsearch query rule with KQL or Lucene i get below error.
The query works when i click on Test query button.

I tried it with both a custom index and .kibana-event-log index, both are failing. But the rule with the same params but with DSL query works.

Error: Cannot read properties of undefined (reading 'map')
    at updateFilterReferences (fetch_search_source_query.ts:198:18)
    at generateLink (fetch_search_source_query.ts:182:26)
    at runMicrotasks (<anonymous>)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
    at fetchSearchSourceQuery (fetch_search_source_query.ts:76:16)
    at executor (executor.ts:67:7)
    at Object.executor (rule_type.ts:186:14)
    at task_runner.ts:340:28
    at TaskRunnerTimer.runWithTimer (task_runner_timer.ts:49:20)
    at TaskRunner.runRule (task_runner.ts:304:38)
    at TaskRunner.run (task_runner.ts:668:31)
    at TaskManagerRunner.run (task_runner.ts:304:22)
@ersin-erdal ersin-erdal added bug Fixes for quality problems that affect the customer experience Team:DataDiscovery Discover, search (e.g. data plugin and KQL), data views, saved searches. For ES|QL, use Team:ES|QL. labels Feb 20, 2023
@elasticmachine
Copy link
Contributor

Pinging @elastic/kibana-data-discovery (Team:DataDiscovery)

@kertal
Copy link
Member

kertal commented Feb 20, 2023

Could you provide some details how you created the rule, which query, filters, data view? And which version of Kibana are you using?

@jughosta jughosta self-assigned this Feb 20, 2023
@ersin-erdal
Copy link
Contributor Author

Sure, I used Kibana UI to create the rule.

First i created a new index in ES and added 2 docs for testing.
Then created a rule with DSL which checks if there are docs more than 1 for the last 5 days.
Rule works.

Then i tried the same things with KQL and Lucene.
Created a DataView for the index i created.
When i click on Test rule button it works,
but when i wait for the rule to run, it fails and returns the error above.

Then I tried with .kibana-event-log-8.8.0-000001 index as well (just in case of my custom index is broken) failed again.
Asked a colleague to test the rule, failed or her too.

I am using the main branch, so v8.8.

dw

rule

Screenshot 2023-02-20 at 16 10 50

@jughosta jughosta mentioned this issue Feb 20, 2023
Merged
jughosta added a commit that referenced this issue Feb 21, 2023
@jughosta
[Discover] Fix Elasticsearch query rule for "KQL or Lucene" query wit…
17ab3a3 
…hout filters (#151632)

Closes #151609 

For testing:
- Go to Rule Management page
- Create a new Rule with "Elasticsearch query" type in "KQL or Lucene"
format
- Save and check the status after the rule finishes the creating process
=> It should say "Succeeded"
kibanamachine pushed a commit to kibanamachine/kibana that referenced this issue Feb 21, 2023
@jughosta
[Discover] Fix Elasticsearch query rule for "KQL or Lucene" query wit…
d2e0c2e 
…hout filters (elastic#151632)

Closes elastic#151609

For testing:
- Go to Rule Management page
- Create a new Rule with "Elasticsearch query" type in "KQL or Lucene"
format
- Save and check the status after the rule finishes the creating process
=> It should say "Succeeded"

(cherry picked from commit 17ab3a3)
kibanamachine added a commit that referenced this issue Feb 21, 2023
@kibanamachine @jughosta
[8.7] [Discover] Fix Elasticsearch query rule for "KQL or Lucene" que…
ba57a80 
…ry without filters (#151632) (#151705)

# Backport

This will backport the following commits from `main` to `8.7`:
- [[Discover] Fix Elasticsearch query rule for "KQL or Lucene" query
without filters
(#151632)](#151632)

<!--- Backport version: 8.9.7 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)

<!--BACKPORT [{"author":{"name":"Julia
Rechkunova","email":"[email protected]"},"sourceCommit":{"committedDate":"2023-02-21T13:46:42Z","message":"[Discover]
Fix Elasticsearch query rule for \"KQL or Lucene\" query without filters
(#151632)\n\nCloses #151609
\r\n\r\nFor testing:\r\n- Go to Rule Management page\r\n- Create a new
Rule with \"Elasticsearch query\" type in \"KQL or
Lucene\"\r\nformat\r\n- Save and check the status after the rule
finishes the creating process\r\n=> It should say
\"Succeeded\"","sha":"17ab3a31dd8528d92d3c4a928b7e0923a48b037a","branchLabelMapping":{"^v8.8.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:fix","Team:DataDiscovery","backport:prev-minor","v8.8.0"],"number":151632,"url":"https://github.com/elastic/kibana/pull/151632","mergeCommit":{"message":"[Discover]
Fix Elasticsearch query rule for \"KQL or Lucene\" query without filters
(#151632)\n\nCloses #151609
\r\n\r\nFor testing:\r\n- Go to Rule Management page\r\n- Create a new
Rule with \"Elasticsearch query\" type in \"KQL or
Lucene\"\r\nformat\r\n- Save and check the status after the rule
finishes the creating process\r\n=> It should say
\"Succeeded\"","sha":"17ab3a31dd8528d92d3c4a928b7e0923a48b037a"}},"sourceBranch":"main","suggestedTargetBranches":[],"targetPullRequestStates":[{"branch":"main","label":"v8.8.0","labelRegex":"^v8.8.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/151632","number":151632,"mergeCommit":{"message":"[Discover]
Fix Elasticsearch query rule for \"KQL or Lucene\" query without filters
(#151632)\n\nCloses #151609
\r\n\r\nFor testing:\r\n- Go to Rule Management page\r\n- Create a new
Rule with \"Elasticsearch query\" type in \"KQL or
Lucene\"\r\nformat\r\n- Save and check the status after the rule
finishes the creating process\r\n=> It should say
\"Succeeded\"","sha":"17ab3a31dd8528d92d3c4a928b7e0923a48b037a"}}]}]
BACKPORT-->

Co-authored-by: Julia Rechkunova <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@jughosta jughosta

Labels
bug Fixes for quality problems that affect the customer experience feedback_needed Team:DataDiscovery Discover, search (e.g. data plugin and KQL), data views, saved searches. For ES|QL, use Team:ES|QL.
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

[Discover] Fix Elasticsearch query rule for "KQL or Lucene" query without filters jughosta/kibana
4 participants
@kertal @jughosta @elasticmachine @ersin-erdal