Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ResponseOps][Alerts] Fix DSL filter issues in search bar #193623

Merged
merged 38 commits into from
Oct 13, 2024

Conversation

js-jankisalvi
Copy link
Contributor

@js-jankisalvi js-jankisalvi commented Sep 20, 2024

Summary

Resolves #183908
Resolves #192557

This PR fixes an issue of DSL filter in

Search bar in Alerts page of Stack management image
Search bar in Alerts page of Observability image
Search bar in "If alerts matches query" action filter image
Search bar in Maintenance window page image

Checklist

How to verify

  • Create a DSL filter in above mentioned pages
  • Verify that filter is applied properly

@js-jankisalvi js-jankisalvi self-assigned this Sep 25, 2024
@js-jankisalvi js-jankisalvi added release_note:skip Skip the PR/issue when compiling release notes Team:ResponseOps Label for the ResponseOps team (formerly the Cases and Alerting teams) v9.0.0 v8.16.0 backport:prev-minor Backport to (8.x) the previous minor version (i.e. one version back from main) labels Sep 25, 2024
@js-jankisalvi js-jankisalvi changed the title [ResponseOps][Alerts] Fix DSL filter issues [ResponseOps][Alerts] Fix DSL filter issues in search bar Sep 25, 2024
@js-jankisalvi js-jankisalvi marked this pull request as ready for review September 25, 2024 10:44
@js-jankisalvi js-jankisalvi requested a review from a team as a code owner September 25, 2024 10:44
@elasticmachine
Copy link
Contributor

Pinging @elastic/response-ops (Team:ResponseOps)

Copy link
Member

@maryam-saeidi maryam-saeidi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@cnasikas nice catch! and thanks @js-jankisalvi for fixing it!

When I tested locally, the first filter was added correctly, but I wasn't able to add a second one. It worked fine in the Discover.

Here is what happens:

Screen.Recording.2024-09-25.at.17.19.13.mov

@js-jankisalvi
Copy link
Contributor Author

js-jankisalvi commented Sep 26, 2024

When I tested locally, the first filter was added correctly, but I wasn't able to add a second one. It worked fine in the Discover.

Fixed

image

@js-jankisalvi js-jankisalvi enabled auto-merge (squash) October 8, 2024 09:46
Copy link
Member

@markov00 markov00 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe there is something wrong with the current unified search custom DSL filter implementation and we should fix that behaviour instead of adding this changes that to me looks like a workaround.
Let me discuss this with the @elastic/kibana-visualizations team and we will be back with a solution

return {
$state,
meta,
query: filter?.query ? { ...filter.query } : { ...rest },
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you please explain what you are trying to achieve here?
Filter type has only 3 properties: $state, meta, query?.
When deconstructing a filter the rest parameter should be just the query param.
In this ternary we are saying that if filter.query doesn't exist then use filter.query that doesn't exist.
So I'm not sure what we are updating here.
It looks like the Filter objects are "hiding" something more hidden from the current Typing. This is definitely not great and cause tech debt that is hard to find

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On an subsequent check I see that the filter is currently constructed in the wrong way and in general it doesn't conform to its type: Filter

type Filter = {
  $state?: {
    store: FilterStateStore;
  };
  meta: FilterMeta;
  query?: Record<string, any>;
};

The parameter query is not populated with the custom DSL, but the custom DSL always pollutes the rest of the object. I see this as a wrong implementation (not yours but the current unified search implementation) that needs to be fixed to make it work correctly in all cases.
IMHO what we are trying to push here is a workaround of a bigger problem, than an actual fix.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The parameter query is not populated with the custom DSL, but the custom DSL always pollutes the rest of the object.

Exactly!
Above changes had to be done because when filters are updated via onFiltersUpdate, the DSL filter value is not wrapped in a query which throws validation error in the rule's action as it is expected to be a query object.

filter_query.bug.mov

IMHO what we are trying to push here is a workaround of a bigger problem, than an actual fix.

Agree, however as per my understanding not all plugins/ scenarios expects custom DSL value to be wrapped in a query object. Applying DSL filter to Alerts table works fine without query object.
Hence I fixed it wherever it was needed.
CC @cnasikas

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if you don't mind I'd like to see what could happen if I change your code like that: 9c8b5a3
let the CI run and see of this change trigger any error in the use of the unified search. This definitely work with your use case, but I'd like to check if this causes problem in the surrounding

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment on lines +122 to +127
const { $state, meta, ...rest } = filter;
return {
$state,
meta,
query: filter?.query ? { ...filter.query } : { ...rest },
};
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same comment as above: rest is query from the Filter type, so what are we trying to achieve here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same reason, validation fails without query object.

@js-jankisalvi js-jankisalvi disabled auto-merge October 8, 2024 10:51
@js-jankisalvi js-jankisalvi requested a review from a team as a code owner October 8, 2024 17:17
@elasticmachine
Copy link
Contributor

elasticmachine commented Oct 8, 2024

💔 Build Failed

Failed CI Steps

Test Failures

  • [job] [logs] Jest Tests #7 / combineQueryAndFilters should accept multiple queries (and play nice with meta filters)
  • [job] [logs] Jest Tests #7 / combineQueryAndFilters should accept multiple queries (and play nice with meta filters)
  • [job] [logs] Jest Tests #7 / combineQueryAndFilters should add ignored filters as disabled
  • [job] [logs] Jest Tests #7 / combineQueryAndFilters should add ignored filters as disabled
  • [job] [logs] Jest Tests #7 / combineQueryAndFilters should append lucene meta filters to an existing lucene query
  • [job] [logs] Jest Tests #7 / combineQueryAndFilters should append lucene meta filters to an existing lucene query
  • [job] [logs] Jest Tests #7 / combineQueryAndFilters should append lucene meta filters to app filters even if existing filters are using kuery
  • [job] [logs] Jest Tests #7 / combineQueryAndFilters should append lucene meta filters to app filters even if existing filters are using kuery
  • [job] [logs] Jest Tests #7 / combineQueryAndFilters should assign kuery meta filters to app filters if existing query is using lucene language
  • [job] [logs] Jest Tests #7 / combineQueryAndFilters should assign kuery meta filters to app filters if existing query is using lucene language
  • [job] [logs] Jest Tests #7 / combineQueryAndFilters should work for complex cases of nested meta filters
  • [job] [logs] Jest Tests #7 / combineQueryAndFilters should work for complex cases of nested meta filters
  • [job] [logs] Jest Tests #7 / combineQueryAndFilters should work for prefix wildcard in disabled KQL filter
  • [job] [logs] Jest Tests #7 / combineQueryAndFilters should work for prefix wildcard in disabled KQL filter
  • [job] [logs] Jest Tests #7 / combineQueryAndFilters should work together with enabled and disabled filters
  • [job] [logs] Jest Tests #7 / combineQueryAndFilters should work together with enabled and disabled filters
  • [job] [logs] FTR Configs #25 / discover/ccs_compatible discover search errors exception on single shard shows warning and results
  • [job] [logs] FTR Configs #92 / discover/ccs_compatible discover search errors exception on single shard shows warning and results
  • [job] [logs] FTR Configs #25 / discover/ccs_compatible discover search errors exception on single shard shows warning and results
  • [job] [logs] FTR Configs #92 / discover/ccs_compatible discover search errors exception on single shard shows warning and results
  • [job] [logs] Jest Tests #14 / ruleActionsAlertsFilter should allow for changing query
  • [job] [logs] Jest Tests #14 / ruleActionsAlertsFilter should allow for changing query
  • [job] [logs] Jest Tests #4 / SearchBar sets filterManager filters correctly
  • [job] [logs] Jest Tests #4 / SearchBar sets filterManager filters correctly
  • [job] [logs] Jest Tests #4 / SearchBar updates filters correctly
  • [job] [logs] Jest Tests #4 / SearchBar updates filters correctly

Metrics [docs]

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id before after diff
alerting 93.6KB 93.7KB +36.0B
triggersActionsUi 1.6MB 1.6MB +74.0B
unifiedSearch 351.8KB 352.0KB +156.0B
total +266.0B

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id before after diff
kbnUiSharedDeps-srcJs 3.4MB 3.4MB -10.0B
triggersActionsUi 123.3KB 123.3KB +36.0B
total +26.0B

History

cc @js-jankisalvi

@markov00 markov00 self-requested a review October 10, 2024 07:55
@js-jankisalvi js-jankisalvi removed the request for review from a team October 10, 2024 11:38
Copy link
Member

@markov00 markov00 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes LGTM

@cnasikas cnasikas requested a review from jcger October 10, 2024 14:33
Copy link
Contributor

@jcger jcger left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:party_parrot:

@js-jankisalvi js-jankisalvi enabled auto-merge (squash) October 11, 2024 14:23
@js-jankisalvi js-jankisalvi merged commit 122647b into elastic:main Oct 13, 2024
37 checks passed
@js-jankisalvi js-jankisalvi deleted the dsl-filter-bug-fix branch October 13, 2024 18:42
@kibanamachine
Copy link
Contributor

Starting backport for target branches: 8.x

https://github.com/elastic/kibana/actions/runs/11316832775

kibanamachine pushed a commit to kibanamachine/kibana that referenced this pull request Oct 13, 2024
…3623)

## Summary

Resolves elastic#183908
Resolves elastic#192557

This PR fixes an issue of DSL filter in
<details><summary>Search bar in Alerts page of Stack
management</summary>
<img width="1237" alt="image"
src="https://github.com/user-attachments/assets/b9b380d2-80a7-4754-95f2-6b6831923c4d">
</details>

<details><summary>Search bar in Alerts page of Observability</summary>
<img width="1237" alt="image"
src="https://github.com/user-attachments/assets/68ceb97f-b958-47f4-b356-757cbafd9170">
</details>

<details><summary>Search bar in "If alerts matches query" action
filter</summary>
<img width="1281" alt="image"
src="https://github.com/user-attachments/assets/1f99ca43-c4b5-4f52-b50f-914f1e205c5b">
</details>

<details><summary>Search bar in Maintenance window page</summary>
<img width="1272" alt="image"
src="https://github.com/user-attachments/assets/ffaa317d-c14b-45a2-9d02-00f7a10239ab">
</details>

### Checklist
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios

### How to verify
- Create a DSL filter in above mentioned pages
- Verify that filter is applied properly

---------

Co-authored-by: kibanamachine <[email protected]>
(cherry picked from commit 122647b)
@kibanamachine
Copy link
Contributor

💚 All backports created successfully

Status Branch Result
8.x

Note: Successful backport PRs will be merged automatically after passing CI.

Questions ?

Please refer to the Backport tool documentation

kibanamachine added a commit that referenced this pull request Oct 13, 2024
) (#196033)

# Backport

This will backport the following commits from `main` to `8.x`:
- [[ResponseOps][Alerts] Fix DSL filter issues in search bar
(#193623)](#193623)

<!--- Backport version: 9.4.3 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)

<!--BACKPORT [{"author":{"name":"Janki
Salvi","email":"[email protected]"},"sourceCommit":{"committedDate":"2024-10-13T18:42:04Z","message":"[ResponseOps][Alerts]
Fix DSL filter issues in search bar (#193623)\n\n##
Summary\r\n\r\nResolves
https://github.com/elastic/kibana/issues/183908\r\nResolves
https://github.com/elastic/kibana/issues/192557\r\n\r\nThis PR fixes an
issue of DSL filter in\r\n<details><summary>Search bar in Alerts page of
Stack\r\nmanagement</summary>\r\n<img width=\"1237\"
alt=\"image\"\r\nsrc=\"https://github.com/user-attachments/assets/b9b380d2-80a7-4754-95f2-6b6831923c4d\">\r\n</details>
\r\n\r\n<details><summary>Search bar in Alerts page of
Observability</summary>\r\n<img width=\"1237\"
alt=\"image\"\r\nsrc=\"https://github.com/user-attachments/assets/68ceb97f-b958-47f4-b356-757cbafd9170\">\r\n</details>
\r\n\r\n<details><summary>Search bar in \"If alerts matches query\"
action\r\nfilter</summary>\r\n<img width=\"1281\"
alt=\"image\"\r\nsrc=\"https://github.com/user-attachments/assets/1f99ca43-c4b5-4f52-b50f-914f1e205c5b\">\r\n</details>
\r\n\r\n<details><summary>Search bar in Maintenance window
page</summary>\r\n<img width=\"1272\"
alt=\"image\"\r\nsrc=\"https://github.com/user-attachments/assets/ffaa317d-c14b-45a2-9d02-00f7a10239ab\">\r\n</details>
\r\n\r\n\r\n### Checklist\r\n- [x] [Unit or
functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere
updated or added to match the most common scenarios\r\n\r\n### How to
verify\r\n- Create a DSL filter in above mentioned pages\r\n- Verify
that filter is applied properly\r\n\r\n---------\r\n\r\nCo-authored-by:
kibanamachine
<[email protected]>","sha":"122647b7c286f5c6a429e73166ff2dd542779f04","branchLabelMapping":{"^v9.0.0$":"main","^v8.16.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","Team:ResponseOps","v9.0.0","backport:prev-minor","v8.16.0"],"title":"[ResponseOps][Alerts]
Fix DSL filter issues in search
bar","number":193623,"url":"https://github.com/elastic/kibana/pull/193623","mergeCommit":{"message":"[ResponseOps][Alerts]
Fix DSL filter issues in search bar (#193623)\n\n##
Summary\r\n\r\nResolves
https://github.com/elastic/kibana/issues/183908\r\nResolves
https://github.com/elastic/kibana/issues/192557\r\n\r\nThis PR fixes an
issue of DSL filter in\r\n<details><summary>Search bar in Alerts page of
Stack\r\nmanagement</summary>\r\n<img width=\"1237\"
alt=\"image\"\r\nsrc=\"https://github.com/user-attachments/assets/b9b380d2-80a7-4754-95f2-6b6831923c4d\">\r\n</details>
\r\n\r\n<details><summary>Search bar in Alerts page of
Observability</summary>\r\n<img width=\"1237\"
alt=\"image\"\r\nsrc=\"https://github.com/user-attachments/assets/68ceb97f-b958-47f4-b356-757cbafd9170\">\r\n</details>
\r\n\r\n<details><summary>Search bar in \"If alerts matches query\"
action\r\nfilter</summary>\r\n<img width=\"1281\"
alt=\"image\"\r\nsrc=\"https://github.com/user-attachments/assets/1f99ca43-c4b5-4f52-b50f-914f1e205c5b\">\r\n</details>
\r\n\r\n<details><summary>Search bar in Maintenance window
page</summary>\r\n<img width=\"1272\"
alt=\"image\"\r\nsrc=\"https://github.com/user-attachments/assets/ffaa317d-c14b-45a2-9d02-00f7a10239ab\">\r\n</details>
\r\n\r\n\r\n### Checklist\r\n- [x] [Unit or
functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere
updated or added to match the most common scenarios\r\n\r\n### How to
verify\r\n- Create a DSL filter in above mentioned pages\r\n- Verify
that filter is applied properly\r\n\r\n---------\r\n\r\nCo-authored-by:
kibanamachine
<[email protected]>","sha":"122647b7c286f5c6a429e73166ff2dd542779f04"}},"sourceBranch":"main","suggestedTargetBranches":["8.x"],"targetPullRequestStates":[{"branch":"main","label":"v9.0.0","branchLabelMappingKey":"^v9.0.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/193623","number":193623,"mergeCommit":{"message":"[ResponseOps][Alerts]
Fix DSL filter issues in search bar (#193623)\n\n##
Summary\r\n\r\nResolves
https://github.com/elastic/kibana/issues/183908\r\nResolves
https://github.com/elastic/kibana/issues/192557\r\n\r\nThis PR fixes an
issue of DSL filter in\r\n<details><summary>Search bar in Alerts page of
Stack\r\nmanagement</summary>\r\n<img width=\"1237\"
alt=\"image\"\r\nsrc=\"https://github.com/user-attachments/assets/b9b380d2-80a7-4754-95f2-6b6831923c4d\">\r\n</details>
\r\n\r\n<details><summary>Search bar in Alerts page of
Observability</summary>\r\n<img width=\"1237\"
alt=\"image\"\r\nsrc=\"https://github.com/user-attachments/assets/68ceb97f-b958-47f4-b356-757cbafd9170\">\r\n</details>
\r\n\r\n<details><summary>Search bar in \"If alerts matches query\"
action\r\nfilter</summary>\r\n<img width=\"1281\"
alt=\"image\"\r\nsrc=\"https://github.com/user-attachments/assets/1f99ca43-c4b5-4f52-b50f-914f1e205c5b\">\r\n</details>
\r\n\r\n<details><summary>Search bar in Maintenance window
page</summary>\r\n<img width=\"1272\"
alt=\"image\"\r\nsrc=\"https://github.com/user-attachments/assets/ffaa317d-c14b-45a2-9d02-00f7a10239ab\">\r\n</details>
\r\n\r\n\r\n### Checklist\r\n- [x] [Unit or
functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere
updated or added to match the most common scenarios\r\n\r\n### How to
verify\r\n- Create a DSL filter in above mentioned pages\r\n- Verify
that filter is applied properly\r\n\r\n---------\r\n\r\nCo-authored-by:
kibanamachine
<[email protected]>","sha":"122647b7c286f5c6a429e73166ff2dd542779f04"}},{"branch":"8.x","label":"v8.16.0","branchLabelMappingKey":"^v8.16.0$","isSourceBranch":false,"state":"NOT_CREATED"}]}]
BACKPORT-->

Co-authored-by: Janki Salvi <[email protected]>
@cnasikas cnasikas mentioned this pull request Nov 1, 2024
12 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport:prev-minor Backport to (8.x) the previous minor version (i.e. one version back from main) release_note:skip Skip the PR/issue when compiling release notes Team:ResponseOps Label for the ResponseOps team (formerly the Cases and Alerting teams) v8.16.0 v9.0.0
Projects
None yet
9 participants