[Embeddable] Avoid rerendering loop due to search context reload #203150
Conversation
botelastic
bot
added
the
ci:project-deploy-observability
🤖 GitHub commentsExpand to view the GitHub comments
Just comment with:
|
dej611
added
release_note:fix
Team:Presentation
|
Pinging @elastic/kibana-presentation (Team:Presentation) |
botelastic
bot
added
the
ci:project-deploy-observability
🤖 GitHub commentsExpand to view the GitHub comments
Just comment with:
|
Flaky Test Runner Stats🟠 Some tests failed. - kibana-flaky-test-suite-runner#7555[❌] test/functional/apps/discover/group1/config.ts: 0/50 tests passed. |
packages/presentation/presentation_publishing/interfaces/fetch/publishes_unified_search.ts
Outdated
Show resolved
Hide resolved
Flaky Test Runner Stats🟠 Some tests failed. - kibana-flaky-test-suite-runner#7557[❌] test/functional/apps/discover/group1/config.ts: 0/50 tests passed. |
botelastic
bot
added
the
Team:obs-ux-management
|
Pinging @elastic/obs-ux-management-team (Team:obs-ux-management) |
Flaky Test Runner Stats🎉 All tests passed! - kibana-flaky-test-suite-runner#7561[✅] test/functional/apps/discover/group1/config.ts: 50/50 tests passed. |
|
The following is a diagnosis of the test failure: ProblemWhen changing the time picker on the Observability Infrastructure Hosts Overview page the KPI charts don't respond until manual / hard refresh is pressed. Root of the problemThe root of this problem is a misunderstanding of the Search sessions were added in the make it slow project years ago, and are meant to allow a previous search to be restored. They are integrated with the unified search bar, and are set up to change any time any of the search context changes (filters, query, time range etc). They are usually stored in the URL. In the Embeddable infrastructure, we had problems with multiple fetches, or fetches from various sources when search session IDs are provided (e.g. changing the time range would tell the embeddable to load data, then update the search session which also tells the embeddable to load data 🔥). To fix this, we made the decision that we would not directly query on changes in the time range etc if a search session ID was provided, and instead query only when the search session ID changes. When the search sessions system is set up correctly, this works great. You can see the code for this here The problem here is that Observability uses a search session ID - but it never changes - in essence this tells the Embeddable framework not to re-fetch data unless the reload button is pressed. In this code you can see that a session is started, but it isn't actually hooked up to anything, so for the whole lifecycle of the page there is just one session. Recommended solutionIt would be possible for Observability to ensure their search session integration works correctly on this page. The above-linked code could be updated to properly regenerate sessions on change. For an example of how this is done, you can see the Dashboard search session integration here That said, it doesn't seem like this page has an actual usage of search session restoration, or makes use of the search session UI (generally placed to the right of the breadcrumb). Because of this, my recommendation would be to remove all of the search session related code from this area. To verify that this works, you can update the const updateSearchSessionId = useCallback(() => {
const sessionId = search.session.start();
setSearchSessionId(undefined);
}, [search.session]);This isn't a long term solution, but it works because the charts will no longer recieve a static search session ID, which restores proper functionality to the embeddable. Next stepsAside from fixing this on the Observability side, we should document this behaviour in |
|
Hey @ThomThomson
In fact, 1 - The date range changes Shouldn't the first time a searchSessionId is assigned make the charts to fetch data?
True. The page doesn't need to restore the session. However, If I'm not mistaken, the If my memory serves me right, we basically tried to replicate what Dashboards UI does on the auto-refresh. Happy to find alternatives cc: @markov00 |
| @@ -138,19 +138,21 @@ export function RuleConditionChart({ | |||
| const errorDiv = document.querySelector('.lnsEmbeddedError'); | |||
| if (errorDiv) { | |||
| const paragraphElements = errorDiv.querySelectorAll('p'); | |||
| if (!paragraphElements || paragraphElements.length < 2) return; | |||
| if (!paragraphElements) return; | |||
There was a problem hiding this comment.
Tested following scenarios, LGTM!
| Error in filter query | Error in equation |
|---|---|
 |
 |
| paragraphElements[1].innerText = i18n.translate( | ||
| 'xpack.observability.ruleCondition.chart.error_equation.description', | ||
| { | ||
| defaultMessage: 'Check the rule equation.', |
There was a problem hiding this comment.
Did you find any example for showing this message, or did you just keep the previous implementation?
There was a problem hiding this comment.
As for now I've kept the same implementation. I did not have all the legacy knowledge of why the 2 errors check, so I've just fixed what I've seen not working for now.
There was a problem hiding this comment.
Could focus the changes on removing the searchSessionId only from embeddable usages?
I'll open a ticket on to refactor what is now called use_search_session.ts, but we need the some of the logic that depends on the change of the searchSessionId to control and force new requests
| @@ -1,34 +0,0 @@ | |||
| /* | |||
There was a problem hiding this comment.
Please keep this file and remove the usages of searchSessionId on Lens charts
| @@ -37,16 +36,11 @@ export const useLoadingState = () => { | |||
| const { | |||
| data: { search }, | |||
| } = services; | |||
| const { updateSearchSessionId } = useSearchSessionContext(); | |||
There was a problem hiding this comment.
We'll refactor this context to be timestamp-based instead of relying on searchSessionId, but removing this now will have consequences beyond the embedabbles. This context is useful to control and force requests
| @@ -44,22 +29,20 @@ export const ContextProviders = ({ | |||
| } = props; | |||
|
|
|||
| return ( | |||
| <RenderWithOptionalSearchSessionProvider renderMode={renderMode}> | |||
There was a problem hiding this comment.
Please leave this context here, it is used to control and force new requests. As said above it will be changed to not rely on searchSessionId.
...lugins/observability_solution/infra/public/components/asset_details/hooks/use_date_picker.ts
Show resolved
Hide resolved
markov00
force-pushed
the
fix/202266-2
branch
from
c1563b9 to
219ccb3
Compare
|
I think this might solve the problem with the test: dej611#23 |
💚 Build Succeeded
Metrics [docs]Async chunks
Page load bundle
History
|
Flaky Test Runner Stats🎉 All tests passed! - kibana-flaky-test-suite-runner#7575[✅] test/functional/apps/discover/group1/config.ts: 50/50 tests passed. |
|
Starting backport for target branches: 8.x https://github.com/elastic/kibana/actions/runs/12279103004 |
…stic#203150) ## Summary Fixes elastic#202266 This PR fixes the underline rerendering issue at the `useSearchApi` hook level, so any embeddable component who uses this hook would benefit from the fix. Ideally the props passed to the Lens component should be memoized, but this assumption would break many integrations as the previous embeddable component did take care of filtering duplicates. To test this: * Go to `Observability > Alerts > Manage rules` and `Add Rule`, pick the `Custom threshold` option and verify the infinite reload does not happen Once fixed this, another problem bubbled up with the brushing use case: when brushing a chart the chart was always one time range step behind. The other bug was due to the `fetch$(api)` function propagating a stale `data` search context who the Lens embeddable was relying on. To solve this other problem the following changes have been applied: * read the `searchSessionId` from the `api` directly (used for `autoRefresh`) * make sure to test the `Refresh` feature with both relative and absolute time ranges * read the `search context` from the `parentApi` directly if implements the `unifiedSearch` API * to test this, brush and check that the final time range matches the correct time range **Note**: the fundamental issue for the latter is `fetch$` not emitting the up-to-date `data` when the parentApi search context updates. The retrieved `data` is stale and one step behind, so it is not reliable. cc @elastic/kibana-presentation As @ThomThomson noticed in his test failure investigation another issue was found in this PR due to the wrong handling of the searchSessionId within the Observability page (for more details see [his analysis](elastic#203150 (comment))). @markov00 and @crespocarlos helped risolve this problem with some additional changes on the Observability side of things: this will lead to some extra searchSessionId to be created, which will be eventually solved by Observability team [shortly moving away from the `searchSessionId` mechanism](elastic#203412) ### Checklist - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios --------- Co-authored-by: Marco Vettorello <[email protected]> Co-authored-by: Carlos Crespo <[email protected]> (cherry picked from commit d4194ba)
💚 All backports created successfully
Note: Successful backport PRs will be merged automatically after passing CI. Questions ?Please refer to the Backport tool documentation |
#203150) (#203818) # Backport This will backport the following commits from `main` to `8.x`: - [[Embeddable] Avoid rerendering loop due to search context reload (#203150)](#203150) <!--- Backport version: 9.4.3 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sqren/backport) <!--BACKPORT [{"author":{"name":"Marco Liberati","email":"[email protected]"},"sourceCommit":{"committedDate":"2024-12-11T15:03:36Z","message":"[Embeddable] Avoid rerendering loop due to search context reload (#203150)\n\n## Summary\r\n\r\nFixes #202266\r\n\r\nThis PR fixes the underline rerendering issue at the `useSearchApi` hook\r\nlevel, so any embeddable component who uses this hook would benefit from\r\nthe fix.\r\n\r\nIdeally the props passed to the Lens component should be memoized, but\r\nthis assumption would break many integrations as the previous embeddable\r\ncomponent did take care of filtering duplicates.\r\nTo test this:\r\n* Go to `Observability > Alerts > Manage rules` and `Add Rule`, pick the\r\n`Custom threshold` option and verify the infinite reload does not happen\r\n\r\nOnce fixed this, another problem bubbled up with the brushing use case:\r\nwhen brushing a chart the chart was always one time range step behind.\r\nThe other bug was due to the `fetch$(api)` function propagating a stale\r\n`data` search context who the Lens embeddable was relying on.\r\nTo solve this other problem the following changes have been applied:\r\n* read the `searchSessionId` from the `api` directly (used for\r\n`autoRefresh`)\r\n* make sure to test the `Refresh` feature with both relative and\r\nabsolute time ranges\r\n* read the `search context` from the `parentApi` directly if implements\r\nthe `unifiedSearch` API\r\n* to test this, brush and check that the final time range matches the\r\ncorrect time range\r\n\r\n**Note**: the fundamental issue for the latter is `fetch# Backport This will backport the following commits from `main` to `8.x`: - [[Embeddable] Avoid rerendering loop due to search context reload (#203150)](#203150) <!--- Backport version: 9.4.3 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sqren/backport) <!--BACKPORT not emitting\r\nthe up-to-date `data` when the parentApi search context updates. The\r\nretrieved `data` is stale and one step behind, so it is not reliable. cc\r\n@elastic/kibana-presentation\r\n\r\nAs @ThomThomson noticed in his test failure investigation another issue\r\nwas found in this PR due to the wrong handling of the searchSessionId\r\nwithin the Observability page (for more details see [his\r\nanalysis](https://github.com/elastic/kibana/pull/203150#issuecomment-2524080129)).\r\n@markov00 and @crespocarlos helped risolve this problem with some\r\nadditional changes on the Observability side of things: this will lead\r\nto some extra searchSessionId to be created, which will be eventually\r\nsolved by Observability team [shortly moving away from the\r\n`searchSessionId`\r\nmechanism](https://github.com/elastic/kibana/issues/203412)\r\n\r\n### Checklist\r\n\r\n- [x] [Unit or functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere updated or added to match the most common scenarios\r\n\r\n---------\r\n\r\nCo-authored-by: Marco Vettorello <[email protected]>\r\nCo-authored-by: Carlos Crespo <[email protected]>","sha":"d4194ba5eb5a72960dad00cf956d9ea6e31219e0","branchLabelMapping":{"^v9.0.0$":"main","^v8.18.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:fix","Team:Presentation","v9.0.0","backport:prev-minor","Feature:Embeddables","ci:project-deploy-observability","Team:obs-ux-management"],"title":"[Embeddable] Avoid rerendering loop due to search context reload","number":203150,"url":"https://github.com/elastic/kibana/pull/203150","mergeCommit":{"message":"[Embeddable] Avoid rerendering loop due to search context reload (#203150)\n\n## Summary\r\n\r\nFixes #202266\r\n\r\nThis PR fixes the underline rerendering issue at the `useSearchApi` hook\r\nlevel, so any embeddable component who uses this hook would benefit from\r\nthe fix.\r\n\r\nIdeally the props passed to the Lens component should be memoized, but\r\nthis assumption would break many integrations as the previous embeddable\r\ncomponent did take care of filtering duplicates.\r\nTo test this:\r\n* Go to `Observability > Alerts > Manage rules` and `Add Rule`, pick the\r\n`Custom threshold` option and verify the infinite reload does not happen\r\n\r\nOnce fixed this, another problem bubbled up with the brushing use case:\r\nwhen brushing a chart the chart was always one time range step behind.\r\nThe other bug was due to the `fetch$(api)` function propagating a stale\r\n`data` search context who the Lens embeddable was relying on.\r\nTo solve this other problem the following changes have been applied:\r\n* read the `searchSessionId` from the `api` directly (used for\r\n`autoRefresh`)\r\n* make sure to test the `Refresh` feature with both relative and\r\nabsolute time ranges\r\n* read the `search context` from the `parentApi` directly if implements\r\nthe `unifiedSearch` API\r\n* to test this, brush and check that the final time range matches the\r\ncorrect time range\r\n\r\n**Note**: the fundamental issue for the latter is `fetch# Backport This will backport the following commits from `main` to `8.x`: - [[Embeddable] Avoid rerendering loop due to search context reload (#203150)](#203150) <!--- Backport version: 9.4.3 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sqren/backport) <!--BACKPORT not emitting\r\nthe up-to-date `data` when the parentApi search context updates. The\r\nretrieved `data` is stale and one step behind, so it is not reliable. cc\r\n@elastic/kibana-presentation\r\n\r\nAs @ThomThomson noticed in his test failure investigation another issue\r\nwas found in this PR due to the wrong handling of the searchSessionId\r\nwithin the Observability page (for more details see [his\r\nanalysis](https://github.com/elastic/kibana/pull/203150#issuecomment-2524080129)).\r\n@markov00 and @crespocarlos helped risolve this problem with some\r\nadditional changes on the Observability side of things: this will lead\r\nto some extra searchSessionId to be created, which will be eventually\r\nsolved by Observability team [shortly moving away from the\r\n`searchSessionId`\r\nmechanism](https://github.com/elastic/kibana/issues/203412)\r\n\r\n### Checklist\r\n\r\n- [x] [Unit or functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere updated or added to match the most common scenarios\r\n\r\n---------\r\n\r\nCo-authored-by: Marco Vettorello <[email protected]>\r\nCo-authored-by: Carlos Crespo <[email protected]>","sha":"d4194ba5eb5a72960dad00cf956d9ea6e31219e0"}},"sourceBranch":"main","suggestedTargetBranches":[],"targetPullRequestStates":[{"branch":"main","label":"v9.0.0","branchLabelMappingKey":"^v9.0.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/203150","number":203150,"mergeCommit":{"message":"[Embeddable] Avoid rerendering loop due to search context reload (#203150)\n\n## Summary\r\n\r\nFixes #202266\r\n\r\nThis PR fixes the underline rerendering issue at the `useSearchApi` hook\r\nlevel, so any embeddable component who uses this hook would benefit from\r\nthe fix.\r\n\r\nIdeally the props passed to the Lens component should be memoized, but\r\nthis assumption would break many integrations as the previous embeddable\r\ncomponent did take care of filtering duplicates.\r\nTo test this:\r\n* Go to `Observability > Alerts > Manage rules` and `Add Rule`, pick the\r\n`Custom threshold` option and verify the infinite reload does not happen\r\n\r\nOnce fixed this, another problem bubbled up with the brushing use case:\r\nwhen brushing a chart the chart was always one time range step behind.\r\nThe other bug was due to the `fetch$(api)` function propagating a stale\r\n`data` search context who the Lens embeddable was relying on.\r\nTo solve this other problem the following changes have been applied:\r\n* read the `searchSessionId` from the `api` directly (used for\r\n`autoRefresh`)\r\n* make sure to test the `Refresh` feature with both relative and\r\nabsolute time ranges\r\n* read the `search context` from the `parentApi` directly if implements\r\nthe `unifiedSearch` API\r\n* to test this, brush and check that the final time range matches the\r\ncorrect time range\r\n\r\n**Note**: the fundamental issue for the latter is `fetch# Backport This will backport the following commits from `main` to `8.x`: - [[Embeddable] Avoid rerendering loop due to search context reload (#203150)](#203150) <!--- Backport version: 9.4.3 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sqren/backport) <!--BACKPORT not emitting\r\nthe up-to-date `data` when the parentApi search context updates. The\r\nretrieved `data` is stale and one step behind, so it is not reliable. cc\r\n@elastic/kibana-presentation\r\n\r\nAs @ThomThomson noticed in his test failure investigation another issue\r\nwas found in this PR due to the wrong handling of the searchSessionId\r\nwithin the Observability page (for more details see [his\r\nanalysis](https://github.com/elastic/kibana/pull/203150#issuecomment-2524080129)).\r\n@markov00 and @crespocarlos helped risolve this problem with some\r\nadditional changes on the Observability side of things: this will lead\r\nto some extra searchSessionId to be created, which will be eventually\r\nsolved by Observability team [shortly moving away from the\r\n`searchSessionId`\r\nmechanism](https://github.com/elastic/kibana/issues/203412)\r\n\r\n### Checklist\r\n\r\n- [x] [Unit or functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere updated or added to match the most common scenarios\r\n\r\n---------\r\n\r\nCo-authored-by: Marco Vettorello <[email protected]>\r\nCo-authored-by: Carlos Crespo <[email protected]>","sha":"d4194ba5eb5a72960dad00cf956d9ea6e31219e0"}}]}] BACKPORT--> Co-authored-by: Marco Liberati <[email protected]>
…stic#203150) ## Summary Fixes elastic#202266 This PR fixes the underline rerendering issue at the `useSearchApi` hook level, so any embeddable component who uses this hook would benefit from the fix. Ideally the props passed to the Lens component should be memoized, but this assumption would break many integrations as the previous embeddable component did take care of filtering duplicates. To test this: * Go to `Observability > Alerts > Manage rules` and `Add Rule`, pick the `Custom threshold` option and verify the infinite reload does not happen Once fixed this, another problem bubbled up with the brushing use case: when brushing a chart the chart was always one time range step behind. The other bug was due to the `fetch$(api)` function propagating a stale `data` search context who the Lens embeddable was relying on. To solve this other problem the following changes have been applied: * read the `searchSessionId` from the `api` directly (used for `autoRefresh`) * make sure to test the `Refresh` feature with both relative and absolute time ranges * read the `search context` from the `parentApi` directly if implements the `unifiedSearch` API * to test this, brush and check that the final time range matches the correct time range **Note**: the fundamental issue for the latter is `fetch$` not emitting the up-to-date `data` when the parentApi search context updates. The retrieved `data` is stale and one step behind, so it is not reliable. cc @elastic/kibana-presentation As @ThomThomson noticed in his test failure investigation another issue was found in this PR due to the wrong handling of the searchSessionId within the Observability page (for more details see [his analysis](elastic#203150 (comment))). @markov00 and @crespocarlos helped risolve this problem with some additional changes on the Observability side of things: this will lead to some extra searchSessionId to be created, which will be eventually solved by Observability team [shortly moving away from the `searchSessionId` mechanism](elastic#203412) ### Checklist - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios --------- Co-authored-by: Marco Vettorello <[email protected]> Co-authored-by: Carlos Crespo <[email protected]>
Summary
Fixes #202266
This PR fixes the underline rerendering issue at the
useSearchApihook level, so any embeddable component who uses this hook would benefit from the fix.Ideally the props passed to the Lens component should be memoized, but this assumption would break many integrations as the previous embeddable component did take care of filtering duplicates.
To test this:
Observability > Alerts > Manage rulesandAdd Rule, pick theCustom thresholdoption and verify the infinite reload does not happenOnce fixed this, another problem bubbled up with the brushing use case: when brushing a chart the chart was always one time range step behind. The other bug was due to the
fetch$(api)function propagating a staledatasearch context who the Lens embeddable was relying on.To solve this other problem the following changes have been applied:
searchSessionIdfrom theapidirectly (used forautoRefresh)Refreshfeature with both relative and absolute time rangessearch contextfrom theparentApidirectly if implements theunifiedSearchAPINote: the fundamental issue for the latter is
fetch$not emitting the up-to-datedatawhen the parentApi search context updates. The retrieveddatais stale and one step behind, so it is not reliable. cc @elastic/kibana-presentationAs @ThomThomson noticed in his test failure investigation another issue was found in this PR due to the wrong handling of the searchSessionId within the Observability page (for more details see his analysis).
@markov00 and @crespocarlos helped risolve this problem with some additional changes on the Observability side of things: this will lead to some extra searchSessionId to be created, which will be eventually solved by Observability team shortly moving away from the
searchSessionIdmechanismChecklist