[Discover] Add contextual awareness framework developer documentation #184698
Assignees
Labels
documentation
Feature:Discover
Discover Application
impact:high
Addressing this issue will have a high level of impact on the quality/strength of our product.
loe:small
Small Level of Effort
Project:OneDiscover
Enrich Discover with contextual awareness
Team:DataDiscovery
Discover, search (e.g. data plugin and KQL), data views, saved searches. For ES|QL, use Team:ES|QL.
Comments
davismcphee
added
Feature:Discover
|
Pinging @elastic/kibana-data-discovery (Team:DataDiscovery) |
This was referenced Jun 4, 2024
|
About examples, do you mean some examples in a readme or something in |
|
@kertal For this issue, I mean some code examples for how to set up new providers, etc. in the readme. But the functional test work I'm doing will also add some example profiles that can be enabled for functional and dev testing, and we can reference those in the docs too. |
|
thx, this makes much of sense 👍 |
davismcphee
added a commit
that referenced
this issue
Jun 10, 2024
## Summary This PR includes the initial implementation of the Discover contextual awareness framework and composable profiles:  ### Context We currently support three levels of context in Discover: - Root context: - Based on the current solution type, navigational parameters, etc. - Resolved at application initialization and on parameter changes. - Runs synchronously or asynchronously. - Data source context: - Based on the current ES|QL query or data view. - Resolved on ES|QL query or data view change, before data fetching occurs. - Runs synchronously or asynchronously. - Document context: - Based on individual ES|QL records or ES documents. - Resolved individually for each ES|QL record or ES document after data fetching runs. - Runs synchronously only. ### Composable profiles To support application extensibility based on context, we've introduced the concept of "composable profiles". Composable profiles are implementations of a core `Profile` interface (or a subset of it) containing all of the available extension points Discover supports. A composable profile can be implemented at any context level through a "profile provider", responsible for defining the composable profile and its associated context resolution method. The context resolution method, named `resolve`, determines if its composable profile is a match for the current Discover context, and returns related metadata in a `context` object. ### Merged accessors Composable profiles operate similarly to middleware in that each of their extension point implementations are passed a `prev` argument, which can be called to access the results from profiles at previous context levels, and allows overwriting or composing a final result from the previous results. The method Discover calls to trigger the extension point merging process and obtain a final result from the combined profiles is referred to as a "merged accessor". The following diagram illustrates the extension point merging process:  ### Supporting services The contextual awareness framework is driven by two main supporting services called `ProfileService` and `ProfilesManager`. Each context level has a dedicated profile service, e.g. `RootProfileService`, which is responsible for accepting profile provider registrations and running through each provider in order during context resolution to identify a matching profile. A single `ProfilesManager` is instantiated on Discover load, or one per saved search panel in a dashboard. The profiles manager is responsible for the following: - Managing state associated with the current Discover context. - Coordinating profile services and exposing resolution methods for each context level. - Providing access to the combined set of resolved profiles. - Deduplicating profile resolution attempts with identical parameters. - Error handling and fallback behaviour on profile resolution failure. ### Bringing it all together The following diagram models the overall Discover contextual awareness framework and how each of the above concepts come together:  ### Followup work - We'll want to add developer documentation as a followup, which I've created an issue for here: #184698. The summary for this PR can be used as the basis for the documentation. - Since we currently have no profile or extension point implementations, this PR does not include any functional tests. We should create example implementations for functional testing and ensure they're only enabled when running the test suite or when developers want them enabled. I've created a followup issue for this work here: #184699. ### Testing notes Testing the framework is tricky since we have no actual profile or extension point implementations yet. However, I previously added example implementations that I was using for testing while building the framework. I've removed the example implementations so they don't get merged, but they can be temporarily restored for testing by reverting the commit where I removed them: `git revert 5752651`. You'll also need to uncomment the following lines in `src/plugins/discover/public/plugin.tsx`: https://github.com/elastic/kibana/blob/ce85a6a35fa3623bfdfac7dae41df2d840394154/src/plugins/discover/public/plugin.tsx#L458-L463 To test the root profile resolution based on solution type, I'd recommend enabling the solution nav locally by adding the following to `kibana.dev.yml`: ```yml xpack.cloud_integrations.experiments.enabled: true xpack.cloud_integrations.experiments.flag_overrides: "solutionNavEnabled": true xpack.cloud.id: "ftr_fake_cloud_id:aGVsbG8uY29tOjQ0MyRFUzEyM2FiYyRrYm4xMjNhYmM=" xpack.cloud.base_url: "https://cloud.elastic.co" xpack.cloud.deployment_url: "/deployments/deploymentId" ``` In order to change the active solution type, modify the `mockSpaceState` in `src/plugins/navigation/public/plugin.tsx`: https://github.com/elastic/kibana/blob/79e51d64f83da6af56107a633a5a3b49947f1ebe/src/plugins/navigation/public/plugin.tsx#L159-L162 For test data, I'd recommend running the following commands to generate sample ECS compliant logs and metrics data: ``` node scripts/synthtrace.js --target "http://elastic:changeme@localhost:9200/" --kibana "http://elastic:changeme@localhost:5601/" --live --logLevel debug simple_logs.ts node scripts/synthtrace.js --target "http://elastic:changeme@localhost:9200/" --kibana "http://elastic:changeme@localhost:5601/" --live --logLevel debug simple_trace.ts ``` And lastly a couple of the ES|QL queries I used for testing: ``` // resolves to the example logs data source context from logs-synth-default // mixed dataset that falls back to vanilla Discover // helpful for testing document context in the doc viewer flyout from logs-synth-default,metrics-* ``` Resolves #181962. ### Checklist - [ ] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md) - [ ] [Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html) was added for features that require explanation or tutorials - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios - [ ] [Flaky Test Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was used on any tests changed - [ ] Any UI touched in this PR is usable by keyboard only (learn more about [keyboard accessibility](https://webaim.org/techniques/keyboard/)) - [ ] Any UI touched in this PR does not create any new axe failures (run axe in browser: [FF](https://addons.mozilla.org/en-US/firefox/addon/axe-devtools/), [Chrome](https://chrome.google.com/webstore/detail/axe-web-accessibility-tes/lhdoppojpmngadmnindnejefpokejbdd?hl=en-US)) - [ ] If a plugin configuration key changed, check if it needs to be allowlisted in the cloud and added to the [docker list](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker) - [ ] This renders correctly on smaller devices using a responsive layout. (You can test this [in your browser](https://www.browserstack.com/guide/responsive-testing-on-local-server)) - [ ] This was checked for [cross-browser compatibility](https://www.elastic.co/support/matrix#matrix_browsers) ### For maintainers - [ ] This was checked for breaking API changes and was [labeled appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process) --------- Co-authored-by: kibanamachine <[email protected]>
10 tasks
kibanamachine
pushed a commit
to kibanamachine/kibana
that referenced
this issue
Sep 13, 2024
) ## Summary This PR adds initial developer documentation for the Discover context awareness framework, with the aim of providing a solid foundation for solution engineers to begin working with it and adding their own profiles and extension point implementations: - Adds a `README.md` file to the Discover `context_awareness` folder including the main developer documentation. - Adds TSDoc comments to all public types and functions used by the context awareness framework. - Reorganizes the `profile_providers` folder to provide a clearer folder structure to support collaboration with solution teams. Resolves elastic#184698. ### Checklist - [ ] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md) - [x] [Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html) was added for features that require explanation or tutorials - [ ] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios - [ ] [Flaky Test Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was used on any tests changed - [ ] Any UI touched in this PR is usable by keyboard only (learn more about [keyboard accessibility](https://webaim.org/techniques/keyboard/)) - [ ] Any UI touched in this PR does not create any new axe failures (run axe in browser: [FF](https://addons.mozilla.org/en-US/firefox/addon/axe-devtools/), [Chrome](https://chrome.google.com/webstore/detail/axe-web-accessibility-tes/lhdoppojpmngadmnindnejefpokejbdd?hl=en-US)) - [ ] If a plugin configuration key changed, check if it needs to be allowlisted in the cloud and added to the [docker list](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker) - [ ] This renders correctly on smaller devices using a responsive layout. (You can test this [in your browser](https://www.browserstack.com/guide/responsive-testing-on-local-server)) - [ ] This was checked for [cross-browser compatibility](https://www.elastic.co/support/matrix#matrix_browsers) ### For maintainers - [ ] This was checked for breaking API changes and was [labeled appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process) (cherry picked from commit f6c8a2e)
kibanamachine
added a commit
that referenced
this issue
Sep 13, 2024
…) (#192909) # Backport This will backport the following commits from `main` to `8.x`: - [[Discover] Add context awareness framework documentation (#192429)](#192429) <!--- Backport version: 9.4.3 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sqren/backport) <!--BACKPORT [{"author":{"name":"Davis McPhee","email":"[email protected]"},"sourceCommit":{"committedDate":"2024-09-13T01:10:50Z","message":"[Discover] Add context awareness framework documentation (#192429)\n\n## Summary\r\n\r\nThis PR adds initial developer documentation for the Discover context\r\nawareness framework, with the aim of providing a solid foundation for\r\nsolution engineers to begin working with it and adding their own\r\nprofiles and extension point implementations:\r\n- Adds a `README.md` file to the Discover `context_awareness` folder\r\nincluding the main developer documentation.\r\n- Adds TSDoc comments to all public types and functions used by the\r\ncontext awareness framework.\r\n- Reorganizes the `profile_providers` folder to provide a clearer folder\r\nstructure to support collaboration with solution teams.\r\n\r\nResolves #184698.\r\n\r\n### Checklist\r\n\r\n- [ ] Any text added follows [EUI's writing\r\nguidelines](https://elastic.github.io/eui/#/guidelines/writing), uses\r\nsentence case text and includes [i18n\r\nsupport](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)\r\n- [x]\r\n[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)\r\nwas added for features that require explanation or tutorials\r\n- [ ] [Unit or functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere updated or added to match the most common scenarios\r\n- [ ] [Flaky Test\r\nRunner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was\r\nused on any tests changed\r\n- [ ] Any UI touched in this PR is usable by keyboard only (learn more\r\nabout [keyboard accessibility](https://webaim.org/techniques/keyboard/))\r\n- [ ] Any UI touched in this PR does not create any new axe failures\r\n(run axe in browser:\r\n[FF](https://addons.mozilla.org/en-US/firefox/addon/axe-devtools/),\r\n[Chrome](https://chrome.google.com/webstore/detail/axe-web-accessibility-tes/lhdoppojpmngadmnindnejefpokejbdd?hl=en-US))\r\n- [ ] If a plugin configuration key changed, check if it needs to be\r\nallowlisted in the cloud and added to the [docker\r\nlist](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker)\r\n- [ ] This renders correctly on smaller devices using a responsive\r\nlayout. (You can test this [in your\r\nbrowser](https://www.browserstack.com/guide/responsive-testing-on-local-server))\r\n- [ ] This was checked for [cross-browser\r\ncompatibility](https://www.elastic.co/support/matrix#matrix_browsers)\r\n\r\n### For maintainers\r\n\r\n- [ ] This was checked for breaking API changes and was [labeled\r\nappropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)","sha":"f6c8a2e5865c389b80d4a2440aaf20ea6fab0617","branchLabelMapping":{"^v9.0.0$":"main","^v8.16.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","v9.0.0","Team:DataDiscovery","backport:prev-minor","v8.16.0"],"title":"[Discover] Add context awareness framework documentation","number":192429,"url":"https://github.com/elastic/kibana/pull/192429","mergeCommit":{"message":"[Discover] Add context awareness framework documentation (#192429)\n\n## Summary\r\n\r\nThis PR adds initial developer documentation for the Discover context\r\nawareness framework, with the aim of providing a solid foundation for\r\nsolution engineers to begin working with it and adding their own\r\nprofiles and extension point implementations:\r\n- Adds a `README.md` file to the Discover `context_awareness` folder\r\nincluding the main developer documentation.\r\n- Adds TSDoc comments to all public types and functions used by the\r\ncontext awareness framework.\r\n- Reorganizes the `profile_providers` folder to provide a clearer folder\r\nstructure to support collaboration with solution teams.\r\n\r\nResolves #184698.\r\n\r\n### Checklist\r\n\r\n- [ ] Any text added follows [EUI's writing\r\nguidelines](https://elastic.github.io/eui/#/guidelines/writing), uses\r\nsentence case text and includes [i18n\r\nsupport](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)\r\n- [x]\r\n[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)\r\nwas added for features that require explanation or tutorials\r\n- [ ] [Unit or functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere updated or added to match the most common scenarios\r\n- [ ] [Flaky Test\r\nRunner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was\r\nused on any tests changed\r\n- [ ] Any UI touched in this PR is usable by keyboard only (learn more\r\nabout [keyboard accessibility](https://webaim.org/techniques/keyboard/))\r\n- [ ] Any UI touched in this PR does not create any new axe failures\r\n(run axe in browser:\r\n[FF](https://addons.mozilla.org/en-US/firefox/addon/axe-devtools/),\r\n[Chrome](https://chrome.google.com/webstore/detail/axe-web-accessibility-tes/lhdoppojpmngadmnindnejefpokejbdd?hl=en-US))\r\n- [ ] If a plugin configuration key changed, check if it needs to be\r\nallowlisted in the cloud and added to the [docker\r\nlist](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker)\r\n- [ ] This renders correctly on smaller devices using a responsive\r\nlayout. (You can test this [in your\r\nbrowser](https://www.browserstack.com/guide/responsive-testing-on-local-server))\r\n- [ ] This was checked for [cross-browser\r\ncompatibility](https://www.elastic.co/support/matrix#matrix_browsers)\r\n\r\n### For maintainers\r\n\r\n- [ ] This was checked for breaking API changes and was [labeled\r\nappropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)","sha":"f6c8a2e5865c389b80d4a2440aaf20ea6fab0617"}},"sourceBranch":"main","suggestedTargetBranches":["8.x"],"targetPullRequestStates":[{"branch":"main","label":"v9.0.0","branchLabelMappingKey":"^v9.0.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/192429","number":192429,"mergeCommit":{"message":"[Discover] Add context awareness framework documentation (#192429)\n\n## Summary\r\n\r\nThis PR adds initial developer documentation for the Discover context\r\nawareness framework, with the aim of providing a solid foundation for\r\nsolution engineers to begin working with it and adding their own\r\nprofiles and extension point implementations:\r\n- Adds a `README.md` file to the Discover `context_awareness` folder\r\nincluding the main developer documentation.\r\n- Adds TSDoc comments to all public types and functions used by the\r\ncontext awareness framework.\r\n- Reorganizes the `profile_providers` folder to provide a clearer folder\r\nstructure to support collaboration with solution teams.\r\n\r\nResolves #184698.\r\n\r\n### Checklist\r\n\r\n- [ ] Any text added follows [EUI's writing\r\nguidelines](https://elastic.github.io/eui/#/guidelines/writing), uses\r\nsentence case text and includes [i18n\r\nsupport](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)\r\n- [x]\r\n[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)\r\nwas added for features that require explanation or tutorials\r\n- [ ] [Unit or functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere updated or added to match the most common scenarios\r\n- [ ] [Flaky Test\r\nRunner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was\r\nused on any tests changed\r\n- [ ] Any UI touched in this PR is usable by keyboard only (learn more\r\nabout [keyboard accessibility](https://webaim.org/techniques/keyboard/))\r\n- [ ] Any UI touched in this PR does not create any new axe failures\r\n(run axe in browser:\r\n[FF](https://addons.mozilla.org/en-US/firefox/addon/axe-devtools/),\r\n[Chrome](https://chrome.google.com/webstore/detail/axe-web-accessibility-tes/lhdoppojpmngadmnindnejefpokejbdd?hl=en-US))\r\n- [ ] If a plugin configuration key changed, check if it needs to be\r\nallowlisted in the cloud and added to the [docker\r\nlist](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker)\r\n- [ ] This renders correctly on smaller devices using a responsive\r\nlayout. (You can test this [in your\r\nbrowser](https://www.browserstack.com/guide/responsive-testing-on-local-server))\r\n- [ ] This was checked for [cross-browser\r\ncompatibility](https://www.elastic.co/support/matrix#matrix_browsers)\r\n\r\n### For maintainers\r\n\r\n- [ ] This was checked for breaking API changes and was [labeled\r\nappropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)","sha":"f6c8a2e5865c389b80d4a2440aaf20ea6fab0617"}},{"branch":"8.x","label":"v8.16.0","branchLabelMappingKey":"^v8.16.0$","isSourceBranch":false,"state":"NOT_CREATED"}]}] BACKPORT--> Co-authored-by: Davis McPhee <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
📓 Summary
This issue covers adding developer documentation for the Discover contextual awareness framework introduced in #183797. We can use the PR summary as the basis for the documentation, and add additional examples and code samples to guide developers in using the framework.
✔️ Acceptance criteria
README.mdfile insrc/plugins/discover/public/context_awarenesswith developer documentation for working with the contextual awareness framework.The text was updated successfully, but these errors were encountered: