Add plugin status API #75819

joshdover · 2020-08-24T21:39:52Z

Summary

Related to #41983

Implements the core.status.set, core.status.plugins$, and core.status.derivedStatus$ APIs as specified in RFC-0010.

I did not write any e2e tests in this PR as I thought it would be easier to write them directly against the migrated status HTTP API that I am working on in parallel.

To Do:

Add complete example docs to StatusServiceSetup that explains how plugin statuses work, how they are inherited, and how to customize them.

Dev Docs

Kibana Platform plugins may now read the status of their dependencies, their plugin's default status, and manually override that status as reported to the end user and on the /api/status endpoint.

class MyPlugin {
  setup(core) {
    // Override default behavior and only elevate severity when elasticsearch is not available
    core.status.set(
      core.status.core$.pipe(core => core.elasticsearch);
    );
  }
}

Checklist

Delete any items that are not applicable to this PR.

Documentation was added for features that require explanation or tutorials
Unit or functional tests were updated or added to match the most common scenarios

For maintainers

This was checked for breaking API changes and was labeled appropriately

joshdover · 2020-08-24T22:57:34Z

src/core/server/status/status_service.test.ts

+          Array [
+            Object {
+              "level": available,
+              "summary": "All services are available",
+            },
+            Object {
+              "level": degraded,
+              "summary": "[a]: This is degraded!",
+            },
+            Object {
+              "level": degraded,
+              "summary": "[2] services are degraded",
+            },


This highlights a current problem with this Observable-based architecture: a single update to one plugin's status may result in many overall updates as it propagates through the dependency tree.

I plan to experiment more tomorrow with RxJS's various schedulers and debouncing utilities, but I'm not sure yet there is a great solution to this. Even if there is, we still would need to provide guidance or utilities so that plugin authors can also avoid this same issue in their custom status implementations.

https://www.learnrxjs.io/learn-rxjs/operators/filtering/debouncetime after combineLatest? Can it be backed in the core in order not to enforce plugins to wrap it manually?

+1 for debounce effect of some kind to the overall$ and plugins$ observables. We probably want to keep the derivedStatus$ exposed to plugins 'fresh' to avoid delay in status recomputation though?

My goal was to avoid any time-based approach in order to make the behavior very consistent for synchronous Observable sources. I was able to get this to work using debounce with a variable number of microtasks based on the number of known observables. Seems to be working correctly! Now the observables should only emit once per macro-task.

I decided to just go with a time-based apporoach after all. Microtasks seemed like too much magic and very fragile to break. It's also easy to write a test which ends up changing the microtasks queue itself, leading to strange results in tests.

We probably want to keep the derivedStatus$ exposed to plugins 'fresh' to avoid delay in status recomputation though?

+1 to that. I am only going to debounce the overall$ and plugins$ observables

elasticmachine · 2020-08-25T23:01:33Z

Pinging @elastic/kibana-platform (Team:Platform)

joshdover · 2020-08-25T23:21:46Z

src/core/server/status/get_summary_status.ts

+      // TODO: include URL to status page
+      detail: status.detail ?? `See the status page for more information`,
+      meta: {
+        affectedServices: { [serviceName]: status },
+      },


Changed the summary output for when only a single service is unavailable to be more consistent with the case where more than one is unavailable.

src/core/server/status/plugins_status.ts

src/core/server/status/plugins_status.test.ts

src/core/server/status/plugins_status.ts

src/core/server/status/status_service.test.ts

mshustov · 2020-08-26T13:37:34Z

src/core/server/status/status_service.ts

+      // We schedule the number of microtasks expected to resolve all of the dependencies of this update.
+      // Waiting for this ensures that we do not emit 'partial updates' to reduce noise.
+      debounce(([coreStatus, pluginsStatus]) =>
+        waitForMicrotasks(1 + Object.keys(pluginsStatus).length)


does it mean that all this.pluginsStatus API consumers have to implement debouncing manually?

Now that debouncing is configured on the Observable used to wrap a plugin's dependencies (the core.status.dependencies$ Observable), I don't believe this will be a problem that "fans out" to consumers, so manual debouncing should be unnecessary.

src/core/server/status/types.ts

src/core/server/status/status_service.ts

src/core/server/status/plugins_status.ts

src/core/server/status/status_service.test.ts

kibanamachine · 2020-09-01T01:32:55Z

💚 Build Succeeded

continuous-integration/kibana-ci/pull-request
Commit: 476fb75

Build metrics

oss distributable file count

id	value	diff	baseline
total	27306	+1	27305

distributable file count

id	value	diff	baseline
total	45456	+1	45455

History

💔 Build #71435 failed 8bee167
💚 Build #70722 succeeded 2aaaf8c
💚 Build #70415 succeeded 3ea7db2
💔 Build #70407 failed a0a091e0349b2836ad3a4a4d67bb060c468a64f9
💔 Build #70182 failed 1440cc5a8fefaff9a621610bdd962fe4a8aa22c2

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

# Conflicts: # rfcs/text/0010_service_status.md

Co-authored-by: Elastic Machine <[email protected]>

This reverts commit 2006eca.

Co-authored-by: spalger <[email protected]>

Co-authored-by: spalger <[email protected]> # Conflicts: # rfcs/text/0010_service_status.md

@ts-expect-error

…rok/new-patterns-component-use-array * 'master' of github.com:elastic/kibana: (75 commits) Remove legacy ui-apps-mixin (elastic#76604) remove unused test_utils (elastic#76528) [ML] Functional tests - add UI permission tests (elastic#76368) [APM] @ts-error -> @ts-expect-error (elastic#76492) [APM] Avoid negative offset for error marker on timeline (elastic#76638) [Reporting] rename interfaces to align with task manager integration (elastic#76716) Revert back ESO migration for alerting, added try/catch logic to avoid failing Kibana on start (elastic#76220) Test reverting "Add plugin status API (elastic#75819)" (elastic#76707) [Security Solution][Detections] Removes ML Job Settings SIEM copy and fixes link to ML app for creating custom jobs (elastic#76595) [Maps] remove region/coordinate-maps visualizations from sample data (elastic#76399) [DOCS] Dashboard-first docs refresh (elastic#76194) Updated ServiceNow description in docs and actions management UI to contains correct info (elastic#76344) [DOCS] Identifies cloud settings in reporting (elastic#76691) [Security Solution] Refactor timeline details to use search strategy (elastic#75591) es-archiver: Drop invalid index settings, support --query flag (elastic#76522) [DOCS] Identifies graph settings available on cloud (elastic#76661) Add more info about a11y tests (elastic#76045) [data.search.SearchSource] Remove legacy ES client APIs. (elastic#75943) [release notes] automatically retry on Github API 5xx errors (elastic#76447) [es_ui_shared] Fix eslint exhaustive deps rule (elastic#76392) ...

…ic#76707)" This reverts commit 5b853b2.

joshdover force-pushed the status/plugin-api branch from f2b5f56 to 1440cc5 Compare August 24, 2020 22:54

joshdover commented Aug 24, 2020

View reviewed changes

joshdover force-pushed the status/plugin-api branch from 1440cc5 to a0a091e Compare August 25, 2020 22:55

joshdover added Feature:New Platform release_note:plugin_api_changes Contains a Plugin API changes section for the breaking plugin API changes section. Team:Core Core services & architecture: plugins, logging, config, saved objects, http, ES client, i18n, etc v7.10.0 labels Aug 25, 2020

joshdover marked this pull request as ready for review August 25, 2020 23:01

joshdover requested a review from a team as a code owner August 25, 2020 23:01

joshdover force-pushed the status/plugin-api branch from a0a091e to edd46bb Compare August 25, 2020 23:19

Add plugin status API

3ea7db2

joshdover force-pushed the status/plugin-api branch from edd46bb to 3ea7db2 Compare August 25, 2020 23:20

joshdover commented Aug 25, 2020

View reviewed changes

mshustov reviewed Aug 26, 2020

View reviewed changes

joshdover added 2 commits August 26, 2020 15:27

Merge remote-tracking branch 'upstream/master' into status/plugin-api

fa70af6

Use debounceTime + PR comments

2aaaf8c

joshdover mentioned this pull request Aug 26, 2020

Migrate status & stats APIs to KP + remove legacy status lib #76054

Merged

3 tasks

mshustov approved these changes Aug 27, 2020

View reviewed changes

src/core/server/status/status_service.ts Outdated Show resolved Hide resolved

src/core/server/status/plugins_status.ts Outdated Show resolved Hide resolved

src/core/server/status/status_service.test.ts Outdated Show resolved Hide resolved

mshustov reviewed Aug 27, 2020

View reviewed changes

src/core/server/status/status_service.test.ts Outdated Show resolved Hide resolved

joshdover added 4 commits August 31, 2020 14:30

Merge remote-tracking branch 'upstream/master' into status/plugin-api

65735fd

PR comments + rename core.status.dependencies$

b26ecb5

Add docs

8bee167

eslint

476fb75

joshdover merged commit 2006eca into elastic:master Sep 1, 2020

joshdover deleted the status/plugin-api branch September 1, 2020 13:36

joshdover mentioned this pull request Sep 1, 2020

[7.x] Add plugin status API (#75819) #76374

Merged

joshdover added a commit to joshdover/kibana that referenced this pull request Sep 1, 2020

Add plugin status API (elastic#75819)

42e3bf6

# Conflicts: # rfcs/text/0010_service_status.md

joshdover added a commit that referenced this pull request Sep 1, 2020

[7.x] Add plugin status API (#75819) (#76374)

622f524

Co-authored-by: Elastic Machine <[email protected]>

spalger added a commit to spalger/kibana that referenced this pull request Sep 3, 2020

Revert "Add plugin status API (elastic#75819)"

f1d3a71

This reverts commit 2006eca.

spalger added a commit that referenced this pull request Sep 3, 2020

Test reverting "Add plugin status API (#75819)" (#76707)

5b853b2

Co-authored-by: spalger <[email protected]>

spalger added the reverted label Sep 3, 2020

spalger mentioned this pull request Sep 3, 2020

[7.x] Test reverting "Add plugin status API (#75819)" (#76707) #76728

Merged

joshdover mentioned this pull request Sep 3, 2020

Add plugin status API - take 2 #76732

Merged

7 tasks

spalger pushed a commit that referenced this pull request Sep 4, 2020

[7.x] Test reverting "Add plugin status API (#75819)" (#76707) (#76728)

71ee3bd

Co-authored-by: spalger <[email protected]> # Conflicts: # rfcs/text/0010_service_status.md

joshdover added a commit to joshdover/kibana that referenced this pull request Sep 9, 2020

Revert "Test reverting "Add plugin status API (elastic#75819)" (elast…

5c875b2

…ic#76707)" This reverts commit 5b853b2.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add plugin status API #75819

Add plugin status API #75819

joshdover commented Aug 24, 2020 •

edited

Loading

joshdover Aug 24, 2020

mshustov Aug 25, 2020 •

edited

Loading

pgayvallet Aug 25, 2020 •

edited

Loading

joshdover Aug 25, 2020

joshdover Aug 26, 2020

elasticmachine commented Aug 25, 2020

joshdover Aug 25, 2020

mshustov Aug 26, 2020

joshdover Aug 31, 2020

kibanamachine commented Sep 1, 2020

Add plugin status API #75819

Add plugin status API #75819

Conversation

joshdover commented Aug 24, 2020 • edited Loading

Summary

Dev Docs

Checklist

For maintainers

joshdover Aug 24, 2020

Choose a reason for hiding this comment

mshustov Aug 25, 2020 • edited Loading

Choose a reason for hiding this comment

pgayvallet Aug 25, 2020 • edited Loading

Choose a reason for hiding this comment

joshdover Aug 25, 2020

Choose a reason for hiding this comment

joshdover Aug 26, 2020

Choose a reason for hiding this comment

elasticmachine commented Aug 25, 2020

joshdover Aug 25, 2020

Choose a reason for hiding this comment

mshustov Aug 26, 2020

Choose a reason for hiding this comment

joshdover Aug 31, 2020

Choose a reason for hiding this comment

kibanamachine commented Sep 1, 2020

💚 Build Succeeded

Build metrics

oss distributable file count

distributable file count

History

joshdover commented Aug 24, 2020 •

edited

Loading

mshustov Aug 25, 2020 •

edited

Loading

pgayvallet Aug 25, 2020 •

edited

Loading