diff --git a/docs/apm/images/apm-services-overview.png b/docs/apm/images/apm-services-overview.png index 85b441d47f0c2..3a56d597abfb7 100644 Binary files a/docs/apm/images/apm-services-overview.png and b/docs/apm/images/apm-services-overview.png differ diff --git a/docs/apm/images/apm-traces.png b/docs/apm/images/apm-traces.png index 97e801606c613..ed15423b42c51 100644 Binary files a/docs/apm/images/apm-traces.png and b/docs/apm/images/apm-traces.png differ diff --git a/docs/apm/images/apm-transactions-overview.png b/docs/apm/images/apm-transactions-overview.png index 80fca19ff96cc..1b25668f0fd92 100644 Binary files a/docs/apm/images/apm-transactions-overview.png and b/docs/apm/images/apm-transactions-overview.png differ diff --git a/docs/apm/images/traffic-transactions.png b/docs/apm/images/traffic-transactions.png index 134bc0e6bcb42..ef429740ceee3 100644 Binary files a/docs/apm/images/traffic-transactions.png and b/docs/apm/images/traffic-transactions.png differ diff --git a/docs/apm/service-overview.asciidoc b/docs/apm/service-overview.asciidoc index 088791e6098e6..5fd214e6ce613 100644 --- a/docs/apm/service-overview.asciidoc +++ b/docs/apm/service-overview.asciidoc @@ -17,8 +17,8 @@ Response times for the service. You can filter the *Latency* chart to display th image::apm/images/latency.png[Service latency] [discrete] -[[service-traffic-transactions]] -=== Traffic and transactions +[[service-throughput-transactions]] +=== Throughput and transactions The *Throughput* chart visualizes the average number of transactions per minute for the selected service. @@ -62,6 +62,9 @@ each dependency. By default, dependencies are sorted by _Impact_ to show the mos If there is a particular dependency you are interested in, click *View service map* to view the related <>. +IMPORTANT: A known issue prevents Real User Monitoring (RUM) dependencies from being shown in the +*Dependencies* table. We are working on a fix for this issue. + [role="screenshot"] image::apm/images/spans-dependencies.png[Span type duration and dependencies] diff --git a/docs/apm/transactions.asciidoc b/docs/apm/transactions.asciidoc index 83ca9e5a10a9b..8c8da81aa577e 100644 --- a/docs/apm/transactions.asciidoc +++ b/docs/apm/transactions.asciidoc @@ -17,10 +17,10 @@ If there's a weird spike that you'd like to investigate, you can simply zoom in on the graph - this will adjust the specific time range, and all of the data on the page will update accordingly. -*Transactions per minute*:: -Visualize response codes: `2xx`, `3xx`, `4xx`, etc., -and is useful for determining if you're serving more of one code than you typically do. -Like in the Transaction duration graph, you can zoom in on anomalies to further investigate them. +*Throughput*:: +Visualize response codes: `2xx`, `3xx`, `4xx`, etc. +Useful for determining if more responses than usual are being served with a particular response code. +Like in the latency graph, you can zoom in on anomalies to further investigate them. *Error rate*:: Visualize the total number of transactions with errors divided by the total number of transactions. @@ -157,4 +157,4 @@ and solve problems. [role="screenshot"] image::apm/images/apm-logs-tab.png[APM logs tab] -// To do: link to log correlation \ No newline at end of file +// To do: link to log correlation diff --git a/docs/apm/troubleshooting.asciidoc b/docs/apm/troubleshooting.asciidoc index 7084777cbb6f9..465a3d652046d 100644 --- a/docs/apm/troubleshooting.asciidoc +++ b/docs/apm/troubleshooting.asciidoc @@ -1,14 +1,30 @@ [[troubleshooting]] -== Troubleshoot common problems +== Troubleshooting ++++ Troubleshooting ++++ -If you have something to add to this section, please consider creating a pull request with -your proposed changes at https://github.com/elastic/kibana. - -Also, check out the https://discuss.elastic.co/c/apm[APM discussion forum]. +This section describes common problems you might encounter with the APM app. +To add to this page, please consider opening a +https://github.com/elastic/kibana/pulls[pull request] with your proposed changes. + +If your issue is potentially related to other components of the APM ecosystem, +don't forget to check our other troubleshooting guides or discussion forum: + +* {apm-server-ref}/troubleshooting.html[APM Server troubleshooting] +* {apm-dotnet-ref}/troubleshooting.html[.NET agent troubleshooting] +* {apm-go-ref}/troubleshooting.html[Go agent troubleshooting] +* {apm-java-ref}/trouble-shooting.html[Java agent troubleshooting] +* {apm-node-ref}/troubleshooting.html[Node.js agent troubleshooting] +* {apm-py-ref}/troubleshooting.html[Python agent troubleshooting] +* {apm-ruby-ref}/debugging.html[Ruby agent troubleshooting] +* {apm-rum-ref/troubleshooting.html[RUM troubleshooting] +* https://discuss.elastic.co/c/apm[APM discussion forum]. + +[discrete] +[[troubleshooting-apm-app]] +== Troubleshoot common APM app problems * <> * <> diff --git a/docs/development/core/server/kibana-plugin-core-server.elasticsearchclientconfig.md b/docs/development/core/server/kibana-plugin-core-server.elasticsearchclientconfig.md index 1ba359e81b9c6..a854e5ddad19a 100644 --- a/docs/development/core/server/kibana-plugin-core-server.elasticsearchclientconfig.md +++ b/docs/development/core/server/kibana-plugin-core-server.elasticsearchclientconfig.md @@ -9,7 +9,7 @@ Configuration options to be used to create a [cluster client](./kibana-plugin-co Signature: ```typescript -export declare type ElasticsearchClientConfig = Pick & { +export declare type ElasticsearchClientConfig = Pick & { pingTimeout?: ElasticsearchConfig['pingTimeout'] | ClientOptions['pingTimeout']; requestTimeout?: ElasticsearchConfig['requestTimeout'] | ClientOptions['requestTimeout']; ssl?: Partial; diff --git a/docs/development/core/server/kibana-plugin-core-server.elasticsearchconfig.logqueries.md b/docs/development/core/server/kibana-plugin-core-server.elasticsearchconfig.logqueries.md deleted file mode 100644 index 001fb7bfeeb97..0000000000000 --- a/docs/development/core/server/kibana-plugin-core-server.elasticsearchconfig.logqueries.md +++ /dev/null @@ -1,13 +0,0 @@ - - -[Home](./index.md) > [kibana-plugin-core-server](./kibana-plugin-core-server.md) > [ElasticsearchConfig](./kibana-plugin-core-server.elasticsearchconfig.md) > [logQueries](./kibana-plugin-core-server.elasticsearchconfig.logqueries.md) - -## ElasticsearchConfig.logQueries property - -Specifies whether all queries to the client should be logged (status code, method, query etc.). - -Signature: - -```typescript -readonly logQueries: boolean; -``` diff --git a/docs/development/core/server/kibana-plugin-core-server.elasticsearchconfig.md b/docs/development/core/server/kibana-plugin-core-server.elasticsearchconfig.md index 5ec3ce7f41859..d87ea63d59b8d 100644 --- a/docs/development/core/server/kibana-plugin-core-server.elasticsearchconfig.md +++ b/docs/development/core/server/kibana-plugin-core-server.elasticsearchconfig.md @@ -27,7 +27,6 @@ export declare class ElasticsearchConfig | [healthCheckDelay](./kibana-plugin-core-server.elasticsearchconfig.healthcheckdelay.md) | | Duration | The interval between health check requests Kibana sends to the Elasticsearch. | | [hosts](./kibana-plugin-core-server.elasticsearchconfig.hosts.md) | | string[] | Hosts that the client will connect to. If sniffing is enabled, this list will be used as seeds to discover the rest of your cluster. | | [ignoreVersionMismatch](./kibana-plugin-core-server.elasticsearchconfig.ignoreversionmismatch.md) | | boolean | Whether to allow kibana to connect to a non-compatible elasticsearch node. | -| [logQueries](./kibana-plugin-core-server.elasticsearchconfig.logqueries.md) | | boolean | Specifies whether all queries to the client should be logged (status code, method, query etc.). | | [password](./kibana-plugin-core-server.elasticsearchconfig.password.md) | | string | If Elasticsearch is protected with basic authentication, this setting provides the password that the Kibana server uses to perform its administrative functions. | | [pingTimeout](./kibana-plugin-core-server.elasticsearchconfig.pingtimeout.md) | | Duration | Timeout after which PING HTTP request will be aborted and retried. | | [requestHeadersWhitelist](./kibana-plugin-core-server.elasticsearchconfig.requestheaderswhitelist.md) | | string[] | List of Kibana client-side headers to send to Elasticsearch when request scoped cluster client is used. If this is an empty array then \*no\* client-side will be sent. | diff --git a/docs/development/core/server/kibana-plugin-core-server.legacyclusterclient._constructor_.md b/docs/development/core/server/kibana-plugin-core-server.legacyclusterclient._constructor_.md index 823f34bd7dd23..ed2763d980279 100644 --- a/docs/development/core/server/kibana-plugin-core-server.legacyclusterclient._constructor_.md +++ b/docs/development/core/server/kibana-plugin-core-server.legacyclusterclient._constructor_.md @@ -9,7 +9,7 @@ Constructs a new instance of the `LegacyClusterClient` class Signature: ```typescript -constructor(config: LegacyElasticsearchClientConfig, log: Logger, getAuthHeaders?: GetAuthHeaders); +constructor(config: LegacyElasticsearchClientConfig, log: Logger, type: string, getAuthHeaders?: GetAuthHeaders); ``` ## Parameters @@ -18,5 +18,6 @@ constructor(config: LegacyElasticsearchClientConfig, log: Logger, getAuthHeaders | --- | --- | --- | | config | LegacyElasticsearchClientConfig | | | log | Logger | | +| type | string | | | getAuthHeaders | GetAuthHeaders | | diff --git a/docs/development/core/server/kibana-plugin-core-server.legacyclusterclient.md b/docs/development/core/server/kibana-plugin-core-server.legacyclusterclient.md index d24aeb44ca86a..0872e5ba7c219 100644 --- a/docs/development/core/server/kibana-plugin-core-server.legacyclusterclient.md +++ b/docs/development/core/server/kibana-plugin-core-server.legacyclusterclient.md @@ -21,7 +21,7 @@ export declare class LegacyClusterClient implements ILegacyClusterClient | Constructor | Modifiers | Description | | --- | --- | --- | -| [(constructor)(config, log, getAuthHeaders)](./kibana-plugin-core-server.legacyclusterclient._constructor_.md) | | Constructs a new instance of the LegacyClusterClient class | +| [(constructor)(config, log, type, getAuthHeaders)](./kibana-plugin-core-server.legacyclusterclient._constructor_.md) | | Constructs a new instance of the LegacyClusterClient class | ## Properties diff --git a/docs/development/core/server/kibana-plugin-core-server.legacyelasticsearchclientconfig.md b/docs/development/core/server/kibana-plugin-core-server.legacyelasticsearchclientconfig.md index 78f7bf582d355..b028a09bee453 100644 --- a/docs/development/core/server/kibana-plugin-core-server.legacyelasticsearchclientconfig.md +++ b/docs/development/core/server/kibana-plugin-core-server.legacyelasticsearchclientconfig.md @@ -11,7 +11,7 @@ Signature: ```typescript -export declare type LegacyElasticsearchClientConfig = Pick & Pick & { +export declare type LegacyElasticsearchClientConfig = Pick & Pick & { pingTimeout?: ElasticsearchConfig['pingTimeout'] | ConfigOptions['pingTimeout']; requestTimeout?: ElasticsearchConfig['requestTimeout'] | ConfigOptions['requestTimeout']; sniffInterval?: ElasticsearchConfig['sniffInterval'] | ConfigOptions['sniffInterval']; diff --git a/docs/development/plugins/data/public/kibana-plugin-plugins-data-public.isearchoptions.legacyhitstotal.md b/docs/development/plugins/data/public/kibana-plugin-plugins-data-public.isearchoptions.legacyhitstotal.md new file mode 100644 index 0000000000000..937e20a7a9579 --- /dev/null +++ b/docs/development/plugins/data/public/kibana-plugin-plugins-data-public.isearchoptions.legacyhitstotal.md @@ -0,0 +1,13 @@ + + +[Home](./index.md) > [kibana-plugin-plugins-data-public](./kibana-plugin-plugins-data-public.md) > [ISearchOptions](./kibana-plugin-plugins-data-public.isearchoptions.md) > [legacyHitsTotal](./kibana-plugin-plugins-data-public.isearchoptions.legacyhitstotal.md) + +## ISearchOptions.legacyHitsTotal property + +Request the legacy format for the total number of hits. If sending `rest_total_hits_as_int` to something other than `true`, this should be set to `false`. + +Signature: + +```typescript +legacyHitsTotal?: boolean; +``` diff --git a/docs/development/plugins/data/public/kibana-plugin-plugins-data-public.isearchoptions.md b/docs/development/plugins/data/public/kibana-plugin-plugins-data-public.isearchoptions.md index 5acd837495dac..fc2767cd0231f 100644 --- a/docs/development/plugins/data/public/kibana-plugin-plugins-data-public.isearchoptions.md +++ b/docs/development/plugins/data/public/kibana-plugin-plugins-data-public.isearchoptions.md @@ -17,6 +17,7 @@ export interface ISearchOptions | [abortSignal](./kibana-plugin-plugins-data-public.isearchoptions.abortsignal.md) | AbortSignal | An AbortSignal that allows the caller of search to abort a search request. | | [isRestore](./kibana-plugin-plugins-data-public.isearchoptions.isrestore.md) | boolean | Whether the session is restored (i.e. search requests should re-use the stored search IDs, rather than starting from scratch) | | [isStored](./kibana-plugin-plugins-data-public.isearchoptions.isstored.md) | boolean | Whether the session is already saved (i.e. sent to background) | +| [legacyHitsTotal](./kibana-plugin-plugins-data-public.isearchoptions.legacyhitstotal.md) | boolean | Request the legacy format for the total number of hits. If sending rest_total_hits_as_int to something other than true, this should be set to false. | | [sessionId](./kibana-plugin-plugins-data-public.isearchoptions.sessionid.md) | string | A session ID, grouping multiple search requests into a single session. | | [strategy](./kibana-plugin-plugins-data-public.isearchoptions.strategy.md) | string | Use this option to force using a specific server side search strategy. Leave empty to use the default strategy. | diff --git a/docs/development/plugins/data/server/kibana-plugin-plugins-data-server.iscopedsessionservice.md b/docs/development/plugins/data/server/kibana-plugin-plugins-data-server.iscopedsessionservice.md new file mode 100644 index 0000000000000..eaac671b9a182 --- /dev/null +++ b/docs/development/plugins/data/server/kibana-plugin-plugins-data-server.iscopedsessionservice.md @@ -0,0 +1,18 @@ + + +[Home](./index.md) > [kibana-plugin-plugins-data-server](./kibana-plugin-plugins-data-server.md) > [IScopedSessionService](./kibana-plugin-plugins-data-server.iscopedsessionservice.md) + +## IScopedSessionService interface + +Signature: + +```typescript +export interface IScopedSessionService +``` + +## Properties + +| Property | Type | Description | +| --- | --- | --- | +| [search](./kibana-plugin-plugins-data-server.iscopedsessionservice.search.md) | <Request extends IKibanaSearchRequest, Response extends IKibanaSearchResponse>(strategy: ISearchStrategy<Request, Response>, ...args: Parameters<ISearchStrategy<Request, Response>['search']>) => Observable<Response> | | + diff --git a/docs/development/plugins/data/server/kibana-plugin-plugins-data-server.iscopedsessionservice.search.md b/docs/development/plugins/data/server/kibana-plugin-plugins-data-server.iscopedsessionservice.search.md new file mode 100644 index 0000000000000..d58a9fd9f3761 --- /dev/null +++ b/docs/development/plugins/data/server/kibana-plugin-plugins-data-server.iscopedsessionservice.search.md @@ -0,0 +1,11 @@ + + +[Home](./index.md) > [kibana-plugin-plugins-data-server](./kibana-plugin-plugins-data-server.md) > [IScopedSessionService](./kibana-plugin-plugins-data-server.iscopedsessionservice.md) > [search](./kibana-plugin-plugins-data-server.iscopedsessionservice.search.md) + +## IScopedSessionService.search property + +Signature: + +```typescript +search: (strategy: ISearchStrategy, ...args: Parameters['search']>) => Observable; +``` diff --git a/docs/development/plugins/data/server/kibana-plugin-plugins-data-server.isearchoptions.legacyhitstotal.md b/docs/development/plugins/data/server/kibana-plugin-plugins-data-server.isearchoptions.legacyhitstotal.md new file mode 100644 index 0000000000000..59b8b2c6b446f --- /dev/null +++ b/docs/development/plugins/data/server/kibana-plugin-plugins-data-server.isearchoptions.legacyhitstotal.md @@ -0,0 +1,13 @@ + + +[Home](./index.md) > [kibana-plugin-plugins-data-server](./kibana-plugin-plugins-data-server.md) > [ISearchOptions](./kibana-plugin-plugins-data-server.isearchoptions.md) > [legacyHitsTotal](./kibana-plugin-plugins-data-server.isearchoptions.legacyhitstotal.md) + +## ISearchOptions.legacyHitsTotal property + +Request the legacy format for the total number of hits. If sending `rest_total_hits_as_int` to something other than `true`, this should be set to `false`. + +Signature: + +```typescript +legacyHitsTotal?: boolean; +``` diff --git a/docs/development/plugins/data/server/kibana-plugin-plugins-data-server.isearchoptions.md b/docs/development/plugins/data/server/kibana-plugin-plugins-data-server.isearchoptions.md index 85847e1c61d25..9de351b2b9019 100644 --- a/docs/development/plugins/data/server/kibana-plugin-plugins-data-server.isearchoptions.md +++ b/docs/development/plugins/data/server/kibana-plugin-plugins-data-server.isearchoptions.md @@ -17,6 +17,7 @@ export interface ISearchOptions | [abortSignal](./kibana-plugin-plugins-data-server.isearchoptions.abortsignal.md) | AbortSignal | An AbortSignal that allows the caller of search to abort a search request. | | [isRestore](./kibana-plugin-plugins-data-server.isearchoptions.isrestore.md) | boolean | Whether the session is restored (i.e. search requests should re-use the stored search IDs, rather than starting from scratch) | | [isStored](./kibana-plugin-plugins-data-server.isearchoptions.isstored.md) | boolean | Whether the session is already saved (i.e. sent to background) | +| [legacyHitsTotal](./kibana-plugin-plugins-data-server.isearchoptions.legacyhitstotal.md) | boolean | Request the legacy format for the total number of hits. If sending rest_total_hits_as_int to something other than true, this should be set to false. | | [sessionId](./kibana-plugin-plugins-data-server.isearchoptions.sessionid.md) | string | A session ID, grouping multiple search requests into a single session. | | [strategy](./kibana-plugin-plugins-data-server.isearchoptions.strategy.md) | string | Use this option to force using a specific server side search strategy. Leave empty to use the default strategy. | diff --git a/docs/development/plugins/data/server/kibana-plugin-plugins-data-server.md b/docs/development/plugins/data/server/kibana-plugin-plugins-data-server.md index 84c7875c26ce8..27a386a714fc1 100644 --- a/docs/development/plugins/data/server/kibana-plugin-plugins-data-server.md +++ b/docs/development/plugins/data/server/kibana-plugin-plugins-data-server.md @@ -53,6 +53,7 @@ | [IFieldSubType](./kibana-plugin-plugins-data-server.ifieldsubtype.md) | | | [IFieldType](./kibana-plugin-plugins-data-server.ifieldtype.md) | | | [IndexPatternAttributes](./kibana-plugin-plugins-data-server.indexpatternattributes.md) | Interface for an index pattern saved object | +| [IScopedSessionService](./kibana-plugin-plugins-data-server.iscopedsessionservice.md) | | | [ISearchOptions](./kibana-plugin-plugins-data-server.isearchoptions.md) | | | [ISearchSetup](./kibana-plugin-plugins-data-server.isearchsetup.md) | | | [ISearchStart](./kibana-plugin-plugins-data-server.isearchstart.md) | | diff --git a/docs/maps/maps-getting-started.asciidoc b/docs/maps/maps-getting-started.asciidoc index 32a81c8e65f56..609a133c92ad1 100644 --- a/docs/maps/maps-getting-started.asciidoc +++ b/docs/maps/maps-getting-started.asciidoc @@ -1,15 +1,11 @@ [role="xpack"] [[maps-getting-started]] -== Create a map with multiple layers and data sources - -++++ -Create a multilayer map -++++ +== Build a map to compare metrics by country or region If you are new to **Maps**, this tutorial is a good place to start. It guides you through the common steps for working with your location data. -You'll learn to: +You will learn to: - Create a map with multiple layers and data sources - Use symbols, colors, and labels to style data values diff --git a/docs/setup/settings.asciidoc b/docs/setup/settings.asciidoc index 26f095c59c644..ecdb41c897b12 100644 --- a/docs/setup/settings.asciidoc +++ b/docs/setup/settings.asciidoc @@ -59,7 +59,7 @@ To enable SSL/TLS for outbound connections to {es}, use the `https` protocol in this setting. | `elasticsearch.logQueries:` - | Log queries sent to {es}. Requires <> set to `true`. + | *deprecated* This setting is no longer used and will get removed in Kibana 8.0. Instead, set <> to `true` This is useful for seeing the query DSL generated by applications that currently do not have an inspector, for example Timelion and Monitoring. *Default: `false`* diff --git a/packages/kbn-es-archiver/src/actions/load.ts b/packages/kbn-es-archiver/src/actions/load.ts index 8afd7d79a98f7..c32496ad42694 100644 --- a/packages/kbn-es-archiver/src/actions/load.ts +++ b/packages/kbn-es-archiver/src/actions/load.ts @@ -98,9 +98,11 @@ export async function loadAction({ // If we affected the Kibana index, we need to ensure it's migrated... if (Object.keys(result).some((k) => k.startsWith('.kibana'))) { await migrateKibanaIndex({ client, kbnClient }); + log.debug('[%s] Migrated Kibana index after loading Kibana data', name); if (kibanaPluginIds.includes('spaces')) { await createDefaultSpace({ client, index: '.kibana' }); + log.debug('[%s] Ensured that default space exists in .kibana', name); } } diff --git a/packages/kbn-es/src/cluster.js b/packages/kbn-es/src/cluster.js index f554dd8a1b8e5..5948e9ecececc 100644 --- a/packages/kbn-es/src/cluster.js +++ b/packages/kbn-es/src/cluster.js @@ -246,7 +246,7 @@ exports.Cluster = class Cluster { this._log.info(chalk.bold('Starting')); this._log.indent(4); - const esArgs = ['indices.query.bool.max_nested_depth=100'].concat(options.esArgs || []); + const esArgs = options.esArgs || []; // Add to esArgs if ssl is enabled if (this._ssl) { diff --git a/packages/kbn-es/src/integration_tests/cluster.test.js b/packages/kbn-es/src/integration_tests/cluster.test.js index c1adc84ddc954..684667355852d 100644 --- a/packages/kbn-es/src/integration_tests/cluster.test.js +++ b/packages/kbn-es/src/integration_tests/cluster.test.js @@ -264,9 +264,7 @@ describe('#start(installPath)', () => { expect(extractConfigFiles.mock.calls).toMatchInlineSnapshot(` Array [ Array [ - Array [ - "indices.query.bool.max_nested_depth=100", - ], + Array [], undefined, Object { "log": , @@ -342,9 +340,7 @@ describe('#run()', () => { expect(extractConfigFiles.mock.calls).toMatchInlineSnapshot(` Array [ Array [ - Array [ - "indices.query.bool.max_nested_depth=100", - ], + Array [], undefined, Object { "log": , diff --git a/packages/kbn-optimizer/limits.yml b/packages/kbn-optimizer/limits.yml index a13976d148738..74ee3f7c46e1b 100644 --- a/packages/kbn-optimizer/limits.yml +++ b/packages/kbn-optimizer/limits.yml @@ -34,7 +34,7 @@ pageLoadAssetSize: indexLifecycleManagement: 107090 indexManagement: 140608 indexPatternManagement: 154222 - infra: 197873 + infra: 204800 fleet: 415829 ingestPipelines: 58003 inputControlVis: 172675 @@ -78,7 +78,7 @@ pageLoadAssetSize: tileMap: 65337 timelion: 29920 transform: 41007 - triggersActionsUi: 170001 + triggersActionsUi: 186732 uiActions: 97717 uiActionsEnhanced: 313011 upgradeAssistant: 81241 diff --git a/packages/kbn-pm/dist/index.js b/packages/kbn-pm/dist/index.js index df04965cd8c32..e8c6fa4d5a013 100644 --- a/packages/kbn-pm/dist/index.js +++ b/packages/kbn-pm/dist/index.js @@ -47961,11 +47961,13 @@ __webpack_require__.r(__webpack_exports__); "use strict"; __webpack_require__.r(__webpack_exports__); /* harmony export (binding) */ __webpack_require__.d(__webpack_exports__, "installBazelTools", function() { return installBazelTools; }); -/* harmony import */ var path__WEBPACK_IMPORTED_MODULE_0__ = __webpack_require__(4); -/* harmony import */ var path__WEBPACK_IMPORTED_MODULE_0___default = /*#__PURE__*/__webpack_require__.n(path__WEBPACK_IMPORTED_MODULE_0__); -/* harmony import */ var _child_process__WEBPACK_IMPORTED_MODULE_1__ = __webpack_require__(319); -/* harmony import */ var _fs__WEBPACK_IMPORTED_MODULE_2__ = __webpack_require__(131); -/* harmony import */ var _log__WEBPACK_IMPORTED_MODULE_3__ = __webpack_require__(246); +/* harmony import */ var dedent__WEBPACK_IMPORTED_MODULE_0__ = __webpack_require__(2); +/* harmony import */ var dedent__WEBPACK_IMPORTED_MODULE_0___default = /*#__PURE__*/__webpack_require__.n(dedent__WEBPACK_IMPORTED_MODULE_0__); +/* harmony import */ var path__WEBPACK_IMPORTED_MODULE_1__ = __webpack_require__(4); +/* harmony import */ var path__WEBPACK_IMPORTED_MODULE_1___default = /*#__PURE__*/__webpack_require__.n(path__WEBPACK_IMPORTED_MODULE_1__); +/* harmony import */ var _child_process__WEBPACK_IMPORTED_MODULE_2__ = __webpack_require__(319); +/* harmony import */ var _fs__WEBPACK_IMPORTED_MODULE_3__ = __webpack_require__(131); +/* harmony import */ var _log__WEBPACK_IMPORTED_MODULE_4__ = __webpack_require__(246); /* * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one * or more contributor license agreements. Licensed under the Elastic License @@ -47978,8 +47980,9 @@ __webpack_require__.r(__webpack_exports__); + async function readBazelToolsVersionFile(repoRootPath, versionFilename) { - const version = (await Object(_fs__WEBPACK_IMPORTED_MODULE_2__["readFile"])(Object(path__WEBPACK_IMPORTED_MODULE_0__["resolve"])(repoRootPath, versionFilename))).toString().split('\n')[0]; + const version = (await Object(_fs__WEBPACK_IMPORTED_MODULE_3__["readFile"])(Object(path__WEBPACK_IMPORTED_MODULE_1__["resolve"])(repoRootPath, versionFilename))).toString().split('\n')[0]; if (!version) { throw new Error(`[bazel_tools] Failed on reading bazel tools versions\n ${versionFilename} file do not contain any version set`); @@ -47988,30 +47991,49 @@ async function readBazelToolsVersionFile(repoRootPath, versionFilename) { return version; } +async function isBazelBinAvailable() { + try { + await Object(_child_process__WEBPACK_IMPORTED_MODULE_2__["spawn"])('bazel', ['--version'], { + stdio: 'pipe' + }); + return true; + } catch { + return false; + } +} + async function installBazelTools(repoRootPath) { - _log__WEBPACK_IMPORTED_MODULE_3__["log"].debug(`[bazel_tools] reading bazel tools versions from version files`); + _log__WEBPACK_IMPORTED_MODULE_4__["log"].debug(`[bazel_tools] reading bazel tools versions from version files`); const bazeliskVersion = await readBazelToolsVersionFile(repoRootPath, '.bazeliskversion'); const bazelVersion = await readBazelToolsVersionFile(repoRootPath, '.bazelversion'); // Check what globals are installed - _log__WEBPACK_IMPORTED_MODULE_3__["log"].debug(`[bazel_tools] verify if bazelisk is installed`); + _log__WEBPACK_IMPORTED_MODULE_4__["log"].debug(`[bazel_tools] verify if bazelisk is installed`); const { - stdout - } = await Object(_child_process__WEBPACK_IMPORTED_MODULE_1__["spawn"])('yarn', ['global', 'list'], { + stdout: bazeliskPkgInstallStdout + } = await Object(_child_process__WEBPACK_IMPORTED_MODULE_2__["spawn"])('yarn', ['global', 'list'], { stdio: 'pipe' - }); // Install bazelisk if not installed + }); + const isBazelBinAlreadyAvailable = await isBazelBinAvailable(); // Install bazelisk if not installed - if (!stdout.includes(`@bazel/bazelisk@${bazeliskVersion}`)) { - _log__WEBPACK_IMPORTED_MODULE_3__["log"].info(`[bazel_tools] installing Bazel tools`); - _log__WEBPACK_IMPORTED_MODULE_3__["log"].debug(`[bazel_tools] bazelisk is not installed. Installing @bazel/bazelisk@${bazeliskVersion} and bazel@${bazelVersion}`); - await Object(_child_process__WEBPACK_IMPORTED_MODULE_1__["spawn"])('yarn', ['global', 'add', `@bazel/bazelisk@${bazeliskVersion}`], { + if (!bazeliskPkgInstallStdout.includes(`@bazel/bazelisk@${bazeliskVersion}`) || !isBazelBinAlreadyAvailable) { + _log__WEBPACK_IMPORTED_MODULE_4__["log"].info(`[bazel_tools] installing Bazel tools`); + _log__WEBPACK_IMPORTED_MODULE_4__["log"].debug(`[bazel_tools] bazelisk is not installed. Installing @bazel/bazelisk@${bazeliskVersion} and bazel@${bazelVersion}`); + await Object(_child_process__WEBPACK_IMPORTED_MODULE_2__["spawn"])('yarn', ['global', 'add', `@bazel/bazelisk@${bazeliskVersion}`], { env: { USE_BAZEL_VERSION: bazelVersion }, stdio: 'pipe' }); + const isBazelBinAvailableAfterInstall = await isBazelBinAvailable(); + + if (!isBazelBinAvailableAfterInstall) { + throw new Error(dedent__WEBPACK_IMPORTED_MODULE_0___default.a` + [bazel_tools] an error occurred when installing the Bazel tools. Please make sure 'yarn global bin' is on your $PATH, otherwise just add it there + `); + } } - _log__WEBPACK_IMPORTED_MODULE_3__["log"].success(`[bazel_tools] all bazel tools are correctly installed`); + _log__WEBPACK_IMPORTED_MODULE_4__["log"].success(`[bazel_tools] all bazel tools are correctly installed`); } /***/ }), diff --git a/packages/kbn-pm/src/utils/bazel/install_tools.ts b/packages/kbn-pm/src/utils/bazel/install_tools.ts index 4e19974590e83..3440d32ee4b51 100644 --- a/packages/kbn-pm/src/utils/bazel/install_tools.ts +++ b/packages/kbn-pm/src/utils/bazel/install_tools.ts @@ -6,6 +6,7 @@ * Public License, v 1. */ +import dedent from 'dedent'; import { resolve } from 'path'; import { spawn } from '../child_process'; import { readFile } from '../fs'; @@ -25,6 +26,16 @@ async function readBazelToolsVersionFile(repoRootPath: string, versionFilename: return version; } +async function isBazelBinAvailable() { + try { + await spawn('bazel', ['--version'], { stdio: 'pipe' }); + + return true; + } catch { + return false; + } +} + export async function installBazelTools(repoRootPath: string) { log.debug(`[bazel_tools] reading bazel tools versions from version files`); const bazeliskVersion = await readBazelToolsVersionFile(repoRootPath, '.bazeliskversion'); @@ -32,10 +43,17 @@ export async function installBazelTools(repoRootPath: string) { // Check what globals are installed log.debug(`[bazel_tools] verify if bazelisk is installed`); - const { stdout } = await spawn('yarn', ['global', 'list'], { stdio: 'pipe' }); + const { stdout: bazeliskPkgInstallStdout } = await spawn('yarn', ['global', 'list'], { + stdio: 'pipe', + }); + + const isBazelBinAlreadyAvailable = await isBazelBinAvailable(); // Install bazelisk if not installed - if (!stdout.includes(`@bazel/bazelisk@${bazeliskVersion}`)) { + if ( + !bazeliskPkgInstallStdout.includes(`@bazel/bazelisk@${bazeliskVersion}`) || + !isBazelBinAlreadyAvailable + ) { log.info(`[bazel_tools] installing Bazel tools`); log.debug( @@ -47,6 +65,13 @@ export async function installBazelTools(repoRootPath: string) { }, stdio: 'pipe', }); + + const isBazelBinAvailableAfterInstall = await isBazelBinAvailable(); + if (!isBazelBinAvailableAfterInstall) { + throw new Error(dedent` + [bazel_tools] an error occurred when installing the Bazel tools. Please make sure 'yarn global bin' is on your $PATH, otherwise just add it there + `); + } } log.success(`[bazel_tools] all bazel tools are correctly installed`); diff --git a/preinstall_check.js b/preinstall_check.js index e761afa91022c..e508f9ecb10c6 100644 --- a/preinstall_check.js +++ b/preinstall_check.js @@ -6,34 +6,37 @@ * Public License, v 1. */ -const isUsingNpm = process.env.npm_config_git !== undefined; +(() => { + const isUsingNpm = process.env.npm_config_git !== undefined; -if (isUsingNpm) { - throw `Use Yarn instead of npm, see Kibana's contributing guidelines`; -} - -// The value of the `npm_config_argv` env for each command: -// -// - `npm install`: '{"remain":[],"cooked":["install"],"original":[]}' -// - `yarn`: '{"remain":[],"cooked":["install"],"original":[]}' -// - `yarn kbn bootstrap`: '{"remain":[],"cooked":["run","kbn"],"original":["kbn","bootstrap"]}' -const rawArgv = process.env.npm_config_argv; - -if (rawArgv === undefined) { - return; -} + if (isUsingNpm) { + throw `Use Yarn instead of npm, see Kibana's contributing guidelines`; + } -try { - const argv = JSON.parse(rawArgv); + // The value of the `npm_config_argv` env for each command: + // + // - `npm install`: '{"remain":[],"cooked":["install"],"original":[]}' + // - `yarn`: '{"remain":[],"cooked":["install"],"original":[]}' + // - `yarn kbn bootstrap`: '{"remain":[],"cooked":["run","kbn"],"original":["kbn","bootstrap"]}' + const rawArgv = process.env.npm_config_argv; - if (argv.cooked.includes('kbn')) { - // all good, trying to install deps using `kbn` + if (rawArgv === undefined) { return; } - if (argv.cooked.includes('install')) { - console.log('\nWARNING: When installing dependencies, prefer `yarn kbn bootstrap`\n'); + try { + const argv = JSON.parse(rawArgv); + + // allow dependencies to be installed with `yarn kbn bootstrap` or `bazel run @nodejs//:yarn` (called under the hood by bazel) + if (argv.cooked.includes('kbn') || !!process.env.BAZEL_YARN_INSTALL) { + // all good, trying to install deps using `kbn` or bazel directly + return; + } + + if (argv.cooked.includes('install')) { + console.log('\nWARNING: When installing dependencies, prefer `yarn kbn bootstrap`\n'); + } + } catch (e) { + // if it fails we do nothing, as this is just intended to be a helpful message } -} catch (e) { - // if it fails we do nothing, as this is just intended to be a helpful message -} +})(); diff --git a/src/core/public/chrome/ui/header/__snapshots__/collapsible_nav.test.tsx.snap b/src/core/public/chrome/ui/header/__snapshots__/collapsible_nav.test.tsx.snap index c836686ec602b..80e23a32ca557 100644 --- a/src/core/public/chrome/ui/header/__snapshots__/collapsible_nav.test.tsx.snap +++ b/src/core/public/chrome/ui/header/__snapshots__/collapsible_nav.test.tsx.snap @@ -543,6 +543,7 @@ exports[`CollapsibleNav renders links grouped by category 1`] = ` listItems={ Array [ Object { + "data-test-subj": "homeLink", "href": "/", "iconType": "home", "label": "Home", @@ -564,6 +565,7 @@ exports[`CollapsibleNav renders links grouped by category 1`] = ` > { if (isModifiedOrPrevented(event)) { return; diff --git a/src/core/server/elasticsearch/client/client_config.test.ts b/src/core/server/elasticsearch/client/client_config.test.ts index 57bc7407a9a0f..768d165d5f8be 100644 --- a/src/core/server/elasticsearch/client/client_config.test.ts +++ b/src/core/server/elasticsearch/client/client_config.test.ts @@ -15,7 +15,6 @@ const createConfig = ( ): ElasticsearchClientConfig => { return { customHeaders: {}, - logQueries: false, sniffOnStart: false, sniffOnConnectionFault: false, sniffInterval: false, diff --git a/src/core/server/elasticsearch/client/client_config.ts b/src/core/server/elasticsearch/client/client_config.ts index 5762ef16704a5..01d2222a45e3a 100644 --- a/src/core/server/elasticsearch/client/client_config.ts +++ b/src/core/server/elasticsearch/client/client_config.ts @@ -22,7 +22,6 @@ import { DEFAULT_HEADERS } from '../default_headers'; export type ElasticsearchClientConfig = Pick< ElasticsearchConfig, | 'customHeaders' - | 'logQueries' | 'sniffOnStart' | 'sniffOnConnectionFault' | 'requestHeadersWhitelist' diff --git a/src/core/server/elasticsearch/client/cluster_client.test.ts b/src/core/server/elasticsearch/client/cluster_client.test.ts index b94bf08f1185b..1d6d373ec185c 100644 --- a/src/core/server/elasticsearch/client/cluster_client.test.ts +++ b/src/core/server/elasticsearch/client/cluster_client.test.ts @@ -19,7 +19,6 @@ const createConfig = ( parts: Partial = {} ): ElasticsearchClientConfig => { return { - logQueries: false, sniffOnStart: false, sniffOnConnectionFault: false, sniffInterval: false, @@ -57,16 +56,25 @@ describe('ClusterClient', () => { it('creates a single internal and scoped client during initialization', () => { const config = createConfig(); - new ClusterClient(config, logger, getAuthHeaders); + new ClusterClient(config, logger, 'custom-type', getAuthHeaders); expect(configureClientMock).toHaveBeenCalledTimes(2); - expect(configureClientMock).toHaveBeenCalledWith(config, { logger }); - expect(configureClientMock).toHaveBeenCalledWith(config, { logger, scoped: true }); + expect(configureClientMock).toHaveBeenCalledWith(config, { logger, type: 'custom-type' }); + expect(configureClientMock).toHaveBeenCalledWith(config, { + logger, + type: 'custom-type', + scoped: true, + }); }); describe('#asInternalUser', () => { it('returns the internal client', () => { - const clusterClient = new ClusterClient(createConfig(), logger, getAuthHeaders); + const clusterClient = new ClusterClient( + createConfig(), + logger, + 'custom-type', + getAuthHeaders + ); expect(clusterClient.asInternalUser).toBe(internalClient); }); @@ -74,7 +82,12 @@ describe('ClusterClient', () => { describe('#asScoped', () => { it('returns a scoped cluster client bound to the request', () => { - const clusterClient = new ClusterClient(createConfig(), logger, getAuthHeaders); + const clusterClient = new ClusterClient( + createConfig(), + logger, + 'custom-type', + getAuthHeaders + ); const request = httpServerMock.createKibanaRequest(); const scopedClusterClient = clusterClient.asScoped(request); @@ -87,7 +100,12 @@ describe('ClusterClient', () => { }); it('returns a distinct scoped cluster client on each call', () => { - const clusterClient = new ClusterClient(createConfig(), logger, getAuthHeaders); + const clusterClient = new ClusterClient( + createConfig(), + logger, + 'custom-type', + getAuthHeaders + ); const request = httpServerMock.createKibanaRequest(); const scopedClusterClient1 = clusterClient.asScoped(request); @@ -105,7 +123,7 @@ describe('ClusterClient', () => { }); getAuthHeaders.mockReturnValue({}); - const clusterClient = new ClusterClient(config, logger, getAuthHeaders); + const clusterClient = new ClusterClient(config, logger, 'custom-type', getAuthHeaders); const request = httpServerMock.createKibanaRequest({ headers: { foo: 'bar', @@ -130,7 +148,7 @@ describe('ClusterClient', () => { other: 'nope', }); - const clusterClient = new ClusterClient(config, logger, getAuthHeaders); + const clusterClient = new ClusterClient(config, logger, 'custom-type', getAuthHeaders); const request = httpServerMock.createKibanaRequest({}); clusterClient.asScoped(request); @@ -150,7 +168,7 @@ describe('ClusterClient', () => { other: 'nope', }); - const clusterClient = new ClusterClient(config, logger, getAuthHeaders); + const clusterClient = new ClusterClient(config, logger, 'custom-type', getAuthHeaders); const request = httpServerMock.createKibanaRequest({ headers: { authorization: 'override', @@ -175,7 +193,7 @@ describe('ClusterClient', () => { }); getAuthHeaders.mockReturnValue({}); - const clusterClient = new ClusterClient(config, logger, getAuthHeaders); + const clusterClient = new ClusterClient(config, logger, 'custom-type', getAuthHeaders); const request = httpServerMock.createKibanaRequest({}); clusterClient.asScoped(request); @@ -195,7 +213,7 @@ describe('ClusterClient', () => { const config = createConfig(); getAuthHeaders.mockReturnValue({}); - const clusterClient = new ClusterClient(config, logger, getAuthHeaders); + const clusterClient = new ClusterClient(config, logger, 'custom-type', getAuthHeaders); const request = httpServerMock.createKibanaRequest({ kibanaRequestState: { requestId: 'my-fake-id', requestUuid: 'ignore-this-id' }, }); @@ -223,7 +241,7 @@ describe('ClusterClient', () => { foo: 'auth', }); - const clusterClient = new ClusterClient(config, logger, getAuthHeaders); + const clusterClient = new ClusterClient(config, logger, 'custom-type', getAuthHeaders); const request = httpServerMock.createKibanaRequest({}); clusterClient.asScoped(request); @@ -249,7 +267,7 @@ describe('ClusterClient', () => { }); getAuthHeaders.mockReturnValue({}); - const clusterClient = new ClusterClient(config, logger, getAuthHeaders); + const clusterClient = new ClusterClient(config, logger, 'custom-type', getAuthHeaders); const request = httpServerMock.createKibanaRequest({ headers: { foo: 'request' }, }); @@ -276,7 +294,7 @@ describe('ClusterClient', () => { }); getAuthHeaders.mockReturnValue({}); - const clusterClient = new ClusterClient(config, logger, getAuthHeaders); + const clusterClient = new ClusterClient(config, logger, 'custom-type', getAuthHeaders); const request = httpServerMock.createKibanaRequest(); clusterClient.asScoped(request); @@ -297,7 +315,7 @@ describe('ClusterClient', () => { }); getAuthHeaders.mockReturnValue({}); - const clusterClient = new ClusterClient(config, logger, getAuthHeaders); + const clusterClient = new ClusterClient(config, logger, 'custom-type', getAuthHeaders); const request = httpServerMock.createKibanaRequest({ headers: { [headerKey]: 'foo' }, }); @@ -321,7 +339,7 @@ describe('ClusterClient', () => { }); getAuthHeaders.mockReturnValue({}); - const clusterClient = new ClusterClient(config, logger, getAuthHeaders); + const clusterClient = new ClusterClient(config, logger, 'custom-type', getAuthHeaders); const request = httpServerMock.createKibanaRequest({ headers: { foo: 'request' }, kibanaRequestState: { requestId: 'from request', requestUuid: 'ignore-this-id' }, @@ -344,7 +362,7 @@ describe('ClusterClient', () => { }); getAuthHeaders.mockReturnValue({}); - const clusterClient = new ClusterClient(config, logger, getAuthHeaders); + const clusterClient = new ClusterClient(config, logger, 'custom-type', getAuthHeaders); const request = { headers: { authorization: 'auth', @@ -368,7 +386,7 @@ describe('ClusterClient', () => { authorization: 'auth', }); - const clusterClient = new ClusterClient(config, logger, getAuthHeaders); + const clusterClient = new ClusterClient(config, logger, 'custom-type', getAuthHeaders); const request = { headers: { foo: 'bar', @@ -387,7 +405,12 @@ describe('ClusterClient', () => { describe('#close', () => { it('closes both underlying clients', async () => { - const clusterClient = new ClusterClient(createConfig(), logger, getAuthHeaders); + const clusterClient = new ClusterClient( + createConfig(), + logger, + 'custom-type', + getAuthHeaders + ); await clusterClient.close(); @@ -398,7 +421,12 @@ describe('ClusterClient', () => { it('waits for both clients to close', async (done) => { expect.assertions(4); - const clusterClient = new ClusterClient(createConfig(), logger, getAuthHeaders); + const clusterClient = new ClusterClient( + createConfig(), + logger, + 'custom-type', + getAuthHeaders + ); let internalClientClosed = false; let scopedClientClosed = false; @@ -436,7 +464,12 @@ describe('ClusterClient', () => { }); it('return a rejected promise is any client rejects', async () => { - const clusterClient = new ClusterClient(createConfig(), logger, getAuthHeaders); + const clusterClient = new ClusterClient( + createConfig(), + logger, + 'custom-type', + getAuthHeaders + ); internalClient.close.mockRejectedValue(new Error('error closing client')); @@ -446,7 +479,12 @@ describe('ClusterClient', () => { }); it('does nothing after the first call', async () => { - const clusterClient = new ClusterClient(createConfig(), logger, getAuthHeaders); + const clusterClient = new ClusterClient( + createConfig(), + logger, + 'custom-type', + getAuthHeaders + ); await clusterClient.close(); diff --git a/src/core/server/elasticsearch/client/cluster_client.ts b/src/core/server/elasticsearch/client/cluster_client.ts index 87d59e7417aa9..7e6a7f8ae53e6 100644 --- a/src/core/server/elasticsearch/client/cluster_client.ts +++ b/src/core/server/elasticsearch/client/cluster_client.ts @@ -60,10 +60,11 @@ export class ClusterClient implements ICustomClusterClient { constructor( private readonly config: ElasticsearchClientConfig, logger: Logger, + type: string, private readonly getAuthHeaders: GetAuthHeaders = noop ) { - this.asInternalUser = configureClient(config, { logger }); - this.rootScopedClient = configureClient(config, { logger, scoped: true }); + this.asInternalUser = configureClient(config, { logger, type }); + this.rootScopedClient = configureClient(config, { logger, type, scoped: true }); } asScoped(request: ScopeableRequest) { diff --git a/src/core/server/elasticsearch/client/configure_client.test.ts b/src/core/server/elasticsearch/client/configure_client.test.ts index 3486c210de1f9..548dc44aa4965 100644 --- a/src/core/server/elasticsearch/client/configure_client.test.ts +++ b/src/core/server/elasticsearch/client/configure_client.test.ts @@ -76,14 +76,14 @@ describe('configureClient', () => { }); it('calls `parseClientOptions` with the correct parameters', () => { - configureClient(config, { logger, scoped: false }); + configureClient(config, { logger, type: 'test', scoped: false }); expect(parseClientOptionsMock).toHaveBeenCalledTimes(1); expect(parseClientOptionsMock).toHaveBeenCalledWith(config, false); parseClientOptionsMock.mockClear(); - configureClient(config, { logger, scoped: true }); + configureClient(config, { logger, type: 'test', scoped: true }); expect(parseClientOptionsMock).toHaveBeenCalledTimes(1); expect(parseClientOptionsMock).toHaveBeenCalledWith(config, true); @@ -95,7 +95,7 @@ describe('configureClient', () => { }; parseClientOptionsMock.mockReturnValue(parsedOptions); - const client = configureClient(config, { logger, scoped: false }); + const client = configureClient(config, { logger, type: 'test', scoped: false }); expect(ClientMock).toHaveBeenCalledTimes(1); expect(ClientMock).toHaveBeenCalledWith(parsedOptions); @@ -103,7 +103,7 @@ describe('configureClient', () => { }); it('listens to client on `response` events', () => { - const client = configureClient(config, { logger, scoped: false }); + const client = configureClient(config, { logger, type: 'test', scoped: false }); expect(client.on).toHaveBeenCalledTimes(1); expect(client.on).toHaveBeenCalledWith('response', expect.any(Function)); @@ -122,38 +122,15 @@ describe('configureClient', () => { }, }); } - describe('does not log whrn "logQueries: false"', () => { - it('response', () => { - const client = configureClient(config, { logger, scoped: false }); - const response = createResponseWithBody({ - seq_no_primary_term: true, - query: { - term: { user: 'kimchy' }, - }, - }); - - client.emit('response', null, response); - expect(loggingSystemMock.collect(logger).debug).toHaveLength(0); - }); - - it('error', () => { - const client = configureClient(config, { logger, scoped: false }); - - const response = createApiResponse({ body: {} }); - client.emit('response', new errors.TimeoutError('message', response), response); - expect(loggingSystemMock.collect(logger).error).toHaveLength(0); + describe('logs each query', () => { + it('creates a query logger context based on the `type` parameter', () => { + configureClient(createFakeConfig(), { logger, type: 'test123' }); + expect(logger.get).toHaveBeenCalledWith('query', 'test123'); }); - }); - describe('logs each queries if `logQueries` is true', () => { it('when request body is an object', () => { - const client = configureClient( - createFakeConfig({ - logQueries: true, - }), - { logger, scoped: false } - ); + const client = configureClient(createFakeConfig(), { logger, type: 'test', scoped: false }); const response = createResponseWithBody({ seq_no_primary_term: true, @@ -169,23 +146,13 @@ describe('configureClient', () => { "200 GET /foo?hello=dolly {\\"seq_no_primary_term\\":true,\\"query\\":{\\"term\\":{\\"user\\":\\"kimchy\\"}}}", - Object { - "tags": Array [ - "query", - ], - }, ], ] `); }); it('when request body is a string', () => { - const client = configureClient( - createFakeConfig({ - logQueries: true, - }), - { logger, scoped: false } - ); + const client = configureClient(createFakeConfig(), { logger, type: 'test', scoped: false }); const response = createResponseWithBody( JSON.stringify({ @@ -203,23 +170,13 @@ describe('configureClient', () => { "200 GET /foo?hello=dolly {\\"seq_no_primary_term\\":true,\\"query\\":{\\"term\\":{\\"user\\":\\"kimchy\\"}}}", - Object { - "tags": Array [ - "query", - ], - }, ], ] `); }); it('when request body is a buffer', () => { - const client = configureClient( - createFakeConfig({ - logQueries: true, - }), - { logger, scoped: false } - ); + const client = configureClient(createFakeConfig(), { logger, type: 'test', scoped: false }); const response = createResponseWithBody( Buffer.from( @@ -239,23 +196,13 @@ describe('configureClient', () => { "200 GET /foo?hello=dolly [buffer]", - Object { - "tags": Array [ - "query", - ], - }, ], ] `); }); it('when request body is a readable stream', () => { - const client = configureClient( - createFakeConfig({ - logQueries: true, - }), - { logger, scoped: false } - ); + const client = configureClient(createFakeConfig(), { logger, type: 'test', scoped: false }); const response = createResponseWithBody( Readable.from( @@ -275,23 +222,13 @@ describe('configureClient', () => { "200 GET /foo?hello=dolly [stream]", - Object { - "tags": Array [ - "query", - ], - }, ], ] `); }); it('when request body is not defined', () => { - const client = configureClient( - createFakeConfig({ - logQueries: true, - }), - { logger, scoped: false } - ); + const client = configureClient(createFakeConfig(), { logger, type: 'test', scoped: false }); const response = createResponseWithBody(); @@ -301,23 +238,13 @@ describe('configureClient', () => { Array [ "200 GET /foo?hello=dolly", - Object { - "tags": Array [ - "query", - ], - }, ], ] `); }); it('properly encode queries', () => { - const client = configureClient( - createFakeConfig({ - logQueries: true, - }), - { logger, scoped: false } - ); + const client = configureClient(createFakeConfig(), { logger, type: 'test', scoped: false }); const response = createApiResponse({ body: {}, @@ -336,23 +263,13 @@ describe('configureClient', () => { Array [ "200 GET /foo?city=M%C3%BCnich", - Object { - "tags": Array [ - "query", - ], - }, ], ] `); }); - it('logs queries even in case of errors if `logQueries` is true', () => { - const client = configureClient( - createFakeConfig({ - logQueries: true, - }), - { logger, scoped: false } - ); + it('logs queries even in case of errors', () => { + const client = configureClient(createFakeConfig(), { logger, type: 'test', scoped: false }); const response = createApiResponse({ statusCode: 500, @@ -375,7 +292,7 @@ describe('configureClient', () => { }); client.emit('response', new errors.ResponseError(response), response); - expect(loggingSystemMock.collect(logger).error).toMatchInlineSnapshot(` + expect(loggingSystemMock.collect(logger).debug).toMatchInlineSnapshot(` Array [ Array [ "500 @@ -386,40 +303,13 @@ describe('configureClient', () => { `); }); - it('does not log queries if `logQueries` is false', () => { - const client = configureClient( - createFakeConfig({ - logQueries: false, - }), - { logger, scoped: false } - ); - - const response = createApiResponse({ - body: {}, - statusCode: 200, - params: { - method: 'GET', - path: '/foo', - }, - }); - - client.emit('response', null, response); - - expect(logger.debug).not.toHaveBeenCalled(); - }); - - it('logs error when the client emits an @elastic/elasticsearch error', () => { - const client = configureClient( - createFakeConfig({ - logQueries: true, - }), - { logger, scoped: false } - ); + it('logs debug when the client emits an @elastic/elasticsearch error', () => { + const client = configureClient(createFakeConfig(), { logger, type: 'test', scoped: false }); const response = createApiResponse({ body: {} }); client.emit('response', new errors.TimeoutError('message', response), response); - expect(loggingSystemMock.collect(logger).error).toMatchInlineSnapshot(` + expect(loggingSystemMock.collect(logger).debug).toMatchInlineSnapshot(` Array [ Array [ "[TimeoutError]: message", @@ -428,13 +318,8 @@ describe('configureClient', () => { `); }); - it('logs error when the client emits an ResponseError returned by elasticsearch', () => { - const client = configureClient( - createFakeConfig({ - logQueries: true, - }), - { logger, scoped: false } - ); + it('logs debug when the client emits an ResponseError returned by elasticsearch', () => { + const client = configureClient(createFakeConfig(), { logger, type: 'test', scoped: false }); const response = createApiResponse({ statusCode: 400, @@ -453,7 +338,7 @@ describe('configureClient', () => { }); client.emit('response', new errors.ResponseError(response), response); - expect(loggingSystemMock.collect(logger).error).toMatchInlineSnapshot(` + expect(loggingSystemMock.collect(logger).debug).toMatchInlineSnapshot(` Array [ Array [ "400 @@ -464,12 +349,7 @@ describe('configureClient', () => { }); it('logs default error info when the error response body is empty', () => { - const client = configureClient( - createFakeConfig({ - logQueries: true, - }), - { logger, scoped: false } - ); + const client = configureClient(createFakeConfig(), { logger, type: 'test', scoped: false }); let response = createApiResponse({ statusCode: 400, @@ -484,7 +364,7 @@ describe('configureClient', () => { }); client.emit('response', new errors.ResponseError(response), response); - expect(loggingSystemMock.collect(logger).error).toMatchInlineSnapshot(` + expect(loggingSystemMock.collect(logger).debug).toMatchInlineSnapshot(` Array [ Array [ "400 @@ -493,7 +373,7 @@ describe('configureClient', () => { ] `); - logger.error.mockClear(); + logger.debug.mockClear(); response = createApiResponse({ statusCode: 400, @@ -506,7 +386,7 @@ describe('configureClient', () => { }); client.emit('response', new errors.ResponseError(response), response); - expect(loggingSystemMock.collect(logger).error).toMatchInlineSnapshot(` + expect(loggingSystemMock.collect(logger).debug).toMatchInlineSnapshot(` Array [ Array [ "400 diff --git a/src/core/server/elasticsearch/client/configure_client.ts b/src/core/server/elasticsearch/client/configure_client.ts index 00cbd1958d817..bac792d1293a6 100644 --- a/src/core/server/elasticsearch/client/configure_client.ts +++ b/src/core/server/elasticsearch/client/configure_client.ts @@ -15,12 +15,12 @@ import { parseClientOptions, ElasticsearchClientConfig } from './client_config'; export const configureClient = ( config: ElasticsearchClientConfig, - { logger, scoped = false }: { logger: Logger; scoped?: boolean } + { logger, type, scoped = false }: { logger: Logger; type: string; scoped?: boolean } ): Client => { const clientOptions = parseClientOptions(config, scoped); const client = new Client(clientOptions); - addLogging(client, logger, config.logQueries); + addLogging(client, logger.get('query', type)); return client; }; @@ -67,15 +67,13 @@ function getResponseMessage(event: RequestEvent): string { return `${event.statusCode}\n${params.method} ${url}${body}`; } -const addLogging = (client: Client, logger: Logger, logQueries: boolean) => { +const addLogging = (client: Client, logger: Logger) => { client.on('response', (error, event) => { - if (event && logQueries) { + if (event) { if (error) { - logger.error(getErrorMessage(error, event)); + logger.debug(getErrorMessage(error, event)); } else { - logger.debug(getResponseMessage(event), { - tags: ['query'], - }); + logger.debug(getResponseMessage(event)); } } }); diff --git a/src/core/server/elasticsearch/elasticsearch_config.test.ts b/src/core/server/elasticsearch/elasticsearch_config.test.ts index 803733fddb71c..e76de913a9d91 100644 --- a/src/core/server/elasticsearch/elasticsearch_config.test.ts +++ b/src/core/server/elasticsearch/elasticsearch_config.test.ts @@ -47,7 +47,6 @@ test('set correct defaults', () => { "http://localhost:9200", ], "ignoreVersionMismatch": false, - "logQueries": false, "password": undefined, "pingTimeout": "PT30S", "requestHeadersWhitelist": Array [ diff --git a/src/core/server/elasticsearch/elasticsearch_config.ts b/src/core/server/elasticsearch/elasticsearch_config.ts index b90ae2609f1e3..afda47ca8851b 100644 --- a/src/core/server/elasticsearch/elasticsearch_config.ts +++ b/src/core/server/elasticsearch/elasticsearch_config.ts @@ -133,6 +133,10 @@ const deprecations: ConfigDeprecationProvider = () => [ log( `Setting [${fromPath}.ssl.certificate] without [${fromPath}.ssl.key] is deprecated. This has no effect, you should use both settings to enable TLS client authentication to Elasticsearch.` ); + } else if (es.logQueries === true) { + log( + `Setting [${fromPath}.logQueries] is deprecated and no longer used. You should set the log level to "debug" for the "elasticsearch.queries" context in "logging.loggers" or use "logging.verbose: true".` + ); } return settings; }, @@ -164,12 +168,6 @@ export class ElasticsearchConfig { */ public readonly apiVersion: string; - /** - * Specifies whether all queries to the client should be logged (status code, - * method, query etc.). - */ - public readonly logQueries: boolean; - /** * Hosts that the client will connect to. If sniffing is enabled, this list will * be used as seeds to discover the rest of your cluster. @@ -248,7 +246,6 @@ export class ElasticsearchConfig { constructor(rawConfig: ElasticsearchConfigType) { this.ignoreVersionMismatch = rawConfig.ignoreVersionMismatch; this.apiVersion = rawConfig.apiVersion; - this.logQueries = rawConfig.logQueries; this.hosts = Array.isArray(rawConfig.hosts) ? rawConfig.hosts : [rawConfig.hosts]; this.requestHeadersWhitelist = Array.isArray(rawConfig.requestHeadersWhitelist) ? rawConfig.requestHeadersWhitelist diff --git a/src/core/server/elasticsearch/elasticsearch_service.test.ts b/src/core/server/elasticsearch/elasticsearch_service.test.ts index a6d966b346072..3129ccfb5a67e 100644 --- a/src/core/server/elasticsearch/elasticsearch_service.test.ts +++ b/src/core/server/elasticsearch/elasticsearch_service.test.ts @@ -92,14 +92,15 @@ describe('#setup', () => { // reset all mocks called during setup phase MockLegacyClusterClient.mockClear(); - const customConfig = { logQueries: true }; + const customConfig = { keepAlive: true }; const clusterClient = setupContract.legacy.createClient('some-custom-type', customConfig); expect(clusterClient).toBe(mockLegacyClusterClientInstance); expect(MockLegacyClusterClient).toHaveBeenCalledWith( expect.objectContaining(customConfig), - expect.objectContaining({ context: ['elasticsearch', 'some-custom-type'] }), + expect.objectContaining({ context: ['elasticsearch'] }), + 'some-custom-type', expect.any(Function) ); }); @@ -267,7 +268,7 @@ describe('#start', () => { // reset all mocks called during setup phase MockClusterClient.mockClear(); - const customConfig = { logQueries: true }; + const customConfig = { keepAlive: true }; const clusterClient = startContract.createClient('custom-type', customConfig); expect(clusterClient).toBe(mockClusterClientInstance); @@ -275,7 +276,8 @@ describe('#start', () => { expect(MockClusterClient).toHaveBeenCalledTimes(1); expect(MockClusterClient).toHaveBeenCalledWith( expect.objectContaining(customConfig), - expect.objectContaining({ context: ['elasticsearch', 'custom-type'] }), + expect.objectContaining({ context: ['elasticsearch'] }), + 'custom-type', expect.any(Function) ); }); @@ -286,7 +288,7 @@ describe('#start', () => { // reset all mocks called during setup phase MockClusterClient.mockClear(); - const customConfig = { logQueries: true }; + const customConfig = { keepAlive: true }; startContract.createClient('custom-type', customConfig); startContract.createClient('another-type', customConfig); diff --git a/src/core/server/elasticsearch/elasticsearch_service.ts b/src/core/server/elasticsearch/elasticsearch_service.ts index 2d97f6e5c3121..fd3d546bb77b9 100644 --- a/src/core/server/elasticsearch/elasticsearch_service.ts +++ b/src/core/server/elasticsearch/elasticsearch_service.ts @@ -126,7 +126,8 @@ export class ElasticsearchService private createClusterClient(type: string, config: ElasticsearchClientConfig) { return new ClusterClient( config, - this.coreContext.logger.get('elasticsearch', type), + this.coreContext.logger.get('elasticsearch'), + type, this.getAuthHeaders ); } @@ -134,7 +135,8 @@ export class ElasticsearchService private createLegacyClusterClient(type: string, config: LegacyElasticsearchClientConfig) { return new LegacyClusterClient( config, - this.coreContext.logger.get('elasticsearch', type), + this.coreContext.logger.get('elasticsearch'), + type, this.getAuthHeaders ); } diff --git a/src/core/server/elasticsearch/legacy/cluster_client.test.ts b/src/core/server/elasticsearch/legacy/cluster_client.test.ts index 97a49cd9eb9f4..177181608bee9 100644 --- a/src/core/server/elasticsearch/legacy/cluster_client.test.ts +++ b/src/core/server/elasticsearch/legacy/cluster_client.test.ts @@ -31,11 +31,15 @@ test('#constructor creates client with parsed config', () => { const mockEsConfig = { apiVersion: 'es-version' } as any; const mockLogger = logger.get(); - const clusterClient = new LegacyClusterClient(mockEsConfig, mockLogger); + const clusterClient = new LegacyClusterClient(mockEsConfig, mockLogger, 'custom-type'); expect(clusterClient).toBeDefined(); expect(mockParseElasticsearchClientConfig).toHaveBeenCalledTimes(1); - expect(mockParseElasticsearchClientConfig).toHaveBeenLastCalledWith(mockEsConfig, mockLogger); + expect(mockParseElasticsearchClientConfig).toHaveBeenLastCalledWith( + mockEsConfig, + mockLogger, + 'custom-type' + ); expect(MockClient).toHaveBeenCalledTimes(1); expect(MockClient).toHaveBeenCalledWith(mockEsClientConfig); @@ -57,7 +61,11 @@ describe('#callAsInternalUser', () => { }; MockClient.mockImplementation(() => mockEsClientInstance); - clusterClient = new LegacyClusterClient({ apiVersion: 'es-version' } as any, logger.get()); + clusterClient = new LegacyClusterClient( + { apiVersion: 'es-version' } as any, + logger.get(), + 'custom-type' + ); }); test('fails if cluster client is closed', async () => { @@ -226,7 +234,7 @@ describe('#asScoped', () => { requestHeadersWhitelist: ['one', 'two'], } as any; - clusterClient = new LegacyClusterClient(mockEsConfig, mockLogger); + clusterClient = new LegacyClusterClient(mockEsConfig, mockLogger, 'custom-type'); jest.clearAllMocks(); }); @@ -237,10 +245,15 @@ describe('#asScoped', () => { expect(firstScopedClusterClient).toBeDefined(); expect(mockParseElasticsearchClientConfig).toHaveBeenCalledTimes(1); - expect(mockParseElasticsearchClientConfig).toHaveBeenLastCalledWith(mockEsConfig, mockLogger, { - auth: false, - ignoreCertAndKey: true, - }); + expect(mockParseElasticsearchClientConfig).toHaveBeenLastCalledWith( + mockEsConfig, + mockLogger, + 'custom-type', + { + auth: false, + ignoreCertAndKey: true, + } + ); expect(MockClient).toHaveBeenCalledTimes(1); expect(MockClient).toHaveBeenCalledWith( @@ -261,42 +274,57 @@ describe('#asScoped', () => { test('properly configures `ignoreCertAndKey` for various configurations', () => { // Config without SSL. - clusterClient = new LegacyClusterClient(mockEsConfig, mockLogger); + clusterClient = new LegacyClusterClient(mockEsConfig, mockLogger, 'custom-type'); mockParseElasticsearchClientConfig.mockClear(); clusterClient.asScoped(httpServerMock.createRawRequest({ headers: { one: '1' } })); expect(mockParseElasticsearchClientConfig).toHaveBeenCalledTimes(1); - expect(mockParseElasticsearchClientConfig).toHaveBeenLastCalledWith(mockEsConfig, mockLogger, { - auth: false, - ignoreCertAndKey: true, - }); + expect(mockParseElasticsearchClientConfig).toHaveBeenLastCalledWith( + mockEsConfig, + mockLogger, + 'custom-type', + { + auth: false, + ignoreCertAndKey: true, + } + ); // Config ssl.alwaysPresentCertificate === false mockEsConfig = { ...mockEsConfig, ssl: { alwaysPresentCertificate: false } } as any; - clusterClient = new LegacyClusterClient(mockEsConfig, mockLogger); + clusterClient = new LegacyClusterClient(mockEsConfig, mockLogger, 'custom-type'); mockParseElasticsearchClientConfig.mockClear(); clusterClient.asScoped(httpServerMock.createRawRequest({ headers: { one: '1' } })); expect(mockParseElasticsearchClientConfig).toHaveBeenCalledTimes(1); - expect(mockParseElasticsearchClientConfig).toHaveBeenLastCalledWith(mockEsConfig, mockLogger, { - auth: false, - ignoreCertAndKey: true, - }); + expect(mockParseElasticsearchClientConfig).toHaveBeenLastCalledWith( + mockEsConfig, + mockLogger, + 'custom-type', + { + auth: false, + ignoreCertAndKey: true, + } + ); // Config ssl.alwaysPresentCertificate === true mockEsConfig = { ...mockEsConfig, ssl: { alwaysPresentCertificate: true } } as any; - clusterClient = new LegacyClusterClient(mockEsConfig, mockLogger); + clusterClient = new LegacyClusterClient(mockEsConfig, mockLogger, 'custom-type'); mockParseElasticsearchClientConfig.mockClear(); clusterClient.asScoped(httpServerMock.createRawRequest({ headers: { one: '1' } })); expect(mockParseElasticsearchClientConfig).toHaveBeenCalledTimes(1); - expect(mockParseElasticsearchClientConfig).toHaveBeenLastCalledWith(mockEsConfig, mockLogger, { - auth: false, - ignoreCertAndKey: false, - }); + expect(mockParseElasticsearchClientConfig).toHaveBeenLastCalledWith( + mockEsConfig, + mockLogger, + 'custom-type', + { + auth: false, + ignoreCertAndKey: false, + } + ); }); test('passes only filtered headers to the scoped cluster client', () => { @@ -345,7 +373,7 @@ describe('#asScoped', () => { }); test('does not fail when scope to not defined request', async () => { - clusterClient = new LegacyClusterClient(mockEsConfig, mockLogger); + clusterClient = new LegacyClusterClient(mockEsConfig, mockLogger, 'custom-type'); clusterClient.asScoped(); expect(MockScopedClusterClient).toHaveBeenCalledTimes(1); expect(MockScopedClusterClient).toHaveBeenCalledWith( @@ -356,7 +384,7 @@ describe('#asScoped', () => { }); test('does not fail when scope to a request without headers', async () => { - clusterClient = new LegacyClusterClient(mockEsConfig, mockLogger); + clusterClient = new LegacyClusterClient(mockEsConfig, mockLogger, 'custom-type'); clusterClient.asScoped({} as any); expect(MockScopedClusterClient).toHaveBeenCalledTimes(1); expect(MockScopedClusterClient).toHaveBeenCalledWith( @@ -367,7 +395,7 @@ describe('#asScoped', () => { }); test('calls getAuthHeaders and filters results for a real request', async () => { - clusterClient = new LegacyClusterClient(mockEsConfig, mockLogger, () => ({ + clusterClient = new LegacyClusterClient(mockEsConfig, mockLogger, 'custom-type', () => ({ one: '1', three: '3', })); @@ -381,7 +409,9 @@ describe('#asScoped', () => { }); test('getAuthHeaders results rewrite extends a request headers', async () => { - clusterClient = new LegacyClusterClient(mockEsConfig, mockLogger, () => ({ one: 'foo' })); + clusterClient = new LegacyClusterClient(mockEsConfig, mockLogger, 'custom-type', () => ({ + one: 'foo', + })); clusterClient.asScoped(httpServerMock.createRawRequest({ headers: { one: '1', two: '2' } })); expect(MockScopedClusterClient).toHaveBeenCalledTimes(1); expect(MockScopedClusterClient).toHaveBeenCalledWith( @@ -392,7 +422,7 @@ describe('#asScoped', () => { }); test("doesn't call getAuthHeaders for a fake request", async () => { - clusterClient = new LegacyClusterClient(mockEsConfig, mockLogger, () => ({})); + clusterClient = new LegacyClusterClient(mockEsConfig, mockLogger, 'custom-type', () => ({})); clusterClient.asScoped({ headers: { one: 'foo' } }); expect(MockScopedClusterClient).toHaveBeenCalledTimes(1); @@ -404,7 +434,7 @@ describe('#asScoped', () => { }); test('filters a fake request headers', async () => { - clusterClient = new LegacyClusterClient(mockEsConfig, mockLogger); + clusterClient = new LegacyClusterClient(mockEsConfig, mockLogger, 'custom-type'); clusterClient.asScoped({ headers: { one: '1', two: '2', three: '3' } }); expect(MockScopedClusterClient).toHaveBeenCalledTimes(1); @@ -431,7 +461,8 @@ describe('#close', () => { clusterClient = new LegacyClusterClient( { apiVersion: 'es-version', requestHeadersWhitelist: [] } as any, - logger.get() + logger.get(), + 'custom-type' ); }); diff --git a/src/core/server/elasticsearch/legacy/cluster_client.ts b/src/core/server/elasticsearch/legacy/cluster_client.ts index 9cac713920331..64e1382fee201 100644 --- a/src/core/server/elasticsearch/legacy/cluster_client.ts +++ b/src/core/server/elasticsearch/legacy/cluster_client.ts @@ -121,9 +121,10 @@ export class LegacyClusterClient implements ILegacyClusterClient { constructor( private readonly config: LegacyElasticsearchClientConfig, private readonly log: Logger, + private readonly type: string, private readonly getAuthHeaders: GetAuthHeaders = noop ) { - this.client = new Client(parseElasticsearchClientConfig(config, log)); + this.client = new Client(parseElasticsearchClientConfig(config, log, type)); } /** @@ -186,7 +187,7 @@ export class LegacyClusterClient implements ILegacyClusterClient { // between all scoped client instances. if (this.scopedClient === undefined) { this.scopedClient = new Client( - parseElasticsearchClientConfig(this.config, this.log, { + parseElasticsearchClientConfig(this.config, this.log, this.type, { auth: false, ignoreCertAndKey: !this.config.ssl || !this.config.ssl.alwaysPresentCertificate, }) diff --git a/src/core/server/elasticsearch/legacy/elasticsearch_client_config.test.ts b/src/core/server/elasticsearch/legacy/elasticsearch_client_config.test.ts index 5dac353c1094c..6c79f2c568caa 100644 --- a/src/core/server/elasticsearch/legacy/elasticsearch_client_config.test.ts +++ b/src/core/server/elasticsearch/legacy/elasticsearch_client_config.test.ts @@ -22,13 +22,13 @@ test('parses minimally specified config', () => { { apiVersion: 'master', customHeaders: { xsrf: 'something' }, - logQueries: false, sniffOnStart: false, sniffOnConnectionFault: false, hosts: ['http://localhost/elasticsearch'], requestHeadersWhitelist: [], }, - logger.get() + logger.get(), + 'custom-type' ) ).toMatchInlineSnapshot(` Object { @@ -58,7 +58,6 @@ test('parses fully specified config', () => { const elasticsearchConfig: LegacyElasticsearchClientConfig = { apiVersion: 'v7.0.0', customHeaders: { xsrf: 'something' }, - logQueries: true, sniffOnStart: true, sniffOnConnectionFault: true, hosts: [ @@ -84,7 +83,8 @@ test('parses fully specified config', () => { const elasticsearchClientConfig = parseElasticsearchClientConfig( elasticsearchConfig, - logger.get() + logger.get(), + 'custom-type' ); // Check that original references aren't used. @@ -163,7 +163,6 @@ test('parses config timeouts of moment.Duration type', () => { { apiVersion: 'master', customHeaders: { xsrf: 'something' }, - logQueries: false, sniffOnStart: false, sniffOnConnectionFault: false, pingTimeout: duration(100, 'ms'), @@ -172,7 +171,8 @@ test('parses config timeouts of moment.Duration type', () => { hosts: ['http://localhost:9200/elasticsearch'], requestHeadersWhitelist: [], }, - logger.get() + logger.get(), + 'custom-type' ) ).toMatchInlineSnapshot(` Object { @@ -208,7 +208,6 @@ describe('#auth', () => { { apiVersion: 'v7.0.0', customHeaders: { xsrf: 'something' }, - logQueries: true, sniffOnStart: true, sniffOnConnectionFault: true, hosts: ['http://user:password@localhost/elasticsearch', 'https://es.local'], @@ -217,6 +216,7 @@ describe('#auth', () => { requestHeadersWhitelist: [], }, logger.get(), + 'custom-type', { auth: false } ) ).toMatchInlineSnapshot(` @@ -260,7 +260,6 @@ describe('#auth', () => { { apiVersion: 'v7.0.0', customHeaders: { xsrf: 'something' }, - logQueries: true, sniffOnStart: true, sniffOnConnectionFault: true, hosts: ['https://es.local'], @@ -268,6 +267,7 @@ describe('#auth', () => { password: 'changeme', }, logger.get(), + 'custom-type', { auth: true } ) ).toMatchInlineSnapshot(` @@ -300,7 +300,6 @@ describe('#auth', () => { { apiVersion: 'v7.0.0', customHeaders: { xsrf: 'something' }, - logQueries: true, sniffOnStart: true, sniffOnConnectionFault: true, hosts: ['https://es.local'], @@ -308,6 +307,7 @@ describe('#auth', () => { username: 'elastic', }, logger.get(), + 'custom-type', { auth: true } ) ).toMatchInlineSnapshot(` @@ -342,13 +342,13 @@ describe('#customHeaders', () => { { apiVersion: 'master', customHeaders: { [headerKey]: 'foo' }, - logQueries: false, sniffOnStart: false, sniffOnConnectionFault: false, hosts: ['http://localhost/elasticsearch'], requestHeadersWhitelist: [], }, - logger.get() + logger.get(), + 'custom-type' ); expect(parsedConfig.hosts[0].headers).toEqual({ [headerKey]: 'foo', @@ -357,62 +357,18 @@ describe('#customHeaders', () => { }); describe('#log', () => { - test('default logger with #logQueries = false', () => { + test('default logger', () => { const parsedConfig = parseElasticsearchClientConfig( { apiVersion: 'master', customHeaders: { xsrf: 'something' }, - logQueries: false, sniffOnStart: false, sniffOnConnectionFault: false, hosts: ['http://localhost/elasticsearch'], requestHeadersWhitelist: [], }, - logger.get() - ); - - const esLogger = new parsedConfig.log(); - esLogger.error('some-error'); - esLogger.warning('some-warning'); - esLogger.trace('some-trace'); - esLogger.info('some-info'); - esLogger.debug('some-debug'); - - expect(typeof esLogger.close).toBe('function'); - - expect(loggingSystemMock.collect(logger)).toMatchInlineSnapshot(` - Object { - "debug": Array [], - "error": Array [ - Array [ - "some-error", - ], - ], - "fatal": Array [], - "info": Array [], - "log": Array [], - "trace": Array [], - "warn": Array [ - Array [ - "some-warning", - ], - ], - } - `); - }); - - test('default logger with #logQueries = true', () => { - const parsedConfig = parseElasticsearchClientConfig( - { - apiVersion: 'master', - customHeaders: { xsrf: 'something' }, - logQueries: true, - sniffOnStart: false, - sniffOnConnectionFault: false, - hosts: ['http://localhost/elasticsearch'], - requestHeadersWhitelist: [], - }, - logger.get() + logger.get(), + 'custom-type' ); const esLogger = new parsedConfig.log(); @@ -433,11 +389,6 @@ describe('#log', () => { "304 METHOD /some-path ?query=2", - Object { - "tags": Array [ - "query", - ], - }, ], ], "error": Array [ @@ -465,14 +416,14 @@ describe('#log', () => { { apiVersion: 'master', customHeaders: { xsrf: 'something' }, - logQueries: true, sniffOnStart: false, sniffOnConnectionFault: false, hosts: ['http://localhost/elasticsearch'], requestHeadersWhitelist: [], log: customLogger, }, - logger.get() + logger.get(), + 'custom-type' ); expect(parsedConfig.log).toBe(customLogger); @@ -486,14 +437,14 @@ describe('#ssl', () => { { apiVersion: 'v7.0.0', customHeaders: {}, - logQueries: true, sniffOnStart: true, sniffOnConnectionFault: true, hosts: ['https://es.local'], requestHeadersWhitelist: [], ssl: { verificationMode: 'none' }, }, - logger.get() + logger.get(), + 'custom-type' ) ).toMatchInlineSnapshot(` Object { @@ -527,14 +478,14 @@ describe('#ssl', () => { { apiVersion: 'v7.0.0', customHeaders: {}, - logQueries: true, sniffOnStart: true, sniffOnConnectionFault: true, hosts: ['https://es.local'], requestHeadersWhitelist: [], ssl: { verificationMode: 'certificate' }, }, - logger.get() + logger.get(), + 'custom-type' ); // `checkServerIdentity` shouldn't check hostname when verificationMode is certificate. @@ -576,14 +527,14 @@ describe('#ssl', () => { { apiVersion: 'v7.0.0', customHeaders: {}, - logQueries: true, sniffOnStart: true, sniffOnConnectionFault: true, hosts: ['https://es.local'], requestHeadersWhitelist: [], ssl: { verificationMode: 'full' }, }, - logger.get() + logger.get(), + 'custom-type' ) ).toMatchInlineSnapshot(` Object { @@ -618,14 +569,14 @@ describe('#ssl', () => { { apiVersion: 'v7.0.0', customHeaders: {}, - logQueries: true, sniffOnStart: true, sniffOnConnectionFault: true, hosts: ['https://es.local'], requestHeadersWhitelist: [], ssl: { verificationMode: 'misspelled' as any }, }, - logger.get() + logger.get(), + 'custom-type' ) ).toThrowErrorMatchingInlineSnapshot(`"Unknown ssl verificationMode: misspelled"`); }); @@ -636,7 +587,6 @@ describe('#ssl', () => { { apiVersion: 'v7.0.0', customHeaders: {}, - logQueries: true, sniffOnStart: true, sniffOnConnectionFault: true, hosts: ['https://es.local'], @@ -651,6 +601,7 @@ describe('#ssl', () => { }, }, logger.get(), + 'custom-type', { ignoreCertAndKey: true } ) ).toMatchInlineSnapshot(` diff --git a/src/core/server/elasticsearch/legacy/elasticsearch_client_config.ts b/src/core/server/elasticsearch/legacy/elasticsearch_client_config.ts index ecd2e30097060..66b6046e4516d 100644 --- a/src/core/server/elasticsearch/legacy/elasticsearch_client_config.ts +++ b/src/core/server/elasticsearch/legacy/elasticsearch_client_config.ts @@ -29,7 +29,6 @@ export type LegacyElasticsearchClientConfig = Pick { + return { + // Create a mock for spying on the constructor + DocumentMigrator: jest.fn().mockImplementation((...args) => { + const { DocumentMigrator: RealDocMigrator } = jest.requireActual('../core/document_migrator'); + return new RealDocMigrator(args[0]); + }), + }; +}); const createRegistry = (types: Array>) => { const registry = new SavedObjectTypeRegistry(); @@ -31,12 +41,16 @@ const createRegistry = (types: Array>) => { }; describe('KibanaMigrator', () => { + beforeEach(() => { + (DocumentMigrator as jest.Mock).mockClear(); + }); describe('constructor', () => { it('coerces the current Kibana version if it has a hyphen', () => { const options = mockOptions(); options.kibanaVersion = '3.2.1-SNAPSHOT'; const migrator = new KibanaMigrator(options); expect(migrator.kibanaVersion).toEqual('3.2.1'); + expect((DocumentMigrator as jest.Mock).mock.calls[0][0].kibanaVersion).toEqual('3.2.1'); }); }); describe('getActiveMappings', () => { @@ -105,8 +119,8 @@ describe('KibanaMigrator', () => { const migrator = new KibanaMigrator(options); - expect(() => migrator.runMigrations()).rejects.toThrow( - /Migrations are not ready. Make sure prepareMigrations is called first./i + await expect(() => migrator.runMigrations()).toThrowErrorMatchingInlineSnapshot( + `"Migrations are not ready. Make sure prepareMigrations is called first."` ); }); diff --git a/src/core/server/saved_objects/migrations/kibana/kibana_migrator.ts b/src/core/server/saved_objects/migrations/kibana/kibana_migrator.ts index ecef84a6e297c..1a4611b491419 100644 --- a/src/core/server/saved_objects/migrations/kibana/kibana_migrator.ts +++ b/src/core/server/saved_objects/migrations/kibana/kibana_migrator.ts @@ -12,6 +12,7 @@ */ import { BehaviorSubject } from 'rxjs'; +import Semver from 'semver'; import { KibanaConfigType } from '../../../kibana_config'; import { ElasticsearchClient } from '../../../elasticsearch'; import { Logger } from '../../../logging'; @@ -97,7 +98,7 @@ export class KibanaMigrator { this.log = logger; this.kibanaVersion = kibanaVersion.split('-')[0]; // coerce a semver-like string (x.y.z-SNAPSHOT) or prerelease version (x.y.z-alpha) to a regular semver (x.y.z); this.documentMigrator = new DocumentMigrator({ - kibanaVersion, + kibanaVersion: this.kibanaVersion, typeRegistry, log: this.log, }); @@ -163,6 +164,15 @@ export class KibanaMigrator { registry: this.typeRegistry, }); + this.log.debug('Applying registered migrations for the following saved object types:'); + Object.entries(this.documentMigrator.migrationVersion) + .sort(([t1, v1], [t2, v2]) => { + return Semver.compare(v1, v2); + }) + .forEach(([type, migrationVersion]) => { + this.log.debug(`migrationVersion: ${migrationVersion} saved object type: ${type}`); + }); + const migrators = Object.keys(indexMap).map((index) => { // TODO migrationsV2: remove old migrations algorithm if (this.savedObjectsConfig.enableV2) { diff --git a/src/core/server/server.api.md b/src/core/server/server.api.md index aadd16bde0ee6..40a12290be31b 100644 --- a/src/core/server/server.api.md +++ b/src/core/server/server.api.md @@ -843,7 +843,7 @@ export type ElasticsearchClient = Omit & { +export type ElasticsearchClientConfig = Pick & { pingTimeout?: ElasticsearchConfig['pingTimeout'] | ClientOptions['pingTimeout']; requestTimeout?: ElasticsearchConfig['requestTimeout'] | ClientOptions['requestTimeout']; ssl?: Partial; @@ -859,7 +859,6 @@ export class ElasticsearchConfig { readonly healthCheckDelay: Duration; readonly hosts: string[]; readonly ignoreVersionMismatch: boolean; - readonly logQueries: boolean; readonly password?: string; readonly pingTimeout: Duration; readonly requestHeadersWhitelist: string[]; @@ -1531,7 +1530,7 @@ export interface LegacyCallAPIOptions { // @public @deprecated export class LegacyClusterClient implements ILegacyClusterClient { - constructor(config: LegacyElasticsearchClientConfig, log: Logger, getAuthHeaders?: GetAuthHeaders); + constructor(config: LegacyElasticsearchClientConfig, log: Logger, type: string, getAuthHeaders?: GetAuthHeaders); asScoped(request?: ScopeableRequest): ILegacyScopedClusterClient; // @deprecated callAsInternalUser: LegacyAPICaller; @@ -1553,7 +1552,7 @@ export interface LegacyConfig { } // @public @deprecated (undocumented) -export type LegacyElasticsearchClientConfig = Pick & Pick & { +export type LegacyElasticsearchClientConfig = Pick & Pick & { pingTimeout?: ElasticsearchConfig['pingTimeout'] | ConfigOptions['pingTimeout']; requestTimeout?: ElasticsearchConfig['requestTimeout'] | ConfigOptions['requestTimeout']; sniffInterval?: ElasticsearchConfig['sniffInterval'] | ConfigOptions['sniffInterval']; diff --git a/src/plugins/dashboard/public/application/dashboard_app.tsx b/src/plugins/dashboard/public/application/dashboard_app.tsx index 7ea181715717b..6955365ebca3f 100644 --- a/src/plugins/dashboard/public/application/dashboard_app.tsx +++ b/src/plugins/dashboard/public/application/dashboard_app.tsx @@ -265,6 +265,13 @@ export function DashboardApp({ }; }, [dashboardStateManager, dashboardContainer, onAppLeave, embeddable]); + // clear search session when leaving dashboard route + useEffect(() => { + return () => { + data.search.session.clear(); + }; + }, [data.search.session]); + return (
{savedDashboard && dashboardStateManager && dashboardContainer && viewMode && ( diff --git a/src/plugins/data/common/es_query/kuery/node_types/node_builder.test.ts b/src/plugins/data/common/es_query/kuery/node_types/node_builder.test.ts new file mode 100644 index 0000000000000..df78d68aaef48 --- /dev/null +++ b/src/plugins/data/common/es_query/kuery/node_types/node_builder.test.ts @@ -0,0 +1,280 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License + * and the Server Side Public License, v 1; you may not use this file except in + * compliance with, at your election, the Elastic License or the Server Side + * Public License, v 1. + */ + +import { nodeBuilder } from './node_builder'; +import { toElasticsearchQuery } from '../index'; + +describe('nodeBuilder', () => { + describe('is method', () => { + test('string value', () => { + const nodes = nodeBuilder.is('foo', 'bar'); + const query = toElasticsearchQuery(nodes); + expect(query).toMatchInlineSnapshot(` + Object { + "bool": Object { + "minimum_should_match": 1, + "should": Array [ + Object { + "match": Object { + "foo": "bar", + }, + }, + ], + }, + } + `); + }); + + test('KueryNode value', () => { + const literalValue = { + type: 'literal' as 'literal', + value: 'bar', + }; + const nodes = nodeBuilder.is('foo', literalValue); + const query = toElasticsearchQuery(nodes); + expect(query).toMatchInlineSnapshot(` + Object { + "bool": Object { + "minimum_should_match": 1, + "should": Array [ + Object { + "match": Object { + "foo": "bar", + }, + }, + ], + }, + } + `); + }); + }); + + describe('and method', () => { + test('single clause', () => { + const nodes = [nodeBuilder.is('foo', 'bar')]; + const query = toElasticsearchQuery(nodeBuilder.and(nodes)); + expect(query).toMatchInlineSnapshot(` + Object { + "bool": Object { + "minimum_should_match": 1, + "should": Array [ + Object { + "match": Object { + "foo": "bar", + }, + }, + ], + }, + } + `); + }); + + test('two clauses', () => { + const nodes = [nodeBuilder.is('foo1', 'bar1'), nodeBuilder.is('foo2', 'bar2')]; + const query = toElasticsearchQuery(nodeBuilder.and(nodes)); + expect(query).toMatchInlineSnapshot(` + Object { + "bool": Object { + "filter": Array [ + Object { + "bool": Object { + "minimum_should_match": 1, + "should": Array [ + Object { + "match": Object { + "foo1": "bar1", + }, + }, + ], + }, + }, + Object { + "bool": Object { + "minimum_should_match": 1, + "should": Array [ + Object { + "match": Object { + "foo2": "bar2", + }, + }, + ], + }, + }, + ], + }, + } + `); + }); + + test('three clauses', () => { + const nodes = [ + nodeBuilder.is('foo1', 'bar1'), + nodeBuilder.is('foo2', 'bar2'), + nodeBuilder.is('foo3', 'bar3'), + ]; + const query = toElasticsearchQuery(nodeBuilder.and(nodes)); + expect(query).toMatchInlineSnapshot(` + Object { + "bool": Object { + "filter": Array [ + Object { + "bool": Object { + "minimum_should_match": 1, + "should": Array [ + Object { + "match": Object { + "foo1": "bar1", + }, + }, + ], + }, + }, + Object { + "bool": Object { + "minimum_should_match": 1, + "should": Array [ + Object { + "match": Object { + "foo2": "bar2", + }, + }, + ], + }, + }, + Object { + "bool": Object { + "minimum_should_match": 1, + "should": Array [ + Object { + "match": Object { + "foo3": "bar3", + }, + }, + ], + }, + }, + ], + }, + } + `); + }); + }); + + describe('or method', () => { + test('single clause', () => { + const nodes = [nodeBuilder.is('foo', 'bar')]; + const query = toElasticsearchQuery(nodeBuilder.or(nodes)); + expect(query).toMatchInlineSnapshot(` + Object { + "bool": Object { + "minimum_should_match": 1, + "should": Array [ + Object { + "match": Object { + "foo": "bar", + }, + }, + ], + }, + } + `); + }); + + test('two clauses', () => { + const nodes = [nodeBuilder.is('foo1', 'bar1'), nodeBuilder.is('foo2', 'bar2')]; + const query = toElasticsearchQuery(nodeBuilder.or(nodes)); + expect(query).toMatchInlineSnapshot(` + Object { + "bool": Object { + "minimum_should_match": 1, + "should": Array [ + Object { + "bool": Object { + "minimum_should_match": 1, + "should": Array [ + Object { + "match": Object { + "foo1": "bar1", + }, + }, + ], + }, + }, + Object { + "bool": Object { + "minimum_should_match": 1, + "should": Array [ + Object { + "match": Object { + "foo2": "bar2", + }, + }, + ], + }, + }, + ], + }, + } + `); + }); + + test('three clauses', () => { + const nodes = [ + nodeBuilder.is('foo1', 'bar1'), + nodeBuilder.is('foo2', 'bar2'), + nodeBuilder.is('foo3', 'bar3'), + ]; + const query = toElasticsearchQuery(nodeBuilder.or(nodes)); + expect(query).toMatchInlineSnapshot(` + Object { + "bool": Object { + "minimum_should_match": 1, + "should": Array [ + Object { + "bool": Object { + "minimum_should_match": 1, + "should": Array [ + Object { + "match": Object { + "foo1": "bar1", + }, + }, + ], + }, + }, + Object { + "bool": Object { + "minimum_should_match": 1, + "should": Array [ + Object { + "match": Object { + "foo2": "bar2", + }, + }, + ], + }, + }, + Object { + "bool": Object { + "minimum_should_match": 1, + "should": Array [ + Object { + "match": Object { + "foo3": "bar3", + }, + }, + ], + }, + }, + ], + }, + } + `); + }); + }); +}); diff --git a/src/plugins/data/common/es_query/kuery/node_types/node_builder.ts b/src/plugins/data/common/es_query/kuery/node_types/node_builder.ts index a72c7f2db41a8..6da9c3aa293ef 100644 --- a/src/plugins/data/common/es_query/kuery/node_types/node_builder.ts +++ b/src/plugins/data/common/es_query/kuery/node_types/node_builder.ts @@ -16,12 +16,10 @@ export const nodeBuilder = { nodeTypes.literal.buildNode(false), ]); }, - or: ([first, ...args]: KueryNode[]): KueryNode => { - return args.length ? nodeTypes.function.buildNode('or', [first, nodeBuilder.or(args)]) : first; + or: (nodes: KueryNode[]): KueryNode => { + return nodes.length > 1 ? nodeTypes.function.buildNode('or', nodes) : nodes[0]; }, - and: ([first, ...args]: KueryNode[]): KueryNode => { - return args.length - ? nodeTypes.function.buildNode('and', [first, nodeBuilder.and(args)]) - : first; + and: (nodes: KueryNode[]): KueryNode => { + return nodes.length > 1 ? nodeTypes.function.buildNode('and', nodes) : nodes[0]; }, }; diff --git a/src/plugins/data/common/search/types.ts b/src/plugins/data/common/search/types.ts index c1293f4415458..38e963591f25c 100644 --- a/src/plugins/data/common/search/types.ts +++ b/src/plugins/data/common/search/types.ts @@ -84,11 +84,18 @@ export interface ISearchOptions { * An `AbortSignal` that allows the caller of `search` to abort a search request. */ abortSignal?: AbortSignal; + /** * Use this option to force using a specific server side search strategy. Leave empty to use the default strategy. */ strategy?: string; + /** + * Request the legacy format for the total number of hits. If sending `rest_total_hits_as_int` to + * something other than `true`, this should be set to `false`. + */ + legacyHitsTotal?: boolean; + /** * A session ID, grouping multiple search requests into a single session. */ diff --git a/src/plugins/data/public/public.api.md b/src/plugins/data/public/public.api.md index f533af2db9672..4f197dd43a83e 100644 --- a/src/plugins/data/public/public.api.md +++ b/src/plugins/data/public/public.api.md @@ -1641,6 +1641,7 @@ export interface ISearchOptions { abortSignal?: AbortSignal; isRestore?: boolean; isStored?: boolean; + legacyHitsTotal?: boolean; sessionId?: string; strategy?: string; } diff --git a/src/plugins/data/server/index.ts b/src/plugins/data/server/index.ts index 27af11674d061..370ff180fa562 100644 --- a/src/plugins/data/server/index.ts +++ b/src/plugins/data/server/index.ts @@ -235,6 +235,7 @@ export { SearchUsage, SessionService, ISessionService, + IScopedSessionService, DataApiRequestHandlerContext, DataRequestHandlerContext, } from './search'; diff --git a/src/plugins/data/server/search/es_search/es_search_strategy.ts b/src/plugins/data/server/search/es_search/es_search_strategy.ts index c176a50627b92..2d9b16ac8b00b 100644 --- a/src/plugins/data/server/search/es_search/es_search_strategy.ts +++ b/src/plugins/data/server/search/es_search/es_search_strategy.ts @@ -13,7 +13,7 @@ import type { Logger, SharedGlobalConfig } from 'kibana/server'; import type { ISearchStrategy } from '../types'; import type { SearchUsage } from '../collectors'; import { getDefaultSearchParams, getShardTimeout, shimAbortSignal } from './request_utils'; -import { toKibanaSearchResponse } from './response_utils'; +import { shimHitsTotal, toKibanaSearchResponse } from './response_utils'; import { searchUsageObserver } from '../collectors/usage'; import { getKbnServerError, KbnServerError } from '../../../../kibana_utils/server'; @@ -29,7 +29,7 @@ export const esSearchStrategyProvider = ( * @throws `KbnServerError` * @returns `Observable>` */ - search: (request, { abortSignal }, { esClient, uiSettingsClient }) => { + search: (request, { abortSignal, ...options }, { esClient, uiSettingsClient }) => { // Only default index pattern type is supported here. // See data_enhanced for other type support. if (request.indexType) { @@ -46,7 +46,8 @@ export const esSearchStrategyProvider = ( }; const promise = esClient.asCurrentUser.search>(params); const { body } = await shimAbortSignal(promise, abortSignal); - return toKibanaSearchResponse(body); + const response = shimHitsTotal(body, options); + return toKibanaSearchResponse(response); } catch (e) { throw getKbnServerError(e); } diff --git a/src/plugins/data/server/search/es_search/response_utils.test.ts b/src/plugins/data/server/search/es_search/response_utils.test.ts index 8c973b92c7ffe..7cb5705ecf8ef 100644 --- a/src/plugins/data/server/search/es_search/response_utils.test.ts +++ b/src/plugins/data/server/search/es_search/response_utils.test.ts @@ -6,7 +6,7 @@ * Public License, v 1. */ -import { getTotalLoaded, toKibanaSearchResponse } from './response_utils'; +import { getTotalLoaded, toKibanaSearchResponse, shimHitsTotal } from './response_utils'; import { SearchResponse } from 'elasticsearch'; describe('response utils', () => { @@ -55,4 +55,79 @@ describe('response utils', () => { }); }); }); + + describe('shimHitsTotal', () => { + test('returns the total if it is already numeric', () => { + const result = shimHitsTotal({ + hits: { + total: 5, + }, + } as any); + expect(result).toEqual({ + hits: { + total: 5, + }, + }); + }); + + test('returns the total if it is inside `value`', () => { + const result = shimHitsTotal({ + hits: { + total: { + value: 5, + }, + }, + } as any); + expect(result).toEqual({ + hits: { + total: 5, + }, + }); + }); + + test('returns other properties from the response', () => { + const result = shimHitsTotal({ + _shards: {}, + hits: { + hits: [], + total: { + value: 5, + }, + }, + } as any); + expect(result).toEqual({ + _shards: {}, + hits: { + hits: [], + total: 5, + }, + }); + }); + + test('returns the response as-is if `legacyHitsTotal` is `false`', () => { + const result = shimHitsTotal( + { + _shards: {}, + hits: { + hits: [], + total: { + value: 5, + relation: 'eq', + }, + }, + } as any, + { legacyHitsTotal: false } + ); + expect(result).toEqual({ + _shards: {}, + hits: { + hits: [], + total: { + value: 5, + relation: 'eq', + }, + }, + }); + }); + }); }); diff --git a/src/plugins/data/server/search/es_search/response_utils.ts b/src/plugins/data/server/search/es_search/response_utils.ts index d4fa14866fd97..3417f24cf420a 100644 --- a/src/plugins/data/server/search/es_search/response_utils.ts +++ b/src/plugins/data/server/search/es_search/response_utils.ts @@ -7,6 +7,7 @@ */ import { SearchResponse } from 'elasticsearch'; +import { ISearchOptions } from '../../../common'; /** * Get the `total`/`loaded` for this response (see `IKibanaSearchResponse`). Note that `skipped` is @@ -31,3 +32,20 @@ export function toKibanaSearchResponse(rawResponse: SearchResponse) { ...getTotalLoaded(rawResponse), }; } + +/** + * Temporary workaround until https://github.com/elastic/kibana/issues/26356 is addressed. + * Since we are setting `track_total_hits` in the request, `hits.total` will be an object + * containing the `value`. + * + * @internal + */ +export function shimHitsTotal( + response: SearchResponse, + { legacyHitsTotal = true }: ISearchOptions = {} +) { + if (!legacyHitsTotal) return response; + const total = (response.hits?.total as any)?.value ?? response.hits?.total; + const hits = { ...response.hits, total }; + return { ...response, hits }; +} diff --git a/src/plugins/data/server/search/index.ts b/src/plugins/data/server/search/index.ts index a333424110065..301b0989b5183 100644 --- a/src/plugins/data/server/search/index.ts +++ b/src/plugins/data/server/search/index.ts @@ -10,5 +10,4 @@ export * from './types'; export * from './es_search'; export { usageProvider, SearchUsage, searchUsageObserver } from './collectors'; export * from './aggs'; -export { shimHitsTotal } from './routes'; export * from './session'; diff --git a/src/plugins/data/server/search/routes/bsearch.ts b/src/plugins/data/server/search/routes/bsearch.ts index e30b7bdaa8402..ba96726b787c0 100644 --- a/src/plugins/data/server/search/routes/bsearch.ts +++ b/src/plugins/data/server/search/routes/bsearch.ts @@ -6,7 +6,7 @@ * Public License, v 1. */ -import { catchError, first, map } from 'rxjs/operators'; +import { catchError, first } from 'rxjs/operators'; import { CoreStart, KibanaRequest } from 'src/core/server'; import { BfetchServerSetup } from 'src/plugins/bfetch/server'; import { @@ -15,7 +15,6 @@ import { ISearchClient, ISearchOptions, } from '../../../common/search'; -import { shimHitsTotal } from './shim_hits_total'; type GetScopedProider = (coreStart: CoreStart) => (request: KibanaRequest) => ISearchClient; @@ -40,14 +39,6 @@ export function registerBsearchRoute( .search(requestData, options) .pipe( first(), - map((response) => { - return { - ...response, - ...{ - rawResponse: shimHitsTotal(response.rawResponse), - }, - }; - }), catchError((err) => { // Re-throw as object, to get attributes passed to the client // eslint-disable-next-line no-throw-literal diff --git a/src/plugins/data/server/search/routes/call_msearch.ts b/src/plugins/data/server/search/routes/call_msearch.ts index fc30e2f29c3ef..e6ff5f454079b 100644 --- a/src/plugins/data/server/search/routes/call_msearch.ts +++ b/src/plugins/data/server/search/routes/call_msearch.ts @@ -12,9 +12,8 @@ import { SearchResponse } from 'elasticsearch'; import { IUiSettingsClient, IScopedClusterClient, SharedGlobalConfig } from 'src/core/server'; import type { MsearchRequestBody, MsearchResponse } from '../../../common/search/search_source'; -import { shimHitsTotal } from './shim_hits_total'; import { getKbnServerError } from '../../../../kibana_utils/server'; -import { getShardTimeout, getDefaultSearchParams, shimAbortSignal } from '..'; +import { getShardTimeout, getDefaultSearchParams, shimAbortSignal, shimHitsTotal } from '..'; /** @internal */ export function convertRequestBody( diff --git a/src/plugins/data/server/search/routes/index.ts b/src/plugins/data/server/search/routes/index.ts index ea20240f6ae19..25e0353fb4a27 100644 --- a/src/plugins/data/server/search/routes/index.ts +++ b/src/plugins/data/server/search/routes/index.ts @@ -9,4 +9,3 @@ export * from './call_msearch'; export * from './msearch'; export * from './search'; -export * from './shim_hits_total'; diff --git a/src/plugins/data/server/search/routes/search.ts b/src/plugins/data/server/search/routes/search.ts index 6d2da4c1e63dd..e556e3ca49ec2 100644 --- a/src/plugins/data/server/search/routes/search.ts +++ b/src/plugins/data/server/search/routes/search.ts @@ -9,7 +9,6 @@ import { first } from 'rxjs/operators'; import { schema } from '@kbn/config-schema'; import { getRequestAbortedSignal } from '../../lib'; -import { shimHitsTotal } from './shim_hits_total'; import { reportServerError } from '../../../../kibana_utils/server'; import type { DataPluginRouter } from '../types'; @@ -27,6 +26,7 @@ export function registerSearchRoute(router: DataPluginRouter): void { body: schema.object( { + legacyHitsTotal: schema.maybe(schema.boolean()), sessionId: schema.maybe(schema.string()), isStored: schema.maybe(schema.boolean()), isRestore: schema.maybe(schema.boolean()), @@ -36,7 +36,13 @@ export function registerSearchRoute(router: DataPluginRouter): void { }, }, async (context, request, res) => { - const { sessionId, isStored, isRestore, ...searchRequest } = request.body; + const { + legacyHitsTotal = true, + sessionId, + isStored, + isRestore, + ...searchRequest + } = request.body; const { strategy, id } = request.params; const abortSignal = getRequestAbortedSignal(request.events.aborted$); @@ -47,6 +53,7 @@ export function registerSearchRoute(router: DataPluginRouter): void { { abortSignal, strategy, + legacyHitsTotal, sessionId, isStored, isRestore, @@ -55,14 +62,7 @@ export function registerSearchRoute(router: DataPluginRouter): void { .pipe(first()) .toPromise(); - return res.ok({ - body: { - ...response, - ...{ - rawResponse: shimHitsTotal(response.rawResponse), - }, - }, - }); + return res.ok({ body: response }); } catch (err) { return reportServerError(res, err); } diff --git a/src/plugins/data/server/search/routes/shim_hits_total.test.ts b/src/plugins/data/server/search/routes/shim_hits_total.test.ts deleted file mode 100644 index 6dcd7c3ff6c70..0000000000000 --- a/src/plugins/data/server/search/routes/shim_hits_total.test.ts +++ /dev/null @@ -1,58 +0,0 @@ -/* - * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one - * or more contributor license agreements. Licensed under the Elastic License - * and the Server Side Public License, v 1; you may not use this file except in - * compliance with, at your election, the Elastic License or the Server Side - * Public License, v 1. - */ - -import { shimHitsTotal } from './shim_hits_total'; - -describe('shimHitsTotal', () => { - test('returns the total if it is already numeric', () => { - const result = shimHitsTotal({ - hits: { - total: 5, - }, - } as any); - expect(result).toEqual({ - hits: { - total: 5, - }, - }); - }); - - test('returns the total if it is inside `value`', () => { - const result = shimHitsTotal({ - hits: { - total: { - value: 5, - }, - }, - } as any); - expect(result).toEqual({ - hits: { - total: 5, - }, - }); - }); - - test('returns other properties from the response', () => { - const result = shimHitsTotal({ - _shards: {}, - hits: { - hits: [], - total: { - value: 5, - }, - }, - } as any); - expect(result).toEqual({ - _shards: {}, - hits: { - hits: [], - total: 5, - }, - }); - }); -}); diff --git a/src/plugins/data/server/search/routes/shim_hits_total.ts b/src/plugins/data/server/search/routes/shim_hits_total.ts deleted file mode 100644 index 4b56d6394e0db..0000000000000 --- a/src/plugins/data/server/search/routes/shim_hits_total.ts +++ /dev/null @@ -1,22 +0,0 @@ -/* - * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one - * or more contributor license agreements. Licensed under the Elastic License - * and the Server Side Public License, v 1; you may not use this file except in - * compliance with, at your election, the Elastic License or the Server Side - * Public License, v 1. - */ - -import { SearchResponse } from 'elasticsearch'; - -/** - * Temporary workaround until https://github.com/elastic/kibana/issues/26356 is addressed. - * Since we are setting `track_total_hits` in the request, `hits.total` will be an object - * containing the `value`. - * - * @internal - */ -export function shimHitsTotal(response: SearchResponse) { - const total = (response.hits?.total as any)?.value ?? response.hits?.total; - const hits = { ...response.hits, total }; - return { ...response, hits }; -} diff --git a/src/plugins/data/server/search/search_service.ts b/src/plugins/data/server/search/search_service.ts index 63593bbe84a08..e9f0edbd4d6c4 100644 --- a/src/plugins/data/server/search/search_service.ts +++ b/src/plugins/data/server/search/search_service.ts @@ -34,7 +34,7 @@ import { AggsService } from './aggs'; import { FieldFormatsStart } from '../field_formats'; import { IndexPatternsServiceStart } from '../index_patterns'; -import { getCallMsearch, registerMsearchRoute, registerSearchRoute, shimHitsTotal } from './routes'; +import { getCallMsearch, registerMsearchRoute, registerSearchRoute } from './routes'; import { ES_SEARCH_STRATEGY, esSearchStrategyProvider } from './es_search'; import { DataPluginStart } from '../plugin'; import { UsageCollectionSetup } from '../../../usage_collection/server'; @@ -62,7 +62,7 @@ import { } from '../../common/search/aggs/buckets/shard_delay'; import { aggShardDelay } from '../../common/search/aggs/buckets/shard_delay_fn'; import { ConfigSchema } from '../../config'; -import { SessionService, IScopedSessionService, ISessionService } from './session'; +import { IScopedSessionService, ISessionService, SessionService } from './session'; import { KbnServerError } from '../../../kibana_utils/server'; import { registerBsearchRoute } from './routes/bsearch'; @@ -209,7 +209,7 @@ export class SearchService implements Plugin { const searchSourceDependencies: SearchSourceDependencies = { getConfig: (key: string): T => uiSettingsCache[key], search: asScoped(request).search, - onResponse: (req, res) => shimHitsTotal(res), + onResponse: (req, res) => res, legacy: { callMsearch: getCallMsearch({ esClient, diff --git a/src/plugins/data/server/server.api.md b/src/plugins/data/server/server.api.md index 9789f3354e9ef..635428f298ab2 100644 --- a/src/plugins/data/server/server.api.md +++ b/src/plugins/data/server/server.api.md @@ -310,8 +310,6 @@ export const config: PluginConfigDescriptor; // // @public (undocumented) export interface DataApiRequestHandlerContext extends ISearchClient { - // Warning: (ae-forgotten-export) The symbol "IScopedSessionService" needs to be exported by the entry point index.d.ts - // // (undocumented) session: IScopedSessionService; } @@ -912,6 +910,16 @@ export class IndexPatternsService implements Plugin_3(strategy: ISearchStrategy, ...args: Parameters['search']>) => Observable; +} + // Warning: (ae-missing-release-tag) "ISearchOptions" is exported by the package, but it is missing a release tag (@alpha, @beta, @public, or @internal) // // @public (undocumented) @@ -919,6 +927,7 @@ export interface ISearchOptions { abortSignal?: AbortSignal; isRestore?: boolean; isStored?: boolean; + legacyHitsTotal?: boolean; sessionId?: string; strategy?: string; } @@ -1284,7 +1293,7 @@ export class SessionService implements ISessionService { export const shimAbortSignal: (promise: TransportRequestPromise, signal?: AbortSignal | undefined) => TransportRequestPromise; // @internal -export function shimHitsTotal(response: SearchResponse): { +export function shimHitsTotal(response: SearchResponse, { legacyHitsTotal }?: ISearchOptions): { hits: { total: any; max_score: number; @@ -1293,7 +1302,7 @@ export function shimHitsTotal(response: SearchResponse): { _type: string; _id: string; _score: number; - _source: any; + _source: unknown; _version?: number | undefined; _explanation?: import("elasticsearch").Explanation | undefined; fields?: any; @@ -1426,20 +1435,20 @@ export function usageProvider(core: CoreSetup_2): SearchUsage; // src/plugins/data/server/index.ts:100:26 - (ae-forgotten-export) The symbol "TruncateFormat" needs to be exported by the entry point index.d.ts // src/plugins/data/server/index.ts:126:27 - (ae-forgotten-export) The symbol "isFilterable" needs to be exported by the entry point index.d.ts // src/plugins/data/server/index.ts:126:27 - (ae-forgotten-export) The symbol "isNestedField" needs to be exported by the entry point index.d.ts -// src/plugins/data/server/index.ts:243:20 - (ae-forgotten-export) The symbol "getRequestInspectorStats" needs to be exported by the entry point index.d.ts -// src/plugins/data/server/index.ts:243:20 - (ae-forgotten-export) The symbol "getResponseInspectorStats" needs to be exported by the entry point index.d.ts -// src/plugins/data/server/index.ts:243:20 - (ae-forgotten-export) The symbol "tabifyAggResponse" needs to be exported by the entry point index.d.ts -// src/plugins/data/server/index.ts:243:20 - (ae-forgotten-export) The symbol "tabifyGetColumns" needs to be exported by the entry point index.d.ts -// src/plugins/data/server/index.ts:245:1 - (ae-forgotten-export) The symbol "CidrMask" needs to be exported by the entry point index.d.ts -// src/plugins/data/server/index.ts:246:1 - (ae-forgotten-export) The symbol "dateHistogramInterval" needs to be exported by the entry point index.d.ts -// src/plugins/data/server/index.ts:255:1 - (ae-forgotten-export) The symbol "InvalidEsCalendarIntervalError" needs to be exported by the entry point index.d.ts -// src/plugins/data/server/index.ts:256:1 - (ae-forgotten-export) The symbol "InvalidEsIntervalFormatError" needs to be exported by the entry point index.d.ts -// src/plugins/data/server/index.ts:257:1 - (ae-forgotten-export) The symbol "Ipv4Address" needs to be exported by the entry point index.d.ts -// src/plugins/data/server/index.ts:261:1 - (ae-forgotten-export) The symbol "isValidEsInterval" needs to be exported by the entry point index.d.ts -// src/plugins/data/server/index.ts:262:1 - (ae-forgotten-export) The symbol "isValidInterval" needs to be exported by the entry point index.d.ts -// src/plugins/data/server/index.ts:266:1 - (ae-forgotten-export) The symbol "propFilter" needs to be exported by the entry point index.d.ts -// src/plugins/data/server/index.ts:269:1 - (ae-forgotten-export) The symbol "toAbsoluteDates" needs to be exported by the entry point index.d.ts -// src/plugins/data/server/index.ts:270:1 - (ae-forgotten-export) The symbol "calcAutoIntervalLessThan" needs to be exported by the entry point index.d.ts +// src/plugins/data/server/index.ts:244:20 - (ae-forgotten-export) The symbol "getRequestInspectorStats" needs to be exported by the entry point index.d.ts +// src/plugins/data/server/index.ts:244:20 - (ae-forgotten-export) The symbol "getResponseInspectorStats" needs to be exported by the entry point index.d.ts +// src/plugins/data/server/index.ts:244:20 - (ae-forgotten-export) The symbol "tabifyAggResponse" needs to be exported by the entry point index.d.ts +// src/plugins/data/server/index.ts:244:20 - (ae-forgotten-export) The symbol "tabifyGetColumns" needs to be exported by the entry point index.d.ts +// src/plugins/data/server/index.ts:246:1 - (ae-forgotten-export) The symbol "CidrMask" needs to be exported by the entry point index.d.ts +// src/plugins/data/server/index.ts:247:1 - (ae-forgotten-export) The symbol "dateHistogramInterval" needs to be exported by the entry point index.d.ts +// src/plugins/data/server/index.ts:256:1 - (ae-forgotten-export) The symbol "InvalidEsCalendarIntervalError" needs to be exported by the entry point index.d.ts +// src/plugins/data/server/index.ts:257:1 - (ae-forgotten-export) The symbol "InvalidEsIntervalFormatError" needs to be exported by the entry point index.d.ts +// src/plugins/data/server/index.ts:258:1 - (ae-forgotten-export) The symbol "Ipv4Address" needs to be exported by the entry point index.d.ts +// src/plugins/data/server/index.ts:262:1 - (ae-forgotten-export) The symbol "isValidEsInterval" needs to be exported by the entry point index.d.ts +// src/plugins/data/server/index.ts:263:1 - (ae-forgotten-export) The symbol "isValidInterval" needs to be exported by the entry point index.d.ts +// src/plugins/data/server/index.ts:267:1 - (ae-forgotten-export) The symbol "propFilter" needs to be exported by the entry point index.d.ts +// src/plugins/data/server/index.ts:270:1 - (ae-forgotten-export) The symbol "toAbsoluteDates" needs to be exported by the entry point index.d.ts +// src/plugins/data/server/index.ts:271:1 - (ae-forgotten-export) The symbol "calcAutoIntervalLessThan" needs to be exported by the entry point index.d.ts // src/plugins/data/server/index_patterns/index_patterns_service.ts:59:14 - (ae-forgotten-export) The symbol "IndexPatternsService" needs to be exported by the entry point index.d.ts // src/plugins/data/server/plugin.ts:79:74 - (ae-forgotten-export) The symbol "DataEnhancements" needs to be exported by the entry point index.d.ts // src/plugins/data/server/search/types.ts:103:5 - (ae-forgotten-export) The symbol "ISearchStartSearchSource" needs to be exported by the entry point index.d.ts diff --git a/src/plugins/home/public/application/components/add_data/__snapshots__/add_data.test.tsx.snap b/src/plugins/home/public/application/components/add_data/__snapshots__/add_data.test.tsx.snap index e7136ac817249..5724d46fca10c 100644 --- a/src/plugins/home/public/application/components/add_data/__snapshots__/add_data.test.tsx.snap +++ b/src/plugins/home/public/application/components/add_data/__snapshots__/add_data.test.tsx.snap @@ -32,6 +32,7 @@ exports[`AddData render 1`] = `
= ({ addBasePath, features }) => {
{ return this.tabs.map((tab, index) => ( this.onSelectedTabChanged(tab.id)} isSelected={tab.id === this.state.selectedTabId} key={index} @@ -203,7 +205,7 @@ class TutorialDirectoryUi extends React.Component { }) .map((tutorial) => { return ( - + = ({ = ({ = ({ = SavedObject & { meta: SavedObjectMetadata; }; +export type SavedObjectRelationKind = 'child' | 'parent'; + /** * Represents a relation between two {@link SavedObject | saved object} */ export interface SavedObjectRelation { id: string; type: string; - relationship: 'child' | 'parent'; + relationship: SavedObjectRelationKind; meta: SavedObjectMetadata; } + +export interface SavedObjectInvalidRelation { + id: string; + type: string; + relationship: SavedObjectRelationKind; + error: string; +} + +export interface SavedObjectGetRelationshipsResponse { + relations: SavedObjectRelation[]; + invalidRelations: SavedObjectInvalidRelation[]; +} diff --git a/src/plugins/saved_objects_management/public/lib/get_relationships.test.ts b/src/plugins/saved_objects_management/public/lib/get_relationships.test.ts index b609fac67dac1..4454907f530fe 100644 --- a/src/plugins/saved_objects_management/public/lib/get_relationships.test.ts +++ b/src/plugins/saved_objects_management/public/lib/get_relationships.test.ts @@ -6,6 +6,7 @@ * Public License, v 1. */ +import { SavedObjectGetRelationshipsResponse } from '../types'; import { httpServiceMock } from '../../../../core/public/mocks'; import { getRelationships } from './get_relationships'; @@ -22,13 +23,17 @@ describe('getRelationships', () => { }); it('should handle successful responses', async () => { - httpMock.get.mockResolvedValue([1, 2]); + const serverResponse: SavedObjectGetRelationshipsResponse = { + relations: [], + invalidRelations: [], + }; + httpMock.get.mockResolvedValue(serverResponse); const response = await getRelationships(httpMock, 'dashboard', '1', [ 'search', 'index-pattern', ]); - expect(response).toEqual([1, 2]); + expect(response).toEqual(serverResponse); }); it('should handle errors', async () => { diff --git a/src/plugins/saved_objects_management/public/lib/get_relationships.ts b/src/plugins/saved_objects_management/public/lib/get_relationships.ts index 0eb97e1052fa4..69aeb6fbf580b 100644 --- a/src/plugins/saved_objects_management/public/lib/get_relationships.ts +++ b/src/plugins/saved_objects_management/public/lib/get_relationships.ts @@ -8,19 +8,19 @@ import { HttpStart } from 'src/core/public'; import { get } from 'lodash'; -import { SavedObjectRelation } from '../types'; +import { SavedObjectGetRelationshipsResponse } from '../types'; export async function getRelationships( http: HttpStart, type: string, id: string, savedObjectTypes: string[] -): Promise { +): Promise { const url = `/api/kibana/management/saved_objects/relationships/${encodeURIComponent( type )}/${encodeURIComponent(id)}`; try { - return await http.get(url, { + return await http.get(url, { query: { savedObjectTypes, }, diff --git a/src/plugins/saved_objects_management/public/management_section/objects_table/components/__snapshots__/relationships.test.tsx.snap b/src/plugins/saved_objects_management/public/management_section/objects_table/components/__snapshots__/relationships.test.tsx.snap index 15e5cb89b622c..c39263f304249 100644 --- a/src/plugins/saved_objects_management/public/management_section/objects_table/components/__snapshots__/relationships.test.tsx.snap +++ b/src/plugins/saved_objects_management/public/management_section/objects_table/components/__snapshots__/relationships.test.tsx.snap @@ -28,133 +28,131 @@ exports[`Relationships should render dashboards normally 1`] = ` -
- -

- Here are the saved objects related to MyDashboard. Deleting this dashboard affects its parent objects, but not its children. -

- - - +

+ Here are the saved objects related to MyDashboard. Deleting this dashboard affects its parent objects, but not its children. +

+ + + -
+ } + tableLayout="fixed" + /> `; @@ -231,138 +229,315 @@ exports[`Relationships should render index patterns normally 1`] = ` -
- -

- Here are the saved objects related to MyIndexPattern*. Deleting this index-pattern affects its parent objects, but not its children. -

- - - +

+ Here are the saved objects related to MyIndexPattern*. Deleting this index-pattern affects its parent objects, but not its children. +

+ + + + + +`; + +exports[`Relationships should render invalid relations 1`] = ` + + + +

+ + + +    + MyIndexPattern* +

+ + + + + + + + +

+ Here are the saved objects related to MyIndexPattern*. Deleting this index-pattern affects its parent objects, but not its children. +

+ + + -
+ } + tableLayout="fixed" + /> `; @@ -395,138 +570,136 @@ exports[`Relationships should render searches normally 1`] = ` -
- -

- Here are the saved objects related to MySearch. Deleting this search affects its parent objects, but not its children. -

- - - +

+ Here are the saved objects related to MySearch. Deleting this search affects its parent objects, but not its children. +

+ + + -
+ } + tableLayout="fixed" + /> `; @@ -559,133 +732,131 @@ exports[`Relationships should render visualizations normally 1`] = ` -
- -

- Here are the saved objects related to MyViz. Deleting this visualization affects its parent objects, but not its children. -

- - - +

+ Here are the saved objects related to MyViz. Deleting this visualization affects its parent objects, but not its children. +

+ + + -
+ } + tableLayout="fixed" + /> `; diff --git a/src/plugins/saved_objects_management/public/management_section/objects_table/components/relationships.test.tsx b/src/plugins/saved_objects_management/public/management_section/objects_table/components/relationships.test.tsx index 72a4b0f2788fa..e590520193bba 100644 --- a/src/plugins/saved_objects_management/public/management_section/objects_table/components/relationships.test.tsx +++ b/src/plugins/saved_objects_management/public/management_section/objects_table/components/relationships.test.tsx @@ -25,36 +25,39 @@ describe('Relationships', () => { goInspectObject: () => {}, canGoInApp: () => true, basePath: httpServiceMock.createSetupContract().basePath, - getRelationships: jest.fn().mockImplementation(() => [ - { - type: 'search', - id: '1', - relationship: 'parent', - meta: { - editUrl: '/management/kibana/objects/savedSearches/1', - icon: 'search', - inAppUrl: { - path: '/app/discover#//1', - uiCapabilitiesPath: 'discover.show', + getRelationships: jest.fn().mockImplementation(() => ({ + relations: [ + { + type: 'search', + id: '1', + relationship: 'parent', + meta: { + editUrl: '/management/kibana/objects/savedSearches/1', + icon: 'search', + inAppUrl: { + path: '/app/discover#//1', + uiCapabilitiesPath: 'discover.show', + }, + title: 'My Search Title', }, - title: 'My Search Title', }, - }, - { - type: 'visualization', - id: '2', - relationship: 'parent', - meta: { - editUrl: '/management/kibana/objects/savedVisualizations/2', - icon: 'visualizeApp', - inAppUrl: { - path: '/app/visualize#/edit/2', - uiCapabilitiesPath: 'visualize.show', + { + type: 'visualization', + id: '2', + relationship: 'parent', + meta: { + editUrl: '/management/kibana/objects/savedVisualizations/2', + icon: 'visualizeApp', + inAppUrl: { + path: '/app/visualize#/edit/2', + uiCapabilitiesPath: 'visualize.show', + }, + title: 'My Visualization Title', }, - title: 'My Visualization Title', }, - }, - ]), + ], + invalidRelations: [], + })), savedObject: { id: '1', type: 'index-pattern', @@ -92,36 +95,39 @@ describe('Relationships', () => { goInspectObject: () => {}, canGoInApp: () => true, basePath: httpServiceMock.createSetupContract().basePath, - getRelationships: jest.fn().mockImplementation(() => [ - { - type: 'index-pattern', - id: '1', - relationship: 'child', - meta: { - editUrl: '/management/kibana/indexPatterns/patterns/1', - icon: 'indexPatternApp', - inAppUrl: { - path: '/app/management/kibana/indexPatterns/patterns/1', - uiCapabilitiesPath: 'management.kibana.indexPatterns', + getRelationships: jest.fn().mockImplementation(() => ({ + relations: [ + { + type: 'index-pattern', + id: '1', + relationship: 'child', + meta: { + editUrl: '/management/kibana/indexPatterns/patterns/1', + icon: 'indexPatternApp', + inAppUrl: { + path: '/app/management/kibana/indexPatterns/patterns/1', + uiCapabilitiesPath: 'management.kibana.indexPatterns', + }, + title: 'My Index Pattern', }, - title: 'My Index Pattern', }, - }, - { - type: 'visualization', - id: '2', - relationship: 'parent', - meta: { - editUrl: '/management/kibana/objects/savedVisualizations/2', - icon: 'visualizeApp', - inAppUrl: { - path: '/app/visualize#/edit/2', - uiCapabilitiesPath: 'visualize.show', + { + type: 'visualization', + id: '2', + relationship: 'parent', + meta: { + editUrl: '/management/kibana/objects/savedVisualizations/2', + icon: 'visualizeApp', + inAppUrl: { + path: '/app/visualize#/edit/2', + uiCapabilitiesPath: 'visualize.show', + }, + title: 'My Visualization Title', }, - title: 'My Visualization Title', }, - }, - ]), + ], + invalidRelations: [], + })), savedObject: { id: '1', type: 'search', @@ -159,36 +165,39 @@ describe('Relationships', () => { goInspectObject: () => {}, canGoInApp: () => true, basePath: httpServiceMock.createSetupContract().basePath, - getRelationships: jest.fn().mockImplementation(() => [ - { - type: 'dashboard', - id: '1', - relationship: 'parent', - meta: { - editUrl: '/management/kibana/objects/savedDashboards/1', - icon: 'dashboardApp', - inAppUrl: { - path: '/app/kibana#/dashboard/1', - uiCapabilitiesPath: 'dashboard.show', + getRelationships: jest.fn().mockImplementation(() => ({ + relations: [ + { + type: 'dashboard', + id: '1', + relationship: 'parent', + meta: { + editUrl: '/management/kibana/objects/savedDashboards/1', + icon: 'dashboardApp', + inAppUrl: { + path: '/app/kibana#/dashboard/1', + uiCapabilitiesPath: 'dashboard.show', + }, + title: 'My Dashboard 1', }, - title: 'My Dashboard 1', }, - }, - { - type: 'dashboard', - id: '2', - relationship: 'parent', - meta: { - editUrl: '/management/kibana/objects/savedDashboards/2', - icon: 'dashboardApp', - inAppUrl: { - path: '/app/kibana#/dashboard/2', - uiCapabilitiesPath: 'dashboard.show', + { + type: 'dashboard', + id: '2', + relationship: 'parent', + meta: { + editUrl: '/management/kibana/objects/savedDashboards/2', + icon: 'dashboardApp', + inAppUrl: { + path: '/app/kibana#/dashboard/2', + uiCapabilitiesPath: 'dashboard.show', + }, + title: 'My Dashboard 2', }, - title: 'My Dashboard 2', }, - }, - ]), + ], + invalidRelations: [], + })), savedObject: { id: '1', type: 'visualization', @@ -226,36 +235,39 @@ describe('Relationships', () => { goInspectObject: () => {}, canGoInApp: () => true, basePath: httpServiceMock.createSetupContract().basePath, - getRelationships: jest.fn().mockImplementation(() => [ - { - type: 'visualization', - id: '1', - relationship: 'child', - meta: { - editUrl: '/management/kibana/objects/savedVisualizations/1', - icon: 'visualizeApp', - inAppUrl: { - path: '/app/visualize#/edit/1', - uiCapabilitiesPath: 'visualize.show', + getRelationships: jest.fn().mockImplementation(() => ({ + relations: [ + { + type: 'visualization', + id: '1', + relationship: 'child', + meta: { + editUrl: '/management/kibana/objects/savedVisualizations/1', + icon: 'visualizeApp', + inAppUrl: { + path: '/app/visualize#/edit/1', + uiCapabilitiesPath: 'visualize.show', + }, + title: 'My Visualization Title 1', }, - title: 'My Visualization Title 1', }, - }, - { - type: 'visualization', - id: '2', - relationship: 'child', - meta: { - editUrl: '/management/kibana/objects/savedVisualizations/2', - icon: 'visualizeApp', - inAppUrl: { - path: '/app/visualize#/edit/2', - uiCapabilitiesPath: 'visualize.show', + { + type: 'visualization', + id: '2', + relationship: 'child', + meta: { + editUrl: '/management/kibana/objects/savedVisualizations/2', + icon: 'visualizeApp', + inAppUrl: { + path: '/app/visualize#/edit/2', + uiCapabilitiesPath: 'visualize.show', + }, + title: 'My Visualization Title 2', }, - title: 'My Visualization Title 2', }, - }, - ]), + ], + invalidRelations: [], + })), savedObject: { id: '1', type: 'dashboard', @@ -324,4 +336,49 @@ describe('Relationships', () => { expect(props.getRelationships).toHaveBeenCalled(); expect(component).toMatchSnapshot(); }); + + it('should render invalid relations', async () => { + const props: RelationshipsProps = { + goInspectObject: () => {}, + canGoInApp: () => true, + basePath: httpServiceMock.createSetupContract().basePath, + getRelationships: jest.fn().mockImplementation(() => ({ + relations: [], + invalidRelations: [ + { + id: '1', + type: 'dashboard', + relationship: 'child', + error: 'Saved object [dashboard/1] not found', + }, + ], + })), + savedObject: { + id: '1', + type: 'index-pattern', + attributes: {}, + references: [], + meta: { + title: 'MyIndexPattern*', + icon: 'indexPatternApp', + editUrl: '#/management/kibana/indexPatterns/patterns/1', + inAppUrl: { + path: '/management/kibana/indexPatterns/patterns/1', + uiCapabilitiesPath: 'management.kibana.indexPatterns', + }, + }, + }, + close: jest.fn(), + }; + + const component = shallowWithI18nProvider(); + + // Ensure all promises resolve + await new Promise((resolve) => process.nextTick(resolve)); + // Ensure the state changes are reflected + component.update(); + + expect(props.getRelationships).toHaveBeenCalled(); + expect(component).toMatchSnapshot(); + }); }); diff --git a/src/plugins/saved_objects_management/public/management_section/objects_table/components/relationships.tsx b/src/plugins/saved_objects_management/public/management_section/objects_table/components/relationships.tsx index 2d62699b6f1f2..aee61f7bc9c7a 100644 --- a/src/plugins/saved_objects_management/public/management_section/objects_table/components/relationships.tsx +++ b/src/plugins/saved_objects_management/public/management_section/objects_table/components/relationships.tsx @@ -26,11 +26,17 @@ import { i18n } from '@kbn/i18n'; import { FormattedMessage } from '@kbn/i18n/react'; import { IBasePath } from 'src/core/public'; import { getDefaultTitle, getSavedObjectLabel } from '../../../lib'; -import { SavedObjectWithMetadata, SavedObjectRelation } from '../../../types'; +import { + SavedObjectWithMetadata, + SavedObjectRelationKind, + SavedObjectRelation, + SavedObjectInvalidRelation, + SavedObjectGetRelationshipsResponse, +} from '../../../types'; export interface RelationshipsProps { basePath: IBasePath; - getRelationships: (type: string, id: string) => Promise; + getRelationships: (type: string, id: string) => Promise; savedObject: SavedObjectWithMetadata; close: () => void; goInspectObject: (obj: SavedObjectWithMetadata) => void; @@ -38,17 +44,47 @@ export interface RelationshipsProps { } export interface RelationshipsState { - relationships: SavedObjectRelation[]; + relations: SavedObjectRelation[]; + invalidRelations: SavedObjectInvalidRelation[]; isLoading: boolean; error?: string; } +const relationshipColumn = { + field: 'relationship', + name: i18n.translate('savedObjectsManagement.objectsTable.relationships.columnRelationshipName', { + defaultMessage: 'Direct relationship', + }), + dataType: 'string', + sortable: false, + width: '125px', + 'data-test-subj': 'directRelationship', + render: (relationship: SavedObjectRelationKind) => { + return ( + + {relationship === 'parent' ? ( + + ) : ( + + )} + + ); + }, +}; + export class Relationships extends Component { constructor(props: RelationshipsProps) { super(props); this.state = { - relationships: [], + relations: [], + invalidRelations: [], isLoading: false, error: undefined, }; @@ -70,8 +106,11 @@ export class Relationships extends Component + + + ({ + 'data-test-subj': `invalidRelationshipsTableRow`, + })} + /> + + + ); + } + + renderRelationshipsTable() { + const { goInspectObject, basePath, savedObject } = this.props; + const { relations, isLoading, error } = this.state; if (error) { return this.renderError(); @@ -137,39 +250,7 @@ export class Relationships extends Component { - if (relationship === 'parent') { - return ( - - - - ); - } - if (relationship === 'child') { - return ( - - - - ); - } - }, - }, + relationshipColumn, { field: 'meta.title', name: i18n.translate('savedObjectsManagement.objectsTable.relationships.columnTitleName', { @@ -224,7 +305,7 @@ export class Relationships extends Component [ + relations.map((relationship) => [ relationship.type, { value: relationship.type, @@ -277,7 +358,7 @@ export class Relationships extends Component + <>

{i18n.translate( @@ -296,7 +377,7 @@ export class Relationships extends Component -

+ ); } @@ -328,8 +409,10 @@ export class Relationships extends Component - - {this.renderRelationships()} + + {this.renderInvalidRelationship()} + {this.renderRelationshipsTable()} + ); } diff --git a/src/plugins/saved_objects_management/public/types.ts b/src/plugins/saved_objects_management/public/types.ts index 37f239227475d..cdfa3c43e5af2 100644 --- a/src/plugins/saved_objects_management/public/types.ts +++ b/src/plugins/saved_objects_management/public/types.ts @@ -6,4 +6,11 @@ * Public License, v 1. */ -export { SavedObjectMetadata, SavedObjectWithMetadata, SavedObjectRelation } from '../common'; +export { + SavedObjectMetadata, + SavedObjectWithMetadata, + SavedObjectRelationKind, + SavedObjectRelation, + SavedObjectInvalidRelation, + SavedObjectGetRelationshipsResponse, +} from '../common'; diff --git a/src/plugins/saved_objects_management/server/lib/find_relationships.test.ts b/src/plugins/saved_objects_management/server/lib/find_relationships.test.ts index 631faf0c23c98..416be7d7e7426 100644 --- a/src/plugins/saved_objects_management/server/lib/find_relationships.test.ts +++ b/src/plugins/saved_objects_management/server/lib/find_relationships.test.ts @@ -6,10 +6,35 @@ * Public License, v 1. */ +import type { SavedObject, SavedObjectError } from 'src/core/types'; +import type { SavedObjectsFindResponse } from 'src/core/server'; import { findRelationships } from './find_relationships'; import { managementMock } from '../services/management.mock'; import { savedObjectsClientMock } from '../../../../core/server/mocks'; +const createObj = (parts: Partial>): SavedObject => ({ + id: 'id', + type: 'type', + attributes: {}, + references: [], + ...parts, +}); + +const createFindResponse = (objs: SavedObject[]): SavedObjectsFindResponse => ({ + saved_objects: objs.map((obj) => ({ ...obj, score: 1 })), + total: objs.length, + per_page: 20, + page: 1, +}); + +const createError = (parts: Partial): SavedObjectError => ({ + error: 'error', + message: 'message', + metadata: {}, + statusCode: 404, + ...parts, +}); + describe('findRelationships', () => { let savedObjectsClient: ReturnType; let managementService: ReturnType; @@ -19,7 +44,7 @@ describe('findRelationships', () => { managementService = managementMock.create(); }); - it('returns the child and parent references of the object', async () => { + it('calls the savedObjectClient APIs with the correct parameters', async () => { const type = 'dashboard'; const id = 'some-id'; const references = [ @@ -36,46 +61,35 @@ describe('findRelationships', () => { ]; const referenceTypes = ['some-type', 'another-type']; - savedObjectsClient.get.mockResolvedValue({ - id, - type, - attributes: {}, - references, - }); - + savedObjectsClient.get.mockResolvedValue( + createObj({ + id, + type, + references, + }) + ); savedObjectsClient.bulkGet.mockResolvedValue({ saved_objects: [ - { + createObj({ type: 'some-type', id: 'ref-1', - attributes: {}, - references: [], - }, - { + }), + createObj({ type: 'another-type', id: 'ref-2', - attributes: {}, - references: [], - }, + }), ], }); - - savedObjectsClient.find.mockResolvedValue({ - saved_objects: [ - { + savedObjectsClient.find.mockResolvedValue( + createFindResponse([ + createObj({ type: 'parent-type', id: 'parent-id', - attributes: {}, - score: 1, - references: [], - }, - ], - total: 1, - per_page: 20, - page: 1, - }); + }), + ]) + ); - const relationships = await findRelationships({ + await findRelationships({ type, id, size: 20, @@ -101,8 +115,63 @@ describe('findRelationships', () => { perPage: 20, type: referenceTypes, }); + }); + + it('returns the child and parent references of the object', async () => { + const type = 'dashboard'; + const id = 'some-id'; + const references = [ + { + type: 'some-type', + id: 'ref-1', + name: 'ref 1', + }, + { + type: 'another-type', + id: 'ref-2', + name: 'ref 2', + }, + ]; + const referenceTypes = ['some-type', 'another-type']; + + savedObjectsClient.get.mockResolvedValue( + createObj({ + id, + type, + references, + }) + ); + savedObjectsClient.bulkGet.mockResolvedValue({ + saved_objects: [ + createObj({ + type: 'some-type', + id: 'ref-1', + }), + createObj({ + type: 'another-type', + id: 'ref-2', + }), + ], + }); + savedObjectsClient.find.mockResolvedValue( + createFindResponse([ + createObj({ + type: 'parent-type', + id: 'parent-id', + }), + ]) + ); + + const { relations, invalidRelations } = await findRelationships({ + type, + id, + size: 20, + client: savedObjectsClient, + referenceTypes, + savedObjectsManagement: managementService, + }); - expect(relationships).toEqual([ + expect(relations).toEqual([ { id: 'ref-1', relationship: 'child', @@ -122,6 +191,70 @@ describe('findRelationships', () => { meta: expect.any(Object), }, ]); + expect(invalidRelations).toHaveLength(0); + }); + + it('returns the invalid relations', async () => { + const type = 'dashboard'; + const id = 'some-id'; + const references = [ + { + type: 'some-type', + id: 'ref-1', + name: 'ref 1', + }, + { + type: 'another-type', + id: 'ref-2', + name: 'ref 2', + }, + ]; + const referenceTypes = ['some-type', 'another-type']; + + savedObjectsClient.get.mockResolvedValue( + createObj({ + id, + type, + references, + }) + ); + const ref1Error = createError({ message: 'Not found' }); + savedObjectsClient.bulkGet.mockResolvedValue({ + saved_objects: [ + createObj({ + type: 'some-type', + id: 'ref-1', + error: ref1Error, + }), + createObj({ + type: 'another-type', + id: 'ref-2', + }), + ], + }); + savedObjectsClient.find.mockResolvedValue(createFindResponse([])); + + const { relations, invalidRelations } = await findRelationships({ + type, + id, + size: 20, + client: savedObjectsClient, + referenceTypes, + savedObjectsManagement: managementService, + }); + + expect(relations).toEqual([ + { + id: 'ref-2', + relationship: 'child', + type: 'another-type', + meta: expect.any(Object), + }, + ]); + + expect(invalidRelations).toEqual([ + { type: 'some-type', id: 'ref-1', relationship: 'child', error: ref1Error.message }, + ]); }); it('uses the management service to consolidate the relationship objects', async () => { @@ -144,32 +277,24 @@ describe('findRelationships', () => { uiCapabilitiesPath: 'uiCapabilitiesPath', }); - savedObjectsClient.get.mockResolvedValue({ - id, - type, - attributes: {}, - references, - }); - + savedObjectsClient.get.mockResolvedValue( + createObj({ + id, + type, + references, + }) + ); savedObjectsClient.bulkGet.mockResolvedValue({ saved_objects: [ - { + createObj({ type: 'some-type', id: 'ref-1', - attributes: {}, - references: [], - }, + }), ], }); + savedObjectsClient.find.mockResolvedValue(createFindResponse([])); - savedObjectsClient.find.mockResolvedValue({ - saved_objects: [], - total: 0, - per_page: 20, - page: 1, - }); - - const relationships = await findRelationships({ + const { relations } = await findRelationships({ type, id, size: 20, @@ -183,7 +308,7 @@ describe('findRelationships', () => { expect(managementService.getEditUrl).toHaveBeenCalledTimes(1); expect(managementService.getInAppUrl).toHaveBeenCalledTimes(1); - expect(relationships).toEqual([ + expect(relations).toEqual([ { id: 'ref-1', relationship: 'child', diff --git a/src/plugins/saved_objects_management/server/lib/find_relationships.ts b/src/plugins/saved_objects_management/server/lib/find_relationships.ts index 0ceef484196a3..bc6568e73c4e2 100644 --- a/src/plugins/saved_objects_management/server/lib/find_relationships.ts +++ b/src/plugins/saved_objects_management/server/lib/find_relationships.ts @@ -9,7 +9,11 @@ import { SavedObjectsClientContract } from 'src/core/server'; import { injectMetaAttributes } from './inject_meta_attributes'; import { ISavedObjectsManagement } from '../services'; -import { SavedObjectRelation, SavedObjectWithMetadata } from '../types'; +import { + SavedObjectInvalidRelation, + SavedObjectWithMetadata, + SavedObjectGetRelationshipsResponse, +} from '../types'; export async function findRelationships({ type, @@ -25,17 +29,19 @@ export async function findRelationships({ client: SavedObjectsClientContract; referenceTypes: string[]; savedObjectsManagement: ISavedObjectsManagement; -}): Promise { +}): Promise { const { references = [] } = await client.get(type, id); // Use a map to avoid duplicates, it does happen but have a different "name" in the reference - const referencedToBulkGetOpts = new Map( - references.map((ref) => [`${ref.type}:${ref.id}`, { id: ref.id, type: ref.type }]) - ); + const childrenReferences = [ + ...new Map( + references.map((ref) => [`${ref.type}:${ref.id}`, { id: ref.id, type: ref.type }]) + ).values(), + ]; const [childReferencesResponse, parentReferencesResponse] = await Promise.all([ - referencedToBulkGetOpts.size > 0 - ? client.bulkGet([...referencedToBulkGetOpts.values()]) + childrenReferences.length > 0 + ? client.bulkGet(childrenReferences) : Promise.resolve({ saved_objects: [] }), client.find({ hasReference: { type, id }, @@ -44,28 +50,37 @@ export async function findRelationships({ }), ]); - return childReferencesResponse.saved_objects - .map((obj) => injectMetaAttributes(obj, savedObjectsManagement)) - .map(extractCommonProperties) - .map( - (obj) => - ({ - ...obj, - relationship: 'child', - } as SavedObjectRelation) - ) - .concat( - parentReferencesResponse.saved_objects - .map((obj) => injectMetaAttributes(obj, savedObjectsManagement)) - .map(extractCommonProperties) - .map( - (obj) => - ({ - ...obj, - relationship: 'parent', - } as SavedObjectRelation) - ) - ); + const invalidRelations: SavedObjectInvalidRelation[] = childReferencesResponse.saved_objects + .filter((obj) => Boolean(obj.error)) + .map((obj) => ({ + id: obj.id, + type: obj.type, + relationship: 'child', + error: obj.error!.message, + })); + + const relations = [ + ...childReferencesResponse.saved_objects + .filter((obj) => !obj.error) + .map((obj) => injectMetaAttributes(obj, savedObjectsManagement)) + .map(extractCommonProperties) + .map((obj) => ({ + ...obj, + relationship: 'child' as const, + })), + ...parentReferencesResponse.saved_objects + .map((obj) => injectMetaAttributes(obj, savedObjectsManagement)) + .map(extractCommonProperties) + .map((obj) => ({ + ...obj, + relationship: 'parent' as const, + })), + ]; + + return { + relations, + invalidRelations, + }; } function extractCommonProperties(savedObject: SavedObjectWithMetadata) { diff --git a/src/plugins/saved_objects_management/server/routes/relationships.ts b/src/plugins/saved_objects_management/server/routes/relationships.ts index 3a52c973fde8d..5417ff2926120 100644 --- a/src/plugins/saved_objects_management/server/routes/relationships.ts +++ b/src/plugins/saved_objects_management/server/routes/relationships.ts @@ -38,7 +38,7 @@ export const registerRelationshipsRoute = ( ? req.query.savedObjectTypes : [req.query.savedObjectTypes]; - const relations = await findRelationships({ + const findRelationsResponse = await findRelationships({ type, id, client, @@ -48,7 +48,7 @@ export const registerRelationshipsRoute = ( }); return res.ok({ - body: relations, + body: findRelationsResponse, }); }) ); diff --git a/src/plugins/saved_objects_management/server/types.ts b/src/plugins/saved_objects_management/server/types.ts index 710bb5db7d1cb..562970d2d2dcd 100644 --- a/src/plugins/saved_objects_management/server/types.ts +++ b/src/plugins/saved_objects_management/server/types.ts @@ -12,4 +12,11 @@ export interface SavedObjectsManagementPluginSetup {} // eslint-disable-next-line @typescript-eslint/no-empty-interface export interface SavedObjectsManagementPluginStart {} -export { SavedObjectMetadata, SavedObjectWithMetadata, SavedObjectRelation } from '../common'; +export { + SavedObjectMetadata, + SavedObjectWithMetadata, + SavedObjectRelationKind, + SavedObjectRelation, + SavedObjectInvalidRelation, + SavedObjectGetRelationshipsResponse, +} from '../common'; diff --git a/src/plugins/vis_type_table/public/components/table_vis_basic.tsx b/src/plugins/vis_type_table/public/components/table_vis_basic.tsx index 6cea4d09c4e7f..8bb5186159b7d 100644 --- a/src/plugins/vis_type_table/public/components/table_vis_basic.tsx +++ b/src/plugins/vis_type_table/public/components/table_vis_basic.tsx @@ -13,15 +13,14 @@ import { orderBy } from 'lodash'; import { IInterpreterRenderHandlers } from 'src/plugins/expressions'; import { createTableVisCell } from './table_vis_cell'; -import { Table } from '../table_vis_response_handler'; -import { TableVisConfig, TableVisUseUiStateProps } from '../types'; -import { useFormattedColumnsAndRows, usePagination } from '../utils'; +import { TableContext, TableVisConfig, TableVisUseUiStateProps } from '../types'; +import { usePagination } from '../utils'; import { TableVisControls } from './table_vis_controls'; import { createGridColumns } from './table_vis_columns'; interface TableVisBasicProps { fireEvent: IInterpreterRenderHandlers['event']; - table: Table; + table: TableContext; visConfig: TableVisConfig; title?: string; uiStateProps: TableVisUseUiStateProps; @@ -35,7 +34,7 @@ export const TableVisBasic = memo( title, uiStateProps: { columnsWidth, sort, setColumnsWidth, setSort }, }: TableVisBasicProps) => { - const { columns, rows } = useFormattedColumnsAndRows(table, visConfig); + const { columns, rows, formattedColumns } = table; // custom sorting is in place until the EuiDataGrid sorting gets rid of flaws -> https://github.com/elastic/eui/issues/4108 const sortedRows = useMemo( @@ -47,13 +46,19 @@ export const TableVisBasic = memo( ); // renderCellValue is a component which renders a cell based on column and row indexes - const renderCellValue = useMemo(() => createTableVisCell(columns, sortedRows), [ - columns, + const renderCellValue = useMemo(() => createTableVisCell(sortedRows, formattedColumns), [ + formattedColumns, sortedRows, ]); // Columns config - const gridColumns = createGridColumns(table, columns, columnsWidth, sortedRows, fireEvent); + const gridColumns = createGridColumns( + columns, + sortedRows, + formattedColumns, + columnsWidth, + fireEvent + ); // Pagination config const pagination = usePagination(visConfig, rows.length); @@ -126,10 +131,9 @@ export const TableVisBasic = memo( additionalControls: ( ), @@ -138,8 +142,7 @@ export const TableVisBasic = memo( renderCellValue={renderCellValue} renderFooterCellValue={ visConfig.showTotal - ? // @ts-expect-error - ({ colIndex }) => columns[colIndex].formattedTotal || null + ? ({ columnId }) => formattedColumns[columnId].formattedTotal || null : undefined } pagination={pagination} diff --git a/src/plugins/vis_type_table/public/components/table_vis_cell.tsx b/src/plugins/vis_type_table/public/components/table_vis_cell.tsx index 0a6aafc84bf26..04df3907c8c9b 100644 --- a/src/plugins/vis_type_table/public/components/table_vis_cell.tsx +++ b/src/plugins/vis_type_table/public/components/table_vis_cell.tsx @@ -9,17 +9,15 @@ import React from 'react'; import { EuiDataGridCellValueElementProps } from '@elastic/eui'; -import { Table } from '../table_vis_response_handler'; -import { FormattedColumn } from '../types'; +import { DatatableRow } from 'src/plugins/expressions'; +import { FormattedColumns } from '../types'; -export const createTableVisCell = (formattedColumns: FormattedColumn[], rows: Table['rows']) => ({ - // @ts-expect-error - colIndex, +export const createTableVisCell = (rows: DatatableRow[], formattedColumns: FormattedColumns) => ({ rowIndex, columnId, }: EuiDataGridCellValueElementProps) => { const rowValue = rows[rowIndex][columnId]; - const column = formattedColumns[colIndex]; + const column = formattedColumns[columnId]; const content = column.formatter.convert(rowValue, 'html'); const cellContent = ( diff --git a/src/plugins/vis_type_table/public/components/table_vis_columns.tsx b/src/plugins/vis_type_table/public/components/table_vis_columns.tsx index 2610677b2491c..6b44a2504ff89 100644 --- a/src/plugins/vis_type_table/public/components/table_vis_columns.tsx +++ b/src/plugins/vis_type_table/public/components/table_vis_columns.tsx @@ -10,9 +10,8 @@ import React from 'react'; import { EuiDataGridColumnCellActionProps, EuiDataGridColumn } from '@elastic/eui'; import { i18n } from '@kbn/i18n'; -import { IInterpreterRenderHandlers } from 'src/plugins/expressions'; -import { Table } from '../table_vis_response_handler'; -import { FormattedColumn, TableVisUiState } from '../types'; +import { DatatableColumn, DatatableRow, IInterpreterRenderHandlers } from 'src/plugins/expressions'; +import { FormattedColumns, TableVisUiState } from '../types'; interface FilterCellData { /** @@ -27,33 +26,24 @@ interface FilterCellData { } export const createGridColumns = ( - table: Table, - columns: FormattedColumn[], + columns: DatatableColumn[], + rows: DatatableRow[], + formattedColumns: FormattedColumns, columnsWidth: TableVisUiState['colWidth'], - rows: Table['rows'], fireEvent: IInterpreterRenderHandlers['event'] ) => { const onFilterClick = (data: FilterCellData, negate: boolean) => { - /** - * Visible column index and the actual one from the source table could be different. - * e.x. a column could be filtered out if it's not a dimension - - * see formattedColumns in use_formatted_columns.ts file, - * or an extra percantage column could be added, which doesn't exist in the raw table - */ - const rawTableActualColumnIndex = table.columns.findIndex( - (c) => c.id === columns[data.column].id - ); fireEvent({ name: 'filterBucket', data: { data: [ { table: { - ...table, + columns, rows, }, ...data, - column: rawTableActualColumnIndex, + column: data.column, }, ], negate, @@ -63,12 +53,13 @@ export const createGridColumns = ( return columns.map( (col, colIndex): EuiDataGridColumn => { - const cellActions = col.filterable + const formattedColumn = formattedColumns[col.id]; + const cellActions = formattedColumn.filterable ? [ ({ rowIndex, columnId, Component, closePopover }: EuiDataGridColumnCellActionProps) => { const rowValue = rows[rowIndex][columnId]; const contentsIsDefined = rowValue !== null && rowValue !== undefined; - const cellContent = col.formatter.convert(rowValue); + const cellContent = formattedColumn.formatter.convert(rowValue); const filterForText = i18n.translate( 'visTypeTable.tableCellFilter.filterForValueText', @@ -105,7 +96,7 @@ export const createGridColumns = ( ({ rowIndex, columnId, Component, closePopover }: EuiDataGridColumnCellActionProps) => { const rowValue = rows[rowIndex][columnId]; const contentsIsDefined = rowValue !== null && rowValue !== undefined; - const cellContent = col.formatter.convert(rowValue); + const cellContent = formattedColumn.formatter.convert(rowValue); const filterOutText = i18n.translate( 'visTypeTable.tableCellFilter.filterOutValueText', @@ -144,8 +135,8 @@ export const createGridColumns = ( const initialWidth = columnsWidth.find((c) => c.colIndex === colIndex); const column: EuiDataGridColumn = { id: col.id, - display: col.title, - displayAsText: col.title, + display: col.name, + displayAsText: col.name, actions: { showHide: false, showMoveLeft: false, diff --git a/src/plugins/vis_type_table/public/components/table_vis_controls.tsx b/src/plugins/vis_type_table/public/components/table_vis_controls.tsx index 1f4f49442957f..3eda73084e41d 100644 --- a/src/plugins/vis_type_table/public/components/table_vis_controls.tsx +++ b/src/plugins/vis_type_table/public/components/table_vis_controls.tsx @@ -11,81 +11,103 @@ import { EuiButtonEmpty, EuiContextMenuItem, EuiContextMenuPanel, EuiPopover } f import { FormattedMessage } from '@kbn/i18n/react'; import { i18n } from '@kbn/i18n'; -import { DatatableRow } from 'src/plugins/expressions'; +import { DatatableColumn, DatatableRow } from 'src/plugins/expressions'; import { CoreStart } from 'kibana/public'; import { useKibana } from '../../../kibana_react/public'; -import { FormattedColumn } from '../types'; -import { Table } from '../table_vis_response_handler'; -import { exportAsCsv } from '../utils'; +import { exporters } from '../../../data/public'; +import { + CSV_SEPARATOR_SETTING, + CSV_QUOTE_VALUES_SETTING, + downloadFileAs, +} from '../../../share/public'; +import { getFormatService } from '../services'; interface TableVisControlsProps { dataGridAriaLabel: string; filename?: string; - cols: FormattedColumn[]; + columns: DatatableColumn[]; rows: DatatableRow[]; - table: Table; } -export const TableVisControls = memo(({ dataGridAriaLabel, ...props }: TableVisControlsProps) => { - const [isPopoverOpen, setIsPopoverOpen] = useState(false); - const togglePopover = useCallback(() => setIsPopoverOpen((state) => !state), []); - const closePopover = useCallback(() => setIsPopoverOpen(false), []); +export const TableVisControls = memo( + ({ dataGridAriaLabel, filename, columns, rows }: TableVisControlsProps) => { + const [isPopoverOpen, setIsPopoverOpen] = useState(false); + const togglePopover = useCallback(() => setIsPopoverOpen((state) => !state), []); + const closePopover = useCallback(() => setIsPopoverOpen(false), []); - const { - services: { uiSettings }, - } = useKibana(); + const { + services: { uiSettings }, + } = useKibana(); - const onClickExport = useCallback( - (formatted: boolean) => - exportAsCsv(formatted, { - ...props, - uiSettings, - }), - [props, uiSettings] - ); + const onClickExport = useCallback( + (formatted: boolean) => { + const csvSeparator = uiSettings.get(CSV_SEPARATOR_SETTING); + const quoteValues = uiSettings.get(CSV_QUOTE_VALUES_SETTING); - const exportBtnAriaLabel = i18n.translate('visTypeTable.vis.controls.exportButtonAriaLabel', { - defaultMessage: 'Export {dataGridAriaLabel} as CSV', - values: { - dataGridAriaLabel, - }, - }); + const content = exporters.datatableToCSV( + { + type: 'datatable', + columns, + rows, + }, + { + csvSeparator, + quoteValues, + formatFactory: getFormatService().deserialize, + raw: !formatted, + } + ); + downloadFileAs(`${filename || 'unsaved'}.csv`, { content, type: exporters.CSV_MIME_TYPE }); + }, + [columns, rows, filename, uiSettings] + ); - const button = ( - - - - ); + const exportBtnAriaLabel = i18n.translate('visTypeTable.vis.controls.exportButtonAriaLabel', { + defaultMessage: 'Export {dataGridAriaLabel} as CSV', + values: { + dataGridAriaLabel, + }, + }); - const items = [ - onClickExport(false)}> - - , - onClickExport(true)}> - - , - ]; + const button = ( + + + + ); - return ( - - - - ); -}); + const items = [ + onClickExport(false)}> + + , + onClickExport(true)}> + + , + ]; + + return ( + + + + ); + } +); diff --git a/src/plugins/vis_type_table/public/components/table_vis_split.tsx b/src/plugins/vis_type_table/public/components/table_vis_split.tsx index be1a918e22c4b..3d1cacd732fae 100644 --- a/src/plugins/vis_type_table/public/components/table_vis_split.tsx +++ b/src/plugins/vis_type_table/public/components/table_vis_split.tsx @@ -9,8 +9,7 @@ import React, { memo } from 'react'; import { IInterpreterRenderHandlers } from 'src/plugins/expressions'; -import { TableGroup } from '../table_vis_response_handler'; -import { TableVisConfig, TableVisUseUiStateProps } from '../types'; +import { TableGroup, TableVisConfig, TableVisUseUiStateProps } from '../types'; import { TableVisBasic } from './table_vis_basic'; interface TableVisSplitProps { @@ -24,11 +23,11 @@ export const TableVisSplit = memo( ({ fireEvent, tables, visConfig, uiStateProps }: TableVisSplitProps) => { return ( <> - {tables.map(({ tables: dataTable, key, title }) => ( -
+ {tables.map(({ table, title }) => ( +
{ diff --git a/src/plugins/vis_type_table/public/table_vis_fn.test.ts b/src/plugins/vis_type_table/public/table_vis_fn.test.ts index ea8030688caed..31b440ffb642f 100644 --- a/src/plugins/vis_type_table/public/table_vis_fn.test.ts +++ b/src/plugins/vis_type_table/public/table_vis_fn.test.ts @@ -7,11 +7,11 @@ */ import { createTableVisFn } from './table_vis_fn'; -import { tableVisResponseHandler } from './table_vis_response_handler'; +import { tableVisResponseHandler } from './utils'; import { functionWrapper } from '../../expressions/common/expression_functions/specs/tests/utils'; -jest.mock('./table_vis_response_handler', () => ({ +jest.mock('./utils', () => ({ tableVisResponseHandler: jest.fn().mockReturnValue({ tables: [{ columns: [], rows: [] }], }), @@ -62,6 +62,6 @@ describe('interpreter/functions#table', () => { it('calls response handler with correct values', async () => { await fn(context, { visConfig: JSON.stringify(visConfig) }, undefined); expect(tableVisResponseHandler).toHaveBeenCalledTimes(1); - expect(tableVisResponseHandler).toHaveBeenCalledWith(context, visConfig.dimensions); + expect(tableVisResponseHandler).toHaveBeenCalledWith(context, visConfig); }); }); diff --git a/src/plugins/vis_type_table/public/table_vis_fn.ts b/src/plugins/vis_type_table/public/table_vis_fn.ts index 99fee424b8bea..3dd8e81fc2ab2 100644 --- a/src/plugins/vis_type_table/public/table_vis_fn.ts +++ b/src/plugins/vis_type_table/public/table_vis_fn.ts @@ -7,10 +7,10 @@ */ import { i18n } from '@kbn/i18n'; -import { tableVisResponseHandler, TableContext } from './table_vis_response_handler'; import { ExpressionFunctionDefinition, Datatable, Render } from '../../expressions/public'; -import { TableVisConfig } from './types'; +import { TableVisData, TableVisConfig } from './types'; import { VIS_TYPE_TABLE } from '../common'; +import { tableVisResponseHandler } from './utils'; export type Input = Datatable; @@ -19,7 +19,7 @@ interface Arguments { } export interface TableVisRenderValue { - visData: TableContext; + visData: TableVisData; visType: typeof VIS_TYPE_TABLE; visConfig: TableVisConfig; } @@ -47,7 +47,7 @@ export const createTableVisFn = (): TableExpressionFunctionDefinition => ({ }, fn(input, args, handlers) { const visConfig = args.visConfig && JSON.parse(args.visConfig); - const convertedData = tableVisResponseHandler(input, visConfig.dimensions); + const convertedData = tableVisResponseHandler(input, visConfig); if (handlers?.inspectorAdapters?.tables) { handlers.inspectorAdapters.tables.logDatatable('default', input); diff --git a/src/plugins/vis_type_table/public/table_vis_response_handler.ts b/src/plugins/vis_type_table/public/table_vis_response_handler.ts deleted file mode 100644 index dbd01f94bd3c5..0000000000000 --- a/src/plugins/vis_type_table/public/table_vis_response_handler.ts +++ /dev/null @@ -1,90 +0,0 @@ -/* - * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one - * or more contributor license agreements. Licensed under the Elastic License - * and the Server Side Public License, v 1; you may not use this file except in - * compliance with, at your election, the Elastic License or the Server Side - * Public License, v 1. - */ - -import { Required } from '@kbn/utility-types'; - -import { getFormatService } from './services'; -import { Input } from './table_vis_fn'; -import { Dimensions } from './types'; - -export interface TableContext { - table?: Table; - tables: TableGroup[]; - direction?: 'row' | 'column'; -} - -export interface TableGroup { - table: Input; - tables: Table[]; - title: string; - name: string; - key: string | number; - column: number; - row: number; -} - -export interface Table { - columns: Input['columns']; - rows: Input['rows']; -} - -export function tableVisResponseHandler(input: Input, dimensions: Dimensions): TableContext { - let table: Table | undefined; - let tables: TableGroup[] = []; - let direction: TableContext['direction']; - - const split = dimensions.splitColumn || dimensions.splitRow; - - if (split) { - tables = []; - direction = dimensions.splitRow ? 'row' : 'column'; - const splitColumnIndex = split[0].accessor; - const splitColumnFormatter = getFormatService().deserialize(split[0].format); - const splitColumn = input.columns[splitColumnIndex]; - const splitMap: { [key: string]: number } = {}; - let splitIndex = 0; - - input.rows.forEach((row, rowIndex) => { - const splitValue: string | number = row[splitColumn.id]; - - if (!splitMap.hasOwnProperty(splitValue)) { - splitMap[splitValue] = splitIndex++; - const tableGroup: Required = { - title: `${splitColumnFormatter.convert(splitValue)}: ${splitColumn.name}`, - name: splitColumn.name, - key: splitValue, - column: splitColumnIndex, - row: rowIndex, - table: input, - tables: [], - }; - - tableGroup.tables.push({ - columns: input.columns, - rows: [], - }); - - tables.push(tableGroup); - } - - const tableIndex = splitMap[splitValue]; - tables[tableIndex].tables[0].rows.push(row); - }); - } else { - table = { - columns: input.columns, - rows: input.rows, - }; - } - - return { - direction, - table, - tables, - }; -} diff --git a/src/plugins/vis_type_table/public/types.ts b/src/plugins/vis_type_table/public/types.ts index 03cf8bb3395d6..61ba7739b9cb1 100644 --- a/src/plugins/vis_type_table/public/types.ts +++ b/src/plugins/vis_type_table/public/types.ts @@ -7,6 +7,7 @@ */ import { IFieldFormat } from 'src/plugins/data/public'; +import { DatatableColumn, DatatableRow } from 'src/plugins/expressions'; import { SchemaConfig } from 'src/plugins/visualizations/public'; import { TableVisParams } from '../common'; @@ -43,7 +44,6 @@ export interface TableVisConfig extends TableVisParams { } export interface FormattedColumn { - id: string; title: string; formatter: IFieldFormat; formattedTotal?: string | number; @@ -51,3 +51,24 @@ export interface FormattedColumn { sumTotal?: number; total?: number; } + +export interface FormattedColumns { + [key: string]: FormattedColumn; +} + +export interface TableContext { + columns: DatatableColumn[]; + rows: DatatableRow[]; + formattedColumns: FormattedColumns; +} + +export interface TableGroup { + table: TableContext; + title: string; +} + +export interface TableVisData { + table?: TableContext; + tables: TableGroup[]; + direction?: 'row' | 'column'; +} diff --git a/src/plugins/vis_type_table/public/utils/add_percentage_column.ts b/src/plugins/vis_type_table/public/utils/add_percentage_column.ts index 0e3879255dd06..11528c76ee300 100644 --- a/src/plugins/vis_type_table/public/utils/add_percentage_column.ts +++ b/src/plugins/vis_type_table/public/utils/add_percentage_column.ts @@ -6,48 +6,59 @@ * Public License, v 1. */ +import { findIndex } from 'lodash'; import { i18n } from '@kbn/i18n'; -import { DatatableRow } from 'src/plugins/expressions'; +import { DatatableColumn } from 'src/plugins/expressions'; import { getFormatService } from '../services'; -import { FormattedColumn } from '../types'; -import { Table } from '../table_vis_response_handler'; +import { FormattedColumns, TableContext } from '../types'; -function insertColumn(arr: FormattedColumn[], index: number, col: FormattedColumn) { +function insertColumn(arr: DatatableColumn[], index: number, col: DatatableColumn) { const newArray = [...arr]; newArray.splice(index + 1, 0, col); return newArray; } /** - * @param columns - the formatted columns that will be displayed - * @param title - the title of the column to add to - * @param rows - the row data for the columns - * @param insertAtIndex - the index to insert the percentage column at - * @returns cols and rows for the table to render now included percentage column(s) + * Adds a brand new column with percentages of selected column to existing data table */ -export function addPercentageColumn( - columns: FormattedColumn[], - title: string, - rows: Table['rows'], - insertAtIndex: number -) { - const { id, sumTotal } = columns[insertAtIndex]; - const newId = `${id}-percents`; +export function addPercentageColumn(table: TableContext, name: string) { + const { columns, rows, formattedColumns } = table; + const insertAtIndex = findIndex(columns, { name }); + // column to show percentage for was removed + if (insertAtIndex < 0) return table; + + const { id } = columns[insertAtIndex]; + const { sumTotal } = formattedColumns[id]; + const percentageColumnId = `${id}-percents`; const formatter = getFormatService().deserialize({ id: 'percent' }); - const i18nTitle = i18n.translate('visTypeTable.params.percentageTableColumnName', { + const percentageColumnName = i18n.translate('visTypeTable.params.percentageTableColumnName', { defaultMessage: '{title} percentages', - values: { title }, + values: { title: name }, }); const newCols = insertColumn(columns, insertAtIndex, { - title: i18nTitle, - id: newId, - formatter, - filterable: false, + name: percentageColumnName, + id: percentageColumnId, + meta: { + type: 'number', + params: { id: 'percent' }, + }, }); - const newRows = rows.map((row) => ({ - [newId]: (row[id] as number) / (sumTotal as number), + const newFormattedColumns: FormattedColumns = { + ...formattedColumns, + [percentageColumnId]: { + title: percentageColumnName, + formatter, + filterable: false, + }, + }; + const newRows = rows.map((row) => ({ + [percentageColumnId]: (row[id] as number) / (sumTotal as number), ...row, })); - return { cols: newCols, rows: newRows }; + return { + columns: newCols, + rows: newRows, + formattedColumns: newFormattedColumns, + }; } diff --git a/src/plugins/vis_type_table/public/utils/create_formatted_table.ts b/src/plugins/vis_type_table/public/utils/create_formatted_table.ts new file mode 100644 index 0000000000000..9dbb6c0c76e25 --- /dev/null +++ b/src/plugins/vis_type_table/public/utils/create_formatted_table.ts @@ -0,0 +1,97 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License + * and the Server Side Public License, v 1; you may not use this file except in + * compliance with, at your election, the Elastic License or the Server Side + * Public License, v 1. + */ + +import { chain } from 'lodash'; +import { Datatable } from 'src/plugins/expressions'; +import { getFormatService } from '../services'; +import { FormattedColumn, FormattedColumns, TableVisConfig, TableContext } from '../types'; +import { AggTypes } from '../../common'; + +export const createFormattedTable = ( + table: Datatable | TableContext, + visConfig: TableVisConfig +) => { + const { buckets, metrics } = visConfig.dimensions; + + const formattedColumns = table.columns.reduce((acc, col, i) => { + const isBucket = buckets.find(({ accessor }) => accessor === i); + const dimension = isBucket || metrics.find(({ accessor }) => accessor === i); + + if (!dimension) return acc; + + const formatter = getFormatService().deserialize(dimension.format); + const formattedColumn: FormattedColumn = { + title: col.name, + formatter, + filterable: !!isBucket, + }; + + const isDate = dimension.format.id === 'date' || dimension.format.params?.id === 'date'; + const allowsNumericalAggregations = formatter.allowsNumericalAggregations; + + if (allowsNumericalAggregations || isDate || visConfig.totalFunc === AggTypes.COUNT) { + const sumOfColumnValues = table.rows.reduce((prev, curr) => { + // some metrics return undefined for some of the values + // derivative is an example of this as it returns undefined in the first row + if (curr[col.id] === undefined) return prev; + return prev + (curr[col.id] as number); + }, 0); + + formattedColumn.sumTotal = sumOfColumnValues; + + switch (visConfig.totalFunc) { + case AggTypes.SUM: { + if (!isDate) { + formattedColumn.formattedTotal = formatter.convert(sumOfColumnValues); + formattedColumn.total = sumOfColumnValues; + } + break; + } + case AggTypes.AVG: { + if (!isDate) { + const total = sumOfColumnValues / table.rows.length; + formattedColumn.formattedTotal = formatter.convert(total); + formattedColumn.total = total; + } + break; + } + case AggTypes.MIN: { + const total = chain(table.rows).map(col.id).min().value() as number; + formattedColumn.formattedTotal = formatter.convert(total); + formattedColumn.total = total; + break; + } + case AggTypes.MAX: { + const total = chain(table.rows).map(col.id).max().value() as number; + formattedColumn.formattedTotal = formatter.convert(total); + formattedColumn.total = total; + break; + } + case AggTypes.COUNT: { + const total = table.rows.length; + formattedColumn.formattedTotal = total; + formattedColumn.total = total; + break; + } + default: + break; + } + } + + acc[col.id] = formattedColumn; + + return acc; + }, {}); + + return { + // filter out columns which are not dimensions + columns: table.columns.filter((col) => formattedColumns[col.id]), + rows: table.rows, + formattedColumns, + }; +}; diff --git a/src/plugins/vis_type_table/public/utils/export_as_csv.ts b/src/plugins/vis_type_table/public/utils/export_as_csv.ts deleted file mode 100644 index 4371a20cfa0da..0000000000000 --- a/src/plugins/vis_type_table/public/utils/export_as_csv.ts +++ /dev/null @@ -1,64 +0,0 @@ -/* - * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one - * or more contributor license agreements. Licensed under the Elastic License - * and the Server Side Public License, v 1; you may not use this file except in - * compliance with, at your election, the Elastic License or the Server Side - * Public License, v 1. - */ - -import { isObject } from 'lodash'; -// @ts-ignore -import { saveAs } from '@elastic/filesaver'; - -import { CoreStart } from 'kibana/public'; -import { DatatableRow } from 'src/plugins/expressions'; -import { CSV_SEPARATOR_SETTING, CSV_QUOTE_VALUES_SETTING } from '../../../share/public'; -import { FormattedColumn } from '../types'; -import { Table } from '../table_vis_response_handler'; - -const nonAlphaNumRE = /[^a-zA-Z0-9]/; -const allDoubleQuoteRE = /"/g; - -interface ToCsvData { - filename?: string; - cols: FormattedColumn[]; - rows: DatatableRow[]; - table: Table; - uiSettings: CoreStart['uiSettings']; -} - -const toCsv = (formatted: boolean, { cols, rows, table, uiSettings }: ToCsvData) => { - const separator = uiSettings.get(CSV_SEPARATOR_SETTING); - const quoteValues = uiSettings.get(CSV_QUOTE_VALUES_SETTING); - - function escape(val: unknown) { - if (!formatted && isObject(val)) val = val.valueOf(); - val = String(val); - if (quoteValues && nonAlphaNumRE.test(val as string)) { - val = '"' + (val as string).replace(allDoubleQuoteRE, '""') + '"'; - } - return val as string; - } - - const csvRows: string[][] = []; - - for (const row of rows) { - const rowArray: string[] = []; - for (const col of cols) { - const value = row[col.id]; - const formattedValue = formatted ? escape(col.formatter.convert(value)) : escape(value); - rowArray.push(formattedValue); - } - csvRows.push(rowArray); - } - - // add headers to the rows - csvRows.unshift(cols.map(({ title }) => escape(title))); - - return csvRows.map((row) => row.join(separator) + '\r\n').join(''); -}; - -export const exportAsCsv = (formatted: boolean, data: ToCsvData) => { - const csv = new Blob([toCsv(formatted, data)], { type: 'text/plain;charset=utf-8' }); - saveAs(csv, `${data.filename || 'unsaved'}.csv`); -}; diff --git a/src/plugins/vis_type_table/public/utils/index.ts b/src/plugins/vis_type_table/public/utils/index.ts index 6a6dda0d12fa3..8731b52a7ba30 100644 --- a/src/plugins/vis_type_table/public/utils/index.ts +++ b/src/plugins/vis_type_table/public/utils/index.ts @@ -7,4 +7,4 @@ */ export * from './use'; -export * from './export_as_csv'; +export * from './table_vis_response_handler'; diff --git a/src/plugins/vis_type_table/public/utils/table_vis_response_handler.ts b/src/plugins/vis_type_table/public/utils/table_vis_response_handler.ts new file mode 100644 index 0000000000000..0a2b8d8180854 --- /dev/null +++ b/src/plugins/vis_type_table/public/utils/table_vis_response_handler.ts @@ -0,0 +1,75 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License + * and the Server Side Public License, v 1; you may not use this file except in + * compliance with, at your election, the Elastic License or the Server Side + * Public License, v 1. + */ + +import { Datatable } from 'src/plugins/expressions'; +import { getFormatService } from '../services'; +import { TableVisData, TableGroup, TableVisConfig, TableContext } from '../types'; +import { addPercentageColumn } from './add_percentage_column'; +import { createFormattedTable } from './create_formatted_table'; + +/** + * Converts datatable input from response into appropriate format for consuming renderer + */ +export function tableVisResponseHandler(input: Datatable, visConfig: TableVisConfig): TableVisData { + const tables: TableGroup[] = []; + let table: TableContext | undefined; + let direction: TableVisData['direction']; + + const split = visConfig.dimensions.splitColumn || visConfig.dimensions.splitRow; + + if (split) { + direction = visConfig.dimensions.splitRow ? 'row' : 'column'; + const splitColumnIndex = split[0].accessor; + const splitColumnFormatter = getFormatService().deserialize(split[0].format); + const splitColumn = input.columns[splitColumnIndex]; + const columns = input.columns.filter((c, idx) => idx !== splitColumnIndex); + const splitMap: { [key: string]: number } = {}; + let splitIndex = 0; + + input.rows.forEach((row) => { + const splitValue: string | number = row[splitColumn.id]; + + if (!splitMap.hasOwnProperty(splitValue)) { + splitMap[splitValue] = splitIndex++; + const tableGroup: TableGroup = { + title: `${splitColumnFormatter.convert(splitValue)}: ${splitColumn.name}`, + table: { + columns, + rows: [], + formattedColumns: {}, + }, + }; + + tables.push(tableGroup); + } + + const tableIndex = splitMap[splitValue]; + tables[tableIndex].table.rows.push(row); + }); + + tables.forEach((tg) => { + tg.table = createFormattedTable({ ...tg.table, columns: input.columns }, visConfig); + + if (visConfig.percentageCol) { + tg.table = addPercentageColumn(tg.table, visConfig.percentageCol); + } + }); + } else { + table = createFormattedTable(input, visConfig); + + if (visConfig.percentageCol) { + table = addPercentageColumn(table, visConfig.percentageCol); + } + } + + return { + direction, + table, + tables, + }; +} diff --git a/src/plugins/vis_type_table/public/utils/use/index.ts b/src/plugins/vis_type_table/public/utils/use/index.ts index 08daf7f28c0e8..9fcc791561046 100644 --- a/src/plugins/vis_type_table/public/utils/use/index.ts +++ b/src/plugins/vis_type_table/public/utils/use/index.ts @@ -6,6 +6,5 @@ * Public License, v 1. */ -export * from './use_formatted_columns'; export * from './use_pagination'; export * from './use_ui_state'; diff --git a/src/plugins/vis_type_table/public/utils/use/use_formatted_columns.ts b/src/plugins/vis_type_table/public/utils/use/use_formatted_columns.ts deleted file mode 100644 index 3a733e7a9a4dc..0000000000000 --- a/src/plugins/vis_type_table/public/utils/use/use_formatted_columns.ts +++ /dev/null @@ -1,115 +0,0 @@ -/* - * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one - * or more contributor license agreements. Licensed under the Elastic License - * and the Server Side Public License, v 1; you may not use this file except in - * compliance with, at your election, the Elastic License or the Server Side - * Public License, v 1. - */ - -import { useMemo } from 'react'; -import { chain, findIndex } from 'lodash'; - -import { AggTypes } from '../../../common'; -import { Table } from '../../table_vis_response_handler'; -import { FormattedColumn, TableVisConfig } from '../../types'; -import { getFormatService } from '../../services'; -import { addPercentageColumn } from '../add_percentage_column'; - -export const useFormattedColumnsAndRows = (table: Table, visConfig: TableVisConfig) => { - const { formattedColumns: columns, formattedRows: rows } = useMemo(() => { - const { buckets, metrics } = visConfig.dimensions; - let formattedRows = table.rows; - - let formattedColumns = table.columns - .map((col, i) => { - const isBucket = buckets.find(({ accessor }) => accessor === i); - const dimension = isBucket || metrics.find(({ accessor }) => accessor === i); - - if (!dimension) return undefined; - - const formatter = getFormatService().deserialize(dimension.format); - const formattedColumn: FormattedColumn = { - id: col.id, - title: col.name, - formatter, - filterable: !!isBucket, - }; - - const isDate = dimension.format.id === 'date' || dimension.format.params?.id === 'date'; - const allowsNumericalAggregations = formatter.allowsNumericalAggregations; - - if (allowsNumericalAggregations || isDate || visConfig.totalFunc === AggTypes.COUNT) { - const sumOfColumnValues = table.rows.reduce((prev, curr) => { - // some metrics return undefined for some of the values - // derivative is an example of this as it returns undefined in the first row - if (curr[col.id] === undefined) return prev; - return prev + (curr[col.id] as number); - }, 0); - - formattedColumn.sumTotal = sumOfColumnValues; - - switch (visConfig.totalFunc) { - case AggTypes.SUM: { - if (!isDate) { - formattedColumn.formattedTotal = formatter.convert(sumOfColumnValues); - formattedColumn.total = sumOfColumnValues; - } - break; - } - case AggTypes.AVG: { - if (!isDate) { - const total = sumOfColumnValues / table.rows.length; - formattedColumn.formattedTotal = formatter.convert(total); - formattedColumn.total = total; - } - break; - } - case AggTypes.MIN: { - const total = chain(table.rows).map(col.id).min().value() as number; - formattedColumn.formattedTotal = formatter.convert(total); - formattedColumn.total = total; - break; - } - case AggTypes.MAX: { - const total = chain(table.rows).map(col.id).max().value() as number; - formattedColumn.formattedTotal = formatter.convert(total); - formattedColumn.total = total; - break; - } - case AggTypes.COUNT: { - const total = table.rows.length; - formattedColumn.formattedTotal = total; - formattedColumn.total = total; - break; - } - default: - break; - } - } - - return formattedColumn; - }) - .filter((column): column is FormattedColumn => !!column); - - if (visConfig.percentageCol) { - const insertAtIndex = findIndex(formattedColumns, { title: visConfig.percentageCol }); - - // column to show percentage for was removed - if (insertAtIndex < 0) return { formattedColumns, formattedRows }; - - const { cols, rows: rowsWithPercentage } = addPercentageColumn( - formattedColumns, - visConfig.percentageCol, - table.rows, - insertAtIndex - ); - - formattedRows = rowsWithPercentage; - formattedColumns = cols; - } - - return { formattedColumns, formattedRows }; - }, [table, visConfig.dimensions, visConfig.percentageCol, visConfig.totalFunc]); - - return { columns, rows }; -}; diff --git a/test/api_integration/apis/saved_objects_management/relationships.ts b/test/api_integration/apis/saved_objects_management/relationships.ts index 185c6ded01de4..6dea461f790e8 100644 --- a/test/api_integration/apis/saved_objects_management/relationships.ts +++ b/test/api_integration/apis/saved_objects_management/relationships.ts @@ -14,23 +14,32 @@ export default function ({ getService }: FtrProviderContext) { const supertest = getService('supertest'); const esArchiver = getService('esArchiver'); - const responseSchema = schema.arrayOf( - schema.object({ - id: schema.string(), - type: schema.string(), - relationship: schema.oneOf([schema.literal('parent'), schema.literal('child')]), - meta: schema.object({ - title: schema.string(), - icon: schema.string(), - editUrl: schema.string(), - inAppUrl: schema.object({ - path: schema.string(), - uiCapabilitiesPath: schema.string(), - }), - namespaceType: schema.string(), + const relationSchema = schema.object({ + id: schema.string(), + type: schema.string(), + relationship: schema.oneOf([schema.literal('parent'), schema.literal('child')]), + meta: schema.object({ + title: schema.string(), + icon: schema.string(), + editUrl: schema.string(), + inAppUrl: schema.object({ + path: schema.string(), + uiCapabilitiesPath: schema.string(), }), - }) - ); + namespaceType: schema.string(), + }), + }); + const invalidRelationSchema = schema.object({ + id: schema.string(), + type: schema.string(), + relationship: schema.oneOf([schema.literal('parent'), schema.literal('child')]), + error: schema.string(), + }); + + const responseSchema = schema.object({ + relations: schema.arrayOf(relationSchema), + invalidRelations: schema.arrayOf(invalidRelationSchema), + }); describe('relationships', () => { before(async () => { @@ -64,7 +73,7 @@ export default function ({ getService }: FtrProviderContext) { .get(relationshipsUrl('search', '960372e0-3224-11e8-a572-ffca06da1357')) .expect(200); - expect(resp.body).to.eql([ + expect(resp.body.relations).to.eql([ { id: '8963ca30-3224-11e8-a572-ffca06da1357', type: 'index-pattern', @@ -108,7 +117,7 @@ export default function ({ getService }: FtrProviderContext) { ) .expect(200); - expect(resp.body).to.eql([ + expect(resp.body.relations).to.eql([ { id: '8963ca30-3224-11e8-a572-ffca06da1357', type: 'index-pattern', @@ -145,8 +154,7 @@ export default function ({ getService }: FtrProviderContext) { ]); }); - // TODO: https://github.com/elastic/kibana/issues/19713 causes this test to fail. - it.skip('should return 404 if search finds no results', async () => { + it('should return 404 if search finds no results', async () => { await supertest .get(relationshipsUrl('search', 'xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx')) .expect(404); @@ -169,7 +177,7 @@ export default function ({ getService }: FtrProviderContext) { .get(relationshipsUrl('dashboard', 'b70c7ae0-3224-11e8-a572-ffca06da1357')) .expect(200); - expect(resp.body).to.eql([ + expect(resp.body.relations).to.eql([ { id: 'add810b0-3224-11e8-a572-ffca06da1357', type: 'visualization', @@ -210,7 +218,7 @@ export default function ({ getService }: FtrProviderContext) { .get(relationshipsUrl('dashboard', 'b70c7ae0-3224-11e8-a572-ffca06da1357', ['search'])) .expect(200); - expect(resp.body).to.eql([ + expect(resp.body.relations).to.eql([ { id: 'add810b0-3224-11e8-a572-ffca06da1357', type: 'visualization', @@ -246,8 +254,7 @@ export default function ({ getService }: FtrProviderContext) { ]); }); - // TODO: https://github.com/elastic/kibana/issues/19713 causes this test to fail. - it.skip('should return 404 if dashboard finds no results', async () => { + it('should return 404 if dashboard finds no results', async () => { await supertest .get(relationshipsUrl('dashboard', 'xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx')) .expect(404); @@ -270,7 +277,7 @@ export default function ({ getService }: FtrProviderContext) { .get(relationshipsUrl('visualization', 'a42c0580-3224-11e8-a572-ffca06da1357')) .expect(200); - expect(resp.body).to.eql([ + expect(resp.body.relations).to.eql([ { id: '960372e0-3224-11e8-a572-ffca06da1357', type: 'search', @@ -313,7 +320,7 @@ export default function ({ getService }: FtrProviderContext) { ) .expect(200); - expect(resp.body).to.eql([ + expect(resp.body.relations).to.eql([ { id: '960372e0-3224-11e8-a572-ffca06da1357', type: 'search', @@ -356,7 +363,7 @@ export default function ({ getService }: FtrProviderContext) { .get(relationshipsUrl('index-pattern', '8963ca30-3224-11e8-a572-ffca06da1357')) .expect(200); - expect(resp.body).to.eql([ + expect(resp.body.relations).to.eql([ { id: '960372e0-3224-11e8-a572-ffca06da1357', type: 'search', @@ -399,7 +406,7 @@ export default function ({ getService }: FtrProviderContext) { ) .expect(200); - expect(resp.body).to.eql([ + expect(resp.body.relations).to.eql([ { id: '960372e0-3224-11e8-a572-ffca06da1357', type: 'search', @@ -425,5 +432,48 @@ export default function ({ getService }: FtrProviderContext) { .expect(404); }); }); + + describe('invalid references', () => { + it('should validate the response schema', async () => { + const resp = await supertest.get(relationshipsUrl('dashboard', 'invalid-refs')).expect(200); + + expect(() => { + responseSchema.validate(resp.body); + }).not.to.throwError(); + }); + + it('should return the invalid relations', async () => { + const resp = await supertest.get(relationshipsUrl('dashboard', 'invalid-refs')).expect(200); + + expect(resp.body).to.eql({ + invalidRelations: [ + { + error: 'Saved object [visualization/invalid-vis] not found', + id: 'invalid-vis', + relationship: 'child', + type: 'visualization', + }, + ], + relations: [ + { + id: 'add810b0-3224-11e8-a572-ffca06da1357', + meta: { + editUrl: + '/management/kibana/objects/savedVisualizations/add810b0-3224-11e8-a572-ffca06da1357', + icon: 'visualizeApp', + inAppUrl: { + path: '/app/visualize#/edit/add810b0-3224-11e8-a572-ffca06da1357', + uiCapabilitiesPath: 'visualize.show', + }, + namespaceType: 'single', + title: 'Visualization', + }, + relationship: 'child', + type: 'visualization', + }, + ], + }); + }); + }); }); } diff --git a/test/api_integration/fixtures/es_archiver/management/saved_objects/relationships/data.json b/test/api_integration/fixtures/es_archiver/management/saved_objects/relationships/data.json new file mode 100644 index 0000000000000..21d84c4b55e55 --- /dev/null +++ b/test/api_integration/fixtures/es_archiver/management/saved_objects/relationships/data.json @@ -0,0 +1,190 @@ +{ + "type": "doc", + "value": { + "index": ".kibana", + "id": "timelion-sheet:190f3e90-2ec3-11e8-ba48-69fc4e41e1f6", + "source": { + "type": "timelion-sheet", + "updated_at": "2018-03-23T17:53:30.872Z", + "timelion-sheet": { + "title": "New TimeLion Sheet", + "hits": 0, + "description": "", + "timelion_sheet": [ + ".es(*)" + ], + "timelion_interval": "auto", + "timelion_chart_height": 275, + "timelion_columns": 2, + "timelion_rows": 2, + "version": 1 + } + } + } +} + +{ + "type": "doc", + "value": { + "index": ".kibana", + "id": "index-pattern:8963ca30-3224-11e8-a572-ffca06da1357", + "source": { + "type": "index-pattern", + "updated_at": "2018-03-28T01:08:34.290Z", + "index-pattern": { + "title": "saved_objects*", + "fields": "[{\"name\":\"_id\",\"type\":\"string\",\"count\":0,\"scripted\":false,\"searchable\":true,\"aggregatable\":true,\"readFromDocValues\":false},{\"name\":\"_index\",\"type\":\"string\",\"count\":0,\"scripted\":false,\"searchable\":true,\"aggregatable\":true,\"readFromDocValues\":false},{\"name\":\"_score\",\"type\":\"number\",\"count\":0,\"scripted\":false,\"searchable\":false,\"aggregatable\":false,\"readFromDocValues\":false},{\"name\":\"_source\",\"type\":\"_source\",\"count\":0,\"scripted\":false,\"searchable\":false,\"aggregatable\":false,\"readFromDocValues\":false},{\"name\":\"_type\",\"type\":\"string\",\"count\":0,\"scripted\":false,\"searchable\":true,\"aggregatable\":true,\"readFromDocValues\":false},{\"name\":\"id\",\"type\":\"number\",\"count\":0,\"scripted\":false,\"searchable\":true,\"aggregatable\":true,\"readFromDocValues\":true}]" + } + } + } +} + +{ + "type": "doc", + "value": { + "index": ".kibana", + "id": "config:7.0.0-alpha1", + "source": { + "type": "config", + "updated_at": "2018-03-28T01:08:39.248Z", + "config": { + "buildNum": 8467, + "telemetry:optIn": false, + "defaultIndex": "8963ca30-3224-11e8-a572-ffca06da1357" + } + } + } +} + +{ + "type": "doc", + "value": { + "index": ".kibana", + "id": "search:960372e0-3224-11e8-a572-ffca06da1357", + "source": { + "type": "search", + "updated_at": "2018-03-28T01:08:55.182Z", + "search": { + "title": "OneRecord", + "description": "", + "hits": 0, + "columns": [ + "_source" + ], + "sort": [ + "_score", + "desc" + ], + "version": 1, + "kibanaSavedObjectMeta": { + "searchSourceJSON": "{\"index\":\"8963ca30-3224-11e8-a572-ffca06da1357\",\"highlightAll\":true,\"version\":true,\"query\":{\"query\":\"id:3\",\"language\":\"lucene\"},\"filter\":[]}" + } + } + } + } +} + +{ + "type": "doc", + "value": { + "index": ".kibana", + "id": "visualization:a42c0580-3224-11e8-a572-ffca06da1357", + "source": { + "type": "visualization", + "updated_at": "2018-03-28T01:09:18.936Z", + "visualization": { + "title": "VisualizationFromSavedSearch", + "visState": "{\"title\":\"VisualizationFromSavedSearch\",\"type\":\"table\",\"params\":{\"perPage\":10,\"showPartialRows\":false,\"showMeticsAtAllLevels\":false,\"sort\":{\"columnIndex\":null,\"direction\":null},\"showTotal\":false,\"totalFunc\":\"sum\"},\"aggs\":[{\"id\":\"1\",\"enabled\":true,\"type\":\"count\",\"schema\":\"metric\",\"params\":{}}]}", + "uiStateJSON": "{\"vis\":{\"params\":{\"sort\":{\"columnIndex\":null,\"direction\":null}}}}", + "description": "", + "savedSearchId": "960372e0-3224-11e8-a572-ffca06da1357", + "version": 1, + "kibanaSavedObjectMeta": { + "searchSourceJSON": "{\"filter\":[],\"query\":{\"query\":\"\",\"language\":\"lucene\"}}" + } + } + } + } +} + +{ + "type": "doc", + "value": { + "index": ".kibana", + "id": "visualization:add810b0-3224-11e8-a572-ffca06da1357", + "source": { + "type": "visualization", + "updated_at": "2018-03-28T01:09:35.163Z", + "visualization": { + "title": "Visualization", + "visState": "{\"title\":\"Visualization\",\"type\":\"metric\",\"params\":{\"addTooltip\":true,\"addLegend\":false,\"type\":\"metric\",\"metric\":{\"percentageMode\":false,\"useRanges\":false,\"colorSchema\":\"Green to Red\",\"metricColorMode\":\"None\",\"colorsRange\":[{\"from\":0,\"to\":10000}],\"labels\":{\"show\":true},\"invertColors\":false,\"style\":{\"bgFill\":\"#000\",\"bgColor\":false,\"labelColor\":false,\"subText\":\"\",\"fontSize\":60}}},\"aggs\":[{\"id\":\"1\",\"enabled\":true,\"type\":\"count\",\"schema\":\"metric\",\"params\":{}}]}", + "uiStateJSON": "{}", + "description": "", + "version": 1, + "kibanaSavedObjectMeta": { + "searchSourceJSON": "{\"index\":\"8963ca30-3224-11e8-a572-ffca06da1357\",\"filter\":[],\"query\":{\"query\":\"\",\"language\":\"lucene\"}}" + } + } + } + } +} + +{ + "type": "doc", + "value": { + "index": ".kibana", + "id": "dashboard:b70c7ae0-3224-11e8-a572-ffca06da1357", + "source": { + "type": "dashboard", + "updated_at": "2018-03-28T01:09:50.606Z", + "dashboard": { + "title": "Dashboard", + "hits": 0, + "description": "", + "panelsJSON": "[{\"gridData\":{\"w\":24,\"h\":15,\"x\":0,\"y\":0,\"i\":\"1\"},\"version\":\"7.0.0-alpha1\",\"panelIndex\":\"1\",\"type\":\"visualization\",\"id\":\"add810b0-3224-11e8-a572-ffca06da1357\",\"embeddableConfig\":{}},{\"gridData\":{\"w\":24,\"h\":15,\"x\":24,\"y\":0,\"i\":\"2\"},\"version\":\"7.0.0-alpha1\",\"panelIndex\":\"2\",\"type\":\"visualization\",\"id\":\"a42c0580-3224-11e8-a572-ffca06da1357\",\"embeddableConfig\":{}}]", + "optionsJSON": "{\"darkTheme\":false,\"useMargins\":true,\"hidePanelTitles\":false}", + "version": 1, + "timeRestore": false, + "kibanaSavedObjectMeta": { + "searchSourceJSON": "{\"query\":{\"query\":\"\",\"language\":\"lucene\"},\"filter\":[],\"highlightAll\":true,\"version\":true}" + } + } + } + } +} + +{ + "type": "doc", + "value": { + "index": ".kibana", + "id": "dashboard:invalid-refs", + "source": { + "type": "dashboard", + "updated_at": "2018-03-28T01:09:50.606Z", + "dashboard": { + "title": "Dashboard", + "hits": 0, + "description": "", + "panelsJSON": "[]", + "optionsJSON": "{}", + "version": 1, + "timeRestore": false, + "kibanaSavedObjectMeta": { + "searchSourceJSON": "{}" + } + }, + "references": [ + { + "type":"visualization", + "id": "add810b0-3224-11e8-a572-ffca06da1357", + "name": "valid-ref" + }, + { + "type":"visualization", + "id": "invalid-vis", + "name": "missing-ref" + } + ] + } + } +} diff --git a/test/api_integration/fixtures/es_archiver/management/saved_objects/relationships/data.json.gz b/test/api_integration/fixtures/es_archiver/management/saved_objects/relationships/data.json.gz deleted file mode 100644 index 0834567abb66b..0000000000000 Binary files a/test/api_integration/fixtures/es_archiver/management/saved_objects/relationships/data.json.gz and /dev/null differ diff --git a/test/api_integration/fixtures/es_archiver/management/saved_objects/relationships/mappings.json b/test/api_integration/fixtures/es_archiver/management/saved_objects/relationships/mappings.json index c670508247b1a..6dd4d198e0f67 100644 --- a/test/api_integration/fixtures/es_archiver/management/saved_objects/relationships/mappings.json +++ b/test/api_integration/fixtures/es_archiver/management/saved_objects/relationships/mappings.json @@ -12,6 +12,20 @@ "mappings": { "dynamic": "strict", "properties": { + "references": { + "properties": { + "id": { + "type": "keyword" + }, + "name": { + "type": "keyword" + }, + "type": { + "type": "keyword" + } + }, + "type": "nested" + }, "config": { "dynamic": "true", "properties": { @@ -280,4 +294,4 @@ } } } -} \ No newline at end of file +} diff --git a/test/functional/apps/saved_objects_management/index.ts b/test/functional/apps/saved_objects_management/index.ts index 9491661de73ef..5e4eaefb7e9d1 100644 --- a/test/functional/apps/saved_objects_management/index.ts +++ b/test/functional/apps/saved_objects_management/index.ts @@ -12,5 +12,6 @@ export default function savedObjectsManagementApp({ loadTestFile }: FtrProviderC describe('saved objects management', function savedObjectsManagementAppTestSuite() { this.tags('ciGroup7'); loadTestFile(require.resolve('./edit_saved_object')); + loadTestFile(require.resolve('./show_relationships')); }); } diff --git a/test/functional/apps/saved_objects_management/show_relationships.ts b/test/functional/apps/saved_objects_management/show_relationships.ts new file mode 100644 index 0000000000000..6f3fb5a4973e2 --- /dev/null +++ b/test/functional/apps/saved_objects_management/show_relationships.ts @@ -0,0 +1,52 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License + * and the Server Side Public License, v 1; you may not use this file except in + * compliance with, at your election, the Elastic License or the Server Side + * Public License, v 1. + */ + +import expect from '@kbn/expect'; +import { FtrProviderContext } from '../../ftr_provider_context'; + +export default function ({ getPageObjects, getService }: FtrProviderContext) { + const esArchiver = getService('esArchiver'); + const PageObjects = getPageObjects(['common', 'settings', 'savedObjects']); + + describe('saved objects relationships flyout', () => { + beforeEach(async () => { + await esArchiver.load('saved_objects_management/show_relationships'); + }); + + afterEach(async () => { + await esArchiver.unload('saved_objects_management/show_relationships'); + }); + + it('displays the invalid references', async () => { + await PageObjects.settings.navigateTo(); + await PageObjects.settings.clickKibanaSavedObjects(); + + const objects = await PageObjects.savedObjects.getRowTitles(); + expect(objects.includes('Dashboard with missing refs')).to.be(true); + + await PageObjects.savedObjects.clickRelationshipsByTitle('Dashboard with missing refs'); + + const invalidRelations = await PageObjects.savedObjects.getInvalidRelations(); + + expect(invalidRelations).to.eql([ + { + error: 'Saved object [visualization/missing-vis-ref] not found', + id: 'missing-vis-ref', + relationship: 'Child', + type: 'visualization', + }, + { + error: 'Saved object [dashboard/missing-dashboard-ref] not found', + id: 'missing-dashboard-ref', + relationship: 'Child', + type: 'dashboard', + }, + ]); + }); + }); +} diff --git a/test/functional/apps/timelion/_expression_typeahead.js b/test/functional/apps/timelion/_expression_typeahead.js index 744f8de15e767..3db5cb48dd38b 100644 --- a/test/functional/apps/timelion/_expression_typeahead.js +++ b/test/functional/apps/timelion/_expression_typeahead.js @@ -75,18 +75,18 @@ export default function ({ getPageObjects }) { await PageObjects.timelion.updateExpression(',split'); await PageObjects.timelion.clickSuggestion(); const suggestions = await PageObjects.timelion.getSuggestionItemsText(); - expect(suggestions.length).to.eql(51); + expect(suggestions.length).not.to.eql(0); expect(suggestions[0].includes('@message.raw')).to.eql(true); - await PageObjects.timelion.clickSuggestion(10, 2000); + await PageObjects.timelion.clickSuggestion(10); }); it('should show field suggestions for metric argument when index pattern set', async () => { await PageObjects.timelion.updateExpression(',metric'); await PageObjects.timelion.clickSuggestion(); await PageObjects.timelion.updateExpression('avg:'); - await PageObjects.timelion.clickSuggestion(0, 2000); + await PageObjects.timelion.clickSuggestion(0); const suggestions = await PageObjects.timelion.getSuggestionItemsText(); - expect(suggestions.length).to.eql(2); + expect(suggestions.length).not.to.eql(0); expect(suggestions[0].includes('avg:bytes')).to.eql(true); }); }); diff --git a/test/functional/fixtures/es_archiver/saved_objects_management/show_relationships/data.json b/test/functional/fixtures/es_archiver/saved_objects_management/show_relationships/data.json new file mode 100644 index 0000000000000..4d5b969a3c931 --- /dev/null +++ b/test/functional/fixtures/es_archiver/saved_objects_management/show_relationships/data.json @@ -0,0 +1,36 @@ +{ + "type": "doc", + "value": { + "index": ".kibana", + "type": "doc", + "id": "dashboard:dash-with-missing-refs", + "source": { + "dashboard": { + "title": "Dashboard with missing refs", + "hits": 0, + "description": "", + "panelsJSON": "[]", + "optionsJSON": "{}", + "version": 1, + "timeRestore": false, + "kibanaSavedObjectMeta": { + "searchSourceJSON": "{}" + } + }, + "type": "dashboard", + "references": [ + { + "type": "visualization", + "id": "missing-vis-ref", + "name": "some missing ref" + }, + { + "type": "dashboard", + "id": "missing-dashboard-ref", + "name": "some other missing ref" + } + ], + "updated_at": "2019-01-22T19:32:47.232Z" + } + } +} diff --git a/test/functional/fixtures/es_archiver/saved_objects_management/show_relationships/mappings.json b/test/functional/fixtures/es_archiver/saved_objects_management/show_relationships/mappings.json new file mode 100644 index 0000000000000..d53e6c96e883e --- /dev/null +++ b/test/functional/fixtures/es_archiver/saved_objects_management/show_relationships/mappings.json @@ -0,0 +1,473 @@ +{ + "type": "index", + "value": { + "index": ".kibana", + "settings": { + "index": { + "number_of_shards": "1", + "auto_expand_replicas": "0-1", + "number_of_replicas": "0" + } + }, + "mappings": { + "dynamic": "strict", + "properties": { + "references": { + "properties": { + "id": { + "type": "keyword" + }, + "name": { + "type": "keyword" + }, + "type": { + "type": "keyword" + } + }, + "type": "nested" + }, + "apm-telemetry": { + "properties": { + "has_any_services": { + "type": "boolean" + }, + "services_per_agent": { + "properties": { + "go": { + "type": "long", + "null_value": 0 + }, + "java": { + "type": "long", + "null_value": 0 + }, + "js-base": { + "type": "long", + "null_value": 0 + }, + "nodejs": { + "type": "long", + "null_value": 0 + }, + "python": { + "type": "long", + "null_value": 0 + }, + "ruby": { + "type": "long", + "null_value": 0 + } + } + } + } + }, + "canvas-workpad": { + "dynamic": "false", + "properties": { + "@created": { + "type": "date" + }, + "@timestamp": { + "type": "date" + }, + "id": { + "type": "text", + "index": false + }, + "name": { + "type": "text", + "fields": { + "keyword": { + "type": "keyword" + } + } + } + } + }, + "config": { + "dynamic": "true", + "properties": { + "accessibility:disableAnimations": { + "type": "boolean" + }, + "buildNum": { + "type": "keyword" + }, + "dateFormat:tz": { + "type": "text", + "fields": { + "keyword": { + "type": "keyword", + "ignore_above": 256 + } + } + }, + "defaultIndex": { + "type": "text", + "fields": { + "keyword": { + "type": "keyword", + "ignore_above": 256 + } + } + }, + "telemetry:optIn": { + "type": "boolean" + } + } + }, + "dashboard": { + "properties": { + "description": { + "type": "text" + }, + "hits": { + "type": "integer" + }, + "kibanaSavedObjectMeta": { + "properties": { + "searchSourceJSON": { + "type": "text" + } + } + }, + "optionsJSON": { + "type": "text" + }, + "panelsJSON": { + "type": "text" + }, + "refreshInterval": { + "properties": { + "display": { + "type": "keyword" + }, + "pause": { + "type": "boolean" + }, + "section": { + "type": "integer" + }, + "value": { + "type": "integer" + } + } + }, + "timeFrom": { + "type": "keyword" + }, + "timeRestore": { + "type": "boolean" + }, + "timeTo": { + "type": "keyword" + }, + "title": { + "type": "text" + }, + "uiStateJSON": { + "type": "text" + }, + "version": { + "type": "integer" + } + } + }, + "map": { + "properties": { + "bounds": { + "type": "geo_shape", + "tree": "quadtree" + }, + "description": { + "type": "text" + }, + "layerListJSON": { + "type": "text" + }, + "mapStateJSON": { + "type": "text" + }, + "title": { + "type": "text" + }, + "uiStateJSON": { + "type": "text" + }, + "version": { + "type": "integer" + } + } + }, + "graph-workspace": { + "properties": { + "description": { + "type": "text" + }, + "kibanaSavedObjectMeta": { + "properties": { + "searchSourceJSON": { + "type": "text" + } + } + }, + "numLinks": { + "type": "integer" + }, + "numVertices": { + "type": "integer" + }, + "title": { + "type": "text" + }, + "version": { + "type": "integer" + }, + "wsState": { + "type": "text" + } + } + }, + "index-pattern": { + "properties": { + "fieldFormatMap": { + "type": "text" + }, + "fields": { + "type": "text" + }, + "intervalName": { + "type": "keyword" + }, + "notExpandable": { + "type": "boolean" + }, + "sourceFilters": { + "type": "text" + }, + "timeFieldName": { + "type": "keyword" + }, + "title": { + "type": "text" + }, + "type": { + "type": "keyword" + }, + "typeMeta": { + "type": "keyword" + } + } + }, + "kql-telemetry": { + "properties": { + "optInCount": { + "type": "long" + }, + "optOutCount": { + "type": "long" + } + } + }, + "migrationVersion": { + "dynamic": "true", + "properties": { + "index-pattern": { + "type": "text", + "fields": { + "keyword": { + "type": "keyword", + "ignore_above": 256 + } + } + }, + "space": { + "type": "text", + "fields": { + "keyword": { + "type": "keyword", + "ignore_above": 256 + } + } + } + } + }, + "namespace": { + "type": "keyword" + }, + "search": { + "properties": { + "columns": { + "type": "keyword" + }, + "description": { + "type": "text" + }, + "hits": { + "type": "integer" + }, + "kibanaSavedObjectMeta": { + "properties": { + "searchSourceJSON": { + "type": "text" + } + } + }, + "sort": { + "type": "keyword" + }, + "title": { + "type": "text" + }, + "version": { + "type": "integer" + } + } + }, + "server": { + "properties": { + "uuid": { + "type": "keyword" + } + } + }, + "space": { + "properties": { + "_reserved": { + "type": "boolean" + }, + "color": { + "type": "keyword" + }, + "description": { + "type": "text" + }, + "disabledFeatures": { + "type": "keyword" + }, + "initials": { + "type": "keyword" + }, + "name": { + "type": "text", + "fields": { + "keyword": { + "type": "keyword", + "ignore_above": 2048 + } + } + } + } + }, + "spaceId": { + "type": "keyword" + }, + "telemetry": { + "properties": { + "enabled": { + "type": "boolean" + } + } + }, + "timelion-sheet": { + "properties": { + "description": { + "type": "text" + }, + "hits": { + "type": "integer" + }, + "kibanaSavedObjectMeta": { + "properties": { + "searchSourceJSON": { + "type": "text" + } + } + }, + "timelion_chart_height": { + "type": "integer" + }, + "timelion_columns": { + "type": "integer" + }, + "timelion_interval": { + "type": "keyword" + }, + "timelion_other_interval": { + "type": "keyword" + }, + "timelion_rows": { + "type": "integer" + }, + "timelion_sheet": { + "type": "text" + }, + "title": { + "type": "text" + }, + "version": { + "type": "integer" + } + } + }, + "type": { + "type": "keyword" + }, + "updated_at": { + "type": "date" + }, + "url": { + "properties": { + "accessCount": { + "type": "long" + }, + "accessDate": { + "type": "date" + }, + "createDate": { + "type": "date" + }, + "url": { + "type": "text", + "fields": { + "keyword": { + "type": "keyword", + "ignore_above": 2048 + } + } + } + } + }, + "visualization": { + "properties": { + "description": { + "type": "text" + }, + "kibanaSavedObjectMeta": { + "properties": { + "searchSourceJSON": { + "type": "text" + } + } + }, + "savedSearchId": { + "type": "keyword" + }, + "title": { + "type": "text" + }, + "uiStateJSON": { + "type": "text" + }, + "version": { + "type": "integer" + }, + "visState": { + "type": "text" + } + } + } + } + } + } +} diff --git a/test/functional/page_objects/management/saved_objects_page.ts b/test/functional/page_objects/management/saved_objects_page.ts index 1cdf76ad58ef0..cf162f12df9d9 100644 --- a/test/functional/page_objects/management/saved_objects_page.ts +++ b/test/functional/page_objects/management/saved_objects_page.ts @@ -257,6 +257,22 @@ export function SavedObjectsPageProvider({ getService, getPageObjects }: FtrProv }); } + async getInvalidRelations() { + const rows = await testSubjects.findAll('invalidRelationshipsTableRow'); + return mapAsync(rows, async (row) => { + const objectType = await row.findByTestSubject('relationshipsObjectType'); + const objectId = await row.findByTestSubject('relationshipsObjectId'); + const relationship = await row.findByTestSubject('directRelationship'); + const error = await row.findByTestSubject('relationshipsError'); + return { + type: await objectType.getVisibleText(), + id: await objectId.getVisibleText(), + relationship: await relationship.getVisibleText(), + error: await error.getVisibleText(), + }; + }); + } + async getTableSummary() { const table = await testSubjects.find('savedObjectsTable'); const $ = await table.parseDomContent(); diff --git a/test/functional/services/common/browser.ts b/test/functional/services/common/browser.ts index 635fde6dad720..4a7e82d5b42c0 100644 --- a/test/functional/services/common/browser.ts +++ b/test/functional/services/common/browser.ts @@ -289,13 +289,13 @@ export async function BrowserProvider({ getService }: FtrProviderContext) { } const origin = document.querySelector(arguments[0]); - const target = document.querySelector(arguments[1]); const dragStartEvent = createEvent('dragstart'); dispatchEvent(origin, dragStartEvent); setTimeout(() => { const dropEvent = createEvent('drop'); + const target = document.querySelector(arguments[1]); dispatchEvent(target, dropEvent, dragStartEvent.dataTransfer); const dragEndEvent = createEvent('dragend'); dispatchEvent(origin, dragEndEvent, dropEvent.dataTransfer); diff --git a/test/scripts/jenkins_build_load_testing.sh b/test/scripts/jenkins_build_load_testing.sh index aeb584b106498..659321f1d3975 100755 --- a/test/scripts/jenkins_build_load_testing.sh +++ b/test/scripts/jenkins_build_load_testing.sh @@ -1,5 +1,13 @@ #!/usr/bin/env bash +while getopts s: flag +do + case "${flag}" in + s) simulations=${OPTARG};; + esac +done +echo "Simulation classes: $simulations"; + cd "$KIBANA_DIR" source src/dev/ci_setup/setup_env.sh @@ -25,6 +33,7 @@ echo " -> test setup" source test/scripts/jenkins_test_setup_xpack.sh echo " -> run gatling load testing" +export GATLING_SIMULATIONS="$simulations" node scripts/functional_tests \ - --kibana-install-dir "$KIBANA_INSTALL_DIR" \ - --config test/load/config.ts + --kibana-install-dir "$KIBANA_INSTALL_DIR" \ + --config test/load/config.ts diff --git a/test/tsconfig.json b/test/tsconfig.json index c8e6e69586ca0..1dc58f7b25c24 100644 --- a/test/tsconfig.json +++ b/test/tsconfig.json @@ -4,7 +4,12 @@ "incremental": false, "types": ["node", "flot"] }, - "include": ["**/*", "../typings/elastic__node_crypto.d.ts", "typings/**/*", "../packages/kbn-test/types/ftr_globals/**/*"], + "include": [ + "**/*", + "../typings/elastic__node_crypto.d.ts", + "typings/**/*", + "../packages/kbn-test/types/ftr_globals/**/*" + ], "exclude": ["plugin_functional/plugins/**/*", "interpreter_functional/plugins/**/*"], "references": [ { "path": "../src/core/tsconfig.json" }, @@ -34,5 +39,7 @@ { "path": "../src/plugins/ui_actions/tsconfig.json" }, { "path": "../src/plugins/url_forwarding/tsconfig.json" }, { "path": "../src/plugins/usage_collection/tsconfig.json" }, + { "path": "../src/plugins/index_pattern_management/tsconfig.json" }, + { "path": "../src/plugins/legacy_export/tsconfig.json" } ] } diff --git a/tsconfig.json b/tsconfig.json index d8fb2804242bc..21760919c89e9 100644 --- a/tsconfig.json +++ b/tsconfig.json @@ -27,6 +27,7 @@ "src/plugins/kibana_react/**/*", "src/plugins/kibana_usage_collection/**/*", "src/plugins/kibana_utils/**/*", + "src/plugins/legacy_export/**/*", "src/plugins/management/**/*", "src/plugins/maps_legacy/**/*", "src/plugins/navigation/**/*", @@ -58,6 +59,7 @@ "src/plugins/vis_type_xy/**/*", "src/plugins/visualizations/**/*", "src/plugins/visualize/**/*", + "src/plugins/index_pattern_management/**/*", // In the build we actually exclude **/public/**/* from this config so that // we can run the TSC on both this and the .browser version of this config // file, but if we did it during development IDEs would not be able to find @@ -85,6 +87,7 @@ { "path": "./src/plugins/kibana_react/tsconfig.json" }, { "path": "./src/plugins/kibana_usage_collection/tsconfig.json" }, { "path": "./src/plugins/kibana_utils/tsconfig.json" }, + { "path": "./src/plugins/legacy_export/tsconfig.json" }, { "path": "./src/plugins/management/tsconfig.json" }, { "path": "./src/plugins/maps_legacy/tsconfig.json" }, { "path": "./src/plugins/navigation/tsconfig.json" }, @@ -115,5 +118,6 @@ { "path": "./src/plugins/vis_type_xy/tsconfig.json" }, { "path": "./src/plugins/visualizations/tsconfig.json" }, { "path": "./src/plugins/visualize/tsconfig.json" }, + { "path": "./src/plugins/index_pattern_management/tsconfig.json" }, ] } diff --git a/tsconfig.refs.json b/tsconfig.refs.json index 9a65b385b7820..1d08e764709ca 100644 --- a/tsconfig.refs.json +++ b/tsconfig.refs.json @@ -20,6 +20,7 @@ { "path": "./src/plugins/kibana_react/tsconfig.json" }, { "path": "./src/plugins/kibana_usage_collection/tsconfig.json" }, { "path": "./src/plugins/kibana_utils/tsconfig.json" }, + { "path": "./src/plugins/legacy_export/tsconfig.json" }, { "path": "./src/plugins/management/tsconfig.json" }, { "path": "./src/plugins/maps_legacy/tsconfig.json" }, { "path": "./src/plugins/navigation/tsconfig.json" }, @@ -52,5 +53,6 @@ { "path": "./src/plugins/vis_type_xy/tsconfig.json" }, { "path": "./src/plugins/visualizations/tsconfig.json" }, { "path": "./src/plugins/visualize/tsconfig.json" }, + { "path": "./src/plugins/index_pattern_management/tsconfig.json" }, ] } diff --git a/vars/githubPr.groovy b/vars/githubPr.groovy index 546a6785ac2f4..5224aa7463d79 100644 --- a/vars/githubPr.groovy +++ b/vars/githubPr.groovy @@ -182,12 +182,14 @@ def getNextCommentMessage(previousCommentInfo = [:], isFinal = false) { ## :green_heart: Build Succeeded * [continuous-integration/kibana-ci/pull-request](${env.BUILD_URL}) * Commit: ${getCommitHash()} + ${getDocsChangesLink()} """ } else if(status == 'UNSTABLE') { def message = """ ## :yellow_heart: Build succeeded, but was flaky * [continuous-integration/kibana-ci/pull-request](${env.BUILD_URL}) * Commit: ${getCommitHash()} + ${getDocsChangesLink()} """.stripIndent() def failures = retryable.getFlakyFailures() @@ -204,6 +206,7 @@ def getNextCommentMessage(previousCommentInfo = [:], isFinal = false) { * Commit: ${getCommitHash()} * [Pipeline Steps](${env.BUILD_URL}flowGraphTable) (look for red circles / failed steps) * [Interpreting CI Failures](https://www.elastic.co/guide/en/kibana/current/interpreting-ci-failures.html) + ${getDocsChangesLink()} """ } @@ -292,6 +295,21 @@ def getCommitHash() { return env.ghprbActualCommit } +def getDocsChangesLink() { + def url = "https://kibana_${env.ghprbPullId}.docs-preview.app.elstc.co/diff" + + try { + // httpRequest throws on status codes >400 and failures + httpRequest([ method: "GET", url: url ]) + return "* [Documentation Changes](${url})" + } catch (ex) { + print "Failed to reach ${url}" + buildUtils.printStacktrace(ex) + } + + return "" +} + def getFailedSteps() { return jenkinsApi.getFailedSteps()?.findAll { step -> step.displayName != 'Check out from version control' diff --git a/x-pack/plugins/actions/README.md b/x-pack/plugins/actions/README.md index 9472cbf400a6a..1eb94af4dddf8 100644 --- a/x-pack/plugins/actions/README.md +++ b/x-pack/plugins/actions/README.md @@ -70,12 +70,14 @@ Table of Contents - [`params`](#params-6) - [`subActionParams (pushToService)`](#subactionparams-pushtoservice) - [`subActionParams (getFields)`](#subactionparams-getfields) + - [`subActionParams (getIncident)`](#subactionparams-getincident) + - [`subActionParams (getChoices)`](#subactionparams-getchoices) - [Jira](#jira) - [`config`](#config-7) - [`secrets`](#secrets-7) - [`params`](#params-7) - [`subActionParams (pushToService)`](#subactionparams-pushtoservice-1) - - [`subActionParams (getIncident)`](#subactionparams-getincident) + - [`subActionParams (getIncident)`](#subactionparams-getincident-1) - [`subActionParams (issueTypes)`](#subactionparams-issuetypes) - [`subActionParams (fieldsByIssueType)`](#subactionparams-fieldsbyissuetype) - [`subActionParams (issues)`](#subactionparams-issues) @@ -347,17 +349,18 @@ const result = await actionsClient.execute({ Kibana ships with a set of built-in action types: -| Type | Id | Description | -| ------------------------------- | ------------- | ------------------------------------------------------------------ | -| [Server log](#server-log) | `.server-log` | Logs messages to the Kibana log using Kibana's logger | -| [Email](#email) | `.email` | Sends an email using SMTP | -| [Slack](#slack) | `.slack` | Posts a message to a slack channel | -| [Index](#index) | `.index` | Indexes document(s) into Elasticsearch | -| [Webhook](#webhook) | `.webhook` | Send a payload to a web service using HTTP POST or PUT | -| [PagerDuty](#pagerduty) | `.pagerduty` | Trigger, resolve, or acknowlege an incident to a PagerDuty service | -| [ServiceNow](#servicenow) | `.servicenow` | Create or update an incident to a ServiceNow instance | -| [Jira](#jira) | `.jira` | Create or update an issue to a Jira instance | -| [IBM Resilient](#ibm-resilient) | `.resilient` | Create or update an incident to a IBM Resilient instance | +| Type | Id | Description | +| ------------------------------- | ----------------- | ------------------------------------------------------------------ | +| [Server log](#server-log) | `.server-log` | Logs messages to the Kibana log using Kibana's logger | +| [Email](#email) | `.email` | Sends an email using SMTP | +| [Slack](#slack) | `.slack` | Posts a message to a slack channel | +| [Index](#index) | `.index` | Indexes document(s) into Elasticsearch | +| [Webhook](#webhook) | `.webhook` | Send a payload to a web service using HTTP POST or PUT | +| [PagerDuty](#pagerduty) | `.pagerduty` | Trigger, resolve, or acknowlege an incident to a PagerDuty service | +| [ServiceNow ITSM](#servicenow) | `.servicenow` | Create or update an incident to a ServiceNow ITSM instance | +| [ServiceNow SIR](#servicenow) | `.servicenow-sir` | Create or update an incident to a ServiceNow SIR instance | +| [Jira](#jira) | `.jira` | Create or update an issue to a Jira instance | +| [IBM Resilient](#ibm-resilient) | `.resilient` | Create or update an incident to a IBM Resilient instance | --- @@ -549,9 +552,11 @@ For more details see [PagerDuty v2 event parameters](https://v2.developer.pagerd ## ServiceNow -ID: `.servicenow` +ServiceNow ITSM ID: `.servicenow` -The ServiceNow action uses the [V2 Table API](https://developer.servicenow.com/app.do#!/rest_api_doc?v=orlando&id=c_TableAPI) to create and update ServiceNow incidents. +ServiceNow SIR ID: `.servicenow-sir` + +The ServiceNow actions use the [V2 Table API](https://developer.servicenow.com/app.do#!/rest_api_doc?v=orlando&id=c_TableAPI) to create and update ServiceNow incidents. Both action types use the same `config`, `secrets`, and `params` schema. ### `config` @@ -568,10 +573,10 @@ The ServiceNow action uses the [V2 Table API](https://developer.servicenow.com/a ### `params` -| Property | Description | Type | -| --------------- | --------------------------------------------------------------------- | ------ | -| subAction | The sub action to perform. It can be `getFields`, and `pushToService` | string | -| subActionParams | The parameters of the sub action | object | +| Property | Description | Type | +| --------------- | -------------------------------------------------------------------------------------------------- | ------ | +| subAction | The sub action to perform. It can be `pushToService`, `getFields`, `getIncident`, and `getChoices` | string | +| subActionParams | The parameters of the sub action | object | #### `subActionParams (pushToService)` @@ -595,6 +600,19 @@ The following table describes the properties of the `incident` object. No parameters for `getFields` sub-action. Provide an empty object `{}`. +#### `subActionParams (getIncident)` + +| Property | Description | Type | +| ---------- | ------------------------------------- | ------ | +| externalId | The id of the incident in ServiceNow. | string | + + +#### `subActionParams (getChoices)` + +| Property | Description | Type | +| -------- | ------------------------------------------------------------ | -------- | +| fields | An array of fields. Example: `[priority, category, impact]`. | string[] | + --- ## Jira diff --git a/x-pack/plugins/actions/server/builtin_action_types/index.ts b/x-pack/plugins/actions/server/builtin_action_types/index.ts index 3a01b875ec4a0..21161ff8ad0dd 100644 --- a/x-pack/plugins/actions/server/builtin_action_types/index.ts +++ b/x-pack/plugins/actions/server/builtin_action_types/index.ts @@ -14,7 +14,7 @@ import { getActionType as getPagerDutyActionType } from './pagerduty'; import { getActionType as getServerLogActionType } from './server_log'; import { getActionType as getSlackActionType } from './slack'; import { getActionType as getWebhookActionType } from './webhook'; -import { getActionType as getServiceNowActionType } from './servicenow'; +import { getServiceNowITSMActionType, getServiceNowSIRActionType } from './servicenow'; import { getActionType as getJiraActionType } from './jira'; import { getActionType as getResilientActionType } from './resilient'; import { getActionType as getTeamsActionType } from './teams'; @@ -38,7 +38,8 @@ export { } from './webhook'; export { ActionParamsType as ServiceNowActionParams, - ActionTypeId as ServiceNowActionTypeId, + ServiceNowITSMActionTypeId, + ServiceNowSIRActionTypeId, } from './servicenow'; export { ActionParamsType as JiraActionParams, ActionTypeId as JiraActionTypeId } from './jira'; export { @@ -66,7 +67,8 @@ export function registerBuiltInActionTypes({ actionTypeRegistry.register(getServerLogActionType({ logger })); actionTypeRegistry.register(getSlackActionType({ logger, configurationUtilities })); actionTypeRegistry.register(getWebhookActionType({ logger, configurationUtilities })); - actionTypeRegistry.register(getServiceNowActionType({ logger, configurationUtilities })); + actionTypeRegistry.register(getServiceNowITSMActionType({ logger, configurationUtilities })); + actionTypeRegistry.register(getServiceNowSIRActionType({ logger, configurationUtilities })); actionTypeRegistry.register(getJiraActionType({ logger, configurationUtilities })); actionTypeRegistry.register(getResilientActionType({ logger, configurationUtilities })); actionTypeRegistry.register(getTeamsActionType({ logger, configurationUtilities })); diff --git a/x-pack/plugins/actions/server/builtin_action_types/servicenow/api.test.ts b/x-pack/plugins/actions/server/builtin_action_types/servicenow/api.test.ts index 772cd16cc4d51..ef5de9fc487bc 100644 --- a/x-pack/plugins/actions/server/builtin_action_types/servicenow/api.test.ts +++ b/x-pack/plugins/actions/server/builtin_action_types/servicenow/api.test.ts @@ -5,7 +5,7 @@ */ import { Logger } from '../../../../../../src/core/server'; -import { externalServiceMock, apiParams, serviceNowCommonFields } from './mocks'; +import { externalServiceMock, apiParams, serviceNowCommonFields, serviceNowChoices } from './mocks'; import { ExternalService } from './types'; import { api } from './api'; let mockedLogger: jest.Mocked; @@ -235,4 +235,14 @@ describe('api', () => { expect(res).toEqual(serviceNowCommonFields); }); }); + + describe('getChoices', () => { + test('it returns the fields correctly', async () => { + const res = await api.getChoices({ + externalService, + params: { fields: ['priority'] }, + }); + expect(res).toEqual(serviceNowChoices); + }); + }); }); diff --git a/x-pack/plugins/actions/server/builtin_action_types/servicenow/api.ts b/x-pack/plugins/actions/server/builtin_action_types/servicenow/api.ts index 9981a8431a736..7f5747277a4e9 100644 --- a/x-pack/plugins/actions/server/builtin_action_types/servicenow/api.ts +++ b/x-pack/plugins/actions/server/builtin_action_types/servicenow/api.ts @@ -5,6 +5,8 @@ */ import { ExternalServiceApi, + GetChoicesHandlerArgs, + GetChoicesResponse, GetCommonFieldsHandlerArgs, GetCommonFieldsResponse, GetIncidentApiHandlerArgs, @@ -71,7 +73,16 @@ const getFieldsHandler = async ({ return res; }; +const getChoicesHandler = async ({ + externalService, + params, +}: GetChoicesHandlerArgs): Promise => { + const res = await externalService.getChoices(params.fields); + return res; +}; + export const api: ExternalServiceApi = { + getChoices: getChoicesHandler, getFields: getFieldsHandler, getIncident: getIncidentHandler, handshake: handshakeHandler, diff --git a/x-pack/plugins/actions/server/builtin_action_types/servicenow/index.ts b/x-pack/plugins/actions/server/builtin_action_types/servicenow/index.ts index 107d86f111deb..fd4991e5f7e98 100644 --- a/x-pack/plugins/actions/server/builtin_action_types/servicenow/index.ts +++ b/x-pack/plugins/actions/server/builtin_action_types/servicenow/index.ts @@ -11,7 +11,8 @@ import { validate } from './validators'; import { ExternalIncidentServiceConfiguration, ExternalIncidentServiceSecretConfiguration, - ExecutorParamsSchema, + ExecutorParamsSchemaITSM, + ExecutorParamsSchemaSIR, } from './schema'; import { ActionsConfigurationUtilities } from '../../actions_config'; import { ActionType, ActionTypeExecutorOptions, ActionTypeExecutorResult } from '../../types'; @@ -27,18 +28,26 @@ import { PushToServiceResponse, ExecutorSubActionCommonFieldsParams, ServiceNowExecutorResultData, + ExecutorSubActionGetChoicesParams, } from './types'; -export type ActionParamsType = TypeOf; +export type ActionParamsType = + | TypeOf + | TypeOf; interface GetActionTypeParams { logger: Logger; configurationUtilities: ActionsConfigurationUtilities; } -export const ActionTypeId = '.servicenow'; +const serviceNowITSMTable = 'incident'; +const serviceNowSIRTable = 'sn_si_incident'; + +export const ServiceNowITSMActionTypeId = '.servicenow'; +export const ServiceNowSIRActionTypeId = '.servicenow-sir'; + // action type definition -export function getActionType( +export function getServiceNowITSMActionType( params: GetActionTypeParams ): ActionType< ServiceNowPublicConfigurationType, @@ -48,9 +57,9 @@ export function getActionType( > { const { logger, configurationUtilities } = params; return { - id: ActionTypeId, + id: ServiceNowITSMActionTypeId, minimumLicenseRequired: 'platinum', - name: i18n.NAME, + name: i18n.SERVICENOW_ITSM, validate: { config: schema.object(ExternalIncidentServiceConfiguration, { validate: curry(validate.config)(configurationUtilities), @@ -58,19 +67,46 @@ export function getActionType( secrets: schema.object(ExternalIncidentServiceSecretConfiguration, { validate: curry(validate.secrets)(configurationUtilities), }), - params: ExecutorParamsSchema, + params: ExecutorParamsSchemaITSM, }, - executor: curry(executor)({ logger, configurationUtilities }), + executor: curry(executor)({ logger, configurationUtilities, table: serviceNowITSMTable }), + }; +} + +export function getServiceNowSIRActionType( + params: GetActionTypeParams +): ActionType< + ServiceNowPublicConfigurationType, + ServiceNowSecretConfigurationType, + ExecutorParams, + PushToServiceResponse | {} +> { + const { logger, configurationUtilities } = params; + return { + id: ServiceNowSIRActionTypeId, + minimumLicenseRequired: 'platinum', + name: i18n.SERVICENOW_SIR, + validate: { + config: schema.object(ExternalIncidentServiceConfiguration, { + validate: curry(validate.config)(configurationUtilities), + }), + secrets: schema.object(ExternalIncidentServiceSecretConfiguration, { + validate: curry(validate.secrets)(configurationUtilities), + }), + params: ExecutorParamsSchemaSIR, + }, + executor: curry(executor)({ logger, configurationUtilities, table: serviceNowSIRTable }), }; } // action executor -const supportedSubActions: string[] = ['getFields', 'pushToService']; +const supportedSubActions: string[] = ['getFields', 'pushToService', 'getChoices', 'getIncident']; async function executor( { logger, configurationUtilities, - }: { logger: Logger; configurationUtilities: ActionsConfigurationUtilities }, + table, + }: { logger: Logger; configurationUtilities: ActionsConfigurationUtilities; table: string }, execOptions: ActionTypeExecutorOptions< ServiceNowPublicConfigurationType, ServiceNowSecretConfigurationType, @@ -82,6 +118,7 @@ async function executor( let data: ServiceNowExecutorResultData | null = null; const externalService = createExternalService( + table, { config, secrets, @@ -122,5 +159,13 @@ async function executor( }); } + if (subAction === 'getChoices') { + const getChoicesParams = subActionParams as ExecutorSubActionGetChoicesParams; + data = await api.getChoices({ + externalService, + params: getChoicesParams, + }); + } + return { status: 'ok', data: data ?? {}, actionId }; } diff --git a/x-pack/plugins/actions/server/builtin_action_types/servicenow/mocks.ts b/x-pack/plugins/actions/server/builtin_action_types/servicenow/mocks.ts index 9d9b1e164e7dd..f958cdb73ebfc 100644 --- a/x-pack/plugins/actions/server/builtin_action_types/servicenow/mocks.ts +++ b/x-pack/plugins/actions/server/builtin_action_types/servicenow/mocks.ts @@ -4,7 +4,7 @@ * you may not use this file except in compliance with the Elastic License. */ -import { ExternalService, PushToServiceApiParams, ExecutorSubActionPushParams } from './types'; +import { ExternalService, ExecutorSubActionPushParams } from './types'; export const serviceNowCommonFields = [ { @@ -33,8 +33,43 @@ export const serviceNowCommonFields = [ element: 'sys_updated_by', }, ]; + +export const serviceNowChoices = [ + { + dependent_value: '', + label: '1 - Critical', + value: '1', + element: 'priority', + }, + { + dependent_value: '', + label: '2 - High', + value: '2', + element: 'priority', + }, + { + dependent_value: '', + label: '3 - Moderate', + value: '3', + element: 'priority', + }, + { + dependent_value: '', + label: '4 - Low', + value: '4', + element: 'priority', + }, + { + dependent_value: '', + label: '5 - Planning', + value: '5', + element: 'priority', + }, +]; + const createMock = (): jest.Mocked => { const service = { + getChoices: jest.fn().mockImplementation(() => Promise.resolve(serviceNowChoices)), getFields: jest.fn().mockImplementation(() => Promise.resolve(serviceNowCommonFields)), getIncident: jest.fn().mockImplementation(() => Promise.resolve({ @@ -89,8 +124,6 @@ const executorParams: ExecutorSubActionPushParams = { ], }; -const apiParams: PushToServiceApiParams = { - ...executorParams, -}; +const apiParams = executorParams; export { externalServiceMock, executorParams, apiParams }; diff --git a/x-pack/plugins/actions/server/builtin_action_types/servicenow/schema.ts b/x-pack/plugins/actions/server/builtin_action_types/servicenow/schema.ts index 1c05fa93f2362..5c7de935223a8 100644 --- a/x-pack/plugins/actions/server/builtin_action_types/servicenow/schema.ts +++ b/x-pack/plugins/actions/server/builtin_action_types/servicenow/schema.ts @@ -28,25 +28,48 @@ export const ExecutorSubActionSchema = schema.oneOf([ schema.literal('getIncident'), schema.literal('pushToService'), schema.literal('handshake'), + schema.literal('getChoices'), ]); -export const ExecutorSubActionPushParamsSchema = schema.object({ +const CommentsSchema = schema.nullable( + schema.arrayOf( + schema.object({ + comment: schema.string(), + commentId: schema.string(), + }) + ) +); + +const CommonAttributes = { + short_description: schema.string(), + description: schema.nullable(schema.string()), + externalId: schema.nullable(schema.string()), +}; + +// Schema for ServiceNow Incident Management (ITSM) +export const ExecutorSubActionPushParamsSchemaITSM = schema.object({ incident: schema.object({ - short_description: schema.string(), - description: schema.nullable(schema.string()), - externalId: schema.nullable(schema.string()), + ...CommonAttributes, severity: schema.nullable(schema.string()), urgency: schema.nullable(schema.string()), impact: schema.nullable(schema.string()), }), - comments: schema.nullable( - schema.arrayOf( - schema.object({ - comment: schema.string(), - commentId: schema.string(), - }) - ) - ), + comments: CommentsSchema, +}); + +// Schema for ServiceNow Security Incident Response (SIR) +export const ExecutorSubActionPushParamsSchemaSIR = schema.object({ + incident: schema.object({ + ...CommonAttributes, + category: schema.nullable(schema.string()), + dest_ip: schema.nullable(schema.string()), + malware_hash: schema.nullable(schema.string()), + malware_url: schema.nullable(schema.string()), + priority: schema.nullable(schema.string()), + source_ip: schema.nullable(schema.string()), + subcategory: schema.nullable(schema.string()), + }), + comments: CommentsSchema, }); export const ExecutorSubActionGetIncidentParamsSchema = schema.object({ @@ -56,8 +79,36 @@ export const ExecutorSubActionGetIncidentParamsSchema = schema.object({ // Reserved for future implementation export const ExecutorSubActionHandshakeParamsSchema = schema.object({}); export const ExecutorSubActionCommonFieldsParamsSchema = schema.object({}); +export const ExecutorSubActionGetChoicesParamsSchema = schema.object({ + fields: schema.arrayOf(schema.string()), +}); + +// Executor parameters for ServiceNow Incident Management (ITSM) +export const ExecutorParamsSchemaITSM = schema.oneOf([ + schema.object({ + subAction: schema.literal('getFields'), + subActionParams: ExecutorSubActionCommonFieldsParamsSchema, + }), + schema.object({ + subAction: schema.literal('getIncident'), + subActionParams: ExecutorSubActionGetIncidentParamsSchema, + }), + schema.object({ + subAction: schema.literal('handshake'), + subActionParams: ExecutorSubActionHandshakeParamsSchema, + }), + schema.object({ + subAction: schema.literal('pushToService'), + subActionParams: ExecutorSubActionPushParamsSchemaITSM, + }), + schema.object({ + subAction: schema.literal('getChoices'), + subActionParams: ExecutorSubActionGetChoicesParamsSchema, + }), +]); -export const ExecutorParamsSchema = schema.oneOf([ +// Executor parameters for ServiceNow Security Incident Response (SIR) +export const ExecutorParamsSchemaSIR = schema.oneOf([ schema.object({ subAction: schema.literal('getFields'), subActionParams: ExecutorSubActionCommonFieldsParamsSchema, @@ -72,6 +123,10 @@ export const ExecutorParamsSchema = schema.oneOf([ }), schema.object({ subAction: schema.literal('pushToService'), - subActionParams: ExecutorSubActionPushParamsSchema, + subActionParams: ExecutorSubActionPushParamsSchemaSIR, + }), + schema.object({ + subAction: schema.literal('getChoices'), + subActionParams: ExecutorSubActionGetChoicesParamsSchema, }), ]); diff --git a/x-pack/plugins/actions/server/builtin_action_types/servicenow/service.test.ts b/x-pack/plugins/actions/server/builtin_action_types/servicenow/service.test.ts index 4ef0e7da166e6..18f3a2f3ff379 100644 --- a/x-pack/plugins/actions/server/builtin_action_types/servicenow/service.test.ts +++ b/x-pack/plugins/actions/server/builtin_action_types/servicenow/service.test.ts @@ -12,7 +12,7 @@ import { ExternalService } from './types'; import { Logger } from '../../../../../../src/core/server'; import { loggingSystemMock } from '../../../../../../src/core/server/mocks'; import { actionsConfigMock } from '../../actions_config.mock'; -import { serviceNowCommonFields } from './mocks'; +import { serviceNowCommonFields, serviceNowChoices } from './mocks'; const logger = loggingSystemMock.create().get() as jest.Mocked; jest.mock('axios'); @@ -29,12 +29,14 @@ axios.create = jest.fn(() => axios); const requestMock = utils.request as jest.Mock; const patchMock = utils.patch as jest.Mock; const configurationUtilities = actionsConfigMock.create(); +const table = 'incident'; describe('ServiceNow service', () => { let service: ExternalService; - beforeAll(() => { + beforeEach(() => { service = createExternalService( + table, { // The trailing slash at the end of the url is intended. // All API calls need to have the trailing slash removed. @@ -54,6 +56,7 @@ describe('ServiceNow service', () => { test('throws without url', () => { expect(() => createExternalService( + table, { config: { apiUrl: null }, secrets: { username: 'admin', password: 'admin' }, @@ -67,6 +70,7 @@ describe('ServiceNow service', () => { test('throws without username', () => { expect(() => createExternalService( + table, { config: { apiUrl: 'test.com' }, secrets: { username: '', password: 'admin' }, @@ -80,6 +84,7 @@ describe('ServiceNow service', () => { test('throws without password', () => { expect(() => createExternalService( + table, { config: { apiUrl: 'test.com' }, secrets: { username: '', password: undefined }, @@ -114,6 +119,30 @@ describe('ServiceNow service', () => { }); }); + test('it should call request with correct arguments when table changes', async () => { + service = createExternalService( + 'sn_si_incident', + { + config: { apiUrl: 'https://dev102283.service-now.com/' }, + secrets: { username: 'admin', password: 'admin' }, + }, + logger, + configurationUtilities + ); + + requestMock.mockImplementation(() => ({ + data: { result: { sys_id: '1', number: 'INC01' } }, + })); + + await service.getIncident('1'); + expect(requestMock).toHaveBeenCalledWith({ + axios, + logger, + configurationUtilities, + url: 'https://dev102283.service-now.com/api/now/v2/table/sn_si_incident/1', + }); + }); + test('it should throw an error', async () => { requestMock.mockImplementation(() => { throw new Error('An error has occurred'); @@ -122,6 +151,17 @@ describe('ServiceNow service', () => { 'Unable to get incident with id 1. Error: An error has occurred' ); }); + + test('it should throw an error when instance is not alive', async () => { + requestMock.mockImplementation(() => ({ + status: 200, + data: {}, + request: { connection: { servername: 'Developer instance' } }, + })); + await expect(service.getIncident('1')).rejects.toThrow( + 'There is an issue with your Service Now Instance. Please check Developer instance.' + ); + }); }); describe('createIncident', () => { @@ -161,6 +201,39 @@ describe('ServiceNow service', () => { }); }); + test('it should call request with correct arguments when table changes', async () => { + service = createExternalService( + 'sn_si_incident', + { + config: { apiUrl: 'https://dev102283.service-now.com/' }, + secrets: { username: 'admin', password: 'admin' }, + }, + logger, + configurationUtilities + ); + + requestMock.mockImplementation(() => ({ + data: { result: { sys_id: '1', number: 'INC01', sys_created_on: '2020-03-10 12:24:20' } }, + })); + + const res = await service.createIncident({ + incident: { short_description: 'title', description: 'desc' }, + }); + + expect(requestMock).toHaveBeenCalledWith({ + axios, + logger, + configurationUtilities, + url: 'https://dev102283.service-now.com/api/now/v2/table/sn_si_incident', + method: 'post', + data: { short_description: 'title', description: 'desc' }, + }); + + expect(res.url).toEqual( + 'https://dev102283.service-now.com/nav_to.do?uri=sn_si_incident.do?sys_id=1' + ); + }); + test('it should throw an error', async () => { requestMock.mockImplementation(() => { throw new Error('An error has occurred'); @@ -174,6 +247,17 @@ describe('ServiceNow service', () => { '[Action][ServiceNow]: Unable to create incident. Error: An error has occurred' ); }); + + test('it should throw an error when instance is not alive', async () => { + requestMock.mockImplementation(() => ({ + status: 200, + data: {}, + request: { connection: { servername: 'Developer instance' } }, + })); + await expect(service.getIncident('1')).rejects.toThrow( + 'There is an issue with your Service Now Instance. Please check Developer instance.' + ); + }); }); describe('updateIncident', () => { @@ -214,6 +298,39 @@ describe('ServiceNow service', () => { }); }); + test('it should call request with correct arguments when table changes', async () => { + service = createExternalService( + 'sn_si_incident', + { + config: { apiUrl: 'https://dev102283.service-now.com/' }, + secrets: { username: 'admin', password: 'admin' }, + }, + logger, + configurationUtilities + ); + + patchMock.mockImplementation(() => ({ + data: { result: { sys_id: '1', number: 'INC01', sys_updated_on: '2020-03-10 12:24:20' } }, + })); + + const res = await service.updateIncident({ + incidentId: '1', + incident: { short_description: 'title', description: 'desc' }, + }); + + expect(patchMock).toHaveBeenCalledWith({ + axios, + logger, + configurationUtilities, + url: 'https://dev102283.service-now.com/api/now/v2/table/sn_si_incident/1', + data: { short_description: 'title', description: 'desc' }, + }); + + expect(res.url).toEqual( + 'https://dev102283.service-now.com/nav_to.do?uri=sn_si_incident.do?sys_id=1' + ); + }); + test('it should throw an error', async () => { patchMock.mockImplementation(() => { throw new Error('An error has occurred'); @@ -228,6 +345,7 @@ describe('ServiceNow service', () => { '[Action][ServiceNow]: Unable to update incident with id 1. Error: An error has occurred' ); }); + test('it creates the comment correctly', async () => { patchMock.mockImplementation(() => ({ data: { result: { sys_id: '11', number: 'INC011', sys_updated_on: '2020-03-10 12:24:20' } }, @@ -245,6 +363,17 @@ describe('ServiceNow service', () => { url: 'https://dev102283.service-now.com/nav_to.do?uri=incident.do?sys_id=11', }); }); + + test('it should throw an error when instance is not alive', async () => { + requestMock.mockImplementation(() => ({ + status: 200, + data: {}, + request: { connection: { servername: 'Developer instance' } }, + })); + await expect(service.getIncident('1')).rejects.toThrow( + 'There is an issue with your Service Now Instance. Please check Developer instance.' + ); + }); }); describe('getFields', () => { @@ -259,9 +388,10 @@ describe('ServiceNow service', () => { logger, configurationUtilities, url: - 'https://dev102283.service-now.com/api/now/v2/table/sys_dictionary?sysparm_query=name=task^internal_type=string&active=true&array=false&read_only=false&sysparm_fields=max_length,element,column_label,mandatory', + 'https://dev102283.service-now.com/api/now/v2/table/sys_dictionary?sysparm_query=name=task^ORname=incident^internal_type=string&active=true&array=false&read_only=false&sysparm_fields=max_length,element,column_label,mandatory', }); }); + test('it returns common fields correctly', async () => { requestMock.mockImplementation(() => ({ data: { result: serviceNowCommonFields }, @@ -270,6 +400,31 @@ describe('ServiceNow service', () => { expect(res).toEqual(serviceNowCommonFields); }); + test('it should call request with correct arguments when table changes', async () => { + service = createExternalService( + 'sn_si_incident', + { + config: { apiUrl: 'https://dev102283.service-now.com/' }, + secrets: { username: 'admin', password: 'admin' }, + }, + logger, + configurationUtilities + ); + + requestMock.mockImplementation(() => ({ + data: { result: serviceNowCommonFields }, + })); + await service.getFields(); + + expect(requestMock).toHaveBeenCalledWith({ + axios, + logger, + configurationUtilities, + url: + 'https://dev102283.service-now.com/api/now/v2/table/sys_dictionary?sysparm_query=name=task^ORname=sn_si_incident^internal_type=string&active=true&array=false&read_only=false&sysparm_fields=max_length,element,column_label,mandatory', + }); + }); + test('it should throw an error', async () => { requestMock.mockImplementation(() => { throw new Error('An error has occurred'); @@ -278,5 +433,87 @@ describe('ServiceNow service', () => { '[Action][ServiceNow]: Unable to get fields. Error: An error has occurred' ); }); + + test('it should throw an error when instance is not alive', async () => { + requestMock.mockImplementation(() => ({ + status: 200, + data: {}, + request: { connection: { servername: 'Developer instance' } }, + })); + await expect(service.getIncident('1')).rejects.toThrow( + 'There is an issue with your Service Now Instance. Please check Developer instance.' + ); + }); + }); + + describe('getChoices', () => { + test('it should call request with correct arguments', async () => { + requestMock.mockImplementation(() => ({ + data: { result: serviceNowChoices }, + })); + await service.getChoices(['priority', 'category']); + + expect(requestMock).toHaveBeenCalledWith({ + axios, + logger, + configurationUtilities, + url: + 'https://dev102283.service-now.com/api/now/v2/table/sys_choice?sysparm_query=name=task^ORname=incident^element=priority^ORelement=category&sysparm_fields=label,value,dependent_value,element', + }); + }); + + test('it returns common fields correctly', async () => { + requestMock.mockImplementation(() => ({ + data: { result: serviceNowChoices }, + })); + const res = await service.getChoices(['priority']); + expect(res).toEqual(serviceNowChoices); + }); + + test('it should call request with correct arguments when table changes', async () => { + service = createExternalService( + 'sn_si_incident', + { + config: { apiUrl: 'https://dev102283.service-now.com/' }, + secrets: { username: 'admin', password: 'admin' }, + }, + logger, + configurationUtilities + ); + + requestMock.mockImplementation(() => ({ + data: { result: serviceNowChoices }, + })); + + await service.getChoices(['priority', 'category']); + + expect(requestMock).toHaveBeenCalledWith({ + axios, + logger, + configurationUtilities, + url: + 'https://dev102283.service-now.com/api/now/v2/table/sys_choice?sysparm_query=name=task^ORname=sn_si_incident^element=priority^ORelement=category&sysparm_fields=label,value,dependent_value,element', + }); + }); + + test('it should throw an error', async () => { + requestMock.mockImplementation(() => { + throw new Error('An error has occurred'); + }); + await expect(service.getChoices(['priority'])).rejects.toThrow( + '[Action][ServiceNow]: Unable to get choices. Error: An error has occurred' + ); + }); + + test('it should throw an error when instance is not alive', async () => { + requestMock.mockImplementation(() => ({ + status: 200, + data: {}, + request: { connection: { servername: 'Developer instance' } }, + })); + await expect(service.getIncident('1')).rejects.toThrow( + 'There is an issue with your Service Now Instance. Please check Developer instance.' + ); + }); }); }); diff --git a/x-pack/plugins/actions/server/builtin_action_types/servicenow/service.ts b/x-pack/plugins/actions/server/builtin_action_types/servicenow/service.ts index 108fe06bcbcaa..7c7723c98a070 100644 --- a/x-pack/plugins/actions/server/builtin_action_types/servicenow/service.ts +++ b/x-pack/plugins/actions/server/builtin_action_types/servicenow/service.ts @@ -15,13 +15,10 @@ import { request, getErrorMessage, addTimeZoneToDate, patch } from '../lib/axios import { ActionsConfigurationUtilities } from '../../actions_config'; const API_VERSION = 'v2'; -const INCIDENT_URL = `api/now/${API_VERSION}/table/incident`; const SYS_DICTIONARY = `api/now/${API_VERSION}/table/sys_dictionary`; -// Based on: https://docs.servicenow.com/bundle/orlando-platform-user-interface/page/use/navigation/reference/r_NavigatingByURLExamples.html -const VIEW_INCIDENT_URL = `nav_to.do?uri=incident.do?sys_id=`; - export const createExternalService = ( + table: string, { config, secrets }: ExternalServiceCredentials, logger: Logger, configurationUtilities: ActionsConfigurationUtilities @@ -30,24 +27,36 @@ export const createExternalService = ( const { username, password } = secrets as ServiceNowSecretConfigurationType; if (!url || !username || !password) { - throw Error(`[Action]${i18n.NAME}: Wrong configuration.`); + throw Error(`[Action]${i18n.SERVICENOW}: Wrong configuration.`); } const urlWithoutTrailingSlash = url.endsWith('/') ? url.slice(0, -1) : url; - const incidentUrl = `${urlWithoutTrailingSlash}/${INCIDENT_URL}`; - const fieldsUrl = `${urlWithoutTrailingSlash}/${SYS_DICTIONARY}?sysparm_query=name=task^internal_type=string&active=true&array=false&read_only=false&sysparm_fields=max_length,element,column_label,mandatory`; + const incidentUrl = `${urlWithoutTrailingSlash}/api/now/${API_VERSION}/table/${table}`; + const fieldsUrl = `${urlWithoutTrailingSlash}/${SYS_DICTIONARY}?sysparm_query=name=task^ORname=${table}^internal_type=string&active=true&array=false&read_only=false&sysparm_fields=max_length,element,column_label,mandatory`; + const choicesUrl = `${urlWithoutTrailingSlash}/api/now/${API_VERSION}/table/sys_choice`; const axiosInstance = axios.create({ auth: { username, password }, }); const getIncidentViewURL = (id: string) => { - return `${urlWithoutTrailingSlash}/${VIEW_INCIDENT_URL}${id}`; + // Based on: https://docs.servicenow.com/bundle/orlando-platform-user-interface/page/use/navigation/reference/r_NavigatingByURLExamples.html + return `${urlWithoutTrailingSlash}/nav_to.do?uri=${table}.do?sys_id=${id}`; + }; + + const getChoicesURL = (fields: string[]) => { + const elements = fields + .slice(1) + .reduce((acc, field) => `${acc}^ORelement=${field}`, `element=${fields[0]}`); + + return `${choicesUrl}?sysparm_query=name=task^ORname=${table}^${elements}&sysparm_fields=label,value,dependent_value,element`; }; const checkInstance = (res: AxiosResponse) => { if (res.status === 200 && res.data.result == null) { throw new Error( - `There is an issue with your Service Now Instance. Please check ${res.request.connection.servername}` + `There is an issue with your Service Now Instance. Please check ${ + res.request?.connection?.servername ?? '' + }.` ); } }; @@ -64,7 +73,10 @@ export const createExternalService = ( return { ...res.data.result }; } catch (error) { throw new Error( - getErrorMessage(i18n.NAME, `Unable to get incident with id ${id}. Error: ${error.message}`) + getErrorMessage( + i18n.SERVICENOW, + `Unable to get incident with id ${id}. Error: ${error.message}` + ) ); } }; @@ -82,7 +94,10 @@ export const createExternalService = ( return res.data.result.length > 0 ? { ...res.data.result } : undefined; } catch (error) { throw new Error( - getErrorMessage(i18n.NAME, `Unable to find incidents by query. Error: ${error.message}`) + getErrorMessage( + i18n.SERVICENOW, + `Unable to find incidents by query. Error: ${error.message}` + ) ); } }; @@ -106,7 +121,7 @@ export const createExternalService = ( }; } catch (error) { throw new Error( - getErrorMessage(i18n.NAME, `Unable to create incident. Error: ${error.message}`) + getErrorMessage(i18n.SERVICENOW, `Unable to create incident. Error: ${error.message}`) ); } }; @@ -130,7 +145,7 @@ export const createExternalService = ( } catch (error) { throw new Error( getErrorMessage( - i18n.NAME, + i18n.SERVICENOW, `Unable to update incident with id ${incidentId}. Error: ${error.message}` ) ); @@ -148,7 +163,26 @@ export const createExternalService = ( checkInstance(res); return res.data.result.length > 0 ? res.data.result : []; } catch (error) { - throw new Error(getErrorMessage(i18n.NAME, `Unable to get fields. Error: ${error.message}`)); + throw new Error( + getErrorMessage(i18n.SERVICENOW, `Unable to get fields. Error: ${error.message}`) + ); + } + }; + + const getChoices = async (fields: string[]) => { + try { + const res = await request({ + axios: axiosInstance, + url: getChoicesURL(fields), + logger, + configurationUtilities, + }); + checkInstance(res); + return res.data.result; + } catch (error) { + throw new Error( + getErrorMessage(i18n.SERVICENOW, `Unable to get choices. Error: ${error.message}`) + ); } }; @@ -158,5 +192,6 @@ export const createExternalService = ( getFields, getIncident, updateIncident, + getChoices, }; }; diff --git a/x-pack/plugins/actions/server/builtin_action_types/servicenow/translations.ts b/x-pack/plugins/actions/server/builtin_action_types/servicenow/translations.ts index 287fe8cacda79..84fe538e0a63a 100644 --- a/x-pack/plugins/actions/server/builtin_action_types/servicenow/translations.ts +++ b/x-pack/plugins/actions/server/builtin_action_types/servicenow/translations.ts @@ -6,10 +6,18 @@ import { i18n } from '@kbn/i18n'; -export const NAME = i18n.translate('xpack.actions.builtin.servicenowTitle', { +export const SERVICENOW = i18n.translate('xpack.actions.builtin.serviceNowTitle', { defaultMessage: 'ServiceNow', }); +export const SERVICENOW_ITSM = i18n.translate('xpack.actions.builtin.serviceNowITSMTitle', { + defaultMessage: 'ServiceNow ITSM', +}); + +export const SERVICENOW_SIR = i18n.translate('xpack.actions.builtin.serviceNowSIRTitle', { + defaultMessage: 'ServiceNow SIR', +}); + export const ALLOWED_HOSTS_ERROR = (message: string) => i18n.translate('xpack.actions.builtin.configuration.apiAllowedHostsError', { defaultMessage: 'error configuring connector action: {message}', diff --git a/x-pack/plugins/actions/server/builtin_action_types/servicenow/types.ts b/x-pack/plugins/actions/server/builtin_action_types/servicenow/types.ts index 9868f5d1bea06..c74d1fbffd759 100644 --- a/x-pack/plugins/actions/server/builtin_action_types/servicenow/types.ts +++ b/x-pack/plugins/actions/server/builtin_action_types/servicenow/types.ts @@ -8,13 +8,16 @@ import { TypeOf } from '@kbn/config-schema'; import { - ExecutorParamsSchema, + ExecutorParamsSchemaITSM, ExecutorSubActionCommonFieldsParamsSchema, ExecutorSubActionGetIncidentParamsSchema, ExecutorSubActionHandshakeParamsSchema, - ExecutorSubActionPushParamsSchema, + ExecutorSubActionPushParamsSchemaITSM, ExternalIncidentServiceConfigurationSchema, ExternalIncidentServiceSecretConfigurationSchema, + ExecutorParamsSchemaSIR, + ExecutorSubActionPushParamsSchemaSIR, + ExecutorSubActionGetChoicesParamsSchema, } from './schema'; import { ActionsConfigurationUtilities } from '../../actions_config'; import { Logger } from '../../../../../../src/core/server'; @@ -30,14 +33,29 @@ export type ExecutorSubActionCommonFieldsParams = TypeOf< typeof ExecutorSubActionCommonFieldsParamsSchema >; -export type ServiceNowExecutorResultData = PushToServiceResponse | GetCommonFieldsResponse; +export type ExecutorSubActionGetChoicesParams = TypeOf< + typeof ExecutorSubActionGetChoicesParamsSchema +>; + +export type ServiceNowExecutorResultData = + | PushToServiceResponse + | GetCommonFieldsResponse + | GetChoicesResponse; export interface CreateCommentRequest { [key: string]: string; } -export type ExecutorParams = TypeOf; -export type ExecutorSubActionPushParams = TypeOf; +export type ExecutorParams = + | TypeOf + | TypeOf; + +export type ExecutorSubActionPushParamsITSM = TypeOf; +export type ExecutorSubActionPushParamsSIR = TypeOf; + +export type ExecutorSubActionPushParams = + | ExecutorSubActionPushParamsITSM + | ExecutorSubActionPushParamsSIR; export interface ExternalServiceCredentials { config: Record; @@ -62,14 +80,17 @@ export interface PushToServiceResponse extends ExternalServiceIncidentResponse { export type ExternalServiceParams = Record; export interface ExternalService { - getFields: () => Promise; + getChoices: (fields: string[]) => Promise; getIncident: (id: string) => Promise; + getFields: () => Promise; createIncident: (params: ExternalServiceParams) => Promise; updateIncident: (params: ExternalServiceParams) => Promise; findIncidents: (params?: Record) => Promise; } export type PushToServiceApiParams = ExecutorSubActionPushParams; +export type PushToServiceApiParamsITSM = ExecutorSubActionPushParamsITSM; +export type PushToServiceApiParamsSIR = ExecutorSubActionPushParamsSIR; export interface ExternalServiceApiHandlerArgs { externalService: ExternalService; @@ -83,7 +104,17 @@ export type ExecutorSubActionHandshakeParams = TypeOf< typeof ExecutorSubActionHandshakeParamsSchema >; -export type Incident = Omit; +export type ServiceNowITSMIncident = Omit< + TypeOf['incident'], + 'externalId' +>; + +export type ServiceNowSIRIncident = Omit< + TypeOf['incident'], + 'externalId' +>; + +export type Incident = ServiceNowITSMIncident | ServiceNowSIRIncident; export interface PushToServiceApiHandlerArgs extends ExternalServiceApiHandlerArgs { params: PushToServiceApiParams; @@ -104,13 +135,29 @@ export interface ExternalServiceFields { max_length: string; element: string; } + +export interface ExternalServiceChoices { + value: string; + label: string; + dependent_value: string; + element: string; +} + export type GetCommonFieldsResponse = ExternalServiceFields[]; +export type GetChoicesResponse = ExternalServiceChoices[]; + export interface GetCommonFieldsHandlerArgs { externalService: ExternalService; params: ExecutorSubActionCommonFieldsParams; } +export interface GetChoicesHandlerArgs { + externalService: ExternalService; + params: ExecutorSubActionGetChoicesParams; +} + export interface ExternalServiceApi { + getChoices: (args: GetChoicesHandlerArgs) => Promise; getFields: (args: GetCommonFieldsHandlerArgs) => Promise; handshake: (args: HandshakeApiHandlerArgs) => Promise; pushToService: (args: PushToServiceApiHandlerArgs) => Promise; diff --git a/x-pack/plugins/actions/server/index.ts b/x-pack/plugins/actions/server/index.ts index 4e59dfd099811..b573bcfc10914 100644 --- a/x-pack/plugins/actions/server/index.ts +++ b/x-pack/plugins/actions/server/index.ts @@ -35,7 +35,8 @@ export type { SlackActionParams, WebhookActionTypeId, WebhookActionParams, - ServiceNowActionTypeId, + ServiceNowITSMActionTypeId, + ServiceNowSIRActionTypeId, ServiceNowActionParams, JiraActionTypeId, JiraActionParams, diff --git a/x-pack/plugins/alerts/server/alerts_client/alerts_client.ts b/x-pack/plugins/alerts/server/alerts_client/alerts_client.ts index 457079229de94..569b54f21f906 100644 --- a/x-pack/plugins/alerts/server/alerts_client/alerts_client.ts +++ b/x-pack/plugins/alerts/server/alerts_client/alerts_client.ts @@ -149,6 +149,7 @@ export interface CreateOptions { | 'executionStatus' > & { actions: NormalizedAlertAction[] }; options?: { + id?: string; migrationVersion?: Record; }; } @@ -226,7 +227,7 @@ export class AlertsClient { data, options, }: CreateOptions): Promise> { - const id = SavedObjectsUtils.generateId(); + const id = options?.id || SavedObjectsUtils.generateId(); try { await this.authorization.ensureAuthorized( diff --git a/x-pack/plugins/alerts/server/alerts_client/tests/create.test.ts b/x-pack/plugins/alerts/server/alerts_client/tests/create.test.ts index 0424a1295c9b9..2e3dac76f72e5 100644 --- a/x-pack/plugins/alerts/server/alerts_client/tests/create.test.ts +++ b/x-pack/plugins/alerts/server/alerts_client/tests/create.test.ts @@ -462,6 +462,73 @@ describe('create()', () => { expect(actionsClient.isActionTypeEnabled).toHaveBeenCalledWith('test', { notifyUsage: true }); }); + test('creates an alert with a custom id', async () => { + const data = getMockData(); + const createdAttributes = { + ...data, + alertTypeId: '123', + schedule: { interval: '10s' }, + params: { + bar: true, + }, + createdAt: '2019-02-12T21:01:22.479Z', + createdBy: 'elastic', + updatedBy: 'elastic', + updatedAt: '2019-02-12T21:01:22.479Z', + muteAll: false, + mutedInstanceIds: [], + actions: [ + { + group: 'default', + actionRef: 'action_0', + actionTypeId: 'test', + params: { + foo: true, + }, + }, + ], + }; + unsecuredSavedObjectsClient.create.mockResolvedValueOnce({ + id: '123', + type: 'alert', + attributes: createdAttributes, + references: [ + { + name: 'action_0', + type: 'action', + id: '1', + }, + ], + }); + taskManager.schedule.mockResolvedValueOnce({ + id: 'task-123', + taskType: 'alerting:123', + scheduledAt: new Date(), + attempts: 1, + status: TaskStatus.Idle, + runAt: new Date(), + startedAt: null, + retryAt: null, + state: {}, + params: {}, + ownerId: null, + }); + const result = await alertsClient.create({ data, options: { id: '123' } }); + expect(result.id).toEqual('123'); + expect(unsecuredSavedObjectsClient.create.mock.calls[0][2]).toMatchInlineSnapshot(` + Object { + "id": "123", + "references": Array [ + Object { + "id": "1", + "name": "action_0", + "type": "action", + }, + ], + } + `); + }); + test('creates an alert with multiple actions', async () => { const data = getMockData({ actions: [ diff --git a/x-pack/plugins/alerts/server/authorization/__snapshots__/alerts_authorization.test.ts.snap b/x-pack/plugins/alerts/server/authorization/__snapshots__/alerts_authorization.test.ts.snap new file mode 100644 index 0000000000000..f9a28dc3eb119 --- /dev/null +++ b/x-pack/plugins/alerts/server/authorization/__snapshots__/alerts_authorization.test.ts.snap @@ -0,0 +1,316 @@ +// Jest Snapshot v1, https://goo.gl/fbAQLP + +exports[`AlertsAuthorization getFindAuthorizationFilter creates a filter based on the privileged types 1`] = ` +Object { + "arguments": Array [ + Object { + "arguments": Array [ + Object { + "arguments": Array [ + Object { + "type": "literal", + "value": "alert.attributes.alertTypeId", + }, + Object { + "type": "literal", + "value": "myAppAlertType", + }, + Object { + "type": "literal", + "value": false, + }, + ], + "function": "is", + "type": "function", + }, + Object { + "arguments": Array [ + Object { + "arguments": Array [ + Object { + "type": "literal", + "value": "alert.attributes.consumer", + }, + Object { + "type": "literal", + "value": "alerts", + }, + Object { + "type": "literal", + "value": false, + }, + ], + "function": "is", + "type": "function", + }, + Object { + "arguments": Array [ + Object { + "type": "literal", + "value": "alert.attributes.consumer", + }, + Object { + "type": "literal", + "value": "myApp", + }, + Object { + "type": "literal", + "value": false, + }, + ], + "function": "is", + "type": "function", + }, + Object { + "arguments": Array [ + Object { + "type": "literal", + "value": "alert.attributes.consumer", + }, + Object { + "type": "literal", + "value": "myOtherApp", + }, + Object { + "type": "literal", + "value": false, + }, + ], + "function": "is", + "type": "function", + }, + Object { + "arguments": Array [ + Object { + "type": "literal", + "value": "alert.attributes.consumer", + }, + Object { + "type": "literal", + "value": "myAppWithSubFeature", + }, + Object { + "type": "literal", + "value": false, + }, + ], + "function": "is", + "type": "function", + }, + ], + "function": "or", + "type": "function", + }, + ], + "function": "and", + "type": "function", + }, + Object { + "arguments": Array [ + Object { + "arguments": Array [ + Object { + "type": "literal", + "value": "alert.attributes.alertTypeId", + }, + Object { + "type": "literal", + "value": "myOtherAppAlertType", + }, + Object { + "type": "literal", + "value": false, + }, + ], + "function": "is", + "type": "function", + }, + Object { + "arguments": Array [ + Object { + "arguments": Array [ + Object { + "type": "literal", + "value": "alert.attributes.consumer", + }, + Object { + "type": "literal", + "value": "alerts", + }, + Object { + "type": "literal", + "value": false, + }, + ], + "function": "is", + "type": "function", + }, + Object { + "arguments": Array [ + Object { + "type": "literal", + "value": "alert.attributes.consumer", + }, + Object { + "type": "literal", + "value": "myApp", + }, + Object { + "type": "literal", + "value": false, + }, + ], + "function": "is", + "type": "function", + }, + Object { + "arguments": Array [ + Object { + "type": "literal", + "value": "alert.attributes.consumer", + }, + Object { + "type": "literal", + "value": "myOtherApp", + }, + Object { + "type": "literal", + "value": false, + }, + ], + "function": "is", + "type": "function", + }, + Object { + "arguments": Array [ + Object { + "type": "literal", + "value": "alert.attributes.consumer", + }, + Object { + "type": "literal", + "value": "myAppWithSubFeature", + }, + Object { + "type": "literal", + "value": false, + }, + ], + "function": "is", + "type": "function", + }, + ], + "function": "or", + "type": "function", + }, + ], + "function": "and", + "type": "function", + }, + Object { + "arguments": Array [ + Object { + "arguments": Array [ + Object { + "type": "literal", + "value": "alert.attributes.alertTypeId", + }, + Object { + "type": "literal", + "value": "mySecondAppAlertType", + }, + Object { + "type": "literal", + "value": false, + }, + ], + "function": "is", + "type": "function", + }, + Object { + "arguments": Array [ + Object { + "arguments": Array [ + Object { + "type": "literal", + "value": "alert.attributes.consumer", + }, + Object { + "type": "literal", + "value": "alerts", + }, + Object { + "type": "literal", + "value": false, + }, + ], + "function": "is", + "type": "function", + }, + Object { + "arguments": Array [ + Object { + "type": "literal", + "value": "alert.attributes.consumer", + }, + Object { + "type": "literal", + "value": "myApp", + }, + Object { + "type": "literal", + "value": false, + }, + ], + "function": "is", + "type": "function", + }, + Object { + "arguments": Array [ + Object { + "type": "literal", + "value": "alert.attributes.consumer", + }, + Object { + "type": "literal", + "value": "myOtherApp", + }, + Object { + "type": "literal", + "value": false, + }, + ], + "function": "is", + "type": "function", + }, + Object { + "arguments": Array [ + Object { + "type": "literal", + "value": "alert.attributes.consumer", + }, + Object { + "type": "literal", + "value": "myAppWithSubFeature", + }, + Object { + "type": "literal", + "value": false, + }, + ], + "function": "is", + "type": "function", + }, + ], + "function": "or", + "type": "function", + }, + ], + "function": "and", + "type": "function", + }, + ], + "function": "or", + "type": "function", +} +`; diff --git a/x-pack/plugins/alerts/server/authorization/__snapshots__/alerts_authorization_kuery.test.ts.snap b/x-pack/plugins/alerts/server/authorization/__snapshots__/alerts_authorization_kuery.test.ts.snap new file mode 100644 index 0000000000000..de01a7b27ef05 --- /dev/null +++ b/x-pack/plugins/alerts/server/authorization/__snapshots__/alerts_authorization_kuery.test.ts.snap @@ -0,0 +1,448 @@ +// Jest Snapshot v1, https://goo.gl/fbAQLP + +exports[`asFiltersByAlertTypeAndConsumer constructs filter for multiple alert types across authorized consumer 1`] = ` +Object { + "arguments": Array [ + Object { + "arguments": Array [ + Object { + "arguments": Array [ + Object { + "type": "literal", + "value": "alert.attributes.alertTypeId", + }, + Object { + "type": "literal", + "value": "myAppAlertType", + }, + Object { + "type": "literal", + "value": false, + }, + ], + "function": "is", + "type": "function", + }, + Object { + "arguments": Array [ + Object { + "arguments": Array [ + Object { + "type": "literal", + "value": "alert.attributes.consumer", + }, + Object { + "type": "literal", + "value": "alerts", + }, + Object { + "type": "literal", + "value": false, + }, + ], + "function": "is", + "type": "function", + }, + Object { + "arguments": Array [ + Object { + "type": "literal", + "value": "alert.attributes.consumer", + }, + Object { + "type": "literal", + "value": "myApp", + }, + Object { + "type": "literal", + "value": false, + }, + ], + "function": "is", + "type": "function", + }, + Object { + "arguments": Array [ + Object { + "type": "literal", + "value": "alert.attributes.consumer", + }, + Object { + "type": "literal", + "value": "myOtherApp", + }, + Object { + "type": "literal", + "value": false, + }, + ], + "function": "is", + "type": "function", + }, + Object { + "arguments": Array [ + Object { + "type": "literal", + "value": "alert.attributes.consumer", + }, + Object { + "type": "literal", + "value": "myAppWithSubFeature", + }, + Object { + "type": "literal", + "value": false, + }, + ], + "function": "is", + "type": "function", + }, + ], + "function": "or", + "type": "function", + }, + ], + "function": "and", + "type": "function", + }, + Object { + "arguments": Array [ + Object { + "arguments": Array [ + Object { + "type": "literal", + "value": "alert.attributes.alertTypeId", + }, + Object { + "type": "literal", + "value": "myOtherAppAlertType", + }, + Object { + "type": "literal", + "value": false, + }, + ], + "function": "is", + "type": "function", + }, + Object { + "arguments": Array [ + Object { + "arguments": Array [ + Object { + "type": "literal", + "value": "alert.attributes.consumer", + }, + Object { + "type": "literal", + "value": "alerts", + }, + Object { + "type": "literal", + "value": false, + }, + ], + "function": "is", + "type": "function", + }, + Object { + "arguments": Array [ + Object { + "type": "literal", + "value": "alert.attributes.consumer", + }, + Object { + "type": "literal", + "value": "myApp", + }, + Object { + "type": "literal", + "value": false, + }, + ], + "function": "is", + "type": "function", + }, + Object { + "arguments": Array [ + Object { + "type": "literal", + "value": "alert.attributes.consumer", + }, + Object { + "type": "literal", + "value": "myOtherApp", + }, + Object { + "type": "literal", + "value": false, + }, + ], + "function": "is", + "type": "function", + }, + Object { + "arguments": Array [ + Object { + "type": "literal", + "value": "alert.attributes.consumer", + }, + Object { + "type": "literal", + "value": "myAppWithSubFeature", + }, + Object { + "type": "literal", + "value": false, + }, + ], + "function": "is", + "type": "function", + }, + ], + "function": "or", + "type": "function", + }, + ], + "function": "and", + "type": "function", + }, + Object { + "arguments": Array [ + Object { + "arguments": Array [ + Object { + "type": "literal", + "value": "alert.attributes.alertTypeId", + }, + Object { + "type": "literal", + "value": "mySecondAppAlertType", + }, + Object { + "type": "literal", + "value": false, + }, + ], + "function": "is", + "type": "function", + }, + Object { + "arguments": Array [ + Object { + "arguments": Array [ + Object { + "type": "literal", + "value": "alert.attributes.consumer", + }, + Object { + "type": "literal", + "value": "alerts", + }, + Object { + "type": "literal", + "value": false, + }, + ], + "function": "is", + "type": "function", + }, + Object { + "arguments": Array [ + Object { + "type": "literal", + "value": "alert.attributes.consumer", + }, + Object { + "type": "literal", + "value": "myApp", + }, + Object { + "type": "literal", + "value": false, + }, + ], + "function": "is", + "type": "function", + }, + Object { + "arguments": Array [ + Object { + "type": "literal", + "value": "alert.attributes.consumer", + }, + Object { + "type": "literal", + "value": "myOtherApp", + }, + Object { + "type": "literal", + "value": false, + }, + ], + "function": "is", + "type": "function", + }, + Object { + "arguments": Array [ + Object { + "type": "literal", + "value": "alert.attributes.consumer", + }, + Object { + "type": "literal", + "value": "myAppWithSubFeature", + }, + Object { + "type": "literal", + "value": false, + }, + ], + "function": "is", + "type": "function", + }, + ], + "function": "or", + "type": "function", + }, + ], + "function": "and", + "type": "function", + }, + ], + "function": "or", + "type": "function", +} +`; + +exports[`asFiltersByAlertTypeAndConsumer constructs filter for single alert type with multiple authorized consumer 1`] = ` +Object { + "arguments": Array [ + Object { + "arguments": Array [ + Object { + "type": "literal", + "value": "alert.attributes.alertTypeId", + }, + Object { + "type": "literal", + "value": "myAppAlertType", + }, + Object { + "type": "literal", + "value": false, + }, + ], + "function": "is", + "type": "function", + }, + Object { + "arguments": Array [ + Object { + "arguments": Array [ + Object { + "type": "literal", + "value": "alert.attributes.consumer", + }, + Object { + "type": "literal", + "value": "alerts", + }, + Object { + "type": "literal", + "value": false, + }, + ], + "function": "is", + "type": "function", + }, + Object { + "arguments": Array [ + Object { + "type": "literal", + "value": "alert.attributes.consumer", + }, + Object { + "type": "literal", + "value": "myApp", + }, + Object { + "type": "literal", + "value": false, + }, + ], + "function": "is", + "type": "function", + }, + Object { + "arguments": Array [ + Object { + "type": "literal", + "value": "alert.attributes.consumer", + }, + Object { + "type": "literal", + "value": "myOtherApp", + }, + Object { + "type": "literal", + "value": false, + }, + ], + "function": "is", + "type": "function", + }, + ], + "function": "or", + "type": "function", + }, + ], + "function": "and", + "type": "function", +} +`; + +exports[`asFiltersByAlertTypeAndConsumer constructs filter for single alert type with single authorized consumer 1`] = ` +Object { + "arguments": Array [ + Object { + "arguments": Array [ + Object { + "type": "literal", + "value": "alert.attributes.alertTypeId", + }, + Object { + "type": "literal", + "value": "myAppAlertType", + }, + Object { + "type": "literal", + "value": false, + }, + ], + "function": "is", + "type": "function", + }, + Object { + "arguments": Array [ + Object { + "type": "literal", + "value": "alert.attributes.consumer", + }, + Object { + "type": "literal", + "value": "myApp", + }, + Object { + "type": "literal", + "value": false, + }, + ], + "function": "is", + "type": "function", + }, + ], + "function": "and", + "type": "function", +} +`; diff --git a/x-pack/plugins/alerts/server/authorization/alerts_authorization.test.ts b/x-pack/plugins/alerts/server/authorization/alerts_authorization.test.ts index a7d9421073483..fc895f3e308f4 100644 --- a/x-pack/plugins/alerts/server/authorization/alerts_authorization.test.ts +++ b/x-pack/plugins/alerts/server/authorization/alerts_authorization.test.ts @@ -6,7 +6,6 @@ import { KibanaRequest } from 'kibana/server'; import { alertTypeRegistryMock } from '../alert_type_registry.mock'; import { securityMock } from '../../../../plugins/security/server/mocks'; -import { esKuery } from '../../../../../src/plugins/data/server'; import { PluginStartContract as FeaturesStartContract, KibanaFeature, @@ -627,11 +626,17 @@ describe('AlertsAuthorization', () => { }); alertTypeRegistry.list.mockReturnValue(setOfAlertTypes); - expect((await alertAuthorization.getFindAuthorizationFilter()).filter).toEqual( - esKuery.fromKueryExpression( - `((alert.attributes.alertTypeId:myAppAlertType and alert.attributes.consumer:(alerts or myApp or myOtherApp or myAppWithSubFeature)) or (alert.attributes.alertTypeId:myOtherAppAlertType and alert.attributes.consumer:(alerts or myApp or myOtherApp or myAppWithSubFeature)) or (alert.attributes.alertTypeId:mySecondAppAlertType and alert.attributes.consumer:(alerts or myApp or myOtherApp or myAppWithSubFeature)))` - ) - ); + // TODO: once issue https://github.com/elastic/kibana/issues/89473 is + // resolved, we can start using this code again, instead of toMatchSnapshot(): + // + // expect((await alertAuthorization.getFindAuthorizationFilter()).filter).toEqual( + // esKuery.fromKueryExpression( + // `((alert.attributes.alertTypeId:myAppAlertType and alert.attributes.consumer:(alerts or myApp or myOtherApp or myAppWithSubFeature)) or (alert.attributes.alertTypeId:myOtherAppAlertType and alert.attributes.consumer:(alerts or myApp or myOtherApp or myAppWithSubFeature)) or (alert.attributes.alertTypeId:mySecondAppAlertType and alert.attributes.consumer:(alerts or myApp or myOtherApp or myAppWithSubFeature)))` + // ) + // ); + + // This code is the replacement code for above + expect((await alertAuthorization.getFindAuthorizationFilter()).filter).toMatchSnapshot(); expect(auditLogger.alertsAuthorizationSuccess).not.toHaveBeenCalled(); }); diff --git a/x-pack/plugins/alerts/server/authorization/alerts_authorization_kuery.test.ts b/x-pack/plugins/alerts/server/authorization/alerts_authorization_kuery.test.ts index 8249047c0ef39..3d80ff0273db7 100644 --- a/x-pack/plugins/alerts/server/authorization/alerts_authorization_kuery.test.ts +++ b/x-pack/plugins/alerts/server/authorization/alerts_authorization_kuery.test.ts @@ -3,7 +3,6 @@ * or more contributor license agreements. Licensed under the Elastic License; * you may not use this file except in compliance with the Elastic License. */ -import { esKuery } from '../../../../../src/plugins/data/server'; import { RecoveredActionGroup } from '../../common'; import { asFiltersByAlertTypeAndConsumer, @@ -30,11 +29,14 @@ describe('asFiltersByAlertTypeAndConsumer', () => { }, ]) ) - ).toEqual( - esKuery.fromKueryExpression( - `((alert.attributes.alertTypeId:myAppAlertType and alert.attributes.consumer:(myApp)))` - ) - ); + ).toMatchSnapshot(); + // TODO: once issue https://github.com/elastic/kibana/issues/89473 is + // resolved, we can start using this code again instead of toMatchSnapshot() + // ).toEqual( + // esKuery.fromKueryExpression( + // `((alert.attributes.alertTypeId:myAppAlertType and alert.attributes.consumer:(myApp)))` + // ) + // ); }); test('constructs filter for single alert type with multiple authorized consumer', async () => { @@ -58,11 +60,14 @@ describe('asFiltersByAlertTypeAndConsumer', () => { }, ]) ) - ).toEqual( - esKuery.fromKueryExpression( - `((alert.attributes.alertTypeId:myAppAlertType and alert.attributes.consumer:(alerts or myApp or myOtherApp)))` - ) - ); + ).toMatchSnapshot(); + // TODO: once issue https://github.com/elastic/kibana/issues/89473 is + // resolved, we can start using this code again, instead of toMatchSnapshot(): + // ).toEqual( + // esKuery.fromKueryExpression( + // `((alert.attributes.alertTypeId:myAppAlertType and alert.attributes.consumer:(alerts or myApp or myOtherApp)))` + // ) + // ); }); test('constructs filter for multiple alert types across authorized consumer', async () => { @@ -119,11 +124,14 @@ describe('asFiltersByAlertTypeAndConsumer', () => { }, ]) ) - ).toEqual( - esKuery.fromKueryExpression( - `((alert.attributes.alertTypeId:myAppAlertType and alert.attributes.consumer:(alerts or myApp or myOtherApp or myAppWithSubFeature)) or (alert.attributes.alertTypeId:myOtherAppAlertType and alert.attributes.consumer:(alerts or myApp or myOtherApp or myAppWithSubFeature)) or (alert.attributes.alertTypeId:mySecondAppAlertType and alert.attributes.consumer:(alerts or myApp or myOtherApp or myAppWithSubFeature)))` - ) - ); + ).toMatchSnapshot(); + // TODO: once issue https://github.com/elastic/kibana/issues/89473 is + // resolved, we can start using this code again, instead of toMatchSnapshot(): + // ).toEqual( + // esKuery.fromKueryExpression( + // `((alert.attributes.alertTypeId:myAppAlertType and alert.attributes.consumer:(alerts or myApp or myOtherApp or myAppWithSubFeature)) or (alert.attributes.alertTypeId:myOtherAppAlertType and alert.attributes.consumer:(alerts or myApp or myOtherApp or myAppWithSubFeature)) or (alert.attributes.alertTypeId:mySecondAppAlertType and alert.attributes.consumer:(alerts or myApp or myOtherApp or myAppWithSubFeature)))` + // ) + // ); }); }); diff --git a/x-pack/plugins/alerts/server/routes/create.test.ts b/x-pack/plugins/alerts/server/routes/create.test.ts index fc531821f25b6..d0e21ac99a264 100644 --- a/x-pack/plugins/alerts/server/routes/create.test.ts +++ b/x-pack/plugins/alerts/server/routes/create.test.ts @@ -82,7 +82,7 @@ describe('createAlertRoute', () => { const [config, handler] = router.post.mock.calls[0]; - expect(config.path).toMatchInlineSnapshot(`"/api/alerts/alert"`); + expect(config.path).toMatchInlineSnapshot(`"/api/alerts/alert/{id?}"`); alertsClient.create.mockResolvedValueOnce(createResult); @@ -125,6 +125,9 @@ describe('createAlertRoute', () => { ], "throttle": "30s", }, + "options": Object { + "id": undefined, + }, }, ] `); @@ -134,6 +137,74 @@ describe('createAlertRoute', () => { }); }); + it('allows providing a custom id', async () => { + const expectedResult = { + ...createResult, + id: 'custom-id', + }; + const licenseState = licenseStateMock.create(); + const router = httpServiceMock.createRouter(); + + createAlertRoute(router, licenseState); + + const [config, handler] = router.post.mock.calls[0]; + + expect(config.path).toMatchInlineSnapshot(`"/api/alerts/alert/{id?}"`); + + alertsClient.create.mockResolvedValueOnce(expectedResult); + + const [context, req, res] = mockHandlerArguments( + { alertsClient }, + { + params: { id: 'custom-id' }, + body: mockedAlert, + }, + ['ok'] + ); + + expect(await handler(context, req, res)).toEqual({ body: expectedResult }); + + expect(alertsClient.create).toHaveBeenCalledTimes(1); + expect(alertsClient.create.mock.calls[0]).toMatchInlineSnapshot(` + Array [ + Object { + "data": Object { + "actions": Array [ + Object { + "group": "default", + "id": "2", + "params": Object { + "foo": true, + }, + }, + ], + "alertTypeId": "1", + "consumer": "bar", + "name": "abc", + "notifyWhen": "onActionGroupChange", + "params": Object { + "bar": true, + }, + "schedule": Object { + "interval": "10s", + }, + "tags": Array [ + "foo", + ], + "throttle": "30s", + }, + "options": Object { + "id": "custom-id", + }, + }, + ] + `); + + expect(res.ok).toHaveBeenCalledWith({ + body: expectedResult, + }); + }); + it('ensures the license allows creating alerts', async () => { const licenseState = licenseStateMock.create(); const router = httpServiceMock.createRouter(); diff --git a/x-pack/plugins/alerts/server/routes/create.ts b/x-pack/plugins/alerts/server/routes/create.ts index 2b6735d9063df..46151893baef5 100644 --- a/x-pack/plugins/alerts/server/routes/create.ts +++ b/x-pack/plugins/alerts/server/routes/create.ts @@ -45,8 +45,13 @@ export const bodySchema = schema.object({ export const createAlertRoute = (router: AlertingRouter, licenseState: ILicenseState) => { router.post( { - path: `${BASE_ALERT_API_PATH}/alert`, + path: `${BASE_ALERT_API_PATH}/alert/{id?}`, validate: { + params: schema.maybe( + schema.object({ + id: schema.maybe(schema.string()), + }) + ), body: bodySchema, }, }, @@ -59,10 +64,12 @@ export const createAlertRoute = (router: AlertingRouter, licenseState: ILicenseS } const alertsClient = context.alerting.getAlertsClient(); const alert = req.body; + const params = req.params; const notifyWhen = alert?.notifyWhen ? (alert.notifyWhen as AlertNotifyWhenType) : null; try { const alertRes: Alert = await alertsClient.create({ data: { ...alert, notifyWhen }, + options: { id: params?.id }, }); return res.ok({ body: alertRes, diff --git a/x-pack/plugins/apm/public/components/app/RumDashboard/VisitorBreakdownMap/useLayerList.ts b/x-pack/plugins/apm/public/components/app/RumDashboard/VisitorBreakdownMap/useLayerList.ts index c8bbe599ca44f..3ff6686138e9a 100644 --- a/x-pack/plugins/apm/public/components/app/RumDashboard/VisitorBreakdownMap/useLayerList.ts +++ b/x-pack/plugins/apm/public/components/app/RumDashboard/VisitorBreakdownMap/useLayerList.ts @@ -16,6 +16,7 @@ import { COLOR_MAP_TYPE, FIELD_ORIGIN, LABEL_BORDER_SIZES, + SOURCE_TYPES, STYLE_TYPE, SYMBOLIZE_AS_TYPES, } from '../../../../../../maps/common/constants'; @@ -29,7 +30,7 @@ import { import { TRANSACTION_PAGE_LOAD } from '../../../../../common/transaction_types'; const ES_TERM_SOURCE_COUNTRY: ESTermSourceDescriptor = { - type: 'ES_TERM_SOURCE', + type: SOURCE_TYPES.ES_TERM_SOURCE, id: '3657625d-17b0-41ef-99ba-3a2b2938655c', indexPatternTitle: 'apm-*', term: 'client.geo.country_iso_code', @@ -46,7 +47,7 @@ const ES_TERM_SOURCE_COUNTRY: ESTermSourceDescriptor = { }; const ES_TERM_SOURCE_REGION: ESTermSourceDescriptor = { - type: 'ES_TERM_SOURCE', + type: SOURCE_TYPES.ES_TERM_SOURCE, id: 'e62a1b9c-d7ff-4fd4-a0f6-0fdc44bb9e41', indexPatternTitle: 'apm-*', term: 'client.geo.region_iso_code', diff --git a/x-pack/plugins/apm/public/components/shared/Links/apm/ExternalLinks.test.ts b/x-pack/plugins/apm/public/components/shared/Links/apm/ExternalLinks.test.ts deleted file mode 100644 index 1f772e0734404..0000000000000 --- a/x-pack/plugins/apm/public/components/shared/Links/apm/ExternalLinks.test.ts +++ /dev/null @@ -1,26 +0,0 @@ -/* - * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one - * or more contributor license agreements. Licensed under the Elastic License; - * you may not use this file except in compliance with the Elastic License. - */ -import { getTraceUrl } from './ExternalLinks'; - -jest.mock('../../../app/Main/route_config/index.tsx', () => ({ - routes: [ - { - name: 'link_to_trace', - path: '/link-to/trace/:traceId', - }, - ], -})); - -describe('ExternalLinks', () => { - afterAll(() => { - jest.clearAllMocks(); - }); - it('trace link', () => { - expect( - getTraceUrl({ traceId: 'foo', rangeFrom: '123', rangeTo: '456' }) - ).toEqual('/link-to/trace/foo?rangeFrom=123&rangeTo=456'); - }); -}); diff --git a/x-pack/plugins/apm/public/components/shared/TransactionActionMenu/CustomLinkMenuSection/index.tsx b/x-pack/plugins/apm/public/components/shared/TransactionActionMenu/CustomLinkMenuSection/index.tsx index ae22718af8b57..43f566a93a89d 100644 --- a/x-pack/plugins/apm/public/components/shared/TransactionActionMenu/CustomLinkMenuSection/index.tsx +++ b/x-pack/plugins/apm/public/components/shared/TransactionActionMenu/CustomLinkMenuSection/index.tsx @@ -107,7 +107,6 @@ export function CustomLinkMenuSection({ - {i18n.translate( 'xpack.apm.transactionActionMenu.customLink.subtitle', diff --git a/x-pack/plugins/apm/public/components/shared/TransactionActionMenu/TransactionActionMenu.test.tsx b/x-pack/plugins/apm/public/components/shared/TransactionActionMenu/TransactionActionMenu.test.tsx index 48c863b460482..3141dc7a5f3c6 100644 --- a/x-pack/plugins/apm/public/components/shared/TransactionActionMenu/TransactionActionMenu.test.tsx +++ b/x-pack/plugins/apm/public/components/shared/TransactionActionMenu/TransactionActionMenu.test.tsx @@ -52,7 +52,7 @@ const renderTransaction = async (transaction: Record) => { } ); - fireEvent.click(rendered.getByText('Actions')); + fireEvent.click(rendered.getByText('Investigate')); return rendered; }; @@ -289,7 +289,7 @@ describe('TransactionActionMenu component', () => { }); const component = renderTransactionActionMenuWithLicense(license); act(() => { - fireEvent.click(component.getByText('Actions')); + fireEvent.click(component.getByText('Investigate')); }); expectTextsNotInDocument(component, ['Custom Links']); }); @@ -313,7 +313,7 @@ describe('TransactionActionMenu component', () => { { wrapper: Wrapper } ); act(() => { - fireEvent.click(component.getByText('Actions')); + fireEvent.click(component.getByText('Investigate')); }); expectTextsNotInDocument(component, ['Custom Links']); }); @@ -330,7 +330,7 @@ describe('TransactionActionMenu component', () => { }); const component = renderTransactionActionMenuWithLicense(license); act(() => { - fireEvent.click(component.getByText('Actions')); + fireEvent.click(component.getByText('Investigate')); }); expectTextsInDocument(component, ['Custom Links']); }); @@ -347,7 +347,7 @@ describe('TransactionActionMenu component', () => { }); const component = renderTransactionActionMenuWithLicense(license); act(() => { - fireEvent.click(component.getByText('Actions')); + fireEvent.click(component.getByText('Investigate')); }); expectTextsInDocument(component, ['Custom Links']); }); @@ -364,7 +364,7 @@ describe('TransactionActionMenu component', () => { }); const component = renderTransactionActionMenuWithLicense(license); act(() => { - fireEvent.click(component.getByText('Actions')); + fireEvent.click(component.getByText('Investigate')); }); expectTextsInDocument(component, ['Custom Links']); act(() => { diff --git a/x-pack/plugins/apm/public/components/shared/TransactionActionMenu/TransactionActionMenu.tsx b/x-pack/plugins/apm/public/components/shared/TransactionActionMenu/TransactionActionMenu.tsx index 312513db80886..22fa25f93b212 100644 --- a/x-pack/plugins/apm/public/components/shared/TransactionActionMenu/TransactionActionMenu.tsx +++ b/x-pack/plugins/apm/public/components/shared/TransactionActionMenu/TransactionActionMenu.tsx @@ -4,7 +4,7 @@ * you may not use this file except in compliance with the Elastic License. */ -import { EuiButtonEmpty } from '@elastic/eui'; +import { EuiButton } from '@elastic/eui'; import { i18n } from '@kbn/i18n'; import React, { useState } from 'react'; import { useLocation } from 'react-router-dom'; @@ -30,11 +30,11 @@ interface Props { function ActionMenuButton({ onClick }: { onClick: () => void }) { return ( - + {i18n.translate('xpack.apm.transactionActionMenu.actionsButtonLabel', { - defaultMessage: 'Actions', + defaultMessage: 'Investigate', })} - + ); } diff --git a/x-pack/plugins/apm/public/components/shared/TransactionActionMenu/__snapshots__/TransactionActionMenu.test.tsx.snap b/x-pack/plugins/apm/public/components/shared/TransactionActionMenu/__snapshots__/TransactionActionMenu.test.tsx.snap index fa6db645d28a8..ea33fb3c3df08 100644 --- a/x-pack/plugins/apm/public/components/shared/TransactionActionMenu/__snapshots__/TransactionActionMenu.test.tsx.snap +++ b/x-pack/plugins/apm/public/components/shared/TransactionActionMenu/__snapshots__/TransactionActionMenu.test.tsx.snap @@ -10,20 +10,20 @@ exports[`TransactionActionMenu component matches the snapshot 1`] = ` class="euiPopover__anchor" > diff --git a/x-pack/plugins/apm/public/index.ts b/x-pack/plugins/apm/public/index.ts index 40992d7b58e65..d7a8573f2080b 100644 --- a/x-pack/plugins/apm/public/index.ts +++ b/x-pack/plugins/apm/public/index.ts @@ -22,4 +22,3 @@ export const plugin: PluginInitializer = ( ) => new ApmPlugin(pluginInitializerContext); export { ApmPluginSetup, ApmPluginStart }; -export { getTraceUrl } from './components/shared/Links/apm/ExternalLinks'; diff --git a/x-pack/plugins/apm/server/lib/alerts/chart_preview/get_transaction_error_rate.ts b/x-pack/plugins/apm/server/lib/alerts/chart_preview/get_transaction_error_rate.ts index fae43ef148cfa..f6ddb15cbffa9 100644 --- a/x-pack/plugins/apm/server/lib/alerts/chart_preview/get_transaction_error_rate.ts +++ b/x-pack/plugins/apm/server/lib/alerts/chart_preview/get_transaction_error_rate.ts @@ -44,9 +44,7 @@ export async function getTransactionErrorRateChartPreview({ }, }; - const outcomes = getOutcomeAggregation({ - searchAggregatedTransactions: false, - }); + const outcomes = getOutcomeAggregation(); const { intervalString } = getBucketSize({ start, end, numBuckets: 20 }); diff --git a/x-pack/plugins/apm/server/lib/correlations/get_correlations_for_failed_transactions/index.ts b/x-pack/plugins/apm/server/lib/correlations/get_correlations_for_failed_transactions/index.ts index 64d9ebb192eb3..9ecf201ede1b7 100644 --- a/x-pack/plugins/apm/server/lib/correlations/get_correlations_for_failed_transactions/index.ts +++ b/x-pack/plugins/apm/server/lib/correlations/get_correlations_for_failed_transactions/index.ts @@ -4,7 +4,7 @@ * you may not use this file except in compliance with the Elastic License. */ -import { isEmpty, omit } from 'lodash'; +import { isEmpty, omit, merge } from 'lodash'; import { EventOutcome } from '../../../../common/event_outcome'; import { processSignificantTermAggs, @@ -134,8 +134,7 @@ export async function getErrorRateTimeSeries({ extended_bounds: { min: start, max: end }, }, aggs: { - // TODO: add support for metrics - outcomes: getOutcomeAggregation({ searchAggregatedTransactions: false }), + outcomes: getOutcomeAggregation(), }, }; @@ -147,13 +146,12 @@ export async function getErrorRateTimeSeries({ }; return acc; }, - {} as Record< - string, - { + {} as { + [key: string]: { filter: AggregationOptionsByType['filter']; aggs: { timeseries: typeof timeseriesAgg }; - } - > + }; + } ); const params = { @@ -162,32 +160,25 @@ export async function getErrorRateTimeSeries({ body: { size: 0, query: { bool: { filter: backgroundFilters } }, - aggs: { - // overall aggs - timeseries: timeseriesAgg, - - // per term aggs - ...perTermAggs, - }, + aggs: merge({ timeseries: timeseriesAgg }, perTermAggs), }, }; const response = await apmEventClient.search(params); - type Agg = NonNullable; + const { aggregations } = response; - if (!response.aggregations) { + if (!aggregations) { return {}; } return { overall: { timeseries: getTransactionErrorRateTimeSeries( - response.aggregations.timeseries.buckets + aggregations.timeseries.buckets ), }, significantTerms: topSigTerms.map((topSig, index) => { - // @ts-expect-error - const agg = response.aggregations[`term_${index}`] as Agg; + const agg = aggregations[`term_${index}`]!; return { ...topSig, diff --git a/x-pack/plugins/apm/server/lib/helpers/calculate_throughput.ts b/x-pack/plugins/apm/server/lib/helpers/calculate_throughput.ts new file mode 100644 index 0000000000000..7fcbe9e798188 --- /dev/null +++ b/x-pack/plugins/apm/server/lib/helpers/calculate_throughput.ts @@ -0,0 +1,16 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License; + * you may not use this file except in compliance with the Elastic License. + */ + +import { SetupTimeRange } from './setup_request'; + +export function calculateThroughput({ + start, + end, + value, +}: SetupTimeRange & { value: number }) { + const durationAsMinutes = (end - start) / 1000 / 60; + return value / durationAsMinutes; +} diff --git a/x-pack/plugins/apm/server/lib/helpers/transaction_error_rate.ts b/x-pack/plugins/apm/server/lib/helpers/transaction_error_rate.ts index 876fc6b822213..2d041006e0e27 100644 --- a/x-pack/plugins/apm/server/lib/helpers/transaction_error_rate.ts +++ b/x-pack/plugins/apm/server/lib/helpers/transaction_error_rate.ts @@ -10,40 +10,21 @@ import { AggregationOptionsByType, AggregationResultOf, } from '../../../../../typings/elasticsearch/aggregations'; -import { getTransactionDurationFieldForAggregatedTransactions } from './aggregated_transactions'; -export function getOutcomeAggregation({ - searchAggregatedTransactions, -}: { - searchAggregatedTransactions: boolean; -}) { - return { - terms: { - field: EVENT_OUTCOME, - include: [EventOutcome.failure, EventOutcome.success], - }, - aggs: { - // simply using the doc count to get the number of requests is not possible for transaction metrics (histograms) - // to work around this we get the number of transactions by counting the number of latency values - count: { - value_count: { - field: getTransactionDurationFieldForAggregatedTransactions( - searchAggregatedTransactions - ), - }, - }, - }, - }; -} +export const getOutcomeAggregation = () => ({ + terms: { + field: EVENT_OUTCOME, + include: [EventOutcome.failure, EventOutcome.success], + }, +}); + +type OutcomeAggregation = ReturnType; export function calculateTransactionErrorPercentage( - outcomeResponse: AggregationResultOf< - ReturnType, - {} - > + outcomeResponse: AggregationResultOf ) { const outcomes = Object.fromEntries( - outcomeResponse.buckets.map(({ key, count }) => [key, count.value]) + outcomeResponse.buckets.map(({ key, doc_count: count }) => [key, count]) ); const failedTransactions = outcomes[EventOutcome.failure] ?? 0; @@ -56,7 +37,7 @@ export function getTransactionErrorRateTimeSeries( buckets: AggregationResultOf< { date_histogram: AggregationOptionsByType['date_histogram']; - aggs: { outcomes: ReturnType }; + aggs: { outcomes: OutcomeAggregation }; }, {} >['buckets'] diff --git a/x-pack/plugins/apm/server/lib/observability_overview/get_transaction_coordinates.ts b/x-pack/plugins/apm/server/lib/observability_overview/get_transaction_coordinates.ts index 5531944fc7180..f844a6ce1c345 100644 --- a/x-pack/plugins/apm/server/lib/observability_overview/get_transaction_coordinates.ts +++ b/x-pack/plugins/apm/server/lib/observability_overview/get_transaction_coordinates.ts @@ -11,10 +11,8 @@ import { rangeFilter } from '../../../common/utils/range_filter'; import { Coordinates } from '../../../../observability/typings/common'; import { Setup, SetupTimeRange } from '../helpers/setup_request'; -import { - getProcessorEventForAggregatedTransactions, - getTransactionDurationFieldForAggregatedTransactions, -} from '../helpers/aggregated_transactions'; +import { getProcessorEventForAggregatedTransactions } from '../helpers/aggregated_transactions'; +import { calculateThroughput } from '../helpers/calculate_throughput'; export async function getTransactionCoordinates({ setup, @@ -49,26 +47,15 @@ export async function getTransactionCoordinates({ fixed_interval: bucketSize, min_doc_count: 0, }, - aggs: { - count: { - value_count: { - field: getTransactionDurationFieldForAggregatedTransactions( - searchAggregatedTransactions - ), - }, - }, - }, }, }, }, }); - const deltaAsMinutes = (end - start) / 1000 / 60; - return ( aggregations?.distribution.buckets.map((bucket) => ({ x: bucket.key, - y: bucket.count.value / deltaAsMinutes, + y: calculateThroughput({ start, end, value: bucket.doc_count }), })) || [] ); } diff --git a/x-pack/plugins/apm/server/lib/service_map/get_service_map_service_node_info.test.ts b/x-pack/plugins/apm/server/lib/service_map/get_service_map_service_node_info.test.ts index f7ca40ef1052c..173de796d47e4 100644 --- a/x-pack/plugins/apm/server/lib/service_map/get_service_map_service_node_info.test.ts +++ b/x-pack/plugins/apm/server/lib/service_map/get_service_map_service_node_info.test.ts @@ -52,8 +52,10 @@ describe('getServiceMapServiceNodeInfo', () => { apmEventClient: { search: () => Promise.resolve({ + hits: { + total: { value: 1 }, + }, aggregations: { - count: { value: 1 }, duration: { value: null }, avgCpuUsage: { value: null }, avgMemoryUsage: { value: null }, diff --git a/x-pack/plugins/apm/server/lib/service_map/get_service_map_service_node_info.ts b/x-pack/plugins/apm/server/lib/service_map/get_service_map_service_node_info.ts index 82d339686f7ec..4fe9a1a75d43f 100644 --- a/x-pack/plugins/apm/server/lib/service_map/get_service_map_service_node_info.ts +++ b/x-pack/plugins/apm/server/lib/service_map/get_service_map_service_node_info.ts @@ -162,19 +162,12 @@ async function getTransactionStats({ ), }, }, - count: { - value_count: { - field: getTransactionDurationFieldForAggregatedTransactions( - searchAggregatedTransactions - ), - }, - }, }, }, }; const response = await apmEventClient.search(params); - const totalRequests = response.aggregations?.count.value ?? 0; + const totalRequests = response.hits.total.value; return { avgTransactionDuration: response.aggregations?.duration.value ?? null, diff --git a/x-pack/plugins/apm/server/lib/services/__snapshots__/queries.test.ts.snap b/x-pack/plugins/apm/server/lib/services/__snapshots__/queries.test.ts.snap index 21402e4c8dac0..239b909e1572c 100644 --- a/x-pack/plugins/apm/server/lib/services/__snapshots__/queries.test.ts.snap +++ b/x-pack/plugins/apm/server/lib/services/__snapshots__/queries.test.ts.snap @@ -122,13 +122,6 @@ Array [ }, }, "outcomes": Object { - "aggs": Object { - "count": Object { - "value_count": Object { - "field": "transaction.duration.us", - }, - }, - }, "terms": Object { "field": "event.outcome", "include": Array [ @@ -137,11 +130,6 @@ Array [ ], }, }, - "real_document_count": Object { - "value_count": Object { - "field": "transaction.duration.us", - }, - }, "timeseries": Object { "aggs": Object { "avg_duration": Object { @@ -150,13 +138,6 @@ Array [ }, }, "outcomes": Object { - "aggs": Object { - "count": Object { - "value_count": Object { - "field": "transaction.duration.us", - }, - }, - }, "terms": Object { "field": "event.outcome", "include": Array [ @@ -165,11 +146,6 @@ Array [ ], }, }, - "real_document_count": Object { - "value_count": Object { - "field": "transaction.duration.us", - }, - }, }, "date_histogram": Object { "extended_bounds": Object { @@ -184,9 +160,6 @@ Array [ }, "terms": Object { "field": "transaction.type", - "order": Object { - "real_document_count": "desc", - }, }, }, }, diff --git a/x-pack/plugins/apm/server/lib/services/get_service_dependencies/index.ts b/x-pack/plugins/apm/server/lib/services/get_service_dependencies/index.ts index 0ac881aeac00e..2b209f8f6a80a 100644 --- a/x-pack/plugins/apm/server/lib/services/get_service_dependencies/index.ts +++ b/x-pack/plugins/apm/server/lib/services/get_service_dependencies/index.ts @@ -13,6 +13,7 @@ import { joinByKey } from '../../../../common/utils/join_by_key'; import { Setup, SetupTimeRange } from '../../helpers/setup_request'; import { getMetrics } from './get_metrics'; import { getDestinationMap } from './get_destination_map'; +import { calculateThroughput } from '../../helpers/calculate_throughput'; export type ServiceDependencyItem = { name: string; @@ -51,7 +52,6 @@ export async function getServiceDependencies({ numBuckets: number; }): Promise { const { start, end } = setup; - const [allMetrics, destinationMap] = await Promise.all([ getMetrics({ setup, @@ -134,8 +134,6 @@ export async function getServiceDependencies({ } ); - const deltaAsMinutes = (end - start) / 60 / 1000; - const destMetrics = { latency: { value: @@ -150,11 +148,18 @@ export async function getServiceDependencies({ throughput: { value: mergedMetrics.value.count > 0 - ? mergedMetrics.value.count / deltaAsMinutes + ? calculateThroughput({ + start, + end, + value: mergedMetrics.value.count, + }) : null, timeseries: mergedMetrics.timeseries.map((point) => ({ x: point.x, - y: point.count > 0 ? point.count / deltaAsMinutes : null, + y: + point.count > 0 + ? calculateThroughput({ start, end, value: point.count }) + : null, })), }, errorRate: { diff --git a/x-pack/plugins/apm/server/lib/services/get_service_instances/get_service_instance_transaction_stats.ts b/x-pack/plugins/apm/server/lib/services/get_service_instances/get_service_instance_transaction_stats.ts index 5880b5cbc9546..118fbc64146a7 100644 --- a/x-pack/plugins/apm/server/lib/services/get_service_instances/get_service_instance_transaction_stats.ts +++ b/x-pack/plugins/apm/server/lib/services/get_service_instances/get_service_instance_transaction_stats.ts @@ -19,6 +19,7 @@ import { getProcessorEventForAggregatedTransactions, getTransactionDurationFieldForAggregatedTransactions, } from '../../helpers/aggregated_transactions'; +import { calculateThroughput } from '../../helpers/calculate_throughput'; export async function getServiceInstanceTransactionStats({ setup, @@ -30,18 +31,17 @@ export async function getServiceInstanceTransactionStats({ }: ServiceInstanceParams) { const { apmEventClient, start, end, esFilter } = setup; - const { intervalString } = getBucketSize({ start, end, numBuckets }); + const { intervalString, bucketSize } = getBucketSize({ + start, + end, + numBuckets, + }); const field = getTransactionDurationFieldForAggregatedTransactions( searchAggregatedTransactions ); const subAggs = { - count: { - value_count: { - field, - }, - }, avg_transaction_duration: { avg: { field, @@ -53,13 +53,6 @@ export async function getServiceInstanceTransactionStats({ [EVENT_OUTCOME]: EventOutcome.failure, }, }, - aggs: { - count: { - value_count: { - field, - }, - }, - }, }, }; @@ -112,13 +105,13 @@ export async function getServiceInstanceTransactionStats({ }, }); - const deltaAsMinutes = (end - start) / 60 / 1000; + const bucketSizeInMinutes = bucketSize / 60; return ( response.aggregations?.[SERVICE_NODE_NAME].buckets.map( (serviceNodeBucket) => { const { - count, + doc_count: count, avg_transaction_duration: avgTransactionDuration, key, failures, @@ -128,17 +121,17 @@ export async function getServiceInstanceTransactionStats({ return { serviceNodeName: String(key), errorRate: { - value: failures.count.value / count.value, + value: failures.doc_count / count, timeseries: timeseries.buckets.map((dateBucket) => ({ x: dateBucket.key, - y: dateBucket.failures.count.value / dateBucket.count.value, + y: dateBucket.failures.doc_count / dateBucket.doc_count, })), }, throughput: { - value: count.value / deltaAsMinutes, + value: calculateThroughput({ start, end, value: count }), timeseries: timeseries.buckets.map((dateBucket) => ({ x: dateBucket.key, - y: dateBucket.count.value / deltaAsMinutes, + y: dateBucket.doc_count / bucketSizeInMinutes, })), }, latency: { diff --git a/x-pack/plugins/apm/server/lib/services/get_service_transaction_groups/get_timeseries_data_for_transaction_groups.ts b/x-pack/plugins/apm/server/lib/services/get_service_transaction_groups/get_timeseries_data_for_transaction_groups.ts index 937155bc31602..745535f261673 100644 --- a/x-pack/plugins/apm/server/lib/services/get_service_transaction_groups/get_timeseries_data_for_transaction_groups.ts +++ b/x-pack/plugins/apm/server/lib/services/get_service_transaction_groups/get_timeseries_data_for_transaction_groups.ts @@ -17,6 +17,7 @@ import { import { ESFilter } from '../../../../../../typings/elasticsearch'; import { + getDocumentTypeFilterForAggregatedTransactions, getProcessorEventForAggregatedTransactions, getTransactionDurationFieldForAggregatedTransactions, } from '../../helpers/aggregated_transactions'; @@ -76,6 +77,9 @@ export async function getTimeseriesDataForTransactionGroups({ { term: { [SERVICE_NAME]: serviceName } }, { term: { [TRANSACTION_TYPE]: transactionType } }, { range: rangeFilter(start, end) }, + ...getDocumentTypeFilterForAggregatedTransactions( + searchAggregatedTransactions + ), ...esFilter, ], }, @@ -99,10 +103,8 @@ export async function getTimeseriesDataForTransactionGroups({ }, aggs: { ...getLatencyAggregation(latencyAggregationType, field), - transaction_count: { value_count: { field } }, [EVENT_OUTCOME]: { filter: { term: { [EVENT_OUTCOME]: EventOutcome.failure } }, - aggs: { transaction_count: { value_count: { field } } }, }, }, }, diff --git a/x-pack/plugins/apm/server/lib/services/get_service_transaction_groups/get_transaction_groups_for_page.ts b/x-pack/plugins/apm/server/lib/services/get_service_transaction_groups/get_transaction_groups_for_page.ts index ccccf946512dd..77642c1f3d65f 100644 --- a/x-pack/plugins/apm/server/lib/services/get_service_transaction_groups/get_transaction_groups_for_page.ts +++ b/x-pack/plugins/apm/server/lib/services/get_service_transaction_groups/get_transaction_groups_for_page.ts @@ -25,6 +25,7 @@ import { getLatencyAggregation, getLatencyValue, } from '../../helpers/latency_aggregation_type'; +import { calculateThroughput } from '../../helpers/calculate_throughput'; export type ServiceOverviewTransactionGroupSortField = | 'name' @@ -64,8 +65,6 @@ export async function getTransactionGroupsForPage({ transactionType: string; latencyAggregationType: LatencyAggregationType; }) { - const deltaAsMinutes = (end - start) / 1000 / 60; - const field = getTransactionDurationFieldForAggregatedTransactions( searchAggregatedTransactions ); @@ -99,10 +98,8 @@ export async function getTransactionGroupsForPage({ }, aggs: { ...getLatencyAggregation(latencyAggregationType, field), - transaction_count: { value_count: { field } }, [EVENT_OUTCOME]: { filter: { term: { [EVENT_OUTCOME]: EventOutcome.failure } }, - aggs: { transaction_count: { value_count: { field } } }, }, }, }, @@ -113,9 +110,8 @@ export async function getTransactionGroupsForPage({ const transactionGroups = response.aggregations?.transaction_groups.buckets.map((bucket) => { const errorRate = - bucket.transaction_count.value > 0 - ? (bucket[EVENT_OUTCOME].transaction_count.value ?? 0) / - bucket.transaction_count.value + bucket.doc_count > 0 + ? bucket[EVENT_OUTCOME].doc_count / bucket.doc_count : null; return { @@ -124,7 +120,11 @@ export async function getTransactionGroupsForPage({ latencyAggregationType, aggregation: bucket.latency, }), - throughput: bucket.transaction_count.value / deltaAsMinutes, + throughput: calculateThroughput({ + start, + end, + value: bucket.doc_count, + }), errorRate, }; }) ?? []; diff --git a/x-pack/plugins/apm/server/lib/services/get_service_transaction_groups/merge_transaction_group_data.ts b/x-pack/plugins/apm/server/lib/services/get_service_transaction_groups/merge_transaction_group_data.ts index a8794e3c09a40..4b8b1aabbbbcc 100644 --- a/x-pack/plugins/apm/server/lib/services/get_service_transaction_groups/merge_transaction_group_data.ts +++ b/x-pack/plugins/apm/server/lib/services/get_service_transaction_groups/merge_transaction_group_data.ts @@ -6,6 +6,7 @@ import { EVENT_OUTCOME } from '../../../../common/elasticsearch_fieldnames'; import { LatencyAggregationType } from '../../../../common/latency_aggregation_types'; +import { calculateThroughput } from '../../helpers/calculate_throughput'; import { getLatencyValue } from '../../helpers/latency_aggregation_type'; import { TransactionGroupTimeseriesData } from './get_timeseries_data_for_transaction_groups'; import { TransactionGroupWithoutTimeseriesData } from './get_transaction_groups_for_page'; @@ -25,8 +26,6 @@ export function mergeTransactionGroupData({ latencyAggregationType: LatencyAggregationType; transactionType: string; }) { - const deltaAsMinutes = (end - start) / 1000 / 60; - return transactionGroups.map((transactionGroup) => { const groupBucket = timeseriesData.find( ({ key }) => key === transactionGroup.name @@ -52,18 +51,18 @@ export function mergeTransactionGroupData({ ...acc.throughput, timeseries: acc.throughput.timeseries.concat({ x: point.key, - y: point.transaction_count.value / deltaAsMinutes, + y: calculateThroughput({ + start, + end, + value: point.doc_count, + }), }), }, errorRate: { ...acc.errorRate, timeseries: acc.errorRate.timeseries.concat({ x: point.key, - y: - point.transaction_count.value > 0 - ? (point[EVENT_OUTCOME].transaction_count.value ?? 0) / - point.transaction_count.value - : null, + y: point[EVENT_OUTCOME].doc_count / point.doc_count, }), }, }; diff --git a/x-pack/plugins/apm/server/lib/services/get_services/get_service_transaction_stats.ts b/x-pack/plugins/apm/server/lib/services/get_services/get_service_transaction_stats.ts index 0ee7080dc0834..d7cd13317fd73 100644 --- a/x-pack/plugins/apm/server/lib/services/get_services/get_service_transaction_stats.ts +++ b/x-pack/plugins/apm/server/lib/services/get_services/get_service_transaction_stats.ts @@ -22,6 +22,7 @@ import { getTransactionDurationFieldForAggregatedTransactions, } from '../../helpers/aggregated_transactions'; import { getBucketSize } from '../../helpers/get_bucket_size'; +import { calculateThroughput } from '../../helpers/calculate_throughput'; import { calculateTransactionErrorPercentage, getOutcomeAggregation, @@ -35,32 +36,15 @@ interface AggregationParams { const MAX_NUMBER_OF_SERVICES = 500; -function calculateAvgDuration({ - value, - deltaAsMinutes, -}: { - value: number; - deltaAsMinutes: number; -}) { - return value / deltaAsMinutes; -} - export async function getServiceTransactionStats({ setup, searchAggregatedTransactions, }: AggregationParams) { const { apmEventClient, start, end, esFilter } = setup; - const outcomes = getOutcomeAggregation({ searchAggregatedTransactions }); + const outcomes = getOutcomeAggregation(); const metrics = { - real_document_count: { - value_count: { - field: getTransactionDurationFieldForAggregatedTransactions( - searchAggregatedTransactions - ), - }, - }, avg_duration: { avg: { field: getTransactionDurationFieldForAggregatedTransactions( @@ -102,7 +86,6 @@ export async function getServiceTransactionStats({ transactionType: { terms: { field: TRANSACTION_TYPE, - order: { real_document_count: 'desc' }, }, aggs: { ...metrics, @@ -139,8 +122,6 @@ export async function getServiceTransactionStats({ }, }); - const deltaAsMinutes = (setup.end - setup.start) / 1000 / 60; - return ( response.aggregations?.services.buckets.map((bucket) => { const topTransactionTypeBucket = @@ -179,16 +160,18 @@ export async function getServiceTransactionStats({ ), }, transactionsPerMinute: { - value: calculateAvgDuration({ - value: topTransactionTypeBucket.real_document_count.value, - deltaAsMinutes, + value: calculateThroughput({ + start, + end, + value: topTransactionTypeBucket.doc_count, }), timeseries: topTransactionTypeBucket.timeseries.buckets.map( (dateBucket) => ({ x: dateBucket.key, - y: calculateAvgDuration({ - value: dateBucket.real_document_count.value, - deltaAsMinutes, + y: calculateThroughput({ + start, + end, + value: dateBucket.doc_count, }), }) ), diff --git a/x-pack/plugins/apm/server/lib/services/get_throughput.ts b/x-pack/plugins/apm/server/lib/services/get_throughput.ts index bde826a568da9..15ecc88a019db 100644 --- a/x-pack/plugins/apm/server/lib/services/get_throughput.ts +++ b/x-pack/plugins/apm/server/lib/services/get_throughput.ts @@ -16,6 +16,7 @@ import { getProcessorEventForAggregatedTransactions, } from '../helpers/aggregated_transactions'; import { getBucketSize } from '../helpers/get_bucket_size'; +import { calculateThroughput } from '../helpers/calculate_throughput'; import { Setup, SetupTimeRange } from '../helpers/setup_request'; interface Options { @@ -27,16 +28,15 @@ interface Options { type ESResponse = PromiseReturnType; -function transform(response: ESResponse, options: Options) { - const { end, start } = options.setup; - const deltaAsMinutes = (end - start) / 1000 / 60; +function transform(options: Options, response: ESResponse) { if (response.hits.total.value === 0) { return []; } + const { start, end } = options.setup; const buckets = response.aggregations?.throughput.buckets ?? []; - return buckets.map(({ key: x, doc_count: y }) => ({ + return buckets.map(({ key: x, doc_count: value }) => ({ x, - y: y / deltaAsMinutes, + y: calculateThroughput({ start, end, value }), })); } @@ -87,6 +87,6 @@ async function fetcher({ export async function getThroughput(options: Options) { return { - throughput: transform(await fetcher(options), options), + throughput: transform(options, await fetcher(options)), }; } diff --git a/x-pack/plugins/apm/server/lib/transaction_groups/__snapshots__/queries.test.ts.snap b/x-pack/plugins/apm/server/lib/transaction_groups/__snapshots__/queries.test.ts.snap index c678e7db711b6..89069d74bacf8 100644 --- a/x-pack/plugins/apm/server/lib/transaction_groups/__snapshots__/queries.test.ts.snap +++ b/x-pack/plugins/apm/server/lib/transaction_groups/__snapshots__/queries.test.ts.snap @@ -12,11 +12,6 @@ Array [ "aggs": Object { "transaction_groups": Object { "aggs": Object { - "count": Object { - "value_count": Object { - "field": "transaction.duration.us", - }, - }, "transaction_type": Object { "top_hits": Object { "_source": Array [ @@ -226,11 +221,6 @@ Array [ "aggs": Object { "transaction_groups": Object { "aggs": Object { - "count": Object { - "value_count": Object { - "field": "transaction.duration.us", - }, - }, "transaction_type": Object { "top_hits": Object { "_source": Array [ diff --git a/x-pack/plugins/apm/server/lib/transaction_groups/get_error_rate.ts b/x-pack/plugins/apm/server/lib/transaction_groups/get_error_rate.ts index dfd11203b87f1..a2388dddc7fd4 100644 --- a/x-pack/plugins/apm/server/lib/transaction_groups/get_error_rate.ts +++ b/x-pack/plugins/apm/server/lib/transaction_groups/get_error_rate.ts @@ -14,7 +14,10 @@ import { } from '../../../common/elasticsearch_fieldnames'; import { EventOutcome } from '../../../common/event_outcome'; import { rangeFilter } from '../../../common/utils/range_filter'; -import { getProcessorEventForAggregatedTransactions } from '../helpers/aggregated_transactions'; +import { + getDocumentTypeFilterForAggregatedTransactions, + getProcessorEventForAggregatedTransactions, +} from '../helpers/aggregated_transactions'; import { getBucketSize } from '../helpers/get_bucket_size'; import { Setup, SetupTimeRange } from '../helpers/setup_request'; import { @@ -55,12 +58,15 @@ export async function getErrorRate({ { terms: { [EVENT_OUTCOME]: [EventOutcome.failure, EventOutcome.success] }, }, + ...getDocumentTypeFilterForAggregatedTransactions( + searchAggregatedTransactions + ), ...transactionNamefilter, ...transactionTypefilter, ...esFilter, ]; - const outcomes = getOutcomeAggregation({ searchAggregatedTransactions }); + const outcomes = getOutcomeAggregation(); const params = { apm: { diff --git a/x-pack/plugins/apm/server/lib/transaction_groups/get_transaction_group_stats.ts b/x-pack/plugins/apm/server/lib/transaction_groups/get_transaction_group_stats.ts index cfd3540446172..dba58cecad79b 100644 --- a/x-pack/plugins/apm/server/lib/transaction_groups/get_transaction_group_stats.ts +++ b/x-pack/plugins/apm/server/lib/transaction_groups/get_transaction_group_stats.ts @@ -66,13 +66,6 @@ export async function getCounts({ searchAggregatedTransactions, }: MetricParams) { const params = mergeRequestWithAggs(request, { - count: { - value_count: { - field: getTransactionDurationFieldForAggregatedTransactions( - searchAggregatedTransactions - ), - }, - }, transaction_type: { top_hits: { size: 1, @@ -92,7 +85,7 @@ export async function getCounts({ return { key: bucket.key as BucketKey, - count: bucket.count.value, + count: bucket.doc_count, transactionType: source.transaction.type, }; }); diff --git a/x-pack/plugins/apm/server/lib/transactions/get_throughput_charts/index.ts b/x-pack/plugins/apm/server/lib/transactions/get_throughput_charts/index.ts index be374ccfe3400..dda6573ea93ef 100644 --- a/x-pack/plugins/apm/server/lib/transactions/get_throughput_charts/index.ts +++ b/x-pack/plugins/apm/server/lib/transactions/get_throughput_charts/index.ts @@ -15,7 +15,6 @@ import { rangeFilter } from '../../../../common/utils/range_filter'; import { getDocumentTypeFilterForAggregatedTransactions, getProcessorEventForAggregatedTransactions, - getTransactionDurationFieldForAggregatedTransactions, } from '../../../lib/helpers/aggregated_transactions'; import { getBucketSize } from '../../../lib/helpers/get_bucket_size'; import { Setup, SetupTimeRange } from '../../../lib/helpers/setup_request'; @@ -56,10 +55,6 @@ async function searchThroughput({ filter.push({ term: { [TRANSACTION_NAME]: transactionName } }); } - const field = getTransactionDurationFieldForAggregatedTransactions( - searchAggregatedTransactions - ); - const params = { apm: { events: [ @@ -82,7 +77,6 @@ async function searchThroughput({ min_doc_count: 0, extended_bounds: { min: start, max: end }, }, - aggs: { count: { value_count: { field } } }, }, }, }, @@ -106,9 +100,7 @@ export async function getThroughputCharts({ setup: Setup & SetupTimeRange; searchAggregatedTransactions: boolean; }) { - const { start, end } = setup; - const { bucketSize, intervalString } = getBucketSize({ start, end }); - const durationAsMinutes = (end - start) / 1000 / 60; + const { bucketSize, intervalString } = getBucketSize(setup); const response = await searchThroughput({ serviceName, @@ -123,7 +115,7 @@ export async function getThroughputCharts({ throughputTimeseries: getThroughputBuckets({ throughputResultBuckets: response.aggregations?.throughput.buckets, bucketSize, - durationAsMinutes, + setupTimeRange: setup, }), }; } diff --git a/x-pack/plugins/apm/server/lib/transactions/get_throughput_charts/transform.ts b/x-pack/plugins/apm/server/lib/transactions/get_throughput_charts/transform.ts index a12e36c0e9de4..35d1b0e901dee 100644 --- a/x-pack/plugins/apm/server/lib/transactions/get_throughput_charts/transform.ts +++ b/x-pack/plugins/apm/server/lib/transactions/get_throughput_charts/transform.ts @@ -7,25 +7,28 @@ import { sortBy } from 'lodash'; import { NOT_AVAILABLE_LABEL } from '../../../../common/i18n'; import { ThroughputChartsResponse } from '.'; +import { calculateThroughput } from '../../helpers/calculate_throughput'; +import { SetupTimeRange } from '../../helpers/setup_request'; type ThroughputResultBuckets = Required['aggregations']['throughput']['buckets']; export function getThroughputBuckets({ throughputResultBuckets = [], bucketSize, - durationAsMinutes, + setupTimeRange, }: { throughputResultBuckets?: ThroughputResultBuckets; bucketSize: number; - durationAsMinutes: number; + setupTimeRange: SetupTimeRange; }) { + const { start, end } = setupTimeRange; const buckets = throughputResultBuckets.map( ({ key: resultKey, timeseries }) => { const dataPoints = timeseries.buckets.map((bucket) => { return { x: bucket.key, // divide by minutes - y: bucket.count.value / (bucketSize / 60), + y: bucket.doc_count / (bucketSize / 60), }; }); @@ -34,11 +37,11 @@ export function getThroughputBuckets({ resultKey === '' ? NOT_AVAILABLE_LABEL : (resultKey as string); const docCountTotal = timeseries.buckets - .map((bucket) => bucket.count.value) + .map((bucket) => bucket.doc_count) .reduce((a, b) => a + b, 0); // calculate average throughput - const avg = docCountTotal / durationAsMinutes; + const avg = calculateThroughput({ start, end, value: docCountTotal }); return { key, dataPoints, avg }; } diff --git a/x-pack/plugins/case/common/api/connectors/mappings.ts b/x-pack/plugins/case/common/api/connectors/mappings.ts index f8e9830fed7c1..b9f84d406a184 100644 --- a/x-pack/plugins/case/common/api/connectors/mappings.ts +++ b/x-pack/plugins/case/common/api/connectors/mappings.ts @@ -16,8 +16,8 @@ import { Incident as ResilientIncident, } from '../../../../actions/server/builtin_action_types/resilient/types'; import { - PushToServiceApiParams as ServiceNowPushToServiceApiParams, - Incident as ServiceNowIncident, + PushToServiceApiParamsITSM as ServiceNowITSMPushToServiceApiParams, + ServiceNowITSMIncident, } from '../../../../actions/server/builtin_action_types/servicenow/types'; import { ResilientFieldsRT } from './resilient'; import { ServiceNowFieldsRT } from './servicenow'; @@ -33,13 +33,13 @@ export interface ElasticUser { export { JiraPushToServiceApiParams, ResilientPushToServiceApiParams, - ServiceNowPushToServiceApiParams, + ServiceNowITSMPushToServiceApiParams, }; -export type Incident = JiraIncident | ResilientIncident | ServiceNowIncident; +export type Incident = JiraIncident | ResilientIncident | ServiceNowITSMIncident; export type PushToServiceApiParams = | JiraPushToServiceApiParams | ResilientPushToServiceApiParams - | ServiceNowPushToServiceApiParams; + | ServiceNowITSMPushToServiceApiParams; const ActionTypeRT = rt.union([ rt.literal('append'), diff --git a/x-pack/plugins/case/server/routes/api/cases/configure/utils.ts b/x-pack/plugins/case/server/routes/api/cases/configure/utils.ts index 89109af4cecb9..9e903b66459a9 100644 --- a/x-pack/plugins/case/server/routes/api/cases/configure/utils.ts +++ b/x-pack/plugins/case/server/routes/api/cases/configure/utils.ts @@ -19,7 +19,7 @@ import { PrepareFieldsForTransformArgs, PushToServiceApiParams, ResilientPushToServiceApiParams, - ServiceNowPushToServiceApiParams, + ServiceNowITSMPushToServiceApiParams, SimpleComment, Transformer, TransformerArgs, @@ -105,7 +105,11 @@ export const serviceFormatter = ( thirdPartyName: 'Resilient', }; case ConnectorTypes.servicenow: - const { severity, urgency, impact } = params as ServiceNowPushToServiceApiParams['incident']; + const { + severity, + urgency, + impact, + } = params as ServiceNowITSMPushToServiceApiParams['incident']; return { incident: { severity, urgency, impact }, thirdPartyName: 'ServiceNow', diff --git a/x-pack/plugins/data_enhanced/common/index.ts b/x-pack/plugins/data_enhanced/common/index.ts index 669c33230a34c..8c500ef21ffcf 100644 --- a/x-pack/plugins/data_enhanced/common/index.ts +++ b/x-pack/plugins/data_enhanced/common/index.ts @@ -5,6 +5,7 @@ */ export { + SEARCH_SESSION_TYPE, ENHANCED_ES_SEARCH_STRATEGY, EQL_SEARCH_STRATEGY, EqlRequestParams, diff --git a/x-pack/plugins/data_enhanced/common/search/session/types.ts b/x-pack/plugins/data_enhanced/common/search/session/types.ts index 9eefdf43aa245..6d07f4b731fae 100644 --- a/x-pack/plugins/data_enhanced/common/search/session/types.ts +++ b/x-pack/plugins/data_enhanced/common/search/session/types.ts @@ -6,19 +6,25 @@ import { SearchSessionStatus } from './'; +export const SEARCH_SESSION_TYPE = 'search-session'; export interface SearchSessionSavedObjectAttributes { + sessionId: string; /** * User-facing session name to be displayed in session management */ - name: string; + name?: string; /** * App that created the session. e.g 'discover' */ - appId: string; + appId?: string; /** * Creation time of the session */ created: string; + /** + * Last touch time of the session + */ + touched: string; /** * Expiration time of the session. Expiration itself is managed by Elasticsearch. */ @@ -30,22 +36,28 @@ export interface SearchSessionSavedObjectAttributes { /** * urlGeneratorId */ - urlGeneratorId: string; + urlGeneratorId?: string; /** * The application state that was used to create the session. * Should be used, for example, to re-load an expired search session. */ - initialState: Record; + initialState?: Record; /** * Application state that should be used to restore the session. * For example, relative dates are conveted to absolute ones. */ - restoreState: Record; + restoreState?: Record; /** * Mapping of search request hashes to their corresponsing info (async search id, etc.) */ idMapping: Record; + + /** + * This value is true if the session was actively stored by the user. If it is false, the session may be purged by the system. + */ + persisted: boolean; } + export interface SearchSessionRequestInfo { /** * ID of the async search request diff --git a/x-pack/plugins/data_enhanced/config.ts b/x-pack/plugins/data_enhanced/config.ts index 981c398019832..d41206733def3 100644 --- a/x-pack/plugins/data_enhanced/config.ts +++ b/x-pack/plugins/data_enhanced/config.ts @@ -12,7 +12,8 @@ export const configSchema = schema.object({ enabled: schema.boolean({ defaultValue: false }), pageSize: schema.number({ defaultValue: 10000 }), trackingInterval: schema.duration({ defaultValue: '10s' }), - inMemTimeout: schema.duration({ defaultValue: '1m' }), + notTouchedTimeout: schema.duration({ defaultValue: '5m' }), + notTouchedInProgressTimeout: schema.duration({ defaultValue: '1m' }), maxUpdateRetries: schema.number({ defaultValue: 3 }), defaultExpiration: schema.duration({ defaultValue: '7d' }), management: schema.object({ diff --git a/x-pack/plugins/data_enhanced/public/search/sessions_mgmt/lib/api.ts b/x-pack/plugins/data_enhanced/public/search/sessions_mgmt/lib/api.ts index c6a3d088b3cda..25c06d1d2e278 100644 --- a/x-pack/plugins/data_enhanced/public/search/sessions_mgmt/lib/api.ts +++ b/x-pack/plugins/data_enhanced/public/search/sessions_mgmt/lib/api.ts @@ -10,12 +10,11 @@ import moment from 'moment'; import { from, race, timer } from 'rxjs'; import { mapTo, tap } from 'rxjs/operators'; import type { SharePluginStart } from 'src/plugins/share/public'; -import { SessionsConfigSchema } from '../'; -import type { ISessionsClient } from '../../../../../../../src/plugins/data/public'; -import type { SearchSessionSavedObjectAttributes } from '../../../../common'; +import { ISessionsClient } from '../../../../../../../src/plugins/data/public'; import { SearchSessionStatus } from '../../../../common/search'; import { ACTION } from '../components/actions'; -import { UISession } from '../types'; +import { PersistedSearchSessionSavedObjectAttributes, UISession } from '../types'; +import { SessionsConfigSchema } from '..'; type UrlGeneratorsStart = SharePluginStart['urlGenerators']; @@ -48,7 +47,7 @@ async function getUrlFromState( // Helper: factory for a function to map server objects to UI objects const mapToUISession = (urls: UrlGeneratorsStart, config: SessionsConfigSchema) => async ( - savedObject: SavedObject + savedObject: SavedObject ): Promise => { const { name, @@ -110,6 +109,8 @@ export class SearchSessionsMgmtAPI { perPage: mgmtConfig.maxSessions, sortField: 'created', sortOrder: 'asc', + searchFields: ['persisted'], + search: 'true', }) ); const timeout$ = timer(refreshTimeout.asMilliseconds()).pipe( @@ -129,7 +130,7 @@ export class SearchSessionsMgmtAPI { const result = await race(fetch$, timeout$).toPromise(); if (result && result.saved_objects) { const savedObjects = result.saved_objects as Array< - SavedObject + SavedObject >; return await Promise.all(savedObjects.map(mapToUISession(this.deps.urls, this.config))); } diff --git a/x-pack/plugins/data_enhanced/public/search/sessions_mgmt/types.ts b/x-pack/plugins/data_enhanced/public/search/sessions_mgmt/types.ts index 78b91f7ca8ac2..3b0159a1e8faa 100644 --- a/x-pack/plugins/data_enhanced/public/search/sessions_mgmt/types.ts +++ b/x-pack/plugins/data_enhanced/public/search/sessions_mgmt/types.ts @@ -4,11 +4,23 @@ * you may not use this file except in compliance with the Elastic License. */ -import { SearchSessionStatus } from '../../../common'; +import { SearchSessionSavedObjectAttributes, SearchSessionStatus } from '../../../common'; import { ACTION } from './components/actions'; export const DATE_STRING_FORMAT = 'D MMM, YYYY, HH:mm:ss'; +/** + * Some properties are optional for a non-persisted Search Session. + * This interface makes them mandatory, because management only shows persisted search sessions. + */ +export type PersistedSearchSessionSavedObjectAttributes = SearchSessionSavedObjectAttributes & + Required< + Pick< + SearchSessionSavedObjectAttributes, + 'name' | 'appId' | 'urlGeneratorId' | 'initialState' | 'restoreState' + > + >; + export interface UISession { id: string; name: string; diff --git a/x-pack/plugins/data_enhanced/server/plugin.ts b/x-pack/plugins/data_enhanced/server/plugin.ts index cff0ee3efd738..834f1669e2d7e 100644 --- a/x-pack/plugins/data_enhanced/server/plugin.ts +++ b/x-pack/plugins/data_enhanced/server/plugin.ts @@ -5,6 +5,7 @@ */ import { CoreSetup, CoreStart, Logger, Plugin, PluginInitializerContext } from 'kibana/server'; +import { Observable } from 'rxjs'; import { TaskManagerSetupContract, TaskManagerStartContract } from '../../task_manager/server'; import { PluginSetup as DataPluginSetup, @@ -22,6 +23,7 @@ import { } from './search'; import { getUiSettings } from './ui_settings'; import type { DataEnhancedRequestHandlerContext } from './type'; +import { ConfigSchema } from '../config'; interface SetupDependencies { data: DataPluginSetup; @@ -37,9 +39,11 @@ export class EnhancedDataServerPlugin implements Plugin { private readonly logger: Logger; private sessionService!: SearchSessionService; + private config$: Observable; - constructor(private initializerContext: PluginInitializerContext) { + constructor(private initializerContext: PluginInitializerContext) { this.logger = initializerContext.logger.get('data_enhanced'); + this.config$ = this.initializerContext.config.create(); } public setup(core: CoreSetup, deps: SetupDependencies) { @@ -51,6 +55,7 @@ export class EnhancedDataServerPlugin deps.data.search.registerSearchStrategy( ENHANCED_ES_SEARCH_STRATEGY, enhancedEsSearchStrategyProvider( + this.config$, this.initializerContext.config.legacy.globalConfig$, this.logger, usage diff --git a/x-pack/plugins/data_enhanced/server/routes/session.ts b/x-pack/plugins/data_enhanced/server/routes/session.ts index cbf683bd18fd2..622d4d68413ca 100644 --- a/x-pack/plugins/data_enhanced/server/routes/session.ts +++ b/x-pack/plugins/data_enhanced/server/routes/session.ts @@ -91,11 +91,13 @@ export function registerSessionRoutes(router: DataEnhancedPluginRouter, logger: sortField: schema.maybe(schema.string()), sortOrder: schema.maybe(schema.string()), filter: schema.maybe(schema.string()), + searchFields: schema.maybe(schema.arrayOf(schema.string())), + search: schema.maybe(schema.string()), }), }, }, async (context, request, res) => { - const { page, perPage, sortField, sortOrder, filter } = request.body; + const { page, perPage, sortField, sortOrder, filter, searchFields, search } = request.body; try { const response = await context.search!.session.find({ page, @@ -103,6 +105,8 @@ export function registerSessionRoutes(router: DataEnhancedPluginRouter, logger: sortField, sortOrder, filter, + searchFields, + search, }); return res.ok({ diff --git a/x-pack/plugins/data_enhanced/server/saved_objects/search_session.ts b/x-pack/plugins/data_enhanced/server/saved_objects/search_session.ts index 4e75ffaeec69a..16472199de4d9 100644 --- a/x-pack/plugins/data_enhanced/server/saved_objects/search_session.ts +++ b/x-pack/plugins/data_enhanced/server/saved_objects/search_session.ts @@ -5,8 +5,7 @@ */ import { SavedObjectsType } from 'kibana/server'; - -export const SEARCH_SESSION_TYPE = 'search-session'; +import { SEARCH_SESSION_TYPE } from '../../common'; export const searchSessionMapping: SavedObjectsType = { name: SEARCH_SESSION_TYPE, @@ -14,6 +13,9 @@ export const searchSessionMapping: SavedObjectsType = { hidden: true, mappings: { properties: { + persisted: { + type: 'boolean', + }, sessionId: { type: 'keyword', }, @@ -26,6 +28,9 @@ export const searchSessionMapping: SavedObjectsType = { expires: { type: 'date', }, + touched: { + type: 'date', + }, status: { type: 'keyword', }, diff --git a/x-pack/plugins/data_enhanced/server/search/eql_search_strategy.test.ts b/x-pack/plugins/data_enhanced/server/search/eql_search_strategy.test.ts index f2d7725954a26..1670b1116eedb 100644 --- a/x-pack/plugins/data_enhanced/server/search/eql_search_strategy.test.ts +++ b/x-pack/plugins/data_enhanced/server/search/eql_search_strategy.test.ts @@ -117,7 +117,6 @@ describe('EQL search strategy', () => { expect(request).toEqual( expect.objectContaining({ wait_for_completion_timeout: '100ms', - keep_alive: '1m', }) ); }); @@ -156,7 +155,6 @@ describe('EQL search strategy', () => { expect(request).toEqual( expect.objectContaining({ wait_for_completion_timeout: '5ms', - keep_alive: '1m', keep_on_completion: false, }) ); diff --git a/x-pack/plugins/data_enhanced/server/search/eql_search_strategy.ts b/x-pack/plugins/data_enhanced/server/search/eql_search_strategy.ts index a0d4e9dcd19b9..65ce5bdf5255c 100644 --- a/x-pack/plugins/data_enhanced/server/search/eql_search_strategy.ts +++ b/x-pack/plugins/data_enhanced/server/search/eql_search_strategy.ts @@ -22,7 +22,8 @@ export const eqlSearchStrategyProvider = ( logger: Logger ): ISearchStrategy => { async function cancelAsyncSearch(id: string, esClient: IScopedClusterClient) { - await esClient.asCurrentUser.asyncSearch.delete({ id }); + const client = esClient.asCurrentUser.eql; + await client.delete({ id }); } return { @@ -41,11 +42,11 @@ export const eqlSearchStrategyProvider = ( uiSettingsClient ); const params = id - ? getDefaultAsyncGetParams() + ? getDefaultAsyncGetParams(options) : { ...(await getIgnoreThrottled(uiSettingsClient)), ...defaultParams, - ...getDefaultAsyncGetParams(), + ...getDefaultAsyncGetParams(options), ...request.params, }; const promise = id diff --git a/x-pack/plugins/data_enhanced/server/search/es_search_strategy.test.ts b/x-pack/plugins/data_enhanced/server/search/es_search_strategy.test.ts index b2ddd0310f8f5..98238f50fa059 100644 --- a/x-pack/plugins/data_enhanced/server/search/es_search_strategy.test.ts +++ b/x-pack/plugins/data_enhanced/server/search/es_search_strategy.test.ts @@ -7,6 +7,7 @@ import { enhancedEsSearchStrategyProvider } from './es_search_strategy'; import { BehaviorSubject } from 'rxjs'; import { SearchStrategyDependencies } from '../../../../../src/plugins/data/server/search'; +import moment from 'moment'; import { KbnServerError } from '../../../../../src/plugins/kibana_utils/server'; import { ElasticsearchClientError, ResponseError } from '@elastic/elasticsearch/lib/errors'; import * as indexNotFoundException from '../../../../../src/plugins/data/common/search/test_data/index_not_found_exception.json'; @@ -60,7 +61,7 @@ describe('ES search strategy', () => { }, }, } as unknown) as SearchStrategyDependencies; - const mockConfig$ = new BehaviorSubject({ + const mockLegacyConfig$ = new BehaviorSubject({ elasticsearch: { shardTimeout: { asMilliseconds: () => { @@ -70,6 +71,14 @@ describe('ES search strategy', () => { }, }); + const mockConfig$ = new BehaviorSubject({ + search: { + sessions: { + defaultExpiration: moment.duration('1', 'm'), + }, + }, + }); + beforeEach(() => { mockApiCaller.mockClear(); mockGetCaller.mockClear(); @@ -78,76 +87,140 @@ describe('ES search strategy', () => { }); it('returns a strategy with `search and `cancel`', async () => { - const esSearch = await enhancedEsSearchStrategyProvider(mockConfig$, mockLogger); + const esSearch = await enhancedEsSearchStrategyProvider( + mockConfig$, + mockLegacyConfig$, + mockLogger + ); expect(typeof esSearch.search).toBe('function'); }); describe('search', () => { - it('makes a POST request to async search with params when no ID is provided', async () => { - mockSubmitCaller.mockResolvedValueOnce(mockAsyncResponse); + describe('no sessionId', () => { + it('makes a POST request with params when no ID provided', async () => { + mockSubmitCaller.mockResolvedValueOnce(mockAsyncResponse); - const params = { index: 'logstash-*', body: { query: {} } }; - const esSearch = await enhancedEsSearchStrategyProvider(mockConfig$, mockLogger); + const params = { index: 'logstash-*', body: { query: {} } }; + const esSearch = await enhancedEsSearchStrategyProvider( + mockConfig$, + mockLegacyConfig$, + mockLogger + ); - await esSearch.search({ params }, {}, mockDeps).toPromise(); + await esSearch.search({ params }, {}, mockDeps).toPromise(); - expect(mockSubmitCaller).toBeCalled(); - const request = mockSubmitCaller.mock.calls[0][0]; - expect(request.index).toEqual(params.index); - expect(request.body).toEqual(params.body); - }); + expect(mockSubmitCaller).toBeCalled(); + const request = mockSubmitCaller.mock.calls[0][0]; + expect(request.index).toEqual(params.index); + expect(request.body).toEqual(params.body); + expect(request).toHaveProperty('keep_alive', '1m'); + }); - it('makes a GET request to async search with ID when ID is provided', async () => { - mockGetCaller.mockResolvedValueOnce(mockAsyncResponse); + it('makes a GET request to async search with ID', async () => { + mockGetCaller.mockResolvedValueOnce(mockAsyncResponse); - const params = { index: 'logstash-*', body: { query: {} } }; - const esSearch = await enhancedEsSearchStrategyProvider(mockConfig$, mockLogger); + const params = { index: 'logstash-*', body: { query: {} } }; + const esSearch = await enhancedEsSearchStrategyProvider( + mockConfig$, + mockLegacyConfig$, + mockLogger + ); - await esSearch.search({ id: 'foo', params }, {}, mockDeps).toPromise(); + await esSearch.search({ id: 'foo', params }, {}, mockDeps).toPromise(); - expect(mockGetCaller).toBeCalled(); - const request = mockGetCaller.mock.calls[0][0]; - expect(request.id).toEqual('foo'); - expect(request).toHaveProperty('wait_for_completion_timeout'); - expect(request).toHaveProperty('keep_alive'); - }); + expect(mockGetCaller).toBeCalled(); + const request = mockGetCaller.mock.calls[0][0]; + expect(request.id).toEqual('foo'); + expect(request).toHaveProperty('wait_for_completion_timeout'); + expect(request).toHaveProperty('keep_alive', '1m'); + }); + + it('sets wait_for_completion_timeout and keep_alive in the request', async () => { + mockSubmitCaller.mockResolvedValueOnce(mockAsyncResponse); + + const params = { index: 'foo-*', body: {} }; + const esSearch = await enhancedEsSearchStrategyProvider( + mockConfig$, + mockLegacyConfig$, + mockLogger + ); + + await esSearch.search({ params }, {}, mockDeps).toPromise(); + + expect(mockSubmitCaller).toBeCalled(); + const request = mockSubmitCaller.mock.calls[0][0]; + expect(request).toHaveProperty('wait_for_completion_timeout'); + expect(request).toHaveProperty('keep_alive'); + }); - it('calls the rollup API if the index is a rollup type', async () => { - mockApiCaller.mockResolvedValueOnce(mockRollupResponse); - - const params = { index: 'foo-程', body: {} }; - const esSearch = await enhancedEsSearchStrategyProvider(mockConfig$, mockLogger); - - await esSearch - .search( - { - indexType: 'rollup', - params, - }, - {}, - mockDeps - ) - .toPromise(); - - expect(mockApiCaller).toBeCalled(); - const { method, path } = mockApiCaller.mock.calls[0][0]; - expect(method).toBe('POST'); - expect(path).toBe('/foo-%E7%A8%8B/_rollup_search'); + it('calls the rollup API if the index is a rollup type', async () => { + mockApiCaller.mockResolvedValueOnce(mockRollupResponse); + + const params = { index: 'foo-程', body: {} }; + const esSearch = await enhancedEsSearchStrategyProvider( + mockConfig$, + mockLegacyConfig$, + mockLogger + ); + + await esSearch + .search( + { + indexType: 'rollup', + params, + }, + {}, + mockDeps + ) + .toPromise(); + + expect(mockApiCaller).toBeCalled(); + const { method, path } = mockApiCaller.mock.calls[0][0]; + expect(method).toBe('POST'); + expect(path).toBe('/foo-%E7%A8%8B/_rollup_search'); + }); }); - it('sets wait_for_completion_timeout and keep_alive in the request', async () => { - mockSubmitCaller.mockResolvedValueOnce(mockAsyncResponse); + describe('with sessionId', () => { + it('makes a POST request with params (long keepalive)', async () => { + mockSubmitCaller.mockResolvedValueOnce(mockAsyncResponse); - const params = { index: 'foo-*', body: {} }; - const esSearch = await enhancedEsSearchStrategyProvider(mockConfig$, mockLogger); + const params = { index: 'logstash-*', body: { query: {} } }; + const esSearch = await enhancedEsSearchStrategyProvider( + mockConfig$, + mockLegacyConfig$, + mockLogger + ); - await esSearch.search({ params }, {}, mockDeps).toPromise(); + await esSearch.search({ params }, { sessionId: '1' }, mockDeps).toPromise(); - expect(mockSubmitCaller).toBeCalled(); - const request = mockSubmitCaller.mock.calls[0][0]; - expect(request).toHaveProperty('wait_for_completion_timeout'); - expect(request).toHaveProperty('keep_alive'); + expect(mockSubmitCaller).toBeCalled(); + const request = mockSubmitCaller.mock.calls[0][0]; + expect(request.index).toEqual(params.index); + expect(request.body).toEqual(params.body); + + expect(request).toHaveProperty('keep_alive', '60000ms'); + }); + + it('makes a GET request to async search without keepalive', async () => { + mockGetCaller.mockResolvedValueOnce(mockAsyncResponse); + + const params = { index: 'logstash-*', body: { query: {} } }; + const esSearch = await enhancedEsSearchStrategyProvider( + mockConfig$, + mockLegacyConfig$, + mockLogger + ); + + await esSearch.search({ id: 'foo', params }, { sessionId: '1' }, mockDeps).toPromise(); + + expect(mockGetCaller).toBeCalled(); + const request = mockGetCaller.mock.calls[0][0]; + expect(request.id).toEqual('foo'); + expect(request).toHaveProperty('wait_for_completion_timeout'); + expect(request).not.toHaveProperty('keep_alive'); + }); }); it('throws normalized error if ResponseError is thrown', async () => { @@ -162,7 +235,11 @@ describe('ES search strategy', () => { mockSubmitCaller.mockRejectedValue(errResponse); const params = { index: 'logstash-*', body: { query: {} } }; - const esSearch = await enhancedEsSearchStrategyProvider(mockConfig$, mockLogger); + const esSearch = await enhancedEsSearchStrategyProvider( + mockConfig$, + mockLegacyConfig$, + mockLogger + ); let err: KbnServerError | undefined; try { @@ -183,7 +260,11 @@ describe('ES search strategy', () => { mockSubmitCaller.mockRejectedValue(errResponse); const params = { index: 'logstash-*', body: { query: {} } }; - const esSearch = await enhancedEsSearchStrategyProvider(mockConfig$, mockLogger); + const esSearch = await enhancedEsSearchStrategyProvider( + mockConfig$, + mockLegacyConfig$, + mockLogger + ); let err: KbnServerError | undefined; try { @@ -204,7 +285,11 @@ describe('ES search strategy', () => { mockDeleteCaller.mockResolvedValueOnce(200); const id = 'some_id'; - const esSearch = await enhancedEsSearchStrategyProvider(mockConfig$, mockLogger); + const esSearch = await enhancedEsSearchStrategyProvider( + mockConfig$, + mockLegacyConfig$, + mockLogger + ); await esSearch.cancel!(id, {}, mockDeps); @@ -224,7 +309,11 @@ describe('ES search strategy', () => { mockDeleteCaller.mockRejectedValue(errResponse); const id = 'some_id'; - const esSearch = await enhancedEsSearchStrategyProvider(mockConfig$, mockLogger); + const esSearch = await enhancedEsSearchStrategyProvider( + mockConfig$, + mockLegacyConfig$, + mockLogger + ); let err: KbnServerError | undefined; try { @@ -247,7 +336,11 @@ describe('ES search strategy', () => { const id = 'some_other_id'; const keepAlive = '1d'; - const esSearch = await enhancedEsSearchStrategyProvider(mockConfig$, mockLogger); + const esSearch = await enhancedEsSearchStrategyProvider( + mockConfig$, + mockLegacyConfig$, + mockLogger + ); await esSearch.extend!(id, keepAlive, {}, mockDeps); @@ -262,7 +355,11 @@ describe('ES search strategy', () => { const id = 'some_other_id'; const keepAlive = '1d'; - const esSearch = await enhancedEsSearchStrategyProvider(mockConfig$, mockLogger); + const esSearch = await enhancedEsSearchStrategyProvider( + mockConfig$, + mockLegacyConfig$, + mockLogger + ); let err: KbnServerError | undefined; try { diff --git a/x-pack/plugins/data_enhanced/server/search/es_search_strategy.ts b/x-pack/plugins/data_enhanced/server/search/es_search_strategy.ts index 694d9807b5a4d..64b1e1a57b489 100644 --- a/x-pack/plugins/data_enhanced/server/search/es_search_strategy.ts +++ b/x-pack/plugins/data_enhanced/server/search/es_search_strategy.ts @@ -23,6 +23,7 @@ import { getTotalLoaded, searchUsageObserver, shimAbortSignal, + shimHitsTotal, } from '../../../../../src/plugins/data/server'; import type { IAsyncSearchOptions } from '../../common'; import { pollSearch } from '../../common'; @@ -33,10 +34,12 @@ import { } from './request_utils'; import { toAsyncKibanaSearchResponse } from './response_utils'; import { AsyncSearchResponse } from './types'; +import { ConfigSchema } from '../../config'; import { getKbnServerError, KbnServerError } from '../../../../../src/plugins/kibana_utils/server'; export const enhancedEsSearchStrategyProvider = ( - config$: Observable, + config$: Observable, + legacyConfig$: Observable, logger: Logger, usage?: SearchUsage ): ISearchStrategy => { @@ -56,14 +59,19 @@ export const enhancedEsSearchStrategyProvider = ( const client = esClient.asCurrentUser.asyncSearch; const search = async () => { + const config = await config$.pipe(first()).toPromise(); const params = id - ? getDefaultAsyncGetParams() - : { ...(await getDefaultAsyncSubmitParams(uiSettingsClient, options)), ...request.params }; + ? getDefaultAsyncGetParams(options) + : { + ...(await getDefaultAsyncSubmitParams(uiSettingsClient, config, options)), + ...request.params, + }; const promise = id ? client.get({ ...params, id }) : client.submit(params); const { body } = await shimAbortSignal(promise, options.abortSignal); - return toAsyncKibanaSearchResponse(body); + const response = shimHitsTotal(body.response, options); + return toAsyncKibanaSearchResponse({ ...body, response }); }; const cancel = async () => { @@ -86,12 +94,12 @@ export const enhancedEsSearchStrategyProvider = ( options: ISearchOptions, { esClient, uiSettingsClient }: SearchStrategyDependencies ): Promise { - const config = await config$.pipe(first()).toPromise(); + const legacyConfig = await legacyConfig$.pipe(first()).toPromise(); const { body, index, ...params } = request.params!; const method = 'POST'; const path = encodeURI(`/${index}/_rollup_search`); const querystring = { - ...getShardTimeout(config), + ...getShardTimeout(legacyConfig), ...(await getIgnoreThrottled(uiSettingsClient)), ...(await getDefaultSearchParams(uiSettingsClient)), ...params, @@ -108,7 +116,7 @@ export const enhancedEsSearchStrategyProvider = ( const esResponse = await shimAbortSignal(promise, options?.abortSignal); const response = esResponse.body as SearchResponse; return { - rawResponse: response, + rawResponse: shimHitsTotal(response, options), ...getTotalLoaded(response), }; } catch (e) { diff --git a/x-pack/plugins/data_enhanced/server/search/request_utils.ts b/x-pack/plugins/data_enhanced/server/search/request_utils.ts index f54ab2199c905..d9ef3ab3292c3 100644 --- a/x-pack/plugins/data_enhanced/server/search/request_utils.ts +++ b/x-pack/plugins/data_enhanced/server/search/request_utils.ts @@ -11,6 +11,7 @@ import { } from '@elastic/elasticsearch/api/requestParams'; import { ISearchOptions, UI_SETTINGS } from '../../../../../src/plugins/data/common'; import { getDefaultSearchParams } from '../../../../../src/plugins/data/server'; +import { ConfigSchema } from '../../config'; /** * @internal @@ -27,6 +28,7 @@ export async function getIgnoreThrottled( */ export async function getDefaultAsyncSubmitParams( uiSettingsClient: IUiSettingsClient, + config: ConfigSchema, options: ISearchOptions ): Promise< Pick< @@ -44,21 +46,30 @@ export async function getDefaultAsyncSubmitParams( return { batched_reduce_size: 64, keep_on_completion: !!options.sessionId, // Always return an ID, even if the request completes quickly - ...getDefaultAsyncGetParams(), + ...getDefaultAsyncGetParams(options), ...(await getIgnoreThrottled(uiSettingsClient)), ...(await getDefaultSearchParams(uiSettingsClient)), + ...(options.sessionId + ? { + keep_alive: `${config.search.sessions.defaultExpiration.asMilliseconds()}ms`, + } + : {}), }; } /** @internal */ -export function getDefaultAsyncGetParams(): Pick< - AsyncSearchGet, - 'keep_alive' | 'wait_for_completion_timeout' -> { +export function getDefaultAsyncGetParams( + options: ISearchOptions +): Pick { return { - keep_alive: '1m', // Extend the TTL for this search request by one minute wait_for_completion_timeout: '100ms', // Wait up to 100ms for the response to return + ...(options.sessionId + ? undefined + : { + keep_alive: '1m', + // We still need to do polling for searches not within the context of a search session + }), }; } diff --git a/x-pack/plugins/data_enhanced/server/search/session/check_running_sessions.test.ts b/x-pack/plugins/data_enhanced/server/search/session/check_running_sessions.test.ts index 4334ab3bc2903..352edc4639631 100644 --- a/x-pack/plugins/data_enhanced/server/search/session/check_running_sessions.test.ts +++ b/x-pack/plugins/data_enhanced/server/search/session/check_running_sessions.test.ts @@ -5,187 +5,569 @@ */ import { checkRunningSessions } from './check_running_sessions'; -import { SearchSessionStatus, SearchSessionSavedObjectAttributes } from '../../../common'; +import { + SearchSessionStatus, + SearchSessionSavedObjectAttributes, + ENHANCED_ES_SEARCH_STRATEGY, + EQL_SEARCH_STRATEGY, +} from '../../../common'; import { savedObjectsClientMock } from '../../../../../../src/core/server/mocks'; import type { SavedObjectsClientContract } from 'kibana/server'; -import { SearchStatus } from './types'; +import { SearchSessionsConfig, SearchStatus } from './types'; +import moment from 'moment'; describe('getSearchStatus', () => { let mockClient: any; let savedObjectsClient: jest.Mocked; + const config: SearchSessionsConfig = { + enabled: true, + pageSize: 5, + notTouchedInProgressTimeout: moment.duration(1, 'm'), + notTouchedTimeout: moment.duration(5, 'm'), + maxUpdateRetries: 3, + defaultExpiration: moment.duration(7, 'd'), + trackingInterval: moment.duration(10, 's'), + management: {} as any, + }; const mockLogger: any = { debug: jest.fn(), warn: jest.fn(), error: jest.fn(), }; + const emptySO = { + persisted: false, + status: SearchSessionStatus.IN_PROGRESS, + created: moment().subtract(moment.duration(3, 'm')), + touched: moment().subtract(moment.duration(10, 's')), + idMapping: {}, + }; + beforeEach(() => { savedObjectsClient = savedObjectsClientMock.create(); mockClient = { asyncSearch: { status: jest.fn(), + delete: jest.fn(), + }, + eql: { + status: jest.fn(), + delete: jest.fn(), }, }; }); test('does nothing if there are no open sessions', async () => { - savedObjectsClient.bulkUpdate = jest.fn(); savedObjectsClient.find.mockResolvedValue({ saved_objects: [], total: 0, } as any); - await checkRunningSessions(savedObjectsClient, mockClient, mockLogger); + await checkRunningSessions( + { + savedObjectsClient, + client: mockClient, + logger: mockLogger, + }, + config + ); expect(savedObjectsClient.bulkUpdate).not.toBeCalled(); + expect(savedObjectsClient.delete).not.toBeCalled(); }); - test('does nothing if there are no searchIds in the saved object', async () => { - savedObjectsClient.bulkUpdate = jest.fn(); - savedObjectsClient.find.mockResolvedValue({ - saved_objects: [ + describe('pagination', () => { + test('fetches one page if not objects exist', async () => { + savedObjectsClient.find.mockResolvedValueOnce({ + saved_objects: [], + total: 0, + } as any); + + await checkRunningSessions( { - attributes: { - idMapping: {}, - }, + savedObjectsClient, + client: mockClient, + logger: mockLogger, }, - ], - total: 1, - } as any); + config + ); - await checkRunningSessions(savedObjectsClient, mockClient, mockLogger); + expect(savedObjectsClient.find).toHaveBeenCalledTimes(1); + }); - expect(savedObjectsClient.bulkUpdate).not.toBeCalled(); + test('fetches one page if less than page size object are returned', async () => { + savedObjectsClient.find.mockResolvedValueOnce({ + saved_objects: [emptySO, emptySO], + total: 5, + } as any); + + await checkRunningSessions( + { + savedObjectsClient, + client: mockClient, + logger: mockLogger, + }, + config + ); + + expect(savedObjectsClient.find).toHaveBeenCalledTimes(1); + }); + + test('fetches two pages if exactly page size objects are returned', async () => { + let i = 0; + savedObjectsClient.find.mockImplementation(() => { + return new Promise((resolve) => { + resolve({ + saved_objects: i++ === 0 ? [emptySO, emptySO, emptySO, emptySO, emptySO] : [], + total: 5, + page: i, + } as any); + }); + }); + + await checkRunningSessions( + { + savedObjectsClient, + client: mockClient, + logger: mockLogger, + }, + config + ); + + expect(savedObjectsClient.find).toHaveBeenCalledTimes(2); + + // validate that page number increases + const { page: page1 } = savedObjectsClient.find.mock.calls[0][0]; + const { page: page2 } = savedObjectsClient.find.mock.calls[1][0]; + expect(page1).toBe(1); + expect(page2).toBe(2); + }); + + test('fetches two pages if page size +1 objects are returned', async () => { + let i = 0; + savedObjectsClient.find.mockImplementation(() => { + return new Promise((resolve) => { + resolve({ + saved_objects: i++ === 0 ? [emptySO, emptySO, emptySO, emptySO, emptySO] : [emptySO], + total: 5, + page: i, + } as any); + }); + }); + + await checkRunningSessions( + { + savedObjectsClient, + client: mockClient, + logger: mockLogger, + }, + config + ); + + expect(savedObjectsClient.find).toHaveBeenCalledTimes(2); + }); }); - test('does nothing if the search is still running', async () => { - savedObjectsClient.bulkUpdate = jest.fn(); - const so = { - attributes: { - idMapping: { - 'search-hash': { - id: 'search-id', - strategy: 'cool', - status: SearchStatus.IN_PROGRESS, + describe('delete', () => { + test('doesnt delete a persisted session', async () => { + savedObjectsClient.find.mockResolvedValue({ + saved_objects: [ + { + id: '123', + attributes: { + persisted: true, + status: SearchSessionStatus.IN_PROGRESS, + created: moment().subtract(moment.duration(30, 'm')), + touched: moment().subtract(moment.duration(10, 'm')), + idMapping: {}, + }, }, + ], + total: 1, + } as any); + await checkRunningSessions( + { + savedObjectsClient, + client: mockClient, + logger: mockLogger, }, - }, - }; - savedObjectsClient.find.mockResolvedValue({ - saved_objects: [so], - total: 1, - } as any); + config + ); - mockClient.asyncSearch.status.mockResolvedValue({ - body: { - is_partial: true, - is_running: true, - }, + expect(savedObjectsClient.bulkUpdate).not.toBeCalled(); + expect(savedObjectsClient.delete).not.toBeCalled(); }); - await checkRunningSessions(savedObjectsClient, mockClient, mockLogger); + test('doesnt delete a non persisted, recently touched session', async () => { + savedObjectsClient.find.mockResolvedValue({ + saved_objects: [ + { + id: '123', + attributes: { + persisted: false, + status: SearchSessionStatus.IN_PROGRESS, + created: moment().subtract(moment.duration(3, 'm')), + touched: moment().subtract(moment.duration(10, 's')), + idMapping: {}, + }, + }, + ], + total: 1, + } as any); + await checkRunningSessions( + { + savedObjectsClient, + client: mockClient, + logger: mockLogger, + }, + config + ); - expect(savedObjectsClient.bulkUpdate).not.toBeCalled(); - }); + expect(savedObjectsClient.bulkUpdate).not.toBeCalled(); + expect(savedObjectsClient.delete).not.toBeCalled(); + }); - test("doesn't re-check completed or errored searches", async () => { - savedObjectsClient.bulkUpdate = jest.fn(); - const so = { - attributes: { - idMapping: { - 'search-hash': { - id: 'search-id', - strategy: 'cool', - status: SearchStatus.COMPLETE, + test('doesnt delete a non persisted, completed session, within on screen time frame', async () => { + savedObjectsClient.find.mockResolvedValue({ + saved_objects: [ + { + id: '123', + attributes: { + persisted: false, + status: SearchSessionStatus.COMPLETE, + created: moment().subtract(moment.duration(3, 'm')), + touched: moment().subtract(moment.duration(1, 'm')), + idMapping: { + 'search-hash': { + id: 'search-id', + strategy: 'cool', + status: SearchStatus.COMPLETE, + }, + }, + }, }, - 'another-search-hash': { - id: 'search-id', - strategy: 'cool', - status: SearchStatus.ERROR, + ], + total: 1, + } as any); + await checkRunningSessions( + { + savedObjectsClient, + client: mockClient, + logger: mockLogger, + }, + config + ); + + expect(savedObjectsClient.bulkUpdate).not.toBeCalled(); + expect(savedObjectsClient.delete).not.toBeCalled(); + }); + + test('deletes a non persisted, abandoned session', async () => { + savedObjectsClient.find.mockResolvedValue({ + saved_objects: [ + { + id: '123', + attributes: { + persisted: false, + status: SearchSessionStatus.IN_PROGRESS, + created: moment().subtract(moment.duration(3, 'm')), + touched: moment().subtract(moment.duration(2, 'm')), + idMapping: { + 'map-key': { + strategy: ENHANCED_ES_SEARCH_STRATEGY, + id: 'async-id', + }, + }, + }, }, + ], + total: 1, + } as any); + + await checkRunningSessions( + { + savedObjectsClient, + client: mockClient, + logger: mockLogger, }, - }, - }; - savedObjectsClient.find.mockResolvedValue({ - saved_objects: [so], - total: 1, - } as any); + config + ); - await checkRunningSessions(savedObjectsClient, mockClient, mockLogger); + expect(savedObjectsClient.bulkUpdate).not.toBeCalled(); + expect(savedObjectsClient.delete).toBeCalled(); - expect(mockClient.asyncSearch.status).not.toBeCalled(); - }); + expect(mockClient.asyncSearch.delete).toBeCalled(); + + const { id } = mockClient.asyncSearch.delete.mock.calls[0][0]; + expect(id).toBe('async-id'); + }); - test('updates to complete if the search is done', async () => { - savedObjectsClient.bulkUpdate = jest.fn(); - const so = { - attributes: { - idMapping: { - 'search-hash': { - id: 'search-id', - strategy: 'cool', - status: SearchStatus.IN_PROGRESS, + test('deletes a completed, not persisted session', async () => { + mockClient.asyncSearch.delete = jest.fn().mockResolvedValue(true); + + savedObjectsClient.find.mockResolvedValue({ + saved_objects: [ + { + id: '123', + attributes: { + persisted: false, + status: SearchSessionStatus.COMPLETE, + created: moment().subtract(moment.duration(30, 'm')), + touched: moment().subtract(moment.duration(6, 'm')), + idMapping: { + 'map-key': { + strategy: ENHANCED_ES_SEARCH_STRATEGY, + id: 'async-id', + status: SearchStatus.COMPLETE, + }, + 'eql-map-key': { + strategy: EQL_SEARCH_STRATEGY, + id: 'eql-async-id', + status: SearchStatus.COMPLETE, + }, + }, + }, }, + ], + total: 1, + } as any); + + await checkRunningSessions( + { + savedObjectsClient, + client: mockClient, + logger: mockLogger, }, - }, - }; - savedObjectsClient.find.mockResolvedValue({ - saved_objects: [so], - total: 1, - } as any); + config + ); - mockClient.asyncSearch.status.mockResolvedValue({ - body: { - is_partial: false, - is_running: false, - completion_status: 200, - }, + expect(savedObjectsClient.bulkUpdate).not.toBeCalled(); + expect(savedObjectsClient.delete).toBeCalled(); + + expect(mockClient.asyncSearch.delete).toBeCalled(); + expect(mockClient.eql.delete).not.toBeCalled(); + + const { id } = mockClient.asyncSearch.delete.mock.calls[0][0]; + expect(id).toBe('async-id'); }); - await checkRunningSessions(savedObjectsClient, mockClient, mockLogger); + test('ignores errors thrown while deleting async searches', async () => { + mockClient.asyncSearch.delete = jest.fn().mockRejectedValueOnce(false); + + savedObjectsClient.find.mockResolvedValue({ + saved_objects: [ + { + id: '123', + attributes: { + persisted: false, + status: SearchSessionStatus.COMPLETE, + created: moment().subtract(moment.duration(30, 'm')), + touched: moment().subtract(moment.duration(6, 'm')), + idMapping: { + 'map-key': { + strategy: ENHANCED_ES_SEARCH_STRATEGY, + id: 'async-id', + status: SearchStatus.COMPLETE, + }, + }, + }, + }, + ], + total: 1, + } as any); + + await checkRunningSessions( + { + savedObjectsClient, + client: mockClient, + logger: mockLogger, + }, + config + ); + + expect(savedObjectsClient.bulkUpdate).not.toBeCalled(); + expect(savedObjectsClient.delete).toBeCalled(); - expect(mockClient.asyncSearch.status).toBeCalledWith({ id: 'search-id' }); - const [updateInput] = savedObjectsClient.bulkUpdate.mock.calls[0]; - const updatedAttributes = updateInput[0].attributes as SearchSessionSavedObjectAttributes; - expect(updatedAttributes.status).toBe(SearchSessionStatus.COMPLETE); - expect(updatedAttributes.idMapping['search-hash'].status).toBe(SearchStatus.COMPLETE); - expect(updatedAttributes.idMapping['search-hash'].error).toBeUndefined(); + expect(mockClient.asyncSearch.delete).toBeCalled(); + + const { id } = mockClient.asyncSearch.delete.mock.calls[0][0]; + expect(id).toBe('async-id'); + }); }); - test('updates to error if the search is errored', async () => { - savedObjectsClient.bulkUpdate = jest.fn(); - const so = { - attributes: { - idMapping: { - 'search-hash': { - id: 'search-id', - strategy: 'cool', - status: SearchStatus.IN_PROGRESS, + describe('update', () => { + test('does nothing if the search is still running', async () => { + const so = { + id: '123', + attributes: { + persisted: false, + status: SearchSessionStatus.IN_PROGRESS, + created: moment().subtract(moment.duration(3, 'm')), + touched: moment().subtract(moment.duration(10, 's')), + idMapping: { + 'search-hash': { + id: 'search-id', + strategy: 'cool', + status: SearchStatus.IN_PROGRESS, + }, }, }, - }, - }; - savedObjectsClient.find.mockResolvedValue({ - saved_objects: [so], - total: 1, - } as any); + }; + savedObjectsClient.find.mockResolvedValue({ + saved_objects: [so], + total: 1, + } as any); - mockClient.asyncSearch.status.mockResolvedValue({ - body: { - is_partial: false, - is_running: false, - completion_status: 500, - }, + mockClient.asyncSearch.status.mockResolvedValue({ + body: { + is_partial: true, + is_running: true, + }, + }); + + await checkRunningSessions( + { + savedObjectsClient, + client: mockClient, + logger: mockLogger, + }, + config + ); + + expect(savedObjectsClient.bulkUpdate).not.toBeCalled(); + expect(savedObjectsClient.delete).not.toBeCalled(); }); - await checkRunningSessions(savedObjectsClient, mockClient, mockLogger); - const [updateInput] = savedObjectsClient.bulkUpdate.mock.calls[0]; + test("doesn't re-check completed or errored searches", async () => { + savedObjectsClient.bulkUpdate = jest.fn(); + savedObjectsClient.delete = jest.fn(); + const so = { + id: '123', + attributes: { + status: SearchSessionStatus.ERROR, + idMapping: { + 'search-hash': { + id: 'search-id', + strategy: 'cool', + status: SearchStatus.COMPLETE, + }, + 'another-search-hash': { + id: 'search-id', + strategy: 'cool', + status: SearchStatus.ERROR, + }, + }, + }, + }; + savedObjectsClient.find.mockResolvedValue({ + saved_objects: [so], + total: 1, + } as any); - const updatedAttributes = updateInput[0].attributes as SearchSessionSavedObjectAttributes; - expect(updatedAttributes.status).toBe(SearchSessionStatus.ERROR); - expect(updatedAttributes.idMapping['search-hash'].status).toBe(SearchStatus.ERROR); - expect(updatedAttributes.idMapping['search-hash'].error).toBe( - 'Search completed with a 500 status' - ); + await checkRunningSessions( + { + savedObjectsClient, + client: mockClient, + logger: mockLogger, + }, + config + ); + + expect(mockClient.asyncSearch.status).not.toBeCalled(); + expect(savedObjectsClient.bulkUpdate).not.toBeCalled(); + expect(savedObjectsClient.delete).not.toBeCalled(); + }); + + test('updates to complete if the search is done', async () => { + savedObjectsClient.bulkUpdate = jest.fn(); + const so = { + attributes: { + status: SearchSessionStatus.IN_PROGRESS, + touched: '123', + idMapping: { + 'search-hash': { + id: 'search-id', + strategy: 'cool', + status: SearchStatus.IN_PROGRESS, + }, + }, + }, + }; + savedObjectsClient.find.mockResolvedValue({ + saved_objects: [so], + total: 1, + } as any); + + mockClient.asyncSearch.status.mockResolvedValue({ + body: { + is_partial: false, + is_running: false, + completion_status: 200, + }, + }); + + await checkRunningSessions( + { + savedObjectsClient, + client: mockClient, + logger: mockLogger, + }, + config + ); + + expect(mockClient.asyncSearch.status).toBeCalledWith({ id: 'search-id' }); + const [updateInput] = savedObjectsClient.bulkUpdate.mock.calls[0]; + const updatedAttributes = updateInput[0].attributes as SearchSessionSavedObjectAttributes; + expect(updatedAttributes.status).toBe(SearchSessionStatus.COMPLETE); + expect(updatedAttributes.touched).not.toBe('123'); + expect(updatedAttributes.idMapping['search-hash'].status).toBe(SearchStatus.COMPLETE); + expect(updatedAttributes.idMapping['search-hash'].error).toBeUndefined(); + + expect(savedObjectsClient.delete).not.toBeCalled(); + }); + + test('updates to error if the search is errored', async () => { + savedObjectsClient.bulkUpdate = jest.fn(); + const so = { + attributes: { + idMapping: { + 'search-hash': { + id: 'search-id', + strategy: 'cool', + status: SearchStatus.IN_PROGRESS, + }, + }, + }, + }; + savedObjectsClient.find.mockResolvedValue({ + saved_objects: [so], + total: 1, + } as any); + + mockClient.asyncSearch.status.mockResolvedValue({ + body: { + is_partial: false, + is_running: false, + completion_status: 500, + }, + }); + + await checkRunningSessions( + { + savedObjectsClient, + client: mockClient, + logger: mockLogger, + }, + config + ); + const [updateInput] = savedObjectsClient.bulkUpdate.mock.calls[0]; + + const updatedAttributes = updateInput[0].attributes as SearchSessionSavedObjectAttributes; + expect(updatedAttributes.status).toBe(SearchSessionStatus.ERROR); + expect(updatedAttributes.idMapping['search-hash'].status).toBe(SearchStatus.ERROR); + expect(updatedAttributes.idMapping['search-hash'].error).toBe( + 'Search completed with a 500 status' + ); + }); }); }); diff --git a/x-pack/plugins/data_enhanced/server/search/session/check_running_sessions.ts b/x-pack/plugins/data_enhanced/server/search/session/check_running_sessions.ts index 71274e15e284d..7258b0ac124e8 100644 --- a/x-pack/plugins/data_enhanced/server/search/session/check_running_sessions.ts +++ b/x-pack/plugins/data_enhanced/server/search/session/check_running_sessions.ts @@ -10,93 +10,198 @@ import { SavedObjectsFindResult, SavedObjectsClientContract, } from 'kibana/server'; +import moment from 'moment'; +import { EMPTY, from } from 'rxjs'; +import { expand, mergeMap } from 'rxjs/operators'; +import { nodeBuilder } from '../../../../../../src/plugins/data/common'; import { SearchSessionStatus, SearchSessionSavedObjectAttributes, SearchSessionRequestInfo, + SEARCH_SESSION_TYPE, + ENHANCED_ES_SEARCH_STRATEGY, } from '../../../common'; -import { SEARCH_SESSION_TYPE } from '../../saved_objects'; import { getSearchStatus } from './get_search_status'; import { getSessionStatus } from './get_session_status'; -import { SearchStatus } from './types'; +import { SearchSessionsConfig, SearchStatus } from './types'; -export async function checkRunningSessions( - savedObjectsClient: SavedObjectsClientContract, +export interface CheckRunningSessionsDeps { + savedObjectsClient: SavedObjectsClientContract; + client: ElasticsearchClient; + logger: Logger; +} + +function isSessionStale( + session: SavedObjectsFindResult, + config: SearchSessionsConfig, + logger: Logger +) { + const curTime = moment(); + // Delete if a running session wasn't polled for in the last notTouchedInProgressTimeout OR + // if a completed \ errored \ canceled session wasn't saved for within notTouchedTimeout + return ( + (session.attributes.status === SearchSessionStatus.IN_PROGRESS && + curTime.diff(moment(session.attributes.touched), 'ms') > + config.notTouchedInProgressTimeout.asMilliseconds()) || + (session.attributes.status !== SearchSessionStatus.IN_PROGRESS && + curTime.diff(moment(session.attributes.touched), 'ms') > + config.notTouchedTimeout.asMilliseconds()) + ); +} + +async function updateSessionStatus( + session: SavedObjectsFindResult, client: ElasticsearchClient, logger: Logger +) { + let sessionUpdated = false; + + // Check statuses of all running searches + await Promise.all( + Object.keys(session.attributes.idMapping).map(async (searchKey: string) => { + const updateSearchRequest = ( + currentStatus: Pick + ) => { + sessionUpdated = true; + session.attributes.idMapping[searchKey] = { + ...session.attributes.idMapping[searchKey], + ...currentStatus, + }; + }; + + const searchInfo = session.attributes.idMapping[searchKey]; + if (searchInfo.status === SearchStatus.IN_PROGRESS) { + try { + const currentStatus = await getSearchStatus(client, searchInfo.id); + + if (currentStatus.status !== searchInfo.status) { + logger.debug(`search ${searchInfo.id} | status changed to ${currentStatus.status}`); + updateSearchRequest(currentStatus); + } + } catch (e) { + logger.error(e); + updateSearchRequest({ + status: SearchStatus.ERROR, + error: e.message || e.meta.error?.caused_by?.reason, + }); + } + } + }) + ); + + // And only then derive the session's status + const sessionStatus = getSessionStatus(session.attributes); + if (sessionStatus !== session.attributes.status) { + session.attributes.status = sessionStatus; + session.attributes.touched = new Date().toISOString(); + sessionUpdated = true; + } + + return sessionUpdated; +} + +function getSavedSearchSessionsPage$( + { savedObjectsClient, logger }: CheckRunningSessionsDeps, + config: SearchSessionsConfig, + page: number +) { + logger.debug(`Fetching saved search sessions page ${page}`); + return from( + savedObjectsClient.find({ + page, + perPage: config.pageSize, + type: SEARCH_SESSION_TYPE, + namespaces: ['*'], + filter: nodeBuilder.or([ + nodeBuilder.and([ + nodeBuilder.is( + `${SEARCH_SESSION_TYPE}.attributes.status`, + SearchSessionStatus.IN_PROGRESS.toString() + ), + nodeBuilder.is(`${SEARCH_SESSION_TYPE}.attributes.persisted`, 'true'), + ]), + nodeBuilder.is(`${SEARCH_SESSION_TYPE}.attributes.persisted`, 'false'), + ]), + }) + ); +} + +function getAllSavedSearchSessions$(deps: CheckRunningSessionsDeps, config: SearchSessionsConfig) { + return getSavedSearchSessionsPage$(deps, config, 1).pipe( + expand((result) => { + if (!result || !result.saved_objects || result.saved_objects.length < config.pageSize) + return EMPTY; + else { + return getSavedSearchSessionsPage$(deps, config, result.page + 1); + } + }) + ); +} + +export async function checkRunningSessions( + deps: CheckRunningSessionsDeps, + config: SearchSessionsConfig ): Promise { + const { logger, client, savedObjectsClient } = deps; try { - const runningSearchSessionsResponse = await savedObjectsClient.find( - { - type: SEARCH_SESSION_TYPE, - search: SearchSessionStatus.IN_PROGRESS.toString(), - searchFields: ['status'], - namespaces: ['*'], - } - ); - - if (!runningSearchSessionsResponse.total) return; - - logger.debug(`Found ${runningSearchSessionsResponse.total} running sessions`); - - const updatedSessions = new Array>(); - - let sessionUpdated = false; - - await Promise.all( - runningSearchSessionsResponse.saved_objects.map(async (session) => { - // Check statuses of all running searches - await Promise.all( - Object.keys(session.attributes.idMapping).map(async (searchKey: string) => { - const updateSearchRequest = ( - currentStatus: Pick - ) => { - sessionUpdated = true; - session.attributes.idMapping[searchKey] = { - ...session.attributes.idMapping[searchKey], - ...currentStatus, - }; - }; - - const searchInfo = session.attributes.idMapping[searchKey]; - if (searchInfo.status === SearchStatus.IN_PROGRESS) { - try { - const currentStatus = await getSearchStatus(client, searchInfo.id); - - if (currentStatus.status !== SearchStatus.IN_PROGRESS) { - updateSearchRequest(currentStatus); + await getAllSavedSearchSessions$(deps, config) + .pipe( + mergeMap(async (runningSearchSessionsResponse) => { + if (!runningSearchSessionsResponse.total) return; + + logger.debug(`Found ${runningSearchSessionsResponse.total} running sessions`); + + const updatedSessions = new Array< + SavedObjectsFindResult + >(); + + await Promise.all( + runningSearchSessionsResponse.saved_objects.map(async (session) => { + const updated = await updateSessionStatus(session, client, logger); + let deleted = false; + + if (!session.attributes.persisted) { + if (isSessionStale(session, config, logger)) { + deleted = true; + // delete saved object to free up memory + // TODO: there's a potential rare edge case of deleting an object and then receiving a new trackId for that same session! + // Maybe we want to change state to deleted and cleanup later? + logger.debug(`Deleting stale session | ${session.id}`); + await savedObjectsClient.delete(SEARCH_SESSION_TYPE, session.id); + + // Send a delete request for each async search to ES + Object.keys(session.attributes.idMapping).map(async (searchKey: string) => { + const searchInfo = session.attributes.idMapping[searchKey]; + if (searchInfo.strategy === ENHANCED_ES_SEARCH_STRATEGY) { + try { + await client.asyncSearch.delete({ id: searchInfo.id }); + } catch (e) { + logger.debug( + `Error ignored while deleting async_search ${searchInfo.id}: ${e.message}` + ); + } + } + }); } - } catch (e) { - logger.error(e); - updateSearchRequest({ - status: SearchStatus.ERROR, - error: e.message || e.meta.error?.caused_by?.reason, - }); } - } - }) - ); - - // And only then derive the session's status - const sessionStatus = getSessionStatus(session.attributes); - if (sessionStatus !== SearchSessionStatus.IN_PROGRESS) { - session.attributes.status = sessionStatus; - sessionUpdated = true; - } - if (sessionUpdated) { - updatedSessions.push(session); - } - }) - ); - - if (updatedSessions.length) { - // If there's an error, we'll try again in the next iteration, so there's no need to check the output. - const updatedResponse = await savedObjectsClient.bulkUpdate( - updatedSessions - ); - logger.debug(`Updated ${updatedResponse.saved_objects.length} background sessions`); - } + if (updated && !deleted) { + updatedSessions.push(session); + } + }) + ); + + // Do a bulk update + if (updatedSessions.length) { + // If there's an error, we'll try again in the next iteration, so there's no need to check the output. + const updatedResponse = await savedObjectsClient.bulkUpdate( + updatedSessions + ); + logger.debug(`Updated ${updatedResponse.saved_objects.length} search sessions`); + } + }) + ) + .toPromise(); } catch (err) { logger.error(err); } diff --git a/x-pack/plugins/data_enhanced/server/search/session/get_search_status.test.ts b/x-pack/plugins/data_enhanced/server/search/session/get_search_status.test.ts index e66ce613b71d9..c4eef0b3ddbb3 100644 --- a/x-pack/plugins/data_enhanced/server/search/session/get_search_status.test.ts +++ b/x-pack/plugins/data_enhanced/server/search/session/get_search_status.test.ts @@ -17,7 +17,7 @@ describe('getSearchStatus', () => { }; }); - test('returns an error status if search is partial and not running', () => { + test('returns an error status if search is partial and not running', async () => { mockClient.asyncSearch.status.mockResolvedValue({ body: { is_partial: true, @@ -25,10 +25,11 @@ describe('getSearchStatus', () => { completion_status: 200, }, }); - expect(getSearchStatus(mockClient, '123')).resolves.toBe(SearchStatus.ERROR); + const res = await getSearchStatus(mockClient, '123'); + expect(res.status).toBe(SearchStatus.ERROR); }); - test('returns an error status if completion_status is an error', () => { + test('returns an error status if completion_status is an error', async () => { mockClient.asyncSearch.status.mockResolvedValue({ body: { is_partial: false, @@ -36,10 +37,11 @@ describe('getSearchStatus', () => { completion_status: 500, }, }); - expect(getSearchStatus(mockClient, '123')).resolves.toBe(SearchStatus.ERROR); + const res = await getSearchStatus(mockClient, '123'); + expect(res.status).toBe(SearchStatus.ERROR); }); - test('returns an error status if gets an ES error', () => { + test('returns an error status if gets an ES error', async () => { mockClient.asyncSearch.status.mockResolvedValue({ error: { root_cause: { @@ -47,15 +49,17 @@ describe('getSearchStatus', () => { }, }, }); - expect(getSearchStatus(mockClient, '123')).resolves.toBe(SearchStatus.ERROR); + const res = await getSearchStatus(mockClient, '123'); + expect(res.status).toBe(SearchStatus.ERROR); }); - test('returns an error status throws', () => { + test('returns an error status throws', async () => { mockClient.asyncSearch.status.mockRejectedValue(new Error('O_o')); - expect(getSearchStatus(mockClient, '123')).resolves.toBe(SearchStatus.ERROR); + const res = await getSearchStatus(mockClient, '123'); + expect(res.status).toBe(SearchStatus.ERROR); }); - test('returns a complete status', () => { + test('returns a complete status', async () => { mockClient.asyncSearch.status.mockResolvedValue({ body: { is_partial: false, @@ -63,10 +67,11 @@ describe('getSearchStatus', () => { completion_status: 200, }, }); - expect(getSearchStatus(mockClient, '123')).resolves.toBe(SearchStatus.COMPLETE); + const res = await getSearchStatus(mockClient, '123'); + expect(res.status).toBe(SearchStatus.COMPLETE); }); - test('returns a running status otherwise', () => { + test('returns a running status otherwise', async () => { mockClient.asyncSearch.status.mockResolvedValue({ body: { is_partial: false, @@ -74,6 +79,7 @@ describe('getSearchStatus', () => { completion_status: undefined, }, }); - expect(getSearchStatus(mockClient, '123')).resolves.toBe(SearchStatus.IN_PROGRESS); + const res = await getSearchStatus(mockClient, '123'); + expect(res.status).toBe(SearchStatus.IN_PROGRESS); }); }); diff --git a/x-pack/plugins/data_enhanced/server/search/session/get_search_status.ts b/x-pack/plugins/data_enhanced/server/search/session/get_search_status.ts index e2b5fc0157b37..3e93ae4e056c7 100644 --- a/x-pack/plugins/data_enhanced/server/search/session/get_search_status.ts +++ b/x-pack/plugins/data_enhanced/server/search/session/get_search_status.ts @@ -16,27 +16,40 @@ export async function getSearchStatus( asyncId: string ): Promise> { // TODO: Handle strategies other than the default one - const apiResponse: ApiResponse = await client.asyncSearch.status({ - id: asyncId, - }); - const response = apiResponse.body; - if ((response.is_partial && !response.is_running) || response.completion_status >= 400) { + try { + const apiResponse: ApiResponse = await client.asyncSearch.status({ + id: asyncId, + }); + const response = apiResponse.body; + if ((response.is_partial && !response.is_running) || response.completion_status >= 400) { + return { + status: SearchStatus.ERROR, + error: i18n.translate('xpack.data.search.statusError', { + defaultMessage: `Search completed with a {errorCode} status`, + values: { errorCode: response.completion_status }, + }), + }; + } else if (!response.is_partial && !response.is_running) { + return { + status: SearchStatus.COMPLETE, + error: undefined, + }; + } else { + return { + status: SearchStatus.IN_PROGRESS, + error: undefined, + }; + } + } catch (e) { return { status: SearchStatus.ERROR, - error: i18n.translate('xpack.data.search.statusError', { - defaultMessage: `Search completed with a {errorCode} status`, - values: { errorCode: response.completion_status }, + error: i18n.translate('xpack.data.search.statusThrow', { + defaultMessage: `Search status threw an error {message} ({errorCode}) status`, + values: { + message: e.message, + errorCode: e.statusCode || 500, + }, }), }; - } else if (!response.is_partial && !response.is_running) { - return { - status: SearchStatus.COMPLETE, - error: undefined, - }; - } else { - return { - status: SearchStatus.IN_PROGRESS, - error: undefined, - }; } } diff --git a/x-pack/plugins/data_enhanced/server/search/session/monitoring_task.ts b/x-pack/plugins/data_enhanced/server/search/session/monitoring_task.ts index 332e69b119bb6..d9f3cdb8debe7 100644 --- a/x-pack/plugins/data_enhanced/server/search/session/monitoring_task.ts +++ b/x-pack/plugins/data_enhanced/server/search/session/monitoring_task.ts @@ -14,10 +14,10 @@ import { } from '../../../../task_manager/server'; import { checkRunningSessions } from './check_running_sessions'; import { CoreSetup, SavedObjectsClient, Logger } from '../../../../../../src/core/server'; -import { SEARCH_SESSION_TYPE } from '../../saved_objects'; import { ConfigSchema } from '../../../config'; +import { SEARCH_SESSION_TYPE } from '../../../common'; -export const SEARCH_SESSIONS_TASK_TYPE = 'bg_monitor'; +export const SEARCH_SESSIONS_TASK_TYPE = 'search_sessions_monitor'; export const SEARCH_SESSIONS_TASK_ID = `data_enhanced_${SEARCH_SESSIONS_TASK_TYPE}`; interface SearchSessionTaskDeps { @@ -31,17 +31,20 @@ function searchSessionRunner(core: CoreSetup, { logger, config$ }: SearchSession return { async run() { const config = await config$.pipe(first()).toPromise(); + const sessionConfig = config.search.sessions; const [coreStart] = await core.getStartServices(); const internalRepo = coreStart.savedObjects.createInternalRepository([SEARCH_SESSION_TYPE]); const internalSavedObjectsClient = new SavedObjectsClient(internalRepo); await checkRunningSessions( - internalSavedObjectsClient, - coreStart.elasticsearch.client.asInternalUser, - logger + { + savedObjectsClient: internalSavedObjectsClient, + client: coreStart.elasticsearch.client.asInternalUser, + logger, + }, + sessionConfig ); return { - runAt: new Date(Date.now() + config.search.sessions.trackingInterval.asMilliseconds()), state: {}, }; }, diff --git a/x-pack/plugins/data_enhanced/server/search/session/session_service.test.ts b/x-pack/plugins/data_enhanced/server/search/session/session_service.test.ts index 1107ed8155080..a0d1e3c87a79c 100644 --- a/x-pack/plugins/data_enhanced/server/search/session/session_service.test.ts +++ b/x-pack/plugins/data_enhanced/server/search/session/session_service.test.ts @@ -5,21 +5,22 @@ */ import { BehaviorSubject, of } from 'rxjs'; -import type { SavedObject, SavedObjectsClientContract } from 'kibana/server'; +import { + SavedObject, + SavedObjectsClientContract, + SavedObjectsErrorHelpers, +} from '../../../../../../src/core/server'; import type { SearchStrategyDependencies } from '../../../../../../src/plugins/data/server'; import { savedObjectsClientMock } from '../../../../../../src/core/server/mocks'; -import { SearchSessionStatus } from '../../../common'; -import { SEARCH_SESSION_TYPE } from '../../saved_objects'; -import { SearchSessionDependencies, SearchSessionService, SessionInfo } from './session_service'; +import { SearchSessionStatus, SEARCH_SESSION_TYPE } from '../../../common'; +import { SearchSessionDependencies, SearchSessionService } from './session_service'; import { createRequestHash } from './utils'; import moment from 'moment'; import { coreMock } from 'src/core/server/mocks'; import { ConfigSchema } from '../../../config'; // @ts-ignore import { taskManagerMock } from '../../../../task_manager/server/mocks'; -import { SearchStatus } from './types'; -const INMEM_TRACKING_INTERVAL = 10000; const MAX_UPDATE_RETRIES = 3; const flushPromises = () => new Promise((resolve) => setImmediate(resolve)); @@ -28,67 +29,7 @@ describe('SearchSessionService', () => { let savedObjectsClient: jest.Mocked; let service: SearchSessionService; - const MOCK_SESSION_ID = 'session-id-mock'; - const MOCK_ASYNC_ID = '123456'; const MOCK_STRATEGY = 'ese'; - const MOCK_KEY_HASH = '608de49a4600dbb5b173492759792e4a'; - - const createMockInternalSavedObjectClient = ( - findSpy?: jest.SpyInstance, - bulkUpdateSpy?: jest.SpyInstance - ) => { - Object.defineProperty(service, 'internalSavedObjectsClient', { - get: () => { - const find = - findSpy || - (() => { - return { - saved_objects: [ - { - attributes: { - sessionId: MOCK_SESSION_ID, - idMapping: { - 'another-key': { - id: 'another-async-id', - strategy: 'another-strategy', - }, - }, - }, - id: MOCK_SESSION_ID, - version: '1', - }, - ], - }; - }); - - const bulkUpdate = - bulkUpdateSpy || - (() => { - return { - saved_objects: [], - }; - }); - return { - find, - bulkUpdate, - }; - }, - }); - }; - - const createMockIdMapping = ( - mapValues: any[], - insertTime?: moment.Moment, - retryCount?: number - ): Map => { - const fakeMap = new Map(); - fakeMap.set(MOCK_SESSION_ID, { - ids: new Map(mapValues), - insertTime: insertTime || moment(), - retryCount: retryCount || 0, - }); - return fakeMap; - }; const sessionId = 'd7170a35-7e2c-48d6-8dec-9a056721b489'; const mockSavedObject: SavedObject = { @@ -110,8 +51,9 @@ describe('SearchSessionService', () => { sessions: { enabled: true, pageSize: 10000, - inMemTimeout: moment.duration(1, 'm'), - maxUpdateRetries: 3, + notTouchedInProgressTimeout: moment.duration(1, 'm'), + notTouchedTimeout: moment.duration(2, 'm'), + maxUpdateRetries: MAX_UPDATE_RETRIES, defaultExpiration: moment.duration(7, 'd'), trackingInterval: moment.duration(10, 's'), management: {} as any, @@ -126,7 +68,6 @@ describe('SearchSessionService', () => { service = new SearchSessionService(mockLogger, config$); const coreStart = coreMock.createStart(); const mockTaskManager = taskManagerMock.createStart(); - jest.useFakeTimers(); await flushPromises(); await service.start(coreStart, { taskManager: mockTaskManager, @@ -135,19 +76,6 @@ describe('SearchSessionService', () => { afterEach(() => { service.stop(); - jest.useRealTimers(); - }); - - it('search throws if `name` is not provided', () => { - expect(() => service.save(sessionId, {}, { savedObjectsClient })).rejects.toMatchInlineSnapshot( - `[Error: Name is required]` - ); - }); - - it('save throws if `name` is not provided', () => { - expect(() => service.save(sessionId, {}, { savedObjectsClient })).rejects.toMatchInlineSnapshot( - `[Error: Name is required]` - ); }); it('get calls saved objects client', async () => { @@ -182,7 +110,7 @@ describe('SearchSessionService', () => { }); }); - it('update calls saved objects client', async () => { + it('update calls saved objects client with added touch time', async () => { const mockUpdateSavedObject = { ...mockSavedObject, attributes: {}, @@ -193,11 +121,13 @@ describe('SearchSessionService', () => { const response = await service.update(sessionId, attributes, { savedObjectsClient }); expect(response).toBe(mockUpdateSavedObject); - expect(savedObjectsClient.update).toHaveBeenCalledWith( - SEARCH_SESSION_TYPE, - sessionId, - attributes - ); + + const [type, id, callAttributes] = savedObjectsClient.update.mock.calls[0]; + + expect(type).toBe(SEARCH_SESSION_TYPE); + expect(id).toBe(sessionId); + expect(callAttributes).toHaveProperty('name', attributes.name); + expect(callAttributes).toHaveProperty('touched'); }); it('delete calls saved objects client', async () => { @@ -256,17 +186,17 @@ describe('SearchSessionService', () => { spyGetId.mockRestore(); }); - it('calls `trackId` once if the response contains an `id` and not restoring', async () => { + it('calls `trackId` for every response, if the response contains an `id` and not restoring', async () => { const searchRequest = { params: {} }; const options = { sessionId, isStored: false, isRestore: false }; - const spyTrackId = jest.spyOn(service, 'trackId').mockResolvedValue(); + const spyTrackId = jest.spyOn(service, 'trackId'); mockSearch.mockReturnValueOnce(of({ id: 'my_id' }, { id: 'my_id' })); await service .search(mockStrategy, searchRequest, options, mockSearchDeps, mockDeps) .toPromise(); - expect(spyTrackId).toBeCalledTimes(1); + expect(spyTrackId).toBeCalledTimes(2); expect(spyTrackId).toBeCalledWith(searchRequest, 'my_id', options, {}); spyTrackId.mockRestore(); @@ -276,7 +206,7 @@ describe('SearchSessionService', () => { const searchRequest = { params: {} }; const options = { sessionId, isStored: true, isRestore: true }; const spyGetId = jest.spyOn(service, 'getId').mockResolvedValueOnce('my_id'); - const spyTrackId = jest.spyOn(service, 'trackId').mockResolvedValue(); + const spyTrackId = jest.spyOn(service, 'trackId'); mockSearch.mockReturnValueOnce(of({ id: 'my_id' })); await service @@ -291,84 +221,194 @@ describe('SearchSessionService', () => { }); describe('trackId', () => { - it('stores hash in memory when `isStored` is `false` for when `save` is called', async () => { + it('updates the saved object if search session already exists', async () => { const searchRequest = { params: {} }; const requestHash = createRequestHash(searchRequest.params); const searchId = 'FnpFYlBpeXdCUTMyZXhCLTc1TWFKX0EbdDFDTzJzTE1Sck9PVTBIcW1iU05CZzo4MDA0'; - const isStored = false; - const name = 'my saved background search session'; - const appId = 'my_app_id'; - const urlGeneratorId = 'my_url_generator_id'; - const created = new Date().toISOString(); - const expires = new Date().toISOString(); - - const mockIdMapping = createMockIdMapping([]); - const setSpy = jest.fn(); - mockIdMapping.set = setSpy; - Object.defineProperty(service, 'sessionSearchMap', { - get: () => mockIdMapping, - }); + + const mockUpdateSavedObject = { + ...mockSavedObject, + attributes: {}, + }; + savedObjectsClient.update.mockResolvedValue(mockUpdateSavedObject); await service.trackId( searchRequest, searchId, - { sessionId, isStored, strategy: MOCK_STRATEGY }, + { sessionId, strategy: MOCK_STRATEGY }, { savedObjectsClient } ); - expect(savedObjectsClient.update).not.toHaveBeenCalled(); + expect(savedObjectsClient.update).toHaveBeenCalled(); + expect(savedObjectsClient.create).not.toHaveBeenCalled(); - await service.save( - sessionId, - { name, created, expires, appId, urlGeneratorId }, + const [type, id, callAttributes] = savedObjectsClient.update.mock.calls[0]; + expect(type).toBe(SEARCH_SESSION_TYPE); + expect(id).toBe(sessionId); + expect(callAttributes).toHaveProperty('idMapping', { + [requestHash]: { + id: searchId, + status: SearchSessionStatus.IN_PROGRESS, + strategy: MOCK_STRATEGY, + }, + }); + expect(callAttributes).toHaveProperty('touched'); + }); + + it('retries updating the saved object if there was a ES conflict 409', async () => { + const searchRequest = { params: {} }; + const searchId = 'FnpFYlBpeXdCUTMyZXhCLTc1TWFKX0EbdDFDTzJzTE1Sck9PVTBIcW1iU05CZzo4MDA0'; + + const mockUpdateSavedObject = { + ...mockSavedObject, + attributes: {}, + }; + + let counter = 0; + + savedObjectsClient.update.mockImplementation(() => { + return new Promise((resolve, reject) => { + if (counter === 0) { + counter++; + reject(SavedObjectsErrorHelpers.createConflictError(SEARCH_SESSION_TYPE, searchId)); + } else { + resolve(mockUpdateSavedObject); + } + }); + }); + + await service.trackId( + searchRequest, + searchId, + { sessionId, strategy: MOCK_STRATEGY }, { savedObjectsClient } ); - expect(savedObjectsClient.create).toHaveBeenCalledWith( - SEARCH_SESSION_TYPE, - { - name, - created, - expires, - initialState: {}, - restoreState: {}, - status: SearchSessionStatus.IN_PROGRESS, - idMapping: {}, - appId, - urlGeneratorId, - sessionId, - }, - { id: sessionId } + expect(savedObjectsClient.update).toHaveBeenCalledTimes(2); + expect(savedObjectsClient.create).not.toHaveBeenCalled(); + }); + + it('retries updating the saved object if theres a ES conflict 409, but stops after MAX_RETRIES times', async () => { + const searchRequest = { params: {} }; + const searchId = 'FnpFYlBpeXdCUTMyZXhCLTc1TWFKX0EbdDFDTzJzTE1Sck9PVTBIcW1iU05CZzo4MDA0'; + + savedObjectsClient.update.mockImplementation(() => { + return new Promise((resolve, reject) => { + reject(SavedObjectsErrorHelpers.createConflictError(SEARCH_SESSION_TYPE, searchId)); + }); + }); + + await service.trackId( + searchRequest, + searchId, + { sessionId, strategy: MOCK_STRATEGY }, + { savedObjectsClient } ); - const [setSessionId, setParams] = setSpy.mock.calls[0]; - expect(setParams.ids.get(requestHash).id).toBe(searchId); - expect(setParams.ids.get(requestHash).strategy).toBe(MOCK_STRATEGY); - expect(setSessionId).toBe(sessionId); + // Track ID doesn't throw errors even in cases of failure! + expect(savedObjectsClient.update).toHaveBeenCalledTimes(MAX_UPDATE_RETRIES); + expect(savedObjectsClient.create).not.toHaveBeenCalled(); }); - it('updates saved object when `isStored` is `true`', async () => { + it('creates the saved object in non persisted state, if search session doesnt exists', async () => { const searchRequest = { params: {} }; const requestHash = createRequestHash(searchRequest.params); const searchId = 'FnpFYlBpeXdCUTMyZXhCLTc1TWFKX0EbdDFDTzJzTE1Sck9PVTBIcW1iU05CZzo4MDA0'; - const isStored = true; + + const mockCreatedSavedObject = { + ...mockSavedObject, + attributes: {}, + }; + + savedObjectsClient.update.mockRejectedValue( + SavedObjectsErrorHelpers.createGenericNotFoundError(sessionId) + ); + savedObjectsClient.create.mockResolvedValue(mockCreatedSavedObject); await service.trackId( searchRequest, searchId, - { sessionId, isStored, strategy: MOCK_STRATEGY }, + { sessionId, strategy: MOCK_STRATEGY }, { savedObjectsClient } ); - expect(savedObjectsClient.update).toHaveBeenCalledWith(SEARCH_SESSION_TYPE, sessionId, { - idMapping: { - [requestHash]: { - id: searchId, - strategy: MOCK_STRATEGY, - status: SearchStatus.IN_PROGRESS, - }, + expect(savedObjectsClient.update).toHaveBeenCalled(); + expect(savedObjectsClient.create).toHaveBeenCalled(); + + const [type, callAttributes, options] = savedObjectsClient.create.mock.calls[0]; + expect(type).toBe(SEARCH_SESSION_TYPE); + expect(options).toStrictEqual({ id: sessionId }); + expect(callAttributes).toHaveProperty('idMapping', { + [requestHash]: { + id: searchId, + status: SearchSessionStatus.IN_PROGRESS, + strategy: MOCK_STRATEGY, }, }); + expect(callAttributes).toHaveProperty('expires'); + expect(callAttributes).toHaveProperty('created'); + expect(callAttributes).toHaveProperty('touched'); + expect(callAttributes).toHaveProperty('sessionId', sessionId); + expect(callAttributes).toHaveProperty('persisted', false); + }); + + it('retries updating if update returned 404 and then update returned conflict 409 (first create race condition)', async () => { + const searchRequest = { params: {} }; + const searchId = 'FnpFYlBpeXdCUTMyZXhCLTc1TWFKX0EbdDFDTzJzTE1Sck9PVTBIcW1iU05CZzo4MDA0'; + + const mockUpdateSavedObject = { + ...mockSavedObject, + attributes: {}, + }; + + let counter = 0; + + savedObjectsClient.update.mockImplementation(() => { + return new Promise((resolve, reject) => { + if (counter === 0) { + counter++; + reject(SavedObjectsErrorHelpers.createGenericNotFoundError(sessionId)); + } else { + resolve(mockUpdateSavedObject); + } + }); + }); + + savedObjectsClient.create.mockRejectedValue( + SavedObjectsErrorHelpers.createConflictError(SEARCH_SESSION_TYPE, searchId) + ); + + await service.trackId( + searchRequest, + searchId, + { sessionId, strategy: MOCK_STRATEGY }, + { savedObjectsClient } + ); + + expect(savedObjectsClient.update).toHaveBeenCalledTimes(2); + expect(savedObjectsClient.create).toHaveBeenCalledTimes(1); + }); + + it('retries everything at most MAX_RETRIES times', async () => { + const searchRequest = { params: {} }; + const searchId = 'FnpFYlBpeXdCUTMyZXhCLTc1TWFKX0EbdDFDTzJzTE1Sck9PVTBIcW1iU05CZzo4MDA0'; + + savedObjectsClient.update.mockRejectedValue( + SavedObjectsErrorHelpers.createGenericNotFoundError(sessionId) + ); + savedObjectsClient.create.mockRejectedValue( + SavedObjectsErrorHelpers.createConflictError(SEARCH_SESSION_TYPE, searchId) + ); + + await service.trackId( + searchRequest, + searchId, + { sessionId, strategy: MOCK_STRATEGY }, + { savedObjectsClient } + ); + + expect(savedObjectsClient.update).toHaveBeenCalledTimes(MAX_UPDATE_RETRIES); + expect(savedObjectsClient.create).toHaveBeenCalledTimes(MAX_UPDATE_RETRIES); }); }); @@ -437,194 +477,95 @@ describe('SearchSessionService', () => { }); }); - describe('Monitor', () => { - it('schedules the next iteration', async () => { - const findSpy = jest.fn().mockResolvedValue({ saved_objects: [] }); - createMockInternalSavedObjectClient(findSpy); - - const mockIdMapping = createMockIdMapping( - [[MOCK_KEY_HASH, { id: MOCK_ASYNC_ID, strategy: MOCK_STRATEGY }]], - moment() + describe('save', () => { + it('save throws if `name` is not provided', () => { + expect(service.save(sessionId, {}, { savedObjectsClient })).rejects.toMatchInlineSnapshot( + `[Error: Name is required]` ); - - Object.defineProperty(service, 'sessionSearchMap', { - get: () => mockIdMapping, - }); - - jest.advanceTimersByTime(INMEM_TRACKING_INTERVAL); - expect(findSpy).toHaveBeenCalledTimes(1); - await flushPromises(); - - jest.advanceTimersByTime(INMEM_TRACKING_INTERVAL); - expect(findSpy).toHaveBeenCalledTimes(2); - }); - - it('should delete expired IDs', async () => { - const findSpy = jest.fn().mockResolvedValueOnce({ saved_objects: [] }); - createMockInternalSavedObjectClient(findSpy); - - const mockIdMapping = createMockIdMapping( - [[MOCK_KEY_HASH, { id: MOCK_ASYNC_ID, strategy: MOCK_STRATEGY }]], - moment().subtract(2, 'm') - ); - - const deleteSpy = jest.spyOn(mockIdMapping, 'delete'); - Object.defineProperty(service, 'sessionSearchMap', { - get: () => mockIdMapping, - }); - - // Get setInterval to fire - jest.advanceTimersByTime(INMEM_TRACKING_INTERVAL); - - expect(findSpy).not.toHaveBeenCalled(); - expect(deleteSpy).toHaveBeenCalledTimes(1); - }); - - it('should delete IDs that passed max retries', async () => { - const findSpy = jest.fn().mockResolvedValueOnce({ saved_objects: [] }); - createMockInternalSavedObjectClient(findSpy); - - const mockIdMapping = createMockIdMapping( - [[MOCK_KEY_HASH, { id: MOCK_ASYNC_ID, strategy: MOCK_STRATEGY }]], - moment(), - MAX_UPDATE_RETRIES - ); - - const deleteSpy = jest.spyOn(mockIdMapping, 'delete'); - Object.defineProperty(service, 'sessionSearchMap', { - get: () => mockIdMapping, - }); - - // Get setInterval to fire - jest.advanceTimersByTime(INMEM_TRACKING_INTERVAL); - - expect(findSpy).not.toHaveBeenCalled(); - expect(deleteSpy).toHaveBeenCalledTimes(1); }); - it('should not fetch when no IDs are mapped', async () => { - const findSpy = jest.fn().mockResolvedValueOnce({ saved_objects: [] }); - createMockInternalSavedObjectClient(findSpy); - - jest.advanceTimersByTime(INMEM_TRACKING_INTERVAL); - expect(findSpy).not.toHaveBeenCalled(); + it('save throws if `appId` is not provided', () => { + expect( + service.save(sessionId, { name: 'banana' }, { savedObjectsClient }) + ).rejects.toMatchInlineSnapshot(`[Error: AppId is required]`); }); - it('should try to fetch saved objects if some ids are mapped', async () => { - const mockIdMapping = createMockIdMapping([[MOCK_KEY_HASH, MOCK_ASYNC_ID]]); - Object.defineProperty(service, 'sessionSearchMap', { - get: () => mockIdMapping, - }); - - const findSpy = jest.fn().mockResolvedValueOnce({ saved_objects: [] }); - const bulkUpdateSpy = jest.fn().mockResolvedValueOnce({ saved_objects: [] }); - createMockInternalSavedObjectClient(findSpy, bulkUpdateSpy); - - jest.advanceTimersByTime(INMEM_TRACKING_INTERVAL); - expect(findSpy).toHaveBeenCalledTimes(1); - expect(bulkUpdateSpy).not.toHaveBeenCalled(); + it('save throws if `generator id` is not provided', () => { + expect( + service.save(sessionId, { name: 'banana', appId: 'nanana' }, { savedObjectsClient }) + ).rejects.toMatchInlineSnapshot(`[Error: UrlGeneratorId is required]`); }); - it('should update saved objects if they are found, and delete session on success', async () => { - const mockIdMapping = createMockIdMapping([[MOCK_KEY_HASH, MOCK_ASYNC_ID]], undefined, 1); - const mockMapDeleteSpy = jest.fn(); - const mockSessionDeleteSpy = jest.fn(); - mockIdMapping.delete = mockMapDeleteSpy; - mockIdMapping.get(MOCK_SESSION_ID)!.ids.delete = mockSessionDeleteSpy; - Object.defineProperty(service, 'sessionSearchMap', { - get: () => mockIdMapping, - }); - - const findSpy = jest.fn().mockResolvedValueOnce({ - saved_objects: [ - { - id: MOCK_SESSION_ID, - attributes: { - idMapping: { - b: 'c', - }, - }, - }, - ], - }); - const bulkUpdateSpy = jest.fn().mockResolvedValueOnce({ - saved_objects: [ - { - id: MOCK_SESSION_ID, - attributes: { - idMapping: { - b: 'c', - [MOCK_KEY_HASH]: { - id: MOCK_ASYNC_ID, - strategy: MOCK_STRATEGY, - }, - }, - }, - }, - ], - }); - createMockInternalSavedObjectClient(findSpy, bulkUpdateSpy); - - jest.advanceTimersByTime(INMEM_TRACKING_INTERVAL); + it('saving updates an existing saved object and persists it', async () => { + const mockUpdateSavedObject = { + ...mockSavedObject, + attributes: {}, + }; + savedObjectsClient.update.mockResolvedValue(mockUpdateSavedObject); - // Release timers to call check after test actions are done. - jest.useRealTimers(); - await new Promise((r) => setTimeout(r, 15)); + await service.save( + sessionId, + { + name: 'banana', + appId: 'nanana', + urlGeneratorId: 'panama', + }, + { savedObjectsClient } + ); - expect(findSpy).toHaveBeenCalledTimes(1); - expect(bulkUpdateSpy).toHaveBeenCalledTimes(1); - expect(mockSessionDeleteSpy).toHaveBeenCalledTimes(2); - expect(mockSessionDeleteSpy).toBeCalledWith('b'); - expect(mockSessionDeleteSpy).toBeCalledWith(MOCK_KEY_HASH); - expect(mockMapDeleteSpy).toBeCalledTimes(1); + expect(savedObjectsClient.update).toHaveBeenCalled(); + expect(savedObjectsClient.create).not.toHaveBeenCalled(); + + const [type, id, callAttributes] = savedObjectsClient.update.mock.calls[0]; + expect(type).toBe(SEARCH_SESSION_TYPE); + expect(id).toBe(sessionId); + expect(callAttributes).not.toHaveProperty('idMapping'); + expect(callAttributes).toHaveProperty('touched'); + expect(callAttributes).toHaveProperty('persisted', true); + expect(callAttributes).toHaveProperty('name', 'banana'); + expect(callAttributes).toHaveProperty('appId', 'nanana'); + expect(callAttributes).toHaveProperty('urlGeneratorId', 'panama'); + expect(callAttributes).toHaveProperty('initialState', {}); + expect(callAttributes).toHaveProperty('restoreState', {}); }); - it('should update saved objects if they are found, and increase retryCount on error', async () => { - const mockIdMapping = createMockIdMapping([[MOCK_KEY_HASH, MOCK_ASYNC_ID]]); - const mockMapDeleteSpy = jest.fn(); - const mockSessionDeleteSpy = jest.fn(); - mockIdMapping.delete = mockMapDeleteSpy; - mockIdMapping.get(MOCK_SESSION_ID)!.ids.delete = mockSessionDeleteSpy; - Object.defineProperty(service, 'sessionSearchMap', { - get: () => mockIdMapping, - }); - - const findSpy = jest.fn().mockResolvedValueOnce({ - saved_objects: [ - { - id: MOCK_SESSION_ID, - attributes: { - idMapping: { - b: { - id: 'c', - strategy: MOCK_STRATEGY, - }, - }, - }, - }, - ], - }); - const bulkUpdateSpy = jest.fn().mockResolvedValueOnce({ - saved_objects: [ - { - id: MOCK_SESSION_ID, - error: 'not ok', - }, - ], - }); - createMockInternalSavedObjectClient(findSpy, bulkUpdateSpy); + it('saving creates a new persisted saved object, if it did not exist', async () => { + const mockCreatedSavedObject = { + ...mockSavedObject, + attributes: {}, + }; - jest.advanceTimersByTime(INMEM_TRACKING_INTERVAL); + savedObjectsClient.update.mockRejectedValue( + SavedObjectsErrorHelpers.createGenericNotFoundError(sessionId) + ); + savedObjectsClient.create.mockResolvedValue(mockCreatedSavedObject); - // Release timers to call check after test actions are done. - jest.useRealTimers(); - await new Promise((r) => setTimeout(r, 15)); + await service.save( + sessionId, + { + name: 'banana', + appId: 'nanana', + urlGeneratorId: 'panama', + }, + { savedObjectsClient } + ); - expect(findSpy).toHaveBeenCalledTimes(1); - expect(bulkUpdateSpy).toHaveBeenCalledTimes(1); - expect(mockSessionDeleteSpy).not.toHaveBeenCalled(); - expect(mockMapDeleteSpy).not.toHaveBeenCalled(); - expect(mockIdMapping.get(MOCK_SESSION_ID)!.retryCount).toBe(1); + expect(savedObjectsClient.update).toHaveBeenCalledTimes(1); + expect(savedObjectsClient.create).toHaveBeenCalledTimes(1); + + const [type, callAttributes, options] = savedObjectsClient.create.mock.calls[0]; + expect(type).toBe(SEARCH_SESSION_TYPE); + expect(options?.id).toBe(sessionId); + expect(callAttributes).toHaveProperty('idMapping', {}); + expect(callAttributes).toHaveProperty('touched'); + expect(callAttributes).toHaveProperty('expires'); + expect(callAttributes).toHaveProperty('created'); + expect(callAttributes).toHaveProperty('persisted', true); + expect(callAttributes).toHaveProperty('name', 'banana'); + expect(callAttributes).toHaveProperty('appId', 'nanana'); + expect(callAttributes).toHaveProperty('urlGeneratorId', 'panama'); + expect(callAttributes).toHaveProperty('initialState', {}); + expect(callAttributes).toHaveProperty('restoreState', {}); }); }); }); diff --git a/x-pack/plugins/data_enhanced/server/search/session/session_service.ts b/x-pack/plugins/data_enhanced/server/search/session/session_service.ts index 794baa21e2f4c..c8a8b58de25bb 100644 --- a/x-pack/plugins/data_enhanced/server/search/session/session_service.ts +++ b/x-pack/plugins/data_enhanced/server/search/session/session_service.ts @@ -4,27 +4,23 @@ * you may not use this file except in compliance with the Elastic License. */ -import moment, { Moment } from 'moment'; import { from, Observable } from 'rxjs'; -import { first, switchMap } from 'rxjs/operators'; +import { first, switchMap, tap } from 'rxjs/operators'; import { CoreStart, KibanaRequest, - SavedObjectsClient, SavedObjectsClientContract, Logger, - SavedObject, CoreSetup, - SavedObjectsBulkUpdateObject, SavedObjectsFindOptions, + SavedObjectsErrorHelpers, + SavedObjectsUpdateResponse, + SavedObject, } from '../../../../../../src/core/server'; import { IKibanaSearchRequest, IKibanaSearchResponse, ISearchOptions, - KueryNode, - nodeBuilder, - tapFirst, } from '../../../../../../src/plugins/data/common'; import { ISearchStrategy, @@ -36,26 +32,19 @@ import { TaskManagerStartContract, } from '../../../../task_manager/server'; import { - SearchSessionSavedObjectAttributes, SearchSessionRequestInfo, + SearchSessionSavedObjectAttributes, SearchSessionStatus, + SEARCH_SESSION_TYPE, } from '../../../common'; -import { SEARCH_SESSION_TYPE } from '../../saved_objects'; import { createRequestHash } from './utils'; import { ConfigSchema } from '../../../config'; import { registerSearchSessionsTask, scheduleSearchSessionsTasks } from './monitoring_task'; -import { SearchStatus } from './types'; +import { SearchSessionsConfig, SearchStatus } from './types'; export interface SearchSessionDependencies { savedObjectsClient: SavedObjectsClientContract; } - -export interface SessionInfo { - insertTime: Moment; - retryCount: number; - ids: Map; -} - interface SetupDependencies { taskManager: TaskManagerSetupContract; } @@ -64,16 +53,10 @@ interface StartDependencies { taskManager: TaskManagerStartContract; } -type SearchSessionsConfig = ConfigSchema['search']['sessions']; - +function sleep(ms: number) { + return new Promise((r) => setTimeout(r, ms)); +} export class SearchSessionService implements ISessionService { - /** - * Map of sessionId to { [requestHash]: searchId } - * @private - */ - private sessionSearchMap = new Map(); - private internalSavedObjectsClient!: SavedObjectsClientContract; - private monitorTimer!: NodeJS.Timeout; private config!: SearchSessionsConfig; constructor( @@ -95,139 +78,14 @@ export class SearchSessionService implements ISessionService { return this.setupMonitoring(core, deps); } - public stop() { - this.sessionSearchMap.clear(); - clearTimeout(this.monitorTimer); - } + public stop() {} private setupMonitoring = async (core: CoreStart, deps: StartDependencies) => { if (this.config.enabled) { scheduleSearchSessionsTasks(deps.taskManager, this.logger, this.config.trackingInterval); - this.logger.debug(`setupMonitoring | Enabling monitoring`); - const internalRepo = core.savedObjects.createInternalRepository([SEARCH_SESSION_TYPE]); - this.internalSavedObjectsClient = new SavedObjectsClient(internalRepo); - this.monitorMappedIds(); } }; - /** - * Compiles a KQL Query to fetch sessions by ID. - * Done as a performance optimization workaround. - */ - private sessionIdsAsFilters(sessionIds: string[]): KueryNode { - return nodeBuilder.or( - sessionIds.map((id) => { - return nodeBuilder.is(`${SEARCH_SESSION_TYPE}.attributes.sessionId`, id); - }) - ); - } - - /** - * Gets all {@link SessionSavedObjectAttributes | Background Searches} that - * currently being tracked by the service. - * - * @remarks - * Uses `internalSavedObjectsClient` as this is called asynchronously, not within the - * context of a user's session. - */ - private async getAllMappedSavedObjects() { - const filter = this.sessionIdsAsFilters(Array.from(this.sessionSearchMap.keys())); - const res = await this.internalSavedObjectsClient.find({ - perPage: this.config.pageSize, // If there are more sessions in memory, they will be synced when some items are cleared out. - type: SEARCH_SESSION_TYPE, - filter, - namespaces: ['*'], - }); - this.logger.debug(`getAllMappedSavedObjects | Got ${res.saved_objects.length} items`); - return res.saved_objects; - } - - private clearSessions = async () => { - const curTime = moment(); - - this.sessionSearchMap.forEach((sessionInfo, sessionId) => { - if ( - moment.duration(curTime.diff(sessionInfo.insertTime)).asMilliseconds() > - this.config.inMemTimeout.asMilliseconds() - ) { - this.logger.debug(`clearSessions | Deleting expired session ${sessionId}`); - this.sessionSearchMap.delete(sessionId); - } else if (sessionInfo.retryCount >= this.config.maxUpdateRetries) { - this.logger.warn(`clearSessions | Deleting failed session ${sessionId}`); - this.sessionSearchMap.delete(sessionId); - } - }); - }; - - private async monitorMappedIds() { - this.monitorTimer = setTimeout(async () => { - try { - this.clearSessions(); - - if (!this.sessionSearchMap.size) return; - this.logger.debug(`monitorMappedIds | Map contains ${this.sessionSearchMap.size} items`); - - const savedSessions = await this.getAllMappedSavedObjects(); - const updatedSessions = await this.updateAllSavedObjects(savedSessions); - - updatedSessions.forEach((updatedSavedObject) => { - const sessionInfo = this.sessionSearchMap.get(updatedSavedObject.id)!; - if (updatedSavedObject.error) { - this.logger.warn( - `monitorMappedIds | update error ${JSON.stringify(updatedSavedObject.error) || ''}` - ); - // Retry next time - sessionInfo.retryCount++; - } else if (updatedSavedObject.attributes.idMapping) { - // Delete the ids that we just saved, avoiding a potential new ids being lost. - Object.keys(updatedSavedObject.attributes.idMapping).forEach((key) => { - sessionInfo.ids.delete(key); - }); - // If the session object is empty, delete it as well - if (!sessionInfo.ids.entries.length) { - this.sessionSearchMap.delete(updatedSavedObject.id); - } else { - sessionInfo.retryCount = 0; - } - } - }); - } catch (e) { - this.logger.error(`monitorMappedIds | Error while updating sessions. ${e}`); - } finally { - this.monitorMappedIds(); - } - }, this.config.trackingInterval.asMilliseconds()); - } - - private async updateAllSavedObjects( - activeMappingObjects: Array> - ) { - if (!activeMappingObjects.length) return []; - - this.logger.debug(`updateAllSavedObjects | Updating ${activeMappingObjects.length} items`); - const updatedSessions: Array< - SavedObjectsBulkUpdateObject - > = activeMappingObjects - .filter((so) => !so.error) - .map((sessionSavedObject) => { - const sessionInfo = this.sessionSearchMap.get(sessionSavedObject.id); - const idMapping = sessionInfo ? Object.fromEntries(sessionInfo.ids.entries()) : {}; - sessionSavedObject.attributes.idMapping = { - ...sessionSavedObject.attributes.idMapping, - ...idMapping, - }; - return { - ...sessionSavedObject, - namespace: sessionSavedObject.namespaces?.[0], - }; - }); - - const updateResults = await this.internalSavedObjectsClient.bulkUpdate( - updatedSessions - ); - return updateResults.saved_objects; - } - public search( strategy: ISearchStrategy, searchRequest: Request, @@ -246,53 +104,112 @@ export class SearchSessionService implements ISessionService { return from(getSearchRequest()).pipe( switchMap((request) => strategy.search(request, options, searchDeps)), - tapFirst((response) => { - if (searchRequest.id || !options.sessionId || !response.id || options.isRestore) return; + tap((response) => { + if (!options.sessionId || !response.id || options.isRestore) return; this.trackId(searchRequest, response.id, options, deps); }) ); } - // TODO: Generate the `userId` from the realm type/realm name/username + private updateOrCreate = async ( + sessionId: string, + attributes: Partial, + deps: SearchSessionDependencies, + retry: number = 1 + ): Promise< + | SavedObjectsUpdateResponse + | SavedObject + | undefined + > => { + const retryOnConflict = async (e: any) => { + this.logger.debug(`Conflict error | ${sessionId}`); + // Randomize sleep to spread updates out in case of conflicts + await sleep(100 + Math.random() * 50); + return await this.updateOrCreate(sessionId, attributes, deps, retry + 1); + }; + + this.logger.debug(`updateOrCreate | ${sessionId} | ${retry}`); + try { + return await this.update(sessionId, attributes, deps); + } catch (e) { + if (SavedObjectsErrorHelpers.isNotFoundError(e)) { + try { + this.logger.debug(`Object not found | ${sessionId}`); + return await this.create(sessionId, attributes, deps); + } catch (createError) { + if ( + SavedObjectsErrorHelpers.isConflictError(createError) && + retry < this.config.maxUpdateRetries + ) { + return await retryOnConflict(createError); + } else { + this.logger.error(createError); + } + } + } else if ( + SavedObjectsErrorHelpers.isConflictError(e) && + retry < this.config.maxUpdateRetries + ) { + return await retryOnConflict(e); + } else { + this.logger.error(e); + } + } + + return undefined; + }; + public save = async ( sessionId: string, { name, appId, - created = new Date().toISOString(), - expires = new Date(Date.now() + this.config.defaultExpiration.asMilliseconds()).toISOString(), - status = SearchSessionStatus.IN_PROGRESS, urlGeneratorId, initialState = {}, restoreState = {}, }: Partial, - { savedObjectsClient }: SearchSessionDependencies + deps: SearchSessionDependencies ) => { if (!name) throw new Error('Name is required'); if (!appId) throw new Error('AppId is required'); if (!urlGeneratorId) throw new Error('UrlGeneratorId is required'); - this.logger.debug(`save | ${sessionId}`); - - const attributes = { - name, - created, - expires, - status, - initialState, - restoreState, - idMapping: {}, - urlGeneratorId, - appId, + return this.updateOrCreate( sessionId, - }; - const session = await savedObjectsClient.create( + { + name, + appId, + urlGeneratorId, + initialState, + restoreState, + persisted: true, + }, + deps + ); + }; + + private create = ( + sessionId: string, + attributes: Partial, + { savedObjectsClient }: SearchSessionDependencies + ) => { + this.logger.debug(`create | ${sessionId}`); + return savedObjectsClient.create( SEARCH_SESSION_TYPE, - attributes, + { + sessionId, + status: SearchSessionStatus.IN_PROGRESS, + expires: new Date( + Date.now() + this.config.defaultExpiration.asMilliseconds() + ).toISOString(), + created: new Date().toISOString(), + touched: new Date().toISOString(), + idMapping: {}, + persisted: false, + ...attributes, + }, { id: sessionId } ); - - return session; }; // TODO: Throw an error if this session doesn't belong to this user @@ -325,7 +242,10 @@ export class SearchSessionService implements ISessionService { return savedObjectsClient.update( SEARCH_SESSION_TYPE, sessionId, - attributes + { + ...attributes, + touched: new Date().toISOString(), + } ); }; @@ -335,41 +255,31 @@ export class SearchSessionService implements ISessionService { }; /** - * Tracks the given search request/search ID in the saved session (if it exists). Otherwise, just - * store it in memory until a saved session exists. + * Tracks the given search request/search ID in the saved session. * @internal */ public trackId = async ( searchRequest: IKibanaSearchRequest, searchId: string, - { sessionId, isStored, strategy }: ISearchOptions, + { sessionId, strategy }: ISearchOptions, deps: SearchSessionDependencies ) => { if (!sessionId || !searchId) return; this.logger.debug(`trackId | ${sessionId} | ${searchId}`); - const requestHash = createRequestHash(searchRequest.params); - const searchInfo = { - id: searchId, - strategy: strategy!, - status: SearchStatus.IN_PROGRESS, - }; - // If there is already a saved object for this session, update it to include this request/ID. - // Otherwise, just update the in-memory mapping for this session for when the session is saved. - if (isStored) { - const attributes = { - idMapping: { [requestHash]: searchInfo }, - }; - await this.update(sessionId, attributes, deps); - } else { - const map = this.sessionSearchMap.get(sessionId) ?? { - insertTime: moment(), - retryCount: 0, - ids: new Map(), + let idMapping: Record = {}; + + if (searchRequest.params) { + const requestHash = createRequestHash(searchRequest.params); + const searchInfo = { + id: searchId, + strategy: strategy!, + status: SearchStatus.IN_PROGRESS, }; - map.ids.set(requestHash, searchInfo); - this.sessionSearchMap.set(sessionId, map); + idMapping = { [requestHash]: searchInfo }; } + + return this.updateOrCreate(sessionId, { idMapping }, deps); }; /** diff --git a/x-pack/plugins/data_enhanced/server/search/session/types.ts b/x-pack/plugins/data_enhanced/server/search/session/types.ts index c30e03f70d2dc..136c37942cb2e 100644 --- a/x-pack/plugins/data_enhanced/server/search/session/types.ts +++ b/x-pack/plugins/data_enhanced/server/search/session/types.ts @@ -4,8 +4,12 @@ * you may not use this file except in compliance with the Elastic License. */ +import { ConfigSchema } from '../../../config'; + export enum SearchStatus { IN_PROGRESS = 'in_progress', ERROR = 'error', COMPLETE = 'complete', } + +export type SearchSessionsConfig = ConfigSchema['search']['sessions']; diff --git a/x-pack/plugins/encrypted_saved_objects/server/saved_objects/encrypted_saved_objects_client_wrapper.test.ts b/x-pack/plugins/encrypted_saved_objects/server/saved_objects/encrypted_saved_objects_client_wrapper.test.ts index 90700f8fa7521..7ec700607f3e0 100644 --- a/x-pack/plugins/encrypted_saved_objects/server/saved_objects/encrypted_saved_objects_client_wrapper.test.ts +++ b/x-pack/plugins/encrypted_saved_objects/server/saved_objects/encrypted_saved_objects_client_wrapper.test.ts @@ -83,11 +83,11 @@ describe('#create', () => { expect(mockBaseClient.create).toHaveBeenCalledWith('unknown-type', attributes, options); }); - it('fails if type is registered and ID is specified', async () => { + it('fails if type is registered and non-UUID ID is specified', async () => { const attributes = { attrOne: 'one', attrSecret: 'secret', attrThree: 'three' }; await expect(wrapper.create('known-type', attributes, { id: 'some-id' })).rejects.toThrowError( - 'Predefined IDs are not allowed for saved objects with encrypted attributes, unless the ID has been generated using `SavedObjectsUtils.generateId`.' + 'Predefined IDs are not allowed for saved objects with encrypted attributes unless the ID is a UUID.' ); expect(mockBaseClient.create).not.toHaveBeenCalled(); @@ -310,7 +310,7 @@ describe('#bulkCreate', () => { ); }); - it('fails if ID is specified for registered type', async () => { + it('fails if non-UUID ID is specified for registered type', async () => { const attributes = { attrOne: 'one', attrSecret: 'secret', attrThree: 'three' }; const bulkCreateParams = [ @@ -319,7 +319,7 @@ describe('#bulkCreate', () => { ]; await expect(wrapper.bulkCreate(bulkCreateParams)).rejects.toThrowError( - 'Predefined IDs are not allowed for saved objects with encrypted attributes, unless the ID has been generated using `SavedObjectsUtils.generateId`.' + 'Predefined IDs are not allowed for saved objects with encrypted attributes unless the ID is a UUID.' ); expect(mockBaseClient.bulkCreate).not.toHaveBeenCalled(); diff --git a/x-pack/plugins/encrypted_saved_objects/server/saved_objects/encrypted_saved_objects_client_wrapper.ts b/x-pack/plugins/encrypted_saved_objects/server/saved_objects/encrypted_saved_objects_client_wrapper.ts index c3008a8e86505..21475f6a4f5d2 100644 --- a/x-pack/plugins/encrypted_saved_objects/server/saved_objects/encrypted_saved_objects_client_wrapper.ts +++ b/x-pack/plugins/encrypted_saved_objects/server/saved_objects/encrypted_saved_objects_client_wrapper.ts @@ -59,7 +59,7 @@ export class EncryptedSavedObjectsClientWrapper implements SavedObjectsClientCon return await this.options.baseClient.create(type, attributes, options); } - const id = getValidId(options.id, options.version, options.overwrite); + const id = this.getValidId(options.id, options.version, options.overwrite); const namespace = getDescriptorNamespace( this.options.baseTypeRegistry, type, @@ -93,7 +93,7 @@ export class EncryptedSavedObjectsClientWrapper implements SavedObjectsClientCon return object; } - const id = getValidId(object.id, object.version, options?.overwrite); + const id = this.getValidId(object.id, object.version, options?.overwrite); const namespace = getDescriptorNamespace( this.options.baseTypeRegistry, object.type, @@ -307,27 +307,27 @@ export class EncryptedSavedObjectsClientWrapper implements SavedObjectsClientCon return response; } -} -// Saved objects with encrypted attributes should have IDs that are hard to guess especially -// since IDs are part of the AAD used during encryption, that's why we control them within this -// wrapper and don't allow consumers to specify their own IDs directly unless overwriting the original document. -function getValidId( - id: string | undefined, - version: string | undefined, - overwrite: boolean | undefined -) { - if (id) { - // only allow a specified ID if we're overwriting an existing ESO with a Version - // this helps us ensure that the document really was previously created using ESO - // and not being used to get around the specified ID limitation - const canSpecifyID = (overwrite && version) || SavedObjectsUtils.isRandomId(id); - if (!canSpecifyID) { - throw new Error( - 'Predefined IDs are not allowed for saved objects with encrypted attributes, unless the ID has been generated using `SavedObjectsUtils.generateId`.' - ); + // Saved objects with encrypted attributes should have IDs that are hard to guess especially + // since IDs are part of the AAD used during encryption, that's why we control them within this + // wrapper and don't allow consumers to specify their own IDs directly unless overwriting the original document. + private getValidId( + id: string | undefined, + version: string | undefined, + overwrite: boolean | undefined + ) { + if (id) { + // only allow a specified ID if we're overwriting an existing ESO with a Version + // this helps us ensure that the document really was previously created using ESO + // and not being used to get around the specified ID limitation + const canSpecifyID = (overwrite && version) || SavedObjectsUtils.isRandomId(id); + if (!canSpecifyID) { + throw this.errors.createBadRequestError( + 'Predefined IDs are not allowed for saved objects with encrypted attributes unless the ID is a UUID.' + ); + } + return id; } - return id; + return SavedObjectsUtils.generateId(); } - return SavedObjectsUtils.generateId(); } diff --git a/x-pack/plugins/enterprise_search/public/applications/workplace_search/views/content_sources/constants.ts b/x-pack/plugins/enterprise_search/public/applications/workplace_search/views/content_sources/constants.ts index 48b8a06b2549c..5e106a7f42f57 100644 --- a/x-pack/plugins/enterprise_search/public/applications/workplace_search/views/content_sources/constants.ts +++ b/x-pack/plugins/enterprise_search/public/applications/workplace_search/views/content_sources/constants.ts @@ -311,3 +311,157 @@ export const SOURCE_NAME_LABEL = i18n.translate( defaultMessage: 'Source name', } ); + +export const ORG_SOURCES_LINK = i18n.translate( + 'xpack.enterpriseSearch.workplaceSearch.sources.org.link', + { + defaultMessage: 'Add an organization content source', + } +); + +export const ORG_SOURCES_HEADER_TITLE = i18n.translate( + 'xpack.enterpriseSearch.workplaceSearch.sources.org.title', + { + defaultMessage: 'Organization sources', + } +); + +export const ORG_SOURCES_HEADER_DESCRIPTION = i18n.translate( + 'xpack.enterpriseSearch.workplaceSearch.sources.org.description', + { + defaultMessage: + 'Organization sources are available to the entire organization and can be assigned to specific user groups.', + } +); + +export const PRIVATE_LINK_TITLE = i18n.translate( + 'xpack.enterpriseSearch.workplaceSearch.sources.private.link', + { + defaultMessage: 'Add a private content source', + } +); + +export const PRIVATE_CAN_CREATE_PAGE_TITLE = i18n.translate( + 'xpack.enterpriseSearch.workplaceSearch.sources.private.canCreate.title', + { + defaultMessage: 'Manage private content sources', + } +); + +export const PRIVATE_VIEW_ONLY_PAGE_TITLE = i18n.translate( + 'xpack.enterpriseSearch.workplaceSearch.sources.private.vewOnly.title', + { + defaultMessage: 'Review Group Sources', + } +); + +export const PRIVATE_VIEW_ONLY_PAGE_DESCRIPTION = i18n.translate( + 'xpack.enterpriseSearch.workplaceSearch.sources.private.vewOnly.description', + { + defaultMessage: 'Review the status of all sources shared with your Group.', + } +); + +export const PRIVATE_CAN_CREATE_PAGE_DESCRIPTION = i18n.translate( + 'xpack.enterpriseSearch.workplaceSearch.sources.private.canCreate.description', + { + defaultMessage: + 'Review the status of all connected private sources, and manage private sources for your account.', + } +); + +export const PRIVATE_HEADER_TITLE = i18n.translate( + 'xpack.enterpriseSearch.workplaceSearch.sources.private.header.title', + { + defaultMessage: 'My private content sources', + } +); + +export const PRIVATE_HEADER_DESCRIPTION = i18n.translate( + 'xpack.enterpriseSearch.workplaceSearch.sources.private.header.description', + { + defaultMessage: 'Private content sources are available only to you.', + } +); + +export const PRIVATE_SHARED_SOURCES_TITLE = i18n.translate( + 'xpack.enterpriseSearch.workplaceSearch.sources.private.privateShared.header.title', + { + defaultMessage: 'Shared content sources', + } +); + +export const PRIVATE_EMPTY_TITLE = i18n.translate( + 'xpack.enterpriseSearch.workplaceSearch.sources.private.empty.title', + { + defaultMessage: 'You have no private sources', + } +); +export const SHARED_EMPTY_TITLE = i18n.translate( + 'xpack.enterpriseSearch.workplaceSearch.sources.shared.empty.title', + { + defaultMessage: 'No content source available', + } +); + +export const SHARED_EMPTY_DESCRIPTION = i18n.translate( + 'xpack.enterpriseSearch.workplaceSearch.sources.shared.empty.description', + { + defaultMessage: + 'Once content sources are shared with you, they will be displayed here, and available via the search experience.', + } +); + +export const AND = i18n.translate('xpack.enterpriseSearch.workplaceSearch.and', { + defaultMessage: 'and', +}); + +export const LICENSE_CALLOUT_TITLE = i18n.translate( + 'xpack.enterpriseSearch.workplaceSearch.sources.licenseCallout.title', + { + defaultMessage: 'Private Sources are no longer available', + } +); + +export const LICENSE_CALLOUT_DESCRIPTION = i18n.translate( + 'xpack.enterpriseSearch.workplaceSearch.sources.licenseCallout.description', + { + defaultMessage: 'Contact your search experience administrator for more information.', + } +); + +export const SOURCE_DISABLED_CALLOUT_TITLE = i18n.translate( + 'xpack.enterpriseSearch.workplaceSearch.sources.sourceDisabled.title', + { + defaultMessage: 'Content source is disabled', + } +); + +export const SOURCE_DISABLED_CALLOUT_DESCRIPTION = i18n.translate( + 'xpack.enterpriseSearch.workplaceSearch.sources.sourceDisabled.description', + { + defaultMessage: + 'Your organization’s license level has changed. Your data is safe, but document-level permissions are no longer supported and searching of this source has been disabled. Upgrade to a Platinum license to re-enable this source.', + } +); + +export const SOURCE_DISABLED_CALLOUT_BUTTON = i18n.translate( + 'xpack.enterpriseSearch.workplaceSearch.sources.sourceDisabled.button', + { + defaultMessage: 'Explore Platinum license', + } +); + +export const DOCUMENT_PERMISSIONS_LINK = i18n.translate( + 'xpack.enterpriseSearch.workplaceSearch.sources.documentPermissionsLink', + { + defaultMessage: 'Learn more about document-level permission configuration', + } +); + +export const UNDERSTAND_BUTTON = i18n.translate( + 'xpack.enterpriseSearch.workplaceSearch.sources.understandButton', + { + defaultMessage: 'I understand', + } +); diff --git a/x-pack/plugins/enterprise_search/public/applications/workplace_search/views/content_sources/organization_sources.tsx b/x-pack/plugins/enterprise_search/public/applications/workplace_search/views/content_sources/organization_sources.tsx index fdb536dd79771..3081301fe0a9f 100644 --- a/x-pack/plugins/enterprise_search/public/applications/workplace_search/views/content_sources/organization_sources.tsx +++ b/x-pack/plugins/enterprise_search/public/applications/workplace_search/views/content_sources/organization_sources.tsx @@ -12,6 +12,12 @@ import { Link, Redirect } from 'react-router-dom'; import { EuiButton } from '@elastic/eui'; import { ADD_SOURCE_PATH, getSourcesPath } from '../../routes'; +import { + ORG_SOURCES_LINK, + ORG_SOURCES_HEADER_TITLE, + ORG_SOURCES_HEADER_DESCRIPTION, +} from './constants'; + import { Loading } from '../../../shared/loading'; import { ContentSection } from '../../components/shared/content_section'; import { SourcesTable } from '../../components/shared/sources_table'; @@ -21,11 +27,6 @@ import { SourcesLogic } from './sources_logic'; import { SourcesView } from './sources_view'; -const ORG_LINK_TITLE = 'Add an organization content source'; -const ORG_HEADER_TITLE = 'Organization sources'; -const ORG_HEADER_DESCRIPTION = - 'Organization sources are available to the entire organization and can be assigned to specific user groups.'; - export const OrganizationSources: React.FC = () => { const { initializeSources, setSourceSearchability, resetSourcesState } = useActions(SourcesLogic); @@ -40,28 +41,22 @@ export const OrganizationSources: React.FC = () => { if (contentSources.length === 0) return ; - const linkTitle = ORG_LINK_TITLE; - const headerTitle = ORG_HEADER_TITLE; - const headerDescription = ORG_HEADER_DESCRIPTION; - const sectionTitle = ''; - const sectionDescription = ''; - return ( - {linkTitle} + {ORG_SOURCES_LINK} } - description={headerDescription} + description={ORG_SOURCES_HEADER_DESCRIPTION} alignItems="flexStart" /> - + { const { hasPlatinumLicense } = useValues(LicensingLogic); const { initializeSources, setSourceSearchability, resetSourcesState } = useActions(SourcesLogic); @@ -112,7 +119,7 @@ export const PrivateSources: React.FC = () => { - You have no private sources} /> + {PRIVATE_EMPTY_TITLE}} /> @@ -124,13 +131,8 @@ export const PrivateSources: React.FC = () => { No content source available} - body={ -

- Once content sources are shared with you, they will be displayed here, and available - via the search experience. -

- } + title={

{SHARED_EMPTY_TITLE}

} + body={

{SHARED_EMPTY_DESCRIPTION}

} /> @@ -140,16 +142,21 @@ export const PrivateSources: React.FC = () => { const hasPrivateSources = privateContentSources?.length > 0; const privateSources = hasPrivateSources ? privateSourcesTable : privateSourcesEmptyState; - const groupsSentence = `${groups.slice(0, groups.length - 1).join(', ')}, and ${groups.slice( + const groupsSentence = `${groups.slice(0, groups.length - 1).join(', ')}, ${AND} ${groups.slice( -1 )}`; const sharedSources = ( @@ -157,8 +164,8 @@ export const PrivateSources: React.FC = () => { const licenseCallout = ( <> - -

Contact your search experience administrator for more information.

+ +

{LICENSE_CALLOUT_DESCRIPTION}

 diff --git a/x-pack/plugins/enterprise_search/public/applications/workplace_search/views/content_sources/source_router.tsx b/x-pack/plugins/enterprise_search/public/applications/workplace_search/views/content_sources/source_router.tsx index f46743778a168..67995a4920925 100644 --- a/x-pack/plugins/enterprise_search/public/applications/workplace_search/views/content_sources/source_router.tsx +++ b/x-pack/plugins/enterprise_search/public/applications/workplace_search/views/content_sources/source_router.tsx @@ -17,6 +17,12 @@ import { SendWorkplaceSearchTelemetry as SendTelemetry } from '../../../shared/t import { NAV } from '../../constants'; +import { + SOURCE_DISABLED_CALLOUT_TITLE, + SOURCE_DISABLED_CALLOUT_DESCRIPTION, + SOURCE_DISABLED_CALLOUT_BUTTON, +} from './constants'; + import { ENT_SEARCH_LICENSE_MANAGEMENT, REINDEX_JOB_PATH, @@ -80,14 +86,10 @@ export const SourceRouter: React.FC = () => { const callout = ( <> - -

- Your organization’s license level has changed. Your data is safe, but document-level - permissions are no longer supported and searching of this source has been disabled. - Upgrade to a Platinum license to re-enable this source. -

+ +

{SOURCE_DISABLED_CALLOUT_DESCRIPTION}

- Explore Platinum license + {SOURCE_DISABLED_CALLOUT_BUTTON} diff --git a/x-pack/plugins/enterprise_search/public/applications/workplace_search/views/content_sources/sources_view.tsx b/x-pack/plugins/enterprise_search/public/applications/workplace_search/views/content_sources/sources_view.tsx index 9e6c8f5b7319e..f8a2d345c8513 100644 --- a/x-pack/plugins/enterprise_search/public/applications/workplace_search/views/content_sources/sources_view.tsx +++ b/x-pack/plugins/enterprise_search/public/applications/workplace_search/views/content_sources/sources_view.tsx @@ -8,6 +8,9 @@ import React from 'react'; import { useActions, useValues } from 'kea'; +import { i18n } from '@kbn/i18n'; +import { FormattedMessage } from '@kbn/i18n/react'; + import { EuiButton, EuiLink, @@ -27,6 +30,12 @@ import { SourceIcon } from '../../components/shared/source_icon'; import { EXTERNAL_IDENTITIES_DOCS_URL, DOCUMENT_PERMISSIONS_DOCS_URL } from '../../routes'; +import { + EXTERNAL_IDENTITIES_LINK, + DOCUMENT_PERMISSIONS_LINK, + UNDERSTAND_BUTTON, +} from './constants'; + import { SourcesLogic } from './sources_logic'; interface SourcesViewProps { @@ -59,35 +68,53 @@ export const SourcesView: React.FC = ({ children }) => { - {addedSourceName} requires additional configuration + + {i18n.translate( + 'xpack.enterpriseSearch.workplaceSearch.sourcesView.modal.heading', + { + defaultMessage: '{addedSourceName} requires additional configuration', + values: { addedSourceName }, + } + )} +

- {addedSourceName} has been successfully connected and initial content synchronization - is already underway. Since you have elected to synchronize document-level permission - information, you must now provide user and group mappings using the  - - External Identities API - - . + + {EXTERNAL_IDENTITIES_LINK} + + ), + }} + />

- Documents will not be searchable from Workplace Search until user and group mappings - have been configured.  - - Learn more about document-level permission configuration - - . + + {DOCUMENT_PERMISSIONS_LINK} + + ), + }} + />

- I understand + {UNDERSTAND_BUTTON} diff --git a/x-pack/plugins/fleet/server/services/epm/elasticsearch/template/template.ts b/x-pack/plugins/fleet/server/services/epm/elasticsearch/template/template.ts index e1fa2a0b18b59..95f9997645176 100644 --- a/x-pack/plugins/fleet/server/services/epm/elasticsearch/template/template.ts +++ b/x-pack/plugins/fleet/server/services/epm/elasticsearch/template/template.ts @@ -11,7 +11,6 @@ import { TemplateRef, IndexTemplate, IndexTemplateMappings, - DataType, } from '../../../../types'; import { getRegistryDataStreamAssetBaseName } from '../index'; @@ -26,8 +25,8 @@ interface MultiFields { export interface IndexTemplateMapping { [key: string]: any; } -export interface CurrentIndex { - indexName: string; +export interface CurrentDataStream { + dataStreamName: string; indexTemplate: IndexTemplate; } const DEFAULT_SCALING_FACTOR = 1000; @@ -348,33 +347,31 @@ export const updateCurrentWriteIndices = async ( ): Promise => { if (!templates.length) return; - const allIndices = await queryIndicesFromTemplates(callCluster, templates); + const allIndices = await queryDataStreamsFromTemplates(callCluster, templates); if (!allIndices.length) return; - return updateAllIndices(allIndices, callCluster); + return updateAllDataStreams(allIndices, callCluster); }; -function isCurrentIndex(item: CurrentIndex[] | undefined): item is CurrentIndex[] { +function isCurrentDataStream(item: CurrentDataStream[] | undefined): item is CurrentDataStream[] { return item !== undefined; } -const queryIndicesFromTemplates = async ( +const queryDataStreamsFromTemplates = async ( callCluster: CallESAsCurrentUser, templates: TemplateRef[] -): Promise => { - const indexPromises = templates.map((template) => { - return getIndices(callCluster, template); +): Promise => { + const dataStreamPromises = templates.map((template) => { + return getDataStreams(callCluster, template); }); - const indexObjects = await Promise.all(indexPromises); - return indexObjects.filter(isCurrentIndex).flat(); + const dataStreamObjects = await Promise.all(dataStreamPromises); + return dataStreamObjects.filter(isCurrentDataStream).flat(); }; -const getIndices = async ( +const getDataStreams = async ( callCluster: CallESAsCurrentUser, template: TemplateRef -): Promise => { +): Promise => { const { templateName, indexTemplate } = template; - // Until ES provides a way to update mappings of a data stream - // get the last index of the data stream, which is the current write index const res = await callCluster('transport.request', { method: 'GET', path: `/_data_stream/${templateName}-*`, @@ -382,26 +379,28 @@ const getIndices = async ( const dataStreams = res.data_streams; if (!dataStreams.length) return; return dataStreams.map((dataStream: any) => ({ - indexName: dataStream.indices[dataStream.indices.length - 1].index_name, + dataStreamName: dataStream.name, indexTemplate, })); }; -const updateAllIndices = async ( - indexNameWithTemplates: CurrentIndex[], +const updateAllDataStreams = async ( + indexNameWithTemplates: CurrentDataStream[], callCluster: CallESAsCurrentUser ): Promise => { - const updateIndexPromises = indexNameWithTemplates.map(({ indexName, indexTemplate }) => { - return updateExistingIndex({ indexName, callCluster, indexTemplate }); - }); - await Promise.all(updateIndexPromises); + const updatedataStreamPromises = indexNameWithTemplates.map( + ({ dataStreamName, indexTemplate }) => { + return updateExistingDataStream({ dataStreamName, callCluster, indexTemplate }); + } + ); + await Promise.all(updatedataStreamPromises); }; -const updateExistingIndex = async ({ - indexName, +const updateExistingDataStream = async ({ + dataStreamName, callCluster, indexTemplate, }: { - indexName: string; + dataStreamName: string; callCluster: CallESAsCurrentUser; indexTemplate: IndexTemplate; }) => { @@ -416,53 +415,13 @@ const updateExistingIndex = async ({ // try to update the mappings first try { await callCluster('indices.putMapping', { - index: indexName, + index: dataStreamName, body: mappings, + write_index_only: true, }); // if update fails, rollover data stream } catch (err) { try { - // get the data_stream values to compose datastream name - const searchDataStreamFieldsResponse = await callCluster('search', { - index: indexTemplate.index_patterns[0], - body: { - size: 1, - _source: ['data_stream.namespace', 'data_stream.type', 'data_stream.dataset'], - query: { - bool: { - filter: [ - { - exists: { - field: 'data_stream.type', - }, - }, - { - exists: { - field: 'data_stream.dataset', - }, - }, - { - exists: { - field: 'data_stream.namespace', - }, - }, - ], - }, - }, - }, - }); - if (searchDataStreamFieldsResponse.hits.total.value === 0) - throw new Error('data_stream fields are missing from datastream indices'); - const { - dataset, - namespace, - type, - }: { - dataset: string; - namespace: string; - type: DataType; - } = searchDataStreamFieldsResponse.hits.hits[0]._source.data_stream; - const dataStreamName = `${type}-${dataset}-${namespace}`; const path = `/${dataStreamName}/_rollover`; await callCluster('transport.request', { method: 'POST', @@ -478,10 +437,10 @@ const updateExistingIndex = async ({ if (!settings.index.default_pipeline) return; try { await callCluster('indices.putSettings', { - index: indexName, + index: dataStreamName, body: { index: { default_pipeline: settings.index.default_pipeline } }, }); } catch (err) { - throw new Error(`could not update index template settings for ${indexName}`); + throw new Error(`could not update index template settings for ${dataStreamName}`); } }; diff --git a/x-pack/plugins/fleet/server/services/epm/registry/registry_url.ts b/x-pack/plugins/fleet/server/services/epm/registry/registry_url.ts index efc25cc2efb5d..4f17a2b88670a 100644 --- a/x-pack/plugins/fleet/server/services/epm/registry/registry_url.ts +++ b/x-pack/plugins/fleet/server/services/epm/registry/registry_url.ts @@ -11,12 +11,10 @@ import { appContextService, licenseService } from '../../'; const PRODUCTION_REGISTRY_URL_CDN = 'https://epr.elastic.co'; // const STAGING_REGISTRY_URL_CDN = 'https://epr-staging.elastic.co'; -// const EXPERIMENTAL_REGISTRY_URL_CDN = 'https://epr-experimental.elastic.co/'; const SNAPSHOT_REGISTRY_URL_CDN = 'https://epr-snapshot.elastic.co'; // const PRODUCTION_REGISTRY_URL_NO_CDN = 'https://epr.ea-web.elastic.dev'; // const STAGING_REGISTRY_URL_NO_CDN = 'https://epr-staging.ea-web.elastic.dev'; -// const EXPERIMENTAL_REGISTRY_URL_NO_CDN = 'https://epr-experimental.ea-web.elastic.dev/'; // const SNAPSHOT_REGISTRY_URL_NO_CDN = 'https://epr-snapshot.ea-web.elastic.dev'; const getDefaultRegistryUrl = (): string => { diff --git a/x-pack/plugins/grokdebugger/tsconfig.json b/x-pack/plugins/grokdebugger/tsconfig.json new file mode 100644 index 0000000000000..34cf8d74c0024 --- /dev/null +++ b/x-pack/plugins/grokdebugger/tsconfig.json @@ -0,0 +1,24 @@ +{ + "extends": "../../../tsconfig.base.json", + "compilerOptions": { + "composite": true, + "outDir": "./target/types", + "emitDeclarationOnly": true, + "declaration": true, + "declarationMap": true + }, + "include": [ + "common/**/*", + "public/**/*", + "server/**/*", + "../../typings/**/*", + ], + "references": [ + { "path": "../../../src/core/tsconfig.json" }, + { "path": "../../../src/plugins/dev_tools/tsconfig.json"}, + { "path": "../../../src/plugins/home/tsconfig.json"}, + { "path": "../../../src/plugins/kibana_react/tsconfig.json" }, + { "path": "../../../src/plugins/es_ui_shared/tsconfig.json" }, + { "path": "../licensing/tsconfig.json" } + ] +} diff --git a/x-pack/plugins/index_lifecycle_management/__jest__/client_integration/edit_policy/edit_policy.helpers.tsx b/x-pack/plugins/index_lifecycle_management/__jest__/client_integration/edit_policy/edit_policy.helpers.tsx index 64b654b030236..d9256ec916ec8 100644 --- a/x-pack/plugins/index_lifecycle_management/__jest__/client_integration/edit_policy/edit_policy.helpers.tsx +++ b/x-pack/plugins/index_lifecycle_management/__jest__/client_integration/edit_policy/edit_policy.helpers.tsx @@ -251,6 +251,7 @@ export const setup = async (arg?: { appServicesContext: Partial exists('timelineHotPhaseRolloverToolTip'), hasHotPhase: () => exists('ilmTimelineHotPhase'), hasWarmPhase: () => exists('ilmTimelineWarmPhase'), hasColdPhase: () => exists('ilmTimelineColdPhase'), diff --git a/x-pack/plugins/index_lifecycle_management/__jest__/client_integration/edit_policy/edit_policy.test.ts b/x-pack/plugins/index_lifecycle_management/__jest__/client_integration/edit_policy/edit_policy.test.ts index bb96e8b4df239..05793a4bed581 100644 --- a/x-pack/plugins/index_lifecycle_management/__jest__/client_integration/edit_policy/edit_policy.test.ts +++ b/x-pack/plugins/index_lifecycle_management/__jest__/client_integration/edit_policy/edit_policy.test.ts @@ -843,5 +843,13 @@ describe('', () => { expect(actions.timeline.hasColdPhase()).toBe(true); expect(actions.timeline.hasDeletePhase()).toBe(true); }); + + test('show and hide rollover indicator on timeline', async () => { + const { actions } = testBed; + expect(actions.timeline.hasRolloverIndicator()).toBe(true); + await actions.hot.toggleDefaultRollover(false); + await actions.hot.toggleRollover(false); + expect(actions.timeline.hasRolloverIndicator()).toBe(false); + }); }); }); diff --git a/x-pack/plugins/index_lifecycle_management/public/application/sections/edit_policy/components/phases/hot_phase/hot_phase.tsx b/x-pack/plugins/index_lifecycle_management/public/application/sections/edit_policy/components/phases/hot_phase/hot_phase.tsx index fb7c9a80acba0..02de47f8c56ef 100644 --- a/x-pack/plugins/index_lifecycle_management/public/application/sections/edit_policy/components/phases/hot_phase/hot_phase.tsx +++ b/x-pack/plugins/index_lifecycle_management/public/application/sections/edit_policy/components/phases/hot_phase/hot_phase.tsx @@ -16,6 +16,7 @@ import { EuiTextColor, EuiSwitch, EuiIconTip, + EuiIcon, } from '@elastic/eui'; import { useFormData, UseField, SelectField, NumericField } from '../../../../../../shared_imports'; @@ -80,6 +81,10 @@ export const HotPhase: FunctionComponent = () => {

+ +   + {i18nTexts.editPolicy.rolloverOffsetsHotPhaseTiming} + path={isUsingDefaultRolloverPath}> {(field) => ( <> diff --git a/x-pack/plugins/ingest_pipelines/public/application/components/pipeline_editor/components/test_pipeline/test_pipeline_tabs/tab_documents/add_documents_accordion/index.ts b/x-pack/plugins/index_lifecycle_management/public/application/sections/edit_policy/components/timeline/components/index.ts similarity index 78% rename from x-pack/plugins/ingest_pipelines/public/application/components/pipeline_editor/components/test_pipeline/test_pipeline_tabs/tab_documents/add_documents_accordion/index.ts rename to x-pack/plugins/index_lifecycle_management/public/application/sections/edit_policy/components/timeline/components/index.ts index cb00ec640b5a6..1c9d5e1abc316 100644 --- a/x-pack/plugins/ingest_pipelines/public/application/components/pipeline_editor/components/test_pipeline/test_pipeline_tabs/tab_documents/add_documents_accordion/index.ts +++ b/x-pack/plugins/index_lifecycle_management/public/application/sections/edit_policy/components/timeline/components/index.ts @@ -4,4 +4,4 @@ * you may not use this file except in compliance with the Elastic License. */ -export { AddDocumentsAccordion } from './add_documents_accordion'; +export { TimelinePhaseText } from './timeline_phase_text'; diff --git a/x-pack/plugins/index_lifecycle_management/public/application/sections/edit_policy/components/timeline/components/timeline_phase_text.tsx b/x-pack/plugins/index_lifecycle_management/public/application/sections/edit_policy/components/timeline/components/timeline_phase_text.tsx new file mode 100644 index 0000000000000..a44e0f2407c52 --- /dev/null +++ b/x-pack/plugins/index_lifecycle_management/public/application/sections/edit_policy/components/timeline/components/timeline_phase_text.tsx @@ -0,0 +1,28 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License; + * you may not use this file except in compliance with the Elastic License. + */ + +import React, { FunctionComponent, ReactNode } from 'react'; +import { EuiFlexGroup, EuiFlexItem, EuiText } from '@elastic/eui'; + +export const TimelinePhaseText: FunctionComponent<{ + phaseName: ReactNode | string; + durationInPhase?: ReactNode | string; +}> = ({ phaseName, durationInPhase }) => ( + + + + {phaseName} + + + + {typeof durationInPhase === 'string' ? ( + {durationInPhase} + ) : ( + durationInPhase + )} + + +); diff --git a/x-pack/plugins/index_lifecycle_management/public/application/sections/edit_policy/components/timeline/index.ts b/x-pack/plugins/index_lifecycle_management/public/application/sections/edit_policy/components/timeline/index.ts index 4664429db37d7..7bcaa6584edf0 100644 --- a/x-pack/plugins/index_lifecycle_management/public/application/sections/edit_policy/components/timeline/index.ts +++ b/x-pack/plugins/index_lifecycle_management/public/application/sections/edit_policy/components/timeline/index.ts @@ -3,4 +3,4 @@ * or more contributor license agreements. Licensed under the Elastic License; * you may not use this file except in compliance with the Elastic License. */ -export { Timeline } from './timeline'; +export { Timeline } from './timeline.container'; diff --git a/x-pack/plugins/index_lifecycle_management/public/application/sections/edit_policy/components/timeline/timeline.container.tsx b/x-pack/plugins/index_lifecycle_management/public/application/sections/edit_policy/components/timeline/timeline.container.tsx new file mode 100644 index 0000000000000..75f53fcb25091 --- /dev/null +++ b/x-pack/plugins/index_lifecycle_management/public/application/sections/edit_policy/components/timeline/timeline.container.tsx @@ -0,0 +1,33 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License; + * you may not use this file except in compliance with the Elastic License. + */ + +import React, { FunctionComponent } from 'react'; + +import { useFormData } from '../../../../../shared_imports'; + +import { formDataToAbsoluteTimings } from '../../lib'; + +import { useConfigurationIssues } from '../../form'; + +import { FormInternal } from '../../types'; + +import { Timeline as ViewComponent } from './timeline'; + +export const Timeline: FunctionComponent = () => { + const [formData] = useFormData(); + const timings = formDataToAbsoluteTimings(formData); + const { isUsingRollover } = useConfigurationIssues(); + return ( + + ); +}; diff --git a/x-pack/plugins/index_lifecycle_management/public/application/sections/edit_policy/components/timeline/timeline.scss b/x-pack/plugins/index_lifecycle_management/public/application/sections/edit_policy/components/timeline/timeline.scss index 452221a29a991..7d65d2cd6b212 100644 --- a/x-pack/plugins/index_lifecycle_management/public/application/sections/edit_policy/components/timeline/timeline.scss +++ b/x-pack/plugins/index_lifecycle_management/public/application/sections/edit_policy/components/timeline/timeline.scss @@ -84,4 +84,8 @@ $ilmDeletePhaseColor: shadeOrTint($euiColorVis5, 40%, 40%); background-color: $euiColorVis1; } } + + &__rolloverIcon { + display: inline-block; + } } diff --git a/x-pack/plugins/index_lifecycle_management/public/application/sections/edit_policy/components/timeline/timeline.tsx b/x-pack/plugins/index_lifecycle_management/public/application/sections/edit_policy/components/timeline/timeline.tsx index 40bab9c676de2..2e2db88e1384d 100644 --- a/x-pack/plugins/index_lifecycle_management/public/application/sections/edit_policy/components/timeline/timeline.tsx +++ b/x-pack/plugins/index_lifecycle_management/public/application/sections/edit_policy/components/timeline/timeline.tsx @@ -4,9 +4,8 @@ * you may not use this file except in compliance with the Elastic License. */ import { i18n } from '@kbn/i18n'; -import React, { FunctionComponent, useMemo } from 'react'; +import React, { FunctionComponent, memo } from 'react'; import { - EuiText, EuiIcon, EuiIconProps, EuiFlexGroup, @@ -16,18 +15,19 @@ import { } from '@elastic/eui'; import { PhasesExceptDelete } from '../../../../../../common/types'; -import { useFormData } from '../../../../../shared_imports'; - -import { FormInternal } from '../../types'; import { - calculateRelativeTimingMs, + calculateRelativeFromAbsoluteMilliseconds, normalizeTimingsToHumanReadable, PhaseAgeInMilliseconds, + AbsoluteTimings, } from '../../lib'; import './timeline.scss'; import { InfinityIconSvg } from './infinity_icon.svg'; +import { TimelinePhaseText } from './components'; + +const exists = (v: unknown) => v != null; const InfinityIcon: FunctionComponent> = (props) => ( @@ -56,6 +56,13 @@ const i18nTexts = { hotPhase: i18n.translate('xpack.indexLifecycleMgmt.timeline.hotPhaseSectionTitle', { defaultMessage: 'Hot phase', }), + rolloverTooltip: i18n.translate( + 'xpack.indexLifecycleMgmt.timeline.hotPhaseRolloverToolTipContent', + { + defaultMessage: + 'How long it takes to reach the rollover criteria in the hot phase can vary. Data moves to the next phase when the time since rollover reaches the minimum age.', + } + ), warmPhase: i18n.translate('xpack.indexLifecycleMgmt.timeline.warmPhaseSectionTitle', { defaultMessage: 'Warm phase', }), @@ -88,121 +95,136 @@ const calculateWidths = (inputs: PhaseAgeInMilliseconds) => { }; }; -const TimelinePhaseText: FunctionComponent<{ - phaseName: string; - durationInPhase?: React.ReactNode | string; -}> = ({ phaseName, durationInPhase }) => ( - - - - {phaseName} - - - - {typeof durationInPhase === 'string' ? ( - {durationInPhase} - ) : ( - durationInPhase - )} - - -); - -export const Timeline: FunctionComponent = () => { - const [formData] = useFormData(); - - const phaseTimingInMs = useMemo(() => { - return calculateRelativeTimingMs(formData); - }, [formData]); +interface Props { + hasDeletePhase: boolean; + /** + * For now we assume the hot phase does not have a min age + */ + hotPhaseMinAge: undefined; + isUsingRollover: boolean; + warmPhaseMinAge?: string; + coldPhaseMinAge?: string; + deletePhaseMinAge?: string; +} - const humanReadableTimings = useMemo(() => normalizeTimingsToHumanReadable(phaseTimingInMs), [ - phaseTimingInMs, - ]); - - const widths = calculateWidths(phaseTimingInMs); - - const getDurationInPhaseContent = (phase: PhasesExceptDelete): string | React.ReactNode => - phaseTimingInMs.phases[phase] === Infinity ? ( - - ) : ( - humanReadableTimings[phase] - ); - - return ( - - - -

- {i18n.translate('xpack.indexLifecycleMgmt.timeline.title', { - defaultMessage: 'Policy Timeline', - })} -

- - - -
{ - if (el) { - el.style.setProperty('--ilm-timeline-hot-phase-width', widths.hot); - el.style.setProperty('--ilm-timeline-warm-phase-width', widths.warm ?? null); - el.style.setProperty('--ilm-timeline-cold-phase-width', widths.cold ?? null); - } - }} - > - - -
- {/* These are the actual color bars for the timeline */} -
-
- -
- {formData._meta?.warm.enabled && ( +/** + * Display a timeline given ILM policy phase information. This component is re-usable and memo-ized + * and should not rely directly on any application-specific context. + */ +export const Timeline: FunctionComponent = memo( + ({ hasDeletePhase, isUsingRollover, ...phasesMinAge }) => { + const absoluteTimings: AbsoluteTimings = { + hot: { min_age: phasesMinAge.hotPhaseMinAge }, + warm: phasesMinAge.warmPhaseMinAge ? { min_age: phasesMinAge.warmPhaseMinAge } : undefined, + cold: phasesMinAge.coldPhaseMinAge ? { min_age: phasesMinAge.coldPhaseMinAge } : undefined, + delete: phasesMinAge.deletePhaseMinAge + ? { min_age: phasesMinAge.deletePhaseMinAge } + : undefined, + }; + + const phaseAgeInMilliseconds = calculateRelativeFromAbsoluteMilliseconds(absoluteTimings); + const humanReadableTimings = normalizeTimingsToHumanReadable(phaseAgeInMilliseconds); + + const widths = calculateWidths(phaseAgeInMilliseconds); + + const getDurationInPhaseContent = (phase: PhasesExceptDelete): string | React.ReactNode => + phaseAgeInMilliseconds.phases[phase] === Infinity ? ( + + ) : ( + humanReadableTimings[phase] + ); + + return ( + + + +

+ {i18n.translate('xpack.indexLifecycleMgmt.timeline.title', { + defaultMessage: 'Policy Timeline', + })} +

+ + + +
{ + if (el) { + el.style.setProperty('--ilm-timeline-hot-phase-width', widths.hot); + el.style.setProperty('--ilm-timeline-warm-phase-width', widths.warm ?? null); + el.style.setProperty('--ilm-timeline-cold-phase-width', widths.cold ?? null); + } + }} + > + + +
+ {/* These are the actual color bars for the timeline */}
-
+
+ {i18nTexts.hotPhase} +   +
+ +
+ + ) : ( + i18nTexts.hotPhase + ) + } + durationInPhase={getDurationInPhaseContent('hot')} />
- )} - {formData._meta?.cold.enabled && ( + {exists(phaseAgeInMilliseconds.phases.warm) && ( +
+
+ +
+ )} + {exists(phaseAgeInMilliseconds.phases.cold) && ( +
+
+ +
+ )} +
+ + {hasDeletePhase && ( +
-
- +
- )} -
- - {formData._meta?.delete.enabled && ( - -
- -
- - )} - -
- - - ); -}; + + )} + +
+ + + ); + } +); diff --git a/x-pack/plugins/index_lifecycle_management/public/application/sections/edit_policy/i18n_texts.ts b/x-pack/plugins/index_lifecycle_management/public/application/sections/edit_policy/i18n_texts.ts index 71085a6d7a2b8..cf8c92b8333d0 100644 --- a/x-pack/plugins/index_lifecycle_management/public/application/sections/edit_policy/i18n_texts.ts +++ b/x-pack/plugins/index_lifecycle_management/public/application/sections/edit_policy/i18n_texts.ts @@ -11,6 +11,13 @@ export const i18nTexts = { shrinkLabel: i18n.translate('xpack.indexLifecycleMgmt.shrink.indexFieldLabel', { defaultMessage: 'Shrink index', }), + rolloverOffsetsHotPhaseTiming: i18n.translate( + 'xpack.indexLifecycleMgmt.rollover.rolloverOffsetsPhaseTimingDescription', + { + defaultMessage: + 'How long it takes to reach the rollover criteria in the hot phase can vary. Data moves to the next phase when the time since rollover reaches the minimum age.', + } + ), searchableSnapshotInHotPhase: { searchableSnapshotDisallowed: { calloutTitle: i18n.translate( diff --git a/x-pack/plugins/index_lifecycle_management/public/application/sections/edit_policy/lib/absolute_timing_to_relative_timing.test.ts b/x-pack/plugins/index_lifecycle_management/public/application/sections/edit_policy/lib/absolute_timing_to_relative_timing.test.ts index 28910871fa33b..405de2b55a2f7 100644 --- a/x-pack/plugins/index_lifecycle_management/public/application/sections/edit_policy/lib/absolute_timing_to_relative_timing.test.ts +++ b/x-pack/plugins/index_lifecycle_management/public/application/sections/edit_policy/lib/absolute_timing_to_relative_timing.test.ts @@ -4,13 +4,20 @@ * you may not use this file except in compliance with the Elastic License. */ +import { flow } from 'fp-ts/function'; import { deserializer } from '../form'; import { + formDataToAbsoluteTimings, + calculateRelativeFromAbsoluteMilliseconds, absoluteTimingToRelativeTiming, - calculateRelativeTimingMs, } from './absolute_timing_to_relative_timing'; +export const calculateRelativeTimingMs = flow( + formDataToAbsoluteTimings, + calculateRelativeFromAbsoluteMilliseconds +); + describe('Conversion of absolute policy timing to relative timing', () => { describe('calculateRelativeTimingMs', () => { describe('policy that never deletes data (keep forever)', () => { diff --git a/x-pack/plugins/index_lifecycle_management/public/application/sections/edit_policy/lib/absolute_timing_to_relative_timing.ts b/x-pack/plugins/index_lifecycle_management/public/application/sections/edit_policy/lib/absolute_timing_to_relative_timing.ts index 2f37608b2d7ae..a44863b2f1ce2 100644 --- a/x-pack/plugins/index_lifecycle_management/public/application/sections/edit_policy/lib/absolute_timing_to_relative_timing.ts +++ b/x-pack/plugins/index_lifecycle_management/public/application/sections/edit_policy/lib/absolute_timing_to_relative_timing.ts @@ -14,16 +14,21 @@ * * This code converts the absolute timings to _relative_ timings of the form: 30 days in hot phase, * 40 days in warm phase then forever in cold phase. + * + * All functions exported from this file can be viewed as utilities for working with form data and + * other defined interfaces to calculate the relative amount of time data will spend in a phase. */ import moment from 'moment'; -import { flow } from 'fp-ts/lib/function'; import { i18n } from '@kbn/i18n'; +import { flow } from 'fp-ts/function'; import { splitSizeAndUnits } from '../../../lib/policies'; import { FormInternal } from '../types'; +/* -===- Private functions and types -===- */ + type MinAgePhase = 'warm' | 'cold' | 'delete'; type Phase = 'hot' | MinAgePhase; @@ -43,7 +48,34 @@ const i18nTexts = { }), }; -interface AbsoluteTimings { +const phaseOrder: Phase[] = ['hot', 'warm', 'cold', 'delete']; + +const getMinAge = (phase: MinAgePhase, formData: FormInternal) => ({ + min_age: formData.phases?.[phase]?.min_age + ? formData.phases[phase]!.min_age! + formData._meta[phase].minAgeUnit + : '0ms', +}); + +/** + * See https://www.elastic.co/guide/en/elasticsearch/reference/current/common-options.html#date-math + * for all date math values. ILM policies also support "micros" and "nanos". + */ +const getPhaseMinAgeInMilliseconds = (phase: { min_age: string }): number => { + let milliseconds: number; + const { units, size } = splitSizeAndUnits(phase.min_age); + if (units === 'micros') { + milliseconds = parseInt(size, 10) / 1e3; + } else if (units === 'nanos') { + milliseconds = parseInt(size, 10) / 1e6; + } else { + milliseconds = moment.duration(size, units as any).asMilliseconds(); + } + return milliseconds; +}; + +/* -===- Public functions and types -===- */ + +export interface AbsoluteTimings { hot: { min_age: undefined; }; @@ -67,16 +99,7 @@ export interface PhaseAgeInMilliseconds { }; } -const phaseOrder: Phase[] = ['hot', 'warm', 'cold', 'delete']; - -const getMinAge = (phase: MinAgePhase, formData: FormInternal) => ({ - min_age: - formData.phases && formData.phases[phase]?.min_age - ? formData.phases[phase]!.min_age! + formData._meta[phase].minAgeUnit - : '0ms', -}); - -const formDataToAbsoluteTimings = (formData: FormInternal): AbsoluteTimings => { +export const formDataToAbsoluteTimings = (formData: FormInternal): AbsoluteTimings => { const { _meta } = formData; if (!_meta) { return { hot: { min_age: undefined } }; @@ -89,28 +112,13 @@ const formDataToAbsoluteTimings = (formData: FormInternal): AbsoluteTimings => { }; }; -/** - * See https://www.elastic.co/guide/en/elasticsearch/reference/current/common-options.html#date-math - * for all date math values. ILM policies also support "micros" and "nanos". - */ -const getPhaseMinAgeInMilliseconds = (phase: { min_age: string }): number => { - let milliseconds: number; - const { units, size } = splitSizeAndUnits(phase.min_age); - if (units === 'micros') { - milliseconds = parseInt(size, 10) / 1e3; - } else if (units === 'nanos') { - milliseconds = parseInt(size, 10) / 1e6; - } else { - milliseconds = moment.duration(size, units as any).asMilliseconds(); - } - return milliseconds; -}; - /** * Given a set of phase minimum age absolute timings, like hot phase 0ms and warm phase 3d, work out * the number of milliseconds data will reside in phase. */ -const calculateMilliseconds = (inputs: AbsoluteTimings): PhaseAgeInMilliseconds => { +export const calculateRelativeFromAbsoluteMilliseconds = ( + inputs: AbsoluteTimings +): PhaseAgeInMilliseconds => { return phaseOrder.reduce( (acc, phaseName, idx) => { // Delete does not have an age associated with it @@ -152,6 +160,8 @@ const calculateMilliseconds = (inputs: AbsoluteTimings): PhaseAgeInMilliseconds ); }; +export type RelativePhaseTimingInMs = ReturnType; + const millisecondsToDays = (milliseconds?: number): string | undefined => { if (milliseconds == null) { return; @@ -177,10 +187,12 @@ export const normalizeTimingsToHumanReadable = ({ }; }; -export const calculateRelativeTimingMs = flow(formDataToAbsoluteTimings, calculateMilliseconds); - +/** + * Given {@link FormInternal}, extract the min_age values for each phase and calculate + * human readable strings for communicating how long data will remain in a phase. + */ export const absoluteTimingToRelativeTiming = flow( formDataToAbsoluteTimings, - calculateMilliseconds, + calculateRelativeFromAbsoluteMilliseconds, normalizeTimingsToHumanReadable ); diff --git a/x-pack/plugins/index_lifecycle_management/public/application/sections/edit_policy/lib/index.ts b/x-pack/plugins/index_lifecycle_management/public/application/sections/edit_policy/lib/index.ts index 9593fcc810a6f..a9372c99a72fc 100644 --- a/x-pack/plugins/index_lifecycle_management/public/application/sections/edit_policy/lib/index.ts +++ b/x-pack/plugins/index_lifecycle_management/public/application/sections/edit_policy/lib/index.ts @@ -6,7 +6,10 @@ export { absoluteTimingToRelativeTiming, - calculateRelativeTimingMs, + calculateRelativeFromAbsoluteMilliseconds, normalizeTimingsToHumanReadable, + formDataToAbsoluteTimings, + AbsoluteTimings, PhaseAgeInMilliseconds, + RelativePhaseTimingInMs, } from './absolute_timing_to_relative_timing'; diff --git a/x-pack/plugins/index_management/__jest__/client_integration/home/data_streams_tab.test.ts b/x-pack/plugins/index_management/__jest__/client_integration/home/data_streams_tab.test.ts index 87f5408f6ca42..3221054839865 100644 --- a/x-pack/plugins/index_management/__jest__/client_integration/home/data_streams_tab.test.ts +++ b/x-pack/plugins/index_management/__jest__/client_integration/home/data_streams_tab.test.ts @@ -71,7 +71,7 @@ describe('Data Streams tab', () => { test('when Fleet is enabled, links to Fleet', async () => { testBed = await setup({ - plugins: { fleet: { hi: 'ok' } }, + plugins: { isFleetEnabled: true }, }); await act(async () => { diff --git a/x-pack/plugins/index_management/public/application/app_context.tsx b/x-pack/plugins/index_management/public/application/app_context.tsx index 91bcfe5ed55c0..4e5164562207d 100644 --- a/x-pack/plugins/index_management/public/application/app_context.tsx +++ b/x-pack/plugins/index_management/public/application/app_context.tsx @@ -8,9 +8,8 @@ import React, { createContext, useContext } from 'react'; import { ScopedHistory } from 'kibana/public'; import { ManagementAppMountParams } from 'src/plugins/management/public'; import { UsageCollectionSetup } from 'src/plugins/usage_collection/public'; -import { CoreSetup, CoreStart } from '../../../../../src/core/public'; -import { FleetSetup } from '../../../fleet/public'; +import { CoreSetup, CoreStart } from '../../../../../src/core/public'; import { UiMetricService, NotificationService, HttpService } from './services'; import { ExtensionsService } from '../services'; import { SharePluginStart } from '../../../../../src/plugins/share/public'; @@ -24,7 +23,7 @@ export interface AppDependencies { }; plugins: { usageCollection: UsageCollectionSetup; - fleet?: FleetSetup; + isFleetEnabled: boolean; }; services: { uiMetricService: UiMetricService; diff --git a/x-pack/plugins/index_management/public/application/mount_management_section.ts b/x-pack/plugins/index_management/public/application/mount_management_section.ts index b94718c14d3aa..f4136a977df1a 100644 --- a/x-pack/plugins/index_management/public/application/mount_management_section.ts +++ b/x-pack/plugins/index_management/public/application/mount_management_section.ts @@ -9,7 +9,6 @@ import { CoreSetup } from 'src/core/public'; import { ManagementAppMountParams } from 'src/plugins/management/public/'; import { UsageCollectionSetup } from 'src/plugins/usage_collection/public'; -import { FleetSetup } from '../../../fleet/public'; import { UIM_APP_NAME } from '../../common/constants'; import { PLUGIN } from '../../common/constants/plugin'; import { ExtensionsService } from '../services'; @@ -50,7 +49,7 @@ export async function mountManagementSection( usageCollection: UsageCollectionSetup, params: ManagementAppMountParams, extensionsService: ExtensionsService, - fleet?: FleetSetup + isFleetEnabled: boolean ) { const { element, setBreadcrumbs, history } = params; const [core, startDependencies] = await coreSetup.getStartServices(); @@ -80,7 +79,7 @@ export async function mountManagementSection( }, plugins: { usageCollection, - fleet, + isFleetEnabled, }, services: { httpService, notificationService, uiMetricService, extensionsService }, history, diff --git a/x-pack/plugins/index_management/public/application/sections/home/data_stream_list/data_stream_list.tsx b/x-pack/plugins/index_management/public/application/sections/home/data_stream_list/data_stream_list.tsx index 64d874c76afb3..07eccd23d9f44 100644 --- a/x-pack/plugins/index_management/public/application/sections/home/data_stream_list/data_stream_list.tsx +++ b/x-pack/plugins/index_management/public/application/sections/home/data_stream_list/data_stream_list.tsx @@ -52,7 +52,7 @@ export const DataStreamList: React.FunctionComponent {' ' /* We need this space to separate these two sentences. */} - {fleet ? ( + {isFleetEnabled ? ( import('./')); +const LazyLogStream = React.lazy(() => import('./log_stream')); export const LazyLogStreamWrapper: React.FC = (props) => ( }> diff --git a/x-pack/plugins/infra/public/components/log_stream/index.tsx b/x-pack/plugins/infra/public/components/log_stream/log_stream.tsx similarity index 98% rename from x-pack/plugins/infra/public/components/log_stream/index.tsx rename to x-pack/plugins/infra/public/components/log_stream/log_stream.tsx index b485a21221af2..b7410fda6f6fd 100644 --- a/x-pack/plugins/infra/public/components/log_stream/index.tsx +++ b/x-pack/plugins/infra/public/components/log_stream/log_stream.tsx @@ -17,6 +17,7 @@ import { useLogStream } from '../../containers/logs/log_stream'; import { ScrollableLogTextStreamView } from '../logging/log_text_stream'; import { LogColumnRenderConfiguration } from '../../utils/log_column_render_configuration'; import { JsonValue } from '../../../../../../src/plugins/kibana_utils/common'; +import { Query } from '../../../../../../src/plugins/data/common'; const PAGE_THRESHOLD = 2; @@ -55,7 +56,7 @@ export interface LogStreamProps { sourceId?: string; startTimestamp: number; endTimestamp: number; - query?: string; + query?: string | Query; center?: LogEntryCursor; highlight?: string; height?: string | number; diff --git a/x-pack/plugins/infra/public/components/log_stream/log_stream_embeddable.tsx b/x-pack/plugins/infra/public/components/log_stream/log_stream_embeddable.tsx new file mode 100644 index 0000000000000..0d6dfc50960f9 --- /dev/null +++ b/x-pack/plugins/infra/public/components/log_stream/log_stream_embeddable.tsx @@ -0,0 +1,91 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License; + * you may not use this file except in compliance with the Elastic License. + */ +import React from 'react'; +import ReactDOM from 'react-dom'; +import { CoreStart } from 'kibana/public'; + +import { I18nProvider } from '@kbn/i18n/react'; +import { KibanaContextProvider } from '../../../../../../src/plugins/kibana_react/public'; +import { EuiThemeProvider } from '../../../../../../src/plugins/kibana_react/common'; +import { Query, TimeRange } from '../../../../../../src/plugins/data/public'; +import { + Embeddable, + EmbeddableInput, + IContainer, +} from '../../../../../../src/plugins/embeddable/public'; +import { datemathToEpochMillis } from '../../utils/datemath'; +import { LazyLogStreamWrapper } from './lazy_log_stream_wrapper'; + +export const LOG_STREAM_EMBEDDABLE = 'LOG_STREAM_EMBEDDABLE'; + +export interface LogStreamEmbeddableInput extends EmbeddableInput { + timeRange: TimeRange; + query: Query; +} + +export class LogStreamEmbeddable extends Embeddable { + public readonly type = LOG_STREAM_EMBEDDABLE; + private node?: HTMLElement; + + constructor( + private services: CoreStart, + initialInput: LogStreamEmbeddableInput, + parent?: IContainer + ) { + super(initialInput, {}, parent); + } + + public render(node: HTMLElement) { + if (this.node) { + ReactDOM.unmountComponentAtNode(this.node); + } + this.node = node; + + this.renderComponent(); + } + + public reload() { + this.renderComponent(); + } + + public destroy() { + super.destroy(); + if (this.node) { + ReactDOM.unmountComponentAtNode(this.node); + } + } + + private renderComponent() { + if (!this.node) { + return; + } + + const startTimestamp = datemathToEpochMillis(this.input.timeRange.from); + const endTimestamp = datemathToEpochMillis(this.input.timeRange.to); + + if (!startTimestamp || !endTimestamp) { + return; + } + + ReactDOM.render( + + + +
+ +
+ + + , + this.node + ); + } +} diff --git a/x-pack/plugins/infra/public/components/log_stream/log_stream_embeddable_factory.ts b/x-pack/plugins/infra/public/components/log_stream/log_stream_embeddable_factory.ts new file mode 100644 index 0000000000000..f4d1b83a07593 --- /dev/null +++ b/x-pack/plugins/infra/public/components/log_stream/log_stream_embeddable_factory.ts @@ -0,0 +1,37 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License; + * you may not use this file except in compliance with the Elastic License. + */ + +import { CoreStart } from 'kibana/public'; +import { + EmbeddableFactoryDefinition, + IContainer, +} from '../../../../../../src/plugins/embeddable/public'; +import { + LogStreamEmbeddable, + LOG_STREAM_EMBEDDABLE, + LogStreamEmbeddableInput, +} from './log_stream_embeddable'; + +export class LogStreamEmbeddableFactoryDefinition + implements EmbeddableFactoryDefinition { + public readonly type = LOG_STREAM_EMBEDDABLE; + + constructor(private getCoreServices: () => Promise) {} + + public async isEditable() { + const { application } = await this.getCoreServices(); + return application.capabilities.logs.save as boolean; + } + + public async create(initialInput: LogStreamEmbeddableInput, parent?: IContainer) { + const services = await this.getCoreServices(); + return new LogStreamEmbeddable(services, initialInput, parent); + } + + public getDisplayName() { + return 'Log stream'; + } +} diff --git a/x-pack/plugins/infra/public/components/logging/log_entry_flyout/log_entry_actions_menu.tsx b/x-pack/plugins/infra/public/components/logging/log_entry_flyout/log_entry_actions_menu.tsx index aa3b4532e878e..6de2fa2029e4b 100644 --- a/x-pack/plugins/infra/public/components/logging/log_entry_flyout/log_entry_actions_menu.tsx +++ b/x-pack/plugins/infra/public/components/logging/log_entry_flyout/log_entry_actions_menu.tsx @@ -4,11 +4,11 @@ * you may not use this file except in compliance with the Elastic License. */ -import { EuiButtonEmpty, EuiContextMenuItem, EuiContextMenuPanel, EuiPopover } from '@elastic/eui'; +import { EuiButton, EuiContextMenuItem, EuiContextMenuPanel, EuiPopover } from '@elastic/eui'; import { FormattedMessage } from '@kbn/i18n/react'; import React, { useMemo } from 'react'; import { useVisibilityState } from '../../../utils/use_visibility_state'; -import { getTraceUrl } from '../../../../../apm/public'; +import { getApmTraceUrl } from '../../../../../observability/public'; import { useLinkProps, LinkDescriptor } from '../../../hooks/use_link_props'; import { LogEntry } from '../../../../common/search_strategies/log_entries/log_entry'; @@ -67,7 +67,7 @@ export const LogEntryActionsMenu: React.FunctionComponent<{ - + } closePopover={hide} id="logEntryActionsMenu" @@ -136,6 +136,6 @@ const getAPMLink = (logEntry: LogEntry): LinkDescriptor | undefined => { return { app: 'apm', - hash: getTraceUrl({ traceId, rangeFrom, rangeTo }), + pathname: getApmTraceUrl({ traceId, rangeFrom, rangeTo }), }; }; diff --git a/x-pack/plugins/infra/public/components/logging/log_entry_flyout/log_entry_flyout.tsx b/x-pack/plugins/infra/public/components/logging/log_entry_flyout/log_entry_flyout.tsx index 5684d4068f3be..7d8ca95f9b93b 100644 --- a/x-pack/plugins/infra/public/components/logging/log_entry_flyout/log_entry_flyout.tsx +++ b/x-pack/plugins/infra/public/components/logging/log_entry_flyout/log_entry_flyout.tsx @@ -88,7 +88,7 @@ export const LogEntryFlyout = ({ ) : null} - + {logEntry ? : null} diff --git a/x-pack/plugins/infra/public/containers/logs/log_stream/index.ts b/x-pack/plugins/infra/public/containers/logs/log_stream/index.ts index da7176125dae4..1d9a7a1b1d777 100644 --- a/x-pack/plugins/infra/public/containers/logs/log_stream/index.ts +++ b/x-pack/plugins/infra/public/containers/logs/log_stream/index.ts @@ -7,7 +7,7 @@ import { useMemo, useEffect } from 'react'; import useSetState from 'react-use/lib/useSetState'; import usePrevious from 'react-use/lib/usePrevious'; -import { esKuery } from '../../../../../../../src/plugins/data/public'; +import { esKuery, esQuery, Query } from '../../../../../../../src/plugins/data/public'; import { fetchLogEntries } from '../log_entries/api/fetch_log_entries'; import { useTrackedPromise } from '../../../utils/use_tracked_promise'; import { LogEntryCursor, LogEntry } from '../../../../common/log_entry'; @@ -18,7 +18,7 @@ interface LogStreamProps { sourceId: string; startTimestamp: number; endTimestamp: number; - query?: string; + query?: string | Query; center?: LogEntryCursor; columns?: LogSourceConfigurationProperties['logColumns']; } @@ -84,9 +84,21 @@ export function useLogStream({ }, [prevEndTimestamp, endTimestamp, setState]); const parsedQuery = useMemo(() => { - return query - ? JSON.stringify(esKuery.toElasticsearchQuery(esKuery.fromKueryExpression(query))) - : null; + if (!query) { + return null; + } + + let q; + + if (typeof query === 'string') { + q = esKuery.toElasticsearchQuery(esKuery.fromKueryExpression(query)); + } else if (query.language === 'kuery') { + q = esKuery.toElasticsearchQuery(esKuery.fromKueryExpression(query.query as string)); + } else if (query.language === 'lucene') { + q = esQuery.luceneStringToDsl(query.query as string); + } + + return JSON.stringify(q); }, [query]); // Callbacks diff --git a/x-pack/plugins/infra/public/pages/metrics/inventory_view/components/waffle/stepped_gradient_legend.tsx b/x-pack/plugins/infra/public/pages/metrics/inventory_view/components/waffle/stepped_gradient_legend.tsx index ed34a32012bd2..71cfab79ba0cc 100644 --- a/x-pack/plugins/infra/public/pages/metrics/inventory_view/components/waffle/stepped_gradient_legend.tsx +++ b/x-pack/plugins/infra/public/pages/metrics/inventory_view/components/waffle/stepped_gradient_legend.tsx @@ -16,13 +16,12 @@ interface Props { bounds: InfraWaffleMapBounds; formatter: InfraFormatter; } - +type TickValue = 0 | 1; export const SteppedGradientLegend: React.FC = ({ legend, bounds, formatter }) => { return ( - @@ -39,7 +38,7 @@ export const SteppedGradientLegend: React.FC = ({ legend, bounds, formatt interface TickProps { bounds: InfraWaffleMapBounds; - value: number; + value: TickValue; formatter: InfraFormatter; } diff --git a/x-pack/plugins/infra/public/pages/metrics/inventory_view/lib/calculate_bounds_from_nodes.test.ts b/x-pack/plugins/infra/public/pages/metrics/inventory_view/lib/calculate_bounds_from_nodes.test.ts index 49f4b56532936..9f1c2f90635a3 100644 --- a/x-pack/plugins/infra/public/pages/metrics/inventory_view/lib/calculate_bounds_from_nodes.test.ts +++ b/x-pack/plugins/infra/public/pages/metrics/inventory_view/lib/calculate_bounds_from_nodes.test.ts @@ -37,14 +37,14 @@ describe('calculateBoundsFromNodes', () => { const bounds = calculateBoundsFromNodes(nodes); expect(bounds).toEqual({ min: 0.2, - max: 1.5, + max: 0.5, }); }); it('should have a minimum of 0 for only a single node', () => { const bounds = calculateBoundsFromNodes([nodes[0]]); expect(bounds).toEqual({ min: 0, - max: 1.5, + max: 0.5, }); }); it('should return zero for empty nodes', () => { diff --git a/x-pack/plugins/infra/public/pages/metrics/inventory_view/lib/calculate_bounds_from_nodes.ts b/x-pack/plugins/infra/public/pages/metrics/inventory_view/lib/calculate_bounds_from_nodes.ts index 6eb64971efbd7..ff1093a795a10 100644 --- a/x-pack/plugins/infra/public/pages/metrics/inventory_view/lib/calculate_bounds_from_nodes.ts +++ b/x-pack/plugins/infra/public/pages/metrics/inventory_view/lib/calculate_bounds_from_nodes.ts @@ -9,23 +9,17 @@ import { SnapshotNode } from '../../../../../common/http_api/snapshot_api'; import { InfraWaffleMapBounds } from '../../../../lib/lib'; export const calculateBoundsFromNodes = (nodes: SnapshotNode[]): InfraWaffleMapBounds => { - const maxValues = nodes.map((node) => { + const values = nodes.map((node) => { const metric = first(node.metrics); - if (!metric) return 0; - return metric.max; - }); - const minValues = nodes.map((node) => { - const metric = first(node.metrics); - if (!metric) return 0; - return metric.value; + return !metric || !metric.value ? 0 : metric.value; }); // if there is only one value then we need to set the bottom range to zero for min // otherwise the legend will look silly since both values are the same for top and // bottom. - if (minValues.length === 1) { - minValues.unshift(0); + if (values.length === 1) { + values.unshift(0); } - const maxValue = max(maxValues) || 0; - const minValue = min(minValues) || 0; + const maxValue = max(values) || 0; + const minValue = min(values) || 0; return { min: isFinite(minValue) ? minValue : 0, max: isFinite(maxValue) ? maxValue : 0 }; }; diff --git a/x-pack/plugins/infra/public/plugin.ts b/x-pack/plugins/infra/public/plugin.ts index 2bbd0067642c0..809046ee1e17b 100644 --- a/x-pack/plugins/infra/public/plugin.ts +++ b/x-pack/plugins/infra/public/plugin.ts @@ -19,6 +19,8 @@ import { } from './types'; import { getLogsHasDataFetcher, getLogsOverviewDataFetcher } from './utils/logs_overview_fetchers'; import { createMetricsHasData, createMetricsFetchData } from './metrics_overview_fetchers'; +import { LOG_STREAM_EMBEDDABLE } from './components/log_stream/log_stream_embeddable'; +import { LogStreamEmbeddableFactoryDefinition } from './components/log_stream/log_stream_embeddable_factory'; export class Plugin implements InfraClientPluginClass { constructor(_context: PluginInitializerContext) {} @@ -46,6 +48,13 @@ export class Plugin implements InfraClientPluginClass { }); } + const getCoreServices = async () => (await core.getStartServices())[0]; + + pluginsSetup.embeddable.registerEmbeddableFactory( + LOG_STREAM_EMBEDDABLE, + new LogStreamEmbeddableFactoryDefinition(getCoreServices) + ); + core.application.register({ id: 'logs', title: i18n.translate('xpack.infra.logs.pluginTitle', { diff --git a/x-pack/plugins/infra/public/types.ts b/x-pack/plugins/infra/public/types.ts index f1052672978d5..037cfa4b7eb2d 100644 --- a/x-pack/plugins/infra/public/types.ts +++ b/x-pack/plugins/infra/public/types.ts @@ -7,6 +7,7 @@ import type { CoreSetup, CoreStart, Plugin as PluginClass } from 'kibana/public'; import type { DataPublicPluginStart } from '../../../../src/plugins/data/public'; import type { HomePublicPluginSetup } from '../../../../src/plugins/home/public'; +import type { EmbeddableSetup } from '../../../../src/plugins/embeddable/public'; import type { UsageCollectionSetup, UsageCollectionStart, @@ -33,6 +34,7 @@ export interface InfraClientSetupDeps { observability: ObservabilityPluginSetup; triggersActionsUi: TriggersAndActionsUIPublicPluginSetup; usageCollection: UsageCollectionSetup; + embeddable: EmbeddableSetup; } export interface InfraClientStartDeps { diff --git a/x-pack/plugins/infra/server/lib/adapters/framework/adapter_types.ts b/x-pack/plugins/infra/server/lib/adapters/framework/adapter_types.ts index 705d7bf34c1c6..a682500e5af18 100644 --- a/x-pack/plugins/infra/server/lib/adapters/framework/adapter_types.ts +++ b/x-pack/plugins/infra/server/lib/adapters/framework/adapter_types.ts @@ -14,7 +14,6 @@ import { } from '../../../../../../../src/plugins/data/server'; import { HomeServerPluginSetup } from '../../../../../../../src/plugins/home/server'; import { VisTypeTimeseriesSetup } from '../../../../../../../src/plugins/vis_type_timeseries/server'; -import { APMPluginSetup } from '../../../../../../plugins/apm/server'; import { PluginSetupContract as FeaturesPluginSetup } from '../../../../../../plugins/features/server'; import { SpacesPluginSetup } from '../../../../../../plugins/spaces/server'; import { PluginSetupContract as AlertingPluginContract } from '../../../../../alerts/server'; @@ -28,7 +27,6 @@ export interface InfraServerPluginSetupDeps { usageCollection: UsageCollectionSetup; visTypeTimeseries: VisTypeTimeseriesSetup; features: FeaturesPluginSetup; - apm: APMPluginSetup; alerts: AlertingPluginContract; ml?: MlPluginSetup; } diff --git a/x-pack/plugins/ingest_pipelines/public/application/components/pipeline_editor/components/test_pipeline/test_pipeline_tabs/tab_documents/add_documents_accordion/add_documents_accordion.scss b/x-pack/plugins/ingest_pipelines/public/application/components/pipeline_editor/components/test_pipeline/test_pipeline_tabs/tab_documents/add_docs_accordion/add_docs_accordion.scss similarity index 100% rename from x-pack/plugins/ingest_pipelines/public/application/components/pipeline_editor/components/test_pipeline/test_pipeline_tabs/tab_documents/add_documents_accordion/add_documents_accordion.scss rename to x-pack/plugins/ingest_pipelines/public/application/components/pipeline_editor/components/test_pipeline/test_pipeline_tabs/tab_documents/add_docs_accordion/add_docs_accordion.scss diff --git a/x-pack/plugins/ingest_pipelines/public/application/components/pipeline_editor/components/test_pipeline/test_pipeline_tabs/tab_documents/add_documents_accordion/add_documents_accordion.tsx b/x-pack/plugins/ingest_pipelines/public/application/components/pipeline_editor/components/test_pipeline/test_pipeline_tabs/tab_documents/add_docs_accordion/add_docs_accordion.tsx similarity index 98% rename from x-pack/plugins/ingest_pipelines/public/application/components/pipeline_editor/components/test_pipeline/test_pipeline_tabs/tab_documents/add_documents_accordion/add_documents_accordion.tsx rename to x-pack/plugins/ingest_pipelines/public/application/components/pipeline_editor/components/test_pipeline/test_pipeline_tabs/tab_documents/add_docs_accordion/add_docs_accordion.tsx index 9519d849e5d90..cbbd032f25b3d 100644 --- a/x-pack/plugins/ingest_pipelines/public/application/components/pipeline_editor/components/test_pipeline/test_pipeline_tabs/tab_documents/add_documents_accordion/add_documents_accordion.tsx +++ b/x-pack/plugins/ingest_pipelines/public/application/components/pipeline_editor/components/test_pipeline/test_pipeline_tabs/tab_documents/add_docs_accordion/add_docs_accordion.tsx @@ -15,7 +15,7 @@ import { useKibana } from '../../../../../../../../shared_imports'; import { useIsMounted } from '../../../../../use_is_mounted'; import { AddDocumentForm } from '../add_document_form'; -import './add_documents_accordion.scss'; +import './add_docs_accordion.scss'; const DISCOVER_URL_GENERATOR_ID = 'DISCOVER_APP_URL_GENERATOR'; diff --git a/x-pack/plugins/ingest_pipelines/public/application/components/pipeline_editor/components/test_pipeline/test_pipeline_tabs/tab_documents/add_docs_accordion/index.ts b/x-pack/plugins/ingest_pipelines/public/application/components/pipeline_editor/components/test_pipeline/test_pipeline_tabs/tab_documents/add_docs_accordion/index.ts new file mode 100644 index 0000000000000..5f7939690fa55 --- /dev/null +++ b/x-pack/plugins/ingest_pipelines/public/application/components/pipeline_editor/components/test_pipeline/test_pipeline_tabs/tab_documents/add_docs_accordion/index.ts @@ -0,0 +1,7 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License; + * you may not use this file except in compliance with the Elastic License. + */ + +export { AddDocumentsAccordion } from './add_docs_accordion'; diff --git a/x-pack/plugins/ingest_pipelines/public/application/components/pipeline_editor/components/test_pipeline/test_pipeline_tabs/tab_documents/tab_documents.tsx b/x-pack/plugins/ingest_pipelines/public/application/components/pipeline_editor/components/test_pipeline/test_pipeline_tabs/tab_documents/tab_documents.tsx index 6888f947b8606..dccc343e9359c 100644 --- a/x-pack/plugins/ingest_pipelines/public/application/components/pipeline_editor/components/test_pipeline/test_pipeline_tabs/tab_documents/tab_documents.tsx +++ b/x-pack/plugins/ingest_pipelines/public/application/components/pipeline_editor/components/test_pipeline/test_pipeline_tabs/tab_documents/tab_documents.tsx @@ -23,7 +23,7 @@ import { Form, } from '../../../../../../../shared_imports'; import { Document } from '../../../../types'; -import { AddDocumentsAccordion } from './add_documents_accordion'; +import { AddDocumentsAccordion } from './add_docs_accordion'; import { ResetDocumentsModal } from './reset_documents_modal'; import './tab_documents.scss'; diff --git a/x-pack/plugins/ingest_pipelines/tsconfig.json b/x-pack/plugins/ingest_pipelines/tsconfig.json new file mode 100644 index 0000000000000..5d78992600e81 --- /dev/null +++ b/x-pack/plugins/ingest_pipelines/tsconfig.json @@ -0,0 +1,28 @@ +{ + "extends": "../../../tsconfig.base.json", + "compilerOptions": { + "composite": true, + "outDir": "./target/types", + "emitDeclarationOnly": true, + "declaration": true, + "declarationMap": true + }, + "include": [ + "common/**/*", + "public/**/*", + "server/**/*", + "__jest__/**/*", + "../../typings/**/*" + ], + "references": [ + { "path": "../../../src/core/tsconfig.json" }, + { "path": "../licensing/tsconfig.json" }, + { "path": "../features/tsconfig.json" }, + { "path": "../security/tsconfig.json" }, + { "path": "../../../src/plugins/es_ui_shared/tsconfig.json"}, + { "path": "../../../src/plugins/kibana_react/tsconfig.json"}, + { "path": "../../../src/plugins/management/tsconfig.json"}, + { "path": "../../../src/plugins/share/tsconfig.json"}, + { "path": "../../../src/plugins/usage_collection/tsconfig.json"}, + ] +} diff --git a/x-pack/plugins/lens/public/drag_drop/__snapshots__/drag_drop.test.tsx.snap b/x-pack/plugins/lens/public/drag_drop/__snapshots__/drag_drop.test.tsx.snap index dc53f3a2bc2a7..6423a9f6190a7 100644 --- a/x-pack/plugins/lens/public/drag_drop/__snapshots__/drag_drop.test.tsx.snap +++ b/x-pack/plugins/lens/public/drag_drop/__snapshots__/drag_drop.test.tsx.snap @@ -13,10 +13,8 @@ exports[`DragDrop items that have droppable=false get special styling when anoth +
+ +
`; diff --git a/x-pack/plugins/lens/public/drag_drop/drag_drop.scss b/x-pack/plugins/lens/public/drag_drop/drag_drop.scss index ded0b4552a4e5..d0a4019055d57 100644 --- a/x-pack/plugins/lens/public/drag_drop/drag_drop.scss +++ b/x-pack/plugins/lens/public/drag_drop/drag_drop.scss @@ -52,7 +52,7 @@ } } -.lnsDragDrop__reorderableContainer { +.lnsDragDrop__container { position: relative; } @@ -63,11 +63,18 @@ height: calc(100% + #{$lnsLayerPanelDimensionMargin}); } -.lnsDragDrop-isReorderable { +.lnsDragDrop-translatableDrop { + transform: translateY(0); transition: transform $euiAnimSpeedFast ease-in-out; pointer-events: none; } +.lnsDragDrop-translatableDrag { + transform: translateY(0); + transition: transform $euiAnimSpeedFast ease-in-out; + position: relative; +} + // Draggable item when it is moving .lnsDragDrop-isHidden { opacity: 0; diff --git a/x-pack/plugins/lens/public/drag_drop/drag_drop.test.tsx b/x-pack/plugins/lens/public/drag_drop/drag_drop.test.tsx index 07b489d29ad06..9e1583b0c6e81 100644 --- a/x-pack/plugins/lens/public/drag_drop/drag_drop.test.tsx +++ b/x-pack/plugins/lens/public/drag_drop/drag_drop.test.tsx @@ -6,11 +6,33 @@ import React from 'react'; import { render, mount } from 'enzyme'; -import { DragDrop, ReorderableDragDrop, DropToHandler, DropHandler } from './drag_drop'; -import { ChildDragDropProvider, ReorderProvider } from './providers'; +import { DragDrop, DropHandler } from './drag_drop'; +import { + ChildDragDropProvider, + DragContextState, + ReorderProvider, + DragDropIdentifier, + ActiveDropTarget, +} from './providers'; +import { act } from 'react-dom/test-utils'; jest.useFakeTimers(); +const defaultContext = { + dragging: undefined, + setDragging: jest.fn(), + setActiveDropTarget: () => {}, + activeDropTarget: undefined, + keyboardMode: false, + setKeyboardMode: () => {}, + setA11yMessage: jest.fn(), +}; + +const dataTransfer = { + setData: jest.fn(), + getData: jest.fn(), +}; + describe('DragDrop', () => { const value = { id: '1', label: 'hello' }; test('renders if nothing is being dragged', () => { @@ -26,7 +48,7 @@ describe('DragDrop', () => { test('dragover calls preventDefault if droppable is true', () => { const preventDefault = jest.fn(); const component = mount( - + ); @@ -39,7 +61,7 @@ describe('DragDrop', () => { test('dragover does not call preventDefault if droppable is false', () => { const preventDefault = jest.fn(); const component = mount( - + ); @@ -51,13 +73,9 @@ describe('DragDrop', () => { test('dragstart sets dragging in the context', async () => { const setDragging = jest.fn(); - const dataTransfer = { - setData: jest.fn(), - getData: jest.fn(), - }; const component = mount( - + @@ -79,7 +97,11 @@ describe('DragDrop', () => { const onDrop = jest.fn(); const component = mount( - + @@ -93,7 +115,7 @@ describe('DragDrop', () => { expect(preventDefault).toBeCalled(); expect(stopPropagation).toBeCalled(); expect(setDragging).toBeCalledWith(undefined); - expect(onDrop).toBeCalledWith({ id: '2', label: 'hi' }); + expect(onDrop).toBeCalledWith({ id: '2', label: 'hi' }, { id: '1', label: 'hello' }); }); test('drop function is not called on droppable=false', async () => { @@ -103,7 +125,7 @@ describe('DragDrop', () => { const onDrop = jest.fn(); const component = mount( - + @@ -127,6 +149,7 @@ describe('DragDrop', () => { throw x; }} droppable + value={value} > @@ -137,11 +160,11 @@ describe('DragDrop', () => { test('items that have droppable=false get special styling when another item is dragged', () => { const component = mount( - {}}> + - {}} droppable={false}> + {}} droppable={false} value={{ id: '2' }}> @@ -153,17 +176,25 @@ describe('DragDrop', () => { test('additional styles are reflected in the className until drop', () => { let dragging: { id: '1' } | undefined; const getAdditionalClasses = jest.fn().mockReturnValue('additional'); + let activeDropTarget; + const component = mount( { dragging = { id: '1' }; }} + setActiveDropTarget={(val) => { + activeDropTarget = { activeDropTarget: val }; + }} + activeDropTarget={activeDropTarget} > {}} droppable getAdditionalClassesOnEnter={getAdditionalClasses} @@ -173,10 +204,6 @@ describe('DragDrop', () => { ); - const dataTransfer = { - setData: jest.fn(), - getData: jest.fn(), - }; component .find('[data-test-subj="lnsDragDrop"]') .first() @@ -184,40 +211,91 @@ describe('DragDrop', () => { jest.runAllTimers(); component.find('[data-test-subj="lnsDragDrop"]').at(1).simulate('dragover'); - expect(component.find('.additional')).toHaveLength(1); - - component.find('[data-test-subj="lnsDragDrop"]').at(1).simulate('dragleave'); + component.find('[data-test-subj="lnsDragDrop"]').at(1).simulate('drop'); expect(component.find('.additional')).toHaveLength(0); + }); + + test('additional enter styles are reflected in the className until dragleave', () => { + let dragging: { id: '1' } | undefined; + const getAdditionalClasses = jest.fn().mockReturnValue('additional'); + const setActiveDropTarget = jest.fn(); + + const component = mount( + { + dragging = { id: '1' }; + }} + setActiveDropTarget={setActiveDropTarget} + activeDropTarget={ + ({ activeDropTarget: value } as unknown) as DragContextState['activeDropTarget'] + } + keyboardMode={false} + setKeyboardMode={(keyboardMode) => true} + > + + + + {}} + droppable + getAdditionalClassesOnEnter={getAdditionalClasses} + > + + + + ); + + component + .find('[data-test-subj="lnsDragDrop"]') + .first() + .simulate('dragstart', { dataTransfer }); + jest.runAllTimers(); component.find('[data-test-subj="lnsDragDrop"]').at(1).simulate('dragover'); - component.find('[data-test-subj="lnsDragDrop"]').at(1).simulate('drop'); - expect(component.find('.additional')).toHaveLength(0); + expect(component.find('.additional')).toHaveLength(1); + + component.find('[data-test-subj="lnsDragDrop"]').at(1).simulate('dragleave'); + expect(setActiveDropTarget).toBeCalledWith(undefined); }); describe('reordering', () => { const mountComponent = ( - dragging: { id: '1' } | undefined, - onDrop: DropHandler = jest.fn(), - dropTo: DropToHandler = jest.fn() - ) => - mount( - { - dragging = { id: '1' }; - }} - > + dragContext: Partial | undefined, + onDrop: DropHandler = jest.fn() + ) => { + let dragging = dragContext?.dragging; + let keyboardMode = !!dragContext?.keyboardMode; + let activeDropTarget = dragContext?.activeDropTarget; + const baseContext = { + dragging, + setDragging: (val?: DragDropIdentifier) => { + dragging = val; + }, + keyboardMode, + setKeyboardMode: jest.fn((mode) => { + keyboardMode = mode; + }), + setActiveDropTarget: (target?: DragDropIdentifier) => { + activeDropTarget = { activeDropTarget: target } as ActiveDropTarget; + }, + activeDropTarget, + setA11yMessage: jest.fn(), + }; + return mount( + 1 @@ -227,12 +305,11 @@ describe('DragDrop', () => { droppable dragType="reorder" dropType="reorder" - itemsInGroup={['1', '2', '3']} + reorderableGroup={[{ id: '1' }, { id: '2' }, { id: '3' }]} value={{ id: '2', }} onDrop={onDrop} - dropTo={dropTo} > 2 @@ -242,132 +319,270 @@ describe('DragDrop', () => { droppable dragType="reorder" dropType="reorder" - itemsInGroup={['1', '2', '3']} + reorderableGroup={[{ id: '1' }, { id: '2' }, { id: '3' }]} value={{ id: '3', }} onDrop={onDrop} - dropTo={dropTo} > 3 ); - test(`ReorderableDragDrop component doesn't appear for groups of 1 or less`, () => { - let dragging; - const component = mount( - { - dragging = { id: '1' }; - }} - > - - -
- - - - ); - expect(component.find(ReorderableDragDrop)).toHaveLength(0); - }); - test(`Reorderable component renders properly`, () => { + }; + test(`Inactive reorderable group renders properly`, () => { const component = mountComponent(undefined, jest.fn()); - expect(component.find(ReorderableDragDrop)).toHaveLength(3); + expect(component.find('.lnsDragDrop-reorderable')).toHaveLength(3); }); - test(`Elements between dragged and drop get extra class to show the reorder effect when dragging`, () => { - const component = mountComponent({ id: '1' }, jest.fn()); - const dataTransfer = { - setData: jest.fn(), - getData: jest.fn(), - }; - component - .find(ReorderableDragDrop) - .first() - .find('[data-test-subj="lnsDragDrop"]') - .simulate('dragstart', { dataTransfer }); - jest.runAllTimers(); - component.find('[data-test-subj="lnsDragDrop-reorderableDrop"]').at(2).simulate('dragover'); + test(`Reorderable group with lifted element renders properly`, () => { + const setDragging = jest.fn(); + const setA11yMessage = jest.fn(); + const component = mountComponent( + { dragging: { id: '1' }, setA11yMessage, setDragging }, + jest.fn() + ); + act(() => { + component + .find('[data-test-subj="lnsDragDrop"]') + .first() + .simulate('dragstart', { dataTransfer }); + jest.runAllTimers(); + }); + + expect(setDragging).toBeCalledWith({ id: '1' }); + expect(setA11yMessage).toBeCalledWith('You have lifted an item 1 in position 1'); + expect( + component + .find('[data-test-subj="lnsDragDrop-reorderableGroup"]') + .hasClass('lnsDragDrop-isActiveGroup') + ).toEqual(true); + }); + + test(`Reordered elements get extra styles to show the reorder effect when dragging`, () => { + const component = mountComponent({ dragging: { id: '1' } }, jest.fn()); + + act(() => { + component + .find('[data-test-subj="lnsDragDrop"]') + .first() + .simulate('dragstart', { dataTransfer }); + jest.runAllTimers(); + }); + + component + .find('[data-test-subj="lnsDragDrop-reorderableDropLayer"]') + .at(1) + .simulate('dragover'); expect( component.find('[data-test-subj="lnsDragDrop-reorderableDrag"]').at(0).prop('style') - ).toEqual({}); + ).toEqual(undefined); expect( - component.find('[data-test-subj="lnsDragDrop-reorderableDrag"]').at(1).prop('style') + component.find('[data-test-subj="lnsDragDrop-translatableDrop"]').at(0).prop('style') ).toEqual({ - transform: 'translateY(-40px)', + transform: 'translateY(-8px)', }); expect( - component.find('[data-test-subj="lnsDragDrop-reorderableDrag"]').at(2).prop('style') + component.find('[data-test-subj="lnsDragDrop-translatableDrop"]').at(1).prop('style') ).toEqual({ - transform: 'translateY(-40px)', + transform: 'translateY(-8px)', }); - component.find('[data-test-subj="lnsDragDrop-reorderableDrop"]').at(2).simulate('dragleave'); + component + .find('[data-test-subj="lnsDragDrop-reorderableDropLayer"]') + .at(1) + .simulate('dragleave'); expect( - component.find('[data-test-subj="lnsDragDrop-reorderableDrag"]').at(1).prop('style') - ).toEqual({}); + component.find('[data-test-subj="lnsDragDrop-reorderableDrag"]').at(0).prop('style') + ).toEqual(undefined); expect( - component.find('[data-test-subj="lnsDragDrop-reorderableDrag"]').at(2).prop('style') - ).toEqual({}); + component.find('[data-test-subj="lnsDragDrop-translatableDrop"]').at(1).prop('style') + ).toEqual(undefined); }); + test(`Dropping an item runs onDrop function`, () => { + const setDragging = jest.fn(); + const setA11yMessage = jest.fn(); const preventDefault = jest.fn(); const stopPropagation = jest.fn(); const onDrop = jest.fn(); - const component = mountComponent({ id: '1' }, onDrop); + const component = mountComponent( + { dragging: { id: '1' }, setA11yMessage, setDragging }, + onDrop + ); component - .find('[data-test-subj="lnsDragDrop-reorderableDrop"]') + .find('[data-test-subj="lnsDragDrop-reorderableDropLayer"]') .at(1) .simulate('drop', { preventDefault, stopPropagation }); + jest.runAllTimers(); + + expect(setA11yMessage).toBeCalledWith( + 'You have dropped the item. You have moved the item from position 1 to positon 3' + ); expect(preventDefault).toBeCalled(); expect(stopPropagation).toBeCalled(); - expect(onDrop).toBeCalledWith({ id: '1' }); + expect(onDrop).toBeCalledWith({ id: '1' }, { id: '3' }); }); - test(`Keyboard navigation: user can reorder an element`, () => { + + test(`Keyboard navigation: user can drop element to an activeDropTarget`, () => { const onDrop = jest.fn(); - const dropTo = jest.fn(); - const component = mountComponent({ id: '1' }, onDrop, dropTo); + const component = mountComponent( + { + dragging: { id: '1' }, + activeDropTarget: { activeDropTarget: { id: '3' } } as ActiveDropTarget, + keyboardMode: true, + }, + onDrop + ); const keyboardHandler = component - .find(ReorderableDragDrop) - .at(1) - .find('[data-test-subj="lnsDragDrop-keyboardHandler"]'); + .find('[data-test-subj="lnsDragDrop-keyboardHandler"]') + .simulate('focus'); + + act(() => { + keyboardHandler.simulate('keydown', { key: 'ArrowDown' }); + keyboardHandler.simulate('keydown', { key: 'ArrowDown' }); + keyboardHandler.simulate('keydown', { key: 'Enter' }); + }); + expect(onDrop).toBeCalledWith({ id: '1' }, { id: '3' }); + }); + + test(`Keyboard Navigation: Reordered elements get extra styles to show the reorder effect`, () => { + const setA11yMessage = jest.fn(); + const component = mountComponent( + { dragging: { id: '1' }, keyboardMode: true, setA11yMessage }, + jest.fn() + ); + const keyboardHandler = component.find('[data-test-subj="lnsDragDrop-keyboardHandler"]'); keyboardHandler.simulate('keydown', { key: 'Space' }); keyboardHandler.simulate('keydown', { key: 'ArrowDown' }); - expect(dropTo).toBeCalledWith('3'); - keyboardHandler.simulate('keydown', { key: 'ArrowUp' }); - expect(dropTo).toBeCalledWith('1'); + expect( + component.find('[data-test-subj="lnsDragDrop-reorderableDrag"]').at(0).prop('style') + ).toEqual({ + transform: 'translateY(+8px)', + }); + expect( + component.find('[data-test-subj="lnsDragDrop-translatableDrop"]').at(0).prop('style') + ).toEqual({ + transform: 'translateY(-40px)', + }); + expect( + component.find('[data-test-subj="lnsDragDrop-translatableDrop"]').at(1).prop('style') + ).toEqual(undefined); + expect(setA11yMessage).toBeCalledWith( + 'You have moved the item 1 from position 1 to position 2' + ); + + component + .find('[data-test-subj="lnsDragDrop-reorderableDropLayer"]') + .at(1) + .simulate('dragleave'); + expect( + component.find('[data-test-subj="lnsDragDrop-reorderableDrag"]').at(0).prop('style') + ).toEqual(undefined); + expect( + component.find('[data-test-subj="lnsDragDrop-translatableDrop"]').at(1).prop('style') + ).toEqual(undefined); }); + test(`Keyboard Navigation: User cannot move an element outside of the group`, () => { const onDrop = jest.fn(); - const dropTo = jest.fn(); - const component = mountComponent({ id: '1' }, onDrop, dropTo); - const keyboardHandler = component - .find(ReorderableDragDrop) - .first() - .find('[data-test-subj="lnsDragDrop-keyboardHandler"]'); + const setActiveDropTarget = jest.fn(); + const setA11yMessage = jest.fn(); + const component = mountComponent( + { dragging: { id: '1' }, keyboardMode: true, setActiveDropTarget, setA11yMessage }, + onDrop + ); + const keyboardHandler = component.find('[data-test-subj="lnsDragDrop-keyboardHandler"]'); keyboardHandler.simulate('keydown', { key: 'Space' }); keyboardHandler.simulate('keydown', { key: 'ArrowUp' }); - expect(dropTo).not.toHaveBeenCalled(); + expect(setActiveDropTarget).not.toHaveBeenCalled(); + keyboardHandler.simulate('keydown', { key: 'Space' }); keyboardHandler.simulate('keydown', { key: 'ArrowDown' }); - expect(dropTo).toBeCalledWith('2'); + + expect(setActiveDropTarget).toBeCalledWith({ id: '2' }); + expect(setA11yMessage).toBeCalledWith( + 'You have moved the item 1 from position 1 to position 2' + ); + }); + + test(`Keyboard Navigation: User cannot drop element to itself`, () => { + const setActiveDropTarget = jest.fn(); + const setA11yMessage = jest.fn(); + const component = mount( + + + + 1 + + + 2 + + + + ); + const keyboardHandler = component.find('[data-test-subj="lnsDragDrop-keyboardHandler"]'); + + keyboardHandler.simulate('keydown', { key: 'Space' }); + keyboardHandler.simulate('keydown', { key: 'ArrowUp' }); + expect(setActiveDropTarget).toBeCalledWith({ id: '1' }); + expect(setA11yMessage).toBeCalledWith('You have moved back the item 1 to position 1'); + }); + + test(`Keyboard Navigation: Doesn't call onDrop when movement is cancelled`, () => { + const setA11yMessage = jest.fn(); + const onDrop = jest.fn(); + + const component = mountComponent({ dragging: { id: '1' }, setA11yMessage }, onDrop); + const keyboardHandler = component.find('[data-test-subj="lnsDragDrop-keyboardHandler"]'); + keyboardHandler.simulate('keydown', { key: 'Space' }); + keyboardHandler.simulate('keydown', { key: 'Escape' }); + + jest.runAllTimers(); + + expect(onDrop).not.toHaveBeenCalled(); + expect(setA11yMessage).toBeCalledWith( + 'Movement cancelled. The item has returned to its starting position 1' + ); + keyboardHandler.simulate('keydown', { key: 'Space' }); + keyboardHandler.simulate('keydown', { key: 'ArrowDown' }); + keyboardHandler.simulate('blur'); + + expect(onDrop).not.toHaveBeenCalled(); + expect(setA11yMessage).toBeCalledWith( + 'Movement cancelled. The item has returned to its starting position 1' + ); }); }); }); diff --git a/x-pack/plugins/lens/public/drag_drop/drag_drop.tsx b/x-pack/plugins/lens/public/drag_drop/drag_drop.tsx index 32facbf8e84a8..2dbcfab8d5738 100644 --- a/x-pack/plugins/lens/public/drag_drop/drag_drop.tsx +++ b/x-pack/plugins/lens/public/drag_drop/drag_drop.tsx @@ -5,11 +5,17 @@ */ import './drag_drop.scss'; -import React, { useState, useContext, useEffect } from 'react'; +import React, { useContext, useEffect, memo } from 'react'; import classNames from 'classnames'; import { keys, EuiScreenReaderOnly } from '@elastic/eui'; -import { i18n } from '@kbn/i18n'; -import { DragContext, DragContextState, ReorderContext, ReorderState } from './providers'; +import { + DragDropIdentifier, + DragContext, + DragContextState, + ReorderContext, + ReorderState, + reorderAnnouncements, +} from './providers'; import { trackUiEvent } from '../lens_ui_telemetry'; export type DroppableEvent = React.DragEvent; @@ -17,12 +23,7 @@ export type DroppableEvent = React.DragEvent; /** * A function that handles a drop event. */ -export type DropHandler = (item: unknown) => void; - -/** - * A function that handles a dropTo event. - */ -export type DropToHandler = (dropTargetId: string) => void; +export type DropHandler = (dropped: DragDropIdentifier, dropTarget: DragDropIdentifier) => void; /** * The base props to the DragDrop component. @@ -32,24 +33,20 @@ interface BaseProps { * The CSS class(es) for the root element. */ className?: string; - /** - * The event handler that fires when this item - * is dropped to the one with passed id - * + * The label for accessibility */ - dropTo?: DropToHandler; + label?: string; + /** * The event handler that fires when an item * is dropped onto this DragDrop component. */ onDrop?: DropHandler; /** - * The value associated with this item, if it is draggable. - * If this component is dragged, this will be the value of - * "dragging" in the root drag/drop context. + * The value associated with this item. */ - value?: DragContextState['dragging']; + value: DragDropIdentifier; /** * Optional comparison function to check whether a value is the dragged one @@ -60,7 +57,10 @@ interface BaseProps { * The React element which will be passed the draggable handlers */ children: React.ReactElement; - + /** + * Indicates whether or not this component is draggable. + */ + draggable?: boolean; /** * Indicates whether or not the currently dragged item * can be dropped onto this component. @@ -75,12 +75,12 @@ interface BaseProps { /** * The optional test subject associated with this DOM element. */ - 'data-test-subj'?: string; + dataTestSubj?: string; /** * items belonging to the same group that can be reordered */ - itemsInGroup?: string[]; + reorderableGroup?: DragDropIdentifier[]; /** * Indicates to the user whether the currently dragged item @@ -93,34 +93,46 @@ interface BaseProps { * replace something that is existing or add a new one */ dropType?: 'add' | 'replace' | 'reorder'; + + /** + * temporary flag to exclude the draggable elements that don't have keyboard nav yet. To be removed along with the feature development + */ + noKeyboardSupportYet?: boolean; } /** * The props for a draggable instance of that component. */ -interface DraggableProps extends BaseProps { - /** - * Indicates whether or not this component is draggable. - */ - draggable: true; +interface DragInnerProps extends BaseProps { /** * The label, which should be attached to the drag event, and which will e.g. * be used if the element will be dropped into a text field. */ - label: string; + label?: string; + isDragging: boolean; + keyboardMode: boolean; + setKeyboardMode: DragContextState['setKeyboardMode']; + setDragging: DragContextState['setDragging']; + setActiveDropTarget: DragContextState['setActiveDropTarget']; + setA11yMessage: DragContextState['setA11yMessage']; + activeDropTarget: DragContextState['activeDropTarget']; + onDragStart?: ( + target?: + | DroppableEvent['currentTarget'] + | React.KeyboardEvent['currentTarget'] + ) => void; + onDragEnd?: () => void; + extraKeyboardHandler?: (e: React.KeyboardEvent) => void; } /** * The props for a non-draggable instance of that component. */ -interface NonDraggableProps extends BaseProps { - /** - * Indicates whether or not this component is draggable. - */ - draggable?: false; -} +interface DropInnerProps extends BaseProps, DragContextState { + isDragging: boolean; -type Props = DraggableProps | NonDraggableProps; + isNotDroppable: boolean; +} /** * A draggable / droppable item. Items can be both draggable and droppable at @@ -129,40 +141,189 @@ type Props = DraggableProps | NonDraggableProps; * @param props */ -export const DragDrop = (props: Props) => { - const { dragging, setDragging } = useContext(DragContext); - const { value, draggable, droppable, isValueEqual } = props; +const lnsLayerPanelDimensionMargin = 8; - return ( - { + const { + dragging, + setDragging, + keyboardMode, + setKeyboardMode, + activeDropTarget, + setActiveDropTarget, + setA11yMessage, + } = useContext(DragContext); + + const { value, draggable, droppable, reorderableGroup } = props; + + const isDragging = !!(draggable && value.id === dragging?.id); + + const dragProps = { + ...props, + isDragging, + keyboardMode: isDragging ? keyboardMode : false, // optimization to not rerender all dragging components + activeDropTarget: isDragging ? activeDropTarget : undefined, // optimization to not rerender all dragging components + setKeyboardMode, + setDragging, + setActiveDropTarget, + setA11yMessage, + }; + + const dropProps = { + ...props, + setKeyboardMode, + keyboardMode, + dragging, + setDragging, + activeDropTarget, + setActiveDropTarget, + isDragging, + setA11yMessage, + isNotDroppable: + // If the configuration has provided a droppable flag, but this particular item is not + // droppable, then it should be less prominent. Ignores items that are both + // draggable and drop targets + !!(droppable === false && dragging && value.id !== dragging.id), + }; + + if (draggable && !droppable) { + if (reorderableGroup && reorderableGroup.length > 1) { + return ( + + ); + } else { + return ; + } + } + if ( + reorderableGroup && + reorderableGroup.length > 1 && + reorderableGroup?.some((i) => i.id === value.id) + ) { + return ; + } + return ; +}; + +const DragInner = memo(function DragDropInner({ + dataTestSubj, + className, + value, + children, + setDragging, + setKeyboardMode, + setActiveDropTarget, + label = '', + keyboardMode, + isDragging, + activeDropTarget, + onDrop, + dragType, + onDragStart, + onDragEnd, + extraKeyboardHandler, + noKeyboardSupportYet, +}: DragInnerProps) { + const dragStart = (e?: DroppableEvent | React.KeyboardEvent) => { + // Setting stopPropgagation causes Chrome failures, so + // we are manually checking if we've already handled this + // in a nested child, and doing nothing if so... + if (e && 'dataTransfer' in e && e.dataTransfer.getData('text')) { + return; + } + + // We only can reach the dragStart method if the element is draggable, + // so we know we have DraggableProps if we reach this code. + if (e && 'dataTransfer' in e) { + e.dataTransfer.setData('text', label); + } + + // Chrome causes issues if you try to render from within a + // dragStart event, so we drop a setTimeout to avoid that. + + const currentTarget = e?.currentTarget; + setTimeout(() => { + setDragging(value); + if (onDragStart) { + onDragStart(currentTarget); } - isNotDroppable={ - // If the configuration has provided a droppable flag, but this particular item is not - // droppable, then it should be less prominent. Ignores items that are both - // draggable and drop targets - droppable === false && Boolean(dragging) && value !== dragging + }); + }; + + const dragEnd = (e?: DroppableEvent) => { + e?.stopPropagation(); + setDragging(undefined); + setActiveDropTarget(undefined); + setKeyboardMode(false); + if (onDragEnd) { + onDragEnd(); + } + }; + + const dropToActiveDropTarget = () => { + if (isDragging && activeDropTarget?.activeDropTarget) { + trackUiEvent('drop_total'); + if (onDrop) { + onDrop(value, activeDropTarget.activeDropTarget); } - /> + } + }; + + return ( +
+ {!noKeyboardSupportYet && ( + +
); -}; +}); -const DragDropInner = React.memo(function DragDropInner( - props: Props & - DragContextState & { - isDragging: boolean; - isNotDroppable: boolean; - } -) { - const [state, setState] = useState({ - isActive: false, - dragEnterClassNames: '', - }); +const DropInner = memo(function DropInner(props: DropInnerProps) { const { + dataTestSubj, className, onDrop, value, @@ -175,10 +336,16 @@ const DragDropInner = React.memo(function DragDropInner( isNotDroppable, dragType = 'copy', dropType = 'add', - dropTo, - itemsInGroup, + keyboardMode, + setKeyboardMode, + activeDropTarget, + setActiveDropTarget, + getAdditionalClassesOnEnter, } = props; + const activeDropTargetMatches = + activeDropTarget?.activeDropTarget && activeDropTarget.activeDropTarget.id === value.id; + const isMoveDragging = isDragging && dragType === 'move'; const classes = classNames( @@ -186,339 +353,364 @@ const DragDropInner = React.memo(function DragDropInner( { 'lnsDragDrop-isDraggable': draggable, 'lnsDragDrop-isDragging': isDragging, - 'lnsDragDrop-isHidden': isMoveDragging, + 'lnsDragDrop-isHidden': isMoveDragging && !keyboardMode, 'lnsDragDrop-isDroppable': !draggable, 'lnsDragDrop-isDropTarget': droppable && dragType !== 'reorder', - 'lnsDragDrop-isActiveDropTarget': droppable && state.isActive && dragType !== 'reorder', + 'lnsDragDrop-isActiveDropTarget': + droppable && activeDropTargetMatches && dragType !== 'reorder', 'lnsDragDrop-isNotDroppable': !isMoveDragging && isNotDroppable, - 'lnsDragDrop-isReplacing': droppable && state.isActive && dropType === 'replace', + 'lnsDragDrop-isReplacing': droppable && activeDropTargetMatches && dropType === 'replace', }, - state.dragEnterClassNames - ); - - const dragStart = (e: DroppableEvent) => { - // Setting stopPropgagation causes Chrome failures, so - // we are manually checking if we've already handled this - // in a nested child, and doing nothing if so... - if (e.dataTransfer.getData('text')) { - return; + getAdditionalClassesOnEnter && { + [getAdditionalClassesOnEnter()]: activeDropTargetMatches, } - - // We only can reach the dragStart method if the element is draggable, - // so we know we have DraggableProps if we reach this code. - e.dataTransfer.setData('text', (props as DraggableProps).label); - - // Chrome causes issues if you try to render from within a - // dragStart event, so we drop a setTimeout to avoid that. - setState({ ...state }); - setTimeout(() => setDragging(value)); - }; - - const dragEnd = (e: DroppableEvent) => { - e.stopPropagation(); - setDragging(undefined); - }; + ); const dragOver = (e: DroppableEvent) => { if (!droppable) { return; } - e.preventDefault(); // An optimization to prevent a bunch of React churn. - if (!state.isActive) { - setState({ - ...state, - isActive: true, - dragEnterClassNames: props.getAdditionalClassesOnEnter - ? props.getAdditionalClassesOnEnter() - : '', - }); + // todo: replace with custom function ? + if (!activeDropTargetMatches) { + setActiveDropTarget(value); } }; const dragLeave = () => { - setState({ ...state, isActive: false, dragEnterClassNames: '' }); + setActiveDropTarget(undefined); }; - const drop = (e: DroppableEvent) => { + const drop = (e: DroppableEvent | React.KeyboardEvent) => { e.preventDefault(); e.stopPropagation(); - setState({ ...state, isActive: false, dragEnterClassNames: '' }); - setDragging(undefined); - - if (onDrop && droppable) { + if (onDrop && droppable && dragging) { trackUiEvent('drop_total'); - onDrop(dragging); + onDrop(dragging, value); } + setActiveDropTarget(undefined); + setDragging(undefined); + setKeyboardMode(false); }; + return ( + <> + {React.cloneElement(children, { + 'data-test-subj': dataTestSubj || 'lnsDragDrop', + className: classNames(children.props.className, classes, className), + onDragOver: dragOver, + onDragLeave: dragLeave, + onDrop: drop, + draggable, + })} + + ); +}); - const isReorderDragging = !!(dragging && itemsInGroup?.includes(dragging.id)); +const ReorderableDrag = memo(function ReorderableDrag( + props: DragInnerProps & { reorderableGroup: DragDropIdentifier[]; dragging?: DragDropIdentifier } +) { + const { + reorderState: { isReorderOn, reorderedItems, direction }, + setReorderState, + } = useContext(ReorderContext); - if ( - draggable && - itemsInGroup?.length && - itemsInGroup.length > 1 && - value?.id && - dropTo && - (!dragging || isReorderDragging) - ) { - const { label } = props as DraggableProps; - return ( - - {children} - - ); - } - return React.cloneElement(children, { - 'data-test-subj': props['data-test-subj'] || 'lnsDragDrop', - className: classNames(children.props.className, classes, className), - onDragOver: dragOver, - onDragLeave: dragLeave, - onDrop: drop, - draggable, - onDragEnd: dragEnd, - onDragStart: dragStart, - }); -}); + const { + value, + setActiveDropTarget, + label = '', + keyboardMode, + isDragging, + activeDropTarget, + reorderableGroup, + onDrop, + setA11yMessage, + } = props; -const getKeyboardReorderMessageMoved = ( - itemLabel: string, - position: number, - prevPosition: number -) => - i18n.translate('xpack.lens.dragDrop.elementMoved', { - defaultMessage: `You have moved the item {itemLabel} from position {prevPosition} to position {position}`, - values: { - itemLabel, - position, - prevPosition, - }, - }); - -const getKeyboardReorderMessageLifted = (itemLabel: string, position: number) => - i18n.translate('xpack.lens.dragDrop.elementLifted', { - defaultMessage: `You have lifted an item {itemLabel} in position {position}`, - values: { - itemLabel, - position, - }, - }); + const currentIndex = reorderableGroup.findIndex((i) => i.id === value.id); + + const isFocusInGroup = keyboardMode + ? isDragging && + (!activeDropTarget?.activeDropTarget || + reorderableGroup.some((i) => i.id === activeDropTarget?.activeDropTarget?.id)) + : isDragging; + + useEffect(() => { + setReorderState((s: ReorderState) => ({ + ...s, + isReorderOn: isFocusInGroup, + })); + }, [setReorderState, isFocusInGroup]); + + const onReorderableDragStart = ( + currentTarget?: + | DroppableEvent['currentTarget'] + | React.KeyboardEvent['currentTarget'] + ) => { + if (currentTarget) { + const height = currentTarget.offsetHeight + lnsLayerPanelDimensionMargin; + setReorderState((s: ReorderState) => ({ + ...s, + draggingHeight: height, + })); + } -const lnsLayerPanelDimensionMargin = 8; + setA11yMessage(reorderAnnouncements.lifted(label, currentIndex + 1)); + }; -export const ReorderableDragDrop = ({ - draggingProps, - dropProps, - children, - label, - dropTo, - className, - dataTestSubj, -}: { - draggingProps: { - className: string; - draggable: Props['draggable']; - onDragEnd: (e: DroppableEvent) => void; - onDragStart: (e: DroppableEvent) => void; - isReorderDragging: boolean; + const onReorderableDragEnd = () => { + resetReorderState(); + setA11yMessage(reorderAnnouncements.cancelled(currentIndex + 1)); }; - dropProps: { - onDrop: (e: DroppableEvent) => void; - onDragOver: (e: DroppableEvent) => void; - onDragLeave: () => void; - dragging: DragContextState['dragging']; - droppable: DraggableProps['droppable']; - itemsInGroup: string[]; - id: string; - isActive: boolean; + + const onReorderableDrop = (dragging: DragDropIdentifier, target: DragDropIdentifier) => { + if (onDrop) { + onDrop(dragging, target); + const targetIndex = reorderableGroup.findIndex( + (i) => i.id === activeDropTarget?.activeDropTarget?.id + ); + + resetReorderState(); + setA11yMessage(reorderAnnouncements.dropped(targetIndex + 1, currentIndex + 1)); + } }; - children: React.ReactElement; - label: string; - dropTo: DropToHandler; - className?: string; - dataTestSubj: string; -}) => { - const { itemsInGroup, dragging, id, droppable } = dropProps; - const { reorderState, setReorderState } = useContext(ReorderContext); - const { isReorderOn, reorderedItems, draggingHeight, direction, groupId } = reorderState; - const currentIndex = itemsInGroup.indexOf(id); + const resetReorderState = () => + setReorderState((s: ReorderState) => ({ + ...s, + reorderedItems: [], + })); + + const extraKeyboardHandler = (e: React.KeyboardEvent) => { + if (isReorderOn && keyboardMode) { + e.stopPropagation(); + e.preventDefault(); + let activeDropTargetIndex = reorderableGroup.findIndex((i) => i.id === value.id); + if (activeDropTarget?.activeDropTarget) { + const index = reorderableGroup.findIndex( + (i) => i.id === activeDropTarget.activeDropTarget?.id + ); + if (index !== -1) activeDropTargetIndex = index; + } + if (keys.ARROW_DOWN === e.key) { + if (activeDropTargetIndex < reorderableGroup.length - 1) { + setA11yMessage( + reorderAnnouncements.moved(label, activeDropTargetIndex + 2, currentIndex + 1) + ); + onReorderableDragOver(reorderableGroup[activeDropTargetIndex + 1]); + } + } else if (keys.ARROW_UP === e.key) { + if (activeDropTargetIndex > 0) { + setA11yMessage( + reorderAnnouncements.moved(label, activeDropTargetIndex, currentIndex + 1) + ); + + onReorderableDragOver(reorderableGroup[activeDropTargetIndex - 1]); + } + } + } + }; - useEffect( - () => + const onReorderableDragOver = (target: DragDropIdentifier) => { + let droppingIndex = currentIndex; + if (keyboardMode && 'id' in target) { + setActiveDropTarget(target); + droppingIndex = reorderableGroup.findIndex((i) => i.id === target.id); + } + const draggingIndex = reorderableGroup.findIndex((i) => i.id === value?.id); + if (draggingIndex === -1) { + return; + } + + if (draggingIndex === droppingIndex) { setReorderState((s: ReorderState) => ({ ...s, - isReorderOn: draggingProps.isReorderDragging, - })), - [draggingProps.isReorderDragging, setReorderState] - ); + reorderedItems: [], + })); + } + + setReorderState((s: ReorderState) => + draggingIndex < droppingIndex + ? { + ...s, + reorderedItems: reorderableGroup.slice(draggingIndex + 1, droppingIndex + 1), + direction: '-', + } + : { + ...s, + reorderedItems: reorderableGroup.slice(droppingIndex, draggingIndex), + direction: '+', + } + ); + }; + + const areItemsReordered = isDragging && keyboardMode && reorderedItems.length; return (
- -
+ ); +}); + +const ReorderableDrop = memo(function ReorderableDrop( + props: DropInnerProps & { reorderableGroup: DragDropIdentifier[] } +) { + const { + onDrop, + value, + droppable, + dragging, + setDragging, + setKeyboardMode, + activeDropTarget, + setActiveDropTarget, + reorderableGroup, + setA11yMessage, + } = props; + + const currentIndex = reorderableGroup.findIndex((i) => i.id === value.id); + const activeDropTargetMatches = + activeDropTarget?.activeDropTarget && activeDropTarget.activeDropTarget.id === value.id; + + const { + reorderState: { isReorderOn, reorderedItems, draggingHeight, direction }, + setReorderState, + } = useContext(ReorderContext); + + const heightRef = React.useRef(null); + + const isReordered = + isReorderOn && reorderedItems.some((el) => el.id === value.id) && reorderedItems.length; + + useEffect(() => { + if (isReordered && heightRef.current?.clientHeight) { + setReorderState((s) => ({ + ...s, + reorderedItems: s.reorderedItems.map((el) => + el.id === value.id + ? { + ...el, + height: heightRef.current?.clientHeight, } - } - }} - /> - - {React.cloneElement(children, { - ['data-test-subj']: 'lnsDragDrop-reorderableDrag', - draggable: draggingProps.draggable, - onDragEnd: draggingProps.onDragEnd, - onDragStart: (e: DroppableEvent) => { - const height = e.currentTarget.offsetHeight + lnsLayerPanelDimensionMargin; - setReorderState((s: ReorderState) => ({ + : el + ), + })); + } + }, [isReordered, setReorderState, value.id]); + + const onReorderableDragOver = (e: DroppableEvent) => { + if (!droppable) { + return; + } + e.preventDefault(); + + // An optimization to prevent a bunch of React churn. + // todo: replace with custom function ? + if (!activeDropTargetMatches) { + setActiveDropTarget(value); + } + + const draggingIndex = reorderableGroup.findIndex((i) => i.id === dragging?.id); + + if (!dragging || draggingIndex === -1) { + return; + } + const droppingIndex = currentIndex; + if (draggingIndex === droppingIndex) { + setReorderState((s: ReorderState) => ({ + ...s, + reorderedItems: [], + })); + } + + setReorderState((s: ReorderState) => + draggingIndex < droppingIndex + ? { ...s, - draggingHeight: height, - })); - draggingProps.onDragStart(e); - }, - className: classNames( - draggingProps.className, - { - 'lnsDragDrop-isKeyboardModeActive': isReorderOn, - }, - { - 'lnsDragDrop-isReorderable': draggingProps.isReorderDragging, + reorderedItems: reorderableGroup.slice(draggingIndex + 1, droppingIndex + 1), + direction: '-', } - ), - style: reorderedItems.includes(id) - ? { - transform: `translateY(${direction}${draggingHeight}px)`, - } - : {}, - })} + : { + ...s, + reorderedItems: reorderableGroup.slice(droppingIndex, draggingIndex), + direction: '+', + } + ); + }; + + const onReorderableDrop = (e: DroppableEvent) => { + e.preventDefault(); + e.stopPropagation(); + + setActiveDropTarget(undefined); + setDragging(undefined); + setKeyboardMode(false); + + if (onDrop && droppable && dragging) { + trackUiEvent('drop_total'); + + onDrop(dragging, value); + const draggingIndex = reorderableGroup.findIndex((i) => i.id === dragging.id); + // setTimeout ensures it will run after dragEnd messaging + setTimeout(() => + setA11yMessage(reorderAnnouncements.dropped(currentIndex + 1, draggingIndex + 1)) + ); + } + }; + + return ( +
i.id === value.id) + ? { + transform: `translateY(${direction}${draggingHeight}px)`, + } + : undefined + } + ref={heightRef} + data-test-subj="lnsDragDrop-translatableDrop" + className="lnsDragDrop-translatableDrop lnsDragDrop-reorderable" + > + +
+ +
{ - dropProps.onDrop(e); - setReorderState((s: ReorderState) => ({ - ...s, - reorderedItems: [], - })); - }} - onDragOver={(e: DroppableEvent) => { - if (!droppable) { - return; - } - dropProps.onDragOver(e); - if (!dropProps.isActive) { - if (!dragging) { - return; - } - const draggingIndex = itemsInGroup.indexOf(dragging.id); - const droppingIndex = currentIndex; - if (draggingIndex === droppingIndex) { - setReorderState((s: ReorderState) => ({ - ...s, - reorderedItems: [], - })); - } - - setReorderState((s: ReorderState) => - draggingIndex < droppingIndex - ? { - ...s, - reorderedItems: itemsInGroup.slice(draggingIndex + 1, droppingIndex + 1), - direction: '-', - } - : { - ...s, - reorderedItems: itemsInGroup.slice(droppingIndex, draggingIndex), - direction: '+', - } - ); - } - }} + onDrop={onReorderableDrop} + onDragOver={onReorderableDragOver} onDragLeave={() => { - dropProps.onDragLeave(); setReorderState((s: ReorderState) => ({ ...s, reorderedItems: [], @@ -527,4 +719,4 @@ export const ReorderableDragDrop = ({ />
); -}; +}); diff --git a/x-pack/plugins/lens/public/drag_drop/providers.tsx b/x-pack/plugins/lens/public/drag_drop/providers.tsx index 5e0fc648454ad..86ff5054520af 100644 --- a/x-pack/plugins/lens/public/drag_drop/providers.tsx +++ b/x-pack/plugins/lens/public/drag_drop/providers.tsx @@ -9,12 +9,13 @@ import classNames from 'classnames'; import { EuiScreenReaderOnly, EuiPortal } from '@elastic/eui'; import { i18n } from '@kbn/i18n'; -export type Dragging = - | (Record & { - id: string; - }) - | undefined; +export type DragDropIdentifier = Record & { + id: string; +}; +export interface ActiveDropTarget { + activeDropTarget?: DragDropIdentifier; +} /** * The shape of the drag / drop context. */ @@ -22,12 +23,26 @@ export interface DragContextState { /** * The item being dragged or undefined. */ - dragging: Dragging; + dragging?: DragDropIdentifier; + /** + * keyboard mode + */ + keyboardMode: boolean; + /** + * keyboard mode + */ + setKeyboardMode: (mode: boolean) => void; /** * Set the item being dragged. */ - setDragging: (dragging: Dragging) => void; + setDragging: (dragging?: DragDropIdentifier) => void; + + activeDropTarget?: ActiveDropTarget; + + setActiveDropTarget: (newTarget?: DragDropIdentifier) => void; + + setA11yMessage: (message: string) => void; } /** @@ -38,28 +53,52 @@ export interface DragContextState { export const DragContext = React.createContext({ dragging: undefined, setDragging: () => {}, + keyboardMode: false, + setKeyboardMode: () => {}, + activeDropTarget: undefined, + setActiveDropTarget: () => {}, + setA11yMessage: () => {}, }); /** * The argument to DragDropProvider. */ export interface ProviderProps { + /** + * keyboard mode + */ + keyboardMode: boolean; + /** + * keyboard mode + */ + setKeyboardMode: (mode: boolean) => void; + /** + * Set the item being dragged. + */ /** * The item being dragged. If unspecified, the provider will * behave as if it is the root provider. */ - dragging: Dragging; + dragging?: DragDropIdentifier; /** * Sets the item being dragged. If unspecified, the provider * will behave as if it is the root provider. */ - setDragging: (dragging: Dragging) => void; + setDragging: (dragging?: DragDropIdentifier) => void; + + activeDropTarget?: { + activeDropTarget?: DragDropIdentifier; + }; + + setActiveDropTarget: (newTarget?: DragDropIdentifier) => void; /** * The React children. */ children: React.ReactNode; + + setA11yMessage: (message: string) => void; } /** @@ -70,15 +109,60 @@ export interface ProviderProps { * @param props */ export function RootDragDropProvider({ children }: { children: React.ReactNode }) { - const [state, setState] = useState<{ dragging: Dragging }>({ + const [draggingState, setDraggingState] = useState<{ dragging?: DragDropIdentifier }>({ dragging: undefined, }); - const setDragging = useMemo(() => (dragging: Dragging) => setState({ dragging }), [setState]); + const [keyboardModeState, setKeyboardModeState] = useState(false); + const [a11yMessageState, setA11yMessageState] = useState(''); + const [activeDropTargetState, setActiveDropTargetState] = useState<{ + activeDropTarget?: DragDropIdentifier; + }>({ + activeDropTarget: undefined, + }); + + const setDragging = useMemo( + () => (dragging?: DragDropIdentifier) => setDraggingState({ dragging }), + [setDraggingState] + ); + + const setA11yMessage = useMemo(() => (message: string) => setA11yMessageState(message), [ + setA11yMessageState, + ]); + + const setActiveDropTarget = useMemo( + () => (activeDropTarget?: DragDropIdentifier) => + setActiveDropTargetState((s) => ({ ...s, activeDropTarget })), + [setActiveDropTargetState] + ); return ( - - {children} - +
+ + {children} + + + +
+

+ {a11yMessageState} +

+

+ {i18n.translate('xpack.lens.dragDrop.keyboardInstructions', { + defaultMessage: `Press enter or space to start reordering the dimension group. When dragging, use arrow keys to reorder. Press enter or space again to finish.`, + })} +

+
+ + +
); } @@ -89,8 +173,36 @@ export function RootDragDropProvider({ children }: { children: React.ReactNode } * * @param props */ -export function ChildDragDropProvider({ dragging, setDragging, children }: ProviderProps) { - const value = useMemo(() => ({ dragging, setDragging }), [setDragging, dragging]); +export function ChildDragDropProvider({ + dragging, + setDragging, + setKeyboardMode, + keyboardMode, + activeDropTarget, + setActiveDropTarget, + setA11yMessage, + children, +}: ProviderProps) { + const value = useMemo( + () => ({ + setKeyboardMode, + keyboardMode, + dragging, + setDragging, + activeDropTarget, + setActiveDropTarget, + setA11yMessage, + }), + [ + setDragging, + dragging, + activeDropTarget, + setActiveDropTarget, + setKeyboardMode, + keyboardMode, + setA11yMessage, + ] + ); return {children}; } @@ -98,7 +210,7 @@ export interface ReorderState { /** * Ids of the elements that are translated up or down */ - reorderedItems: string[]; + reorderedItems: DragDropIdentifier[]; /** * Direction of the move of dragged element in the reordered list @@ -112,10 +224,6 @@ export interface ReorderState { * indicates that user is in keyboard mode */ isReorderOn: boolean; - /** - * aria-live message for changes in reordering - */ - keyboardReorderMessage: string; /** * reorder group needed for screen reader aria-described-by attribute */ @@ -135,7 +243,6 @@ export const ReorderContext = React.createContext({ direction: '-', draggingHeight: 40, isReorderOn: false, - keyboardReorderMessage: '', groupId: '', }, setReorderState: () => () => {}, @@ -155,33 +262,70 @@ export function ReorderProvider({ direction: '-', draggingHeight: 40, isReorderOn: false, - keyboardReorderMessage: '', groupId: id, }); const setReorderState = useMemo(() => (dispatch: SetReorderStateDispatch) => setState(dispatch), [ setState, ]); - return ( -
+
1, + })} + > {children} - - -
-

- {state.keyboardReorderMessage} -

-

- {i18n.translate('xpack.lens.dragDrop.reorderInstructions', { - defaultMessage: `Press space bar to start a drag. When dragging, use arrow keys to reorder. Press space bar again to finish.`, - })} -

-
- -
); } + +export const reorderAnnouncements = { + moved: (itemLabel: string, position: number, prevPosition: number) => { + return prevPosition === position + ? i18n.translate('xpack.lens.dragDrop.elementMovedBack', { + defaultMessage: `You have moved back the item {itemLabel} to position {prevPosition}`, + values: { + itemLabel, + prevPosition, + }, + }) + : i18n.translate('xpack.lens.dragDrop.elementMoved', { + defaultMessage: `You have moved the item {itemLabel} from position {prevPosition} to position {position}`, + values: { + itemLabel, + position, + prevPosition, + }, + }); + }, + + lifted: (itemLabel: string, position: number) => + i18n.translate('xpack.lens.dragDrop.elementLifted', { + defaultMessage: `You have lifted an item {itemLabel} in position {position}`, + values: { + itemLabel, + position, + }, + }), + + cancelled: (position: number) => + i18n.translate('xpack.lens.dragDrop.abortMessageReorder', { + defaultMessage: + 'Movement cancelled. The item has returned to its starting position {position}', + values: { + position, + }, + }), + dropped: (position: number, prevPosition: number) => + i18n.translate('xpack.lens.dragDrop.dropMessageReorder', { + defaultMessage: + 'You have dropped the item. You have moved the item from position {prevPosition} to positon {position}', + values: { + position, + prevPosition, + }, + }), +}; diff --git a/x-pack/plugins/lens/public/drag_drop/readme.md b/x-pack/plugins/lens/public/drag_drop/readme.md index 1e812c7adac27..e48564a074986 100644 --- a/x-pack/plugins/lens/public/drag_drop/readme.md +++ b/x-pack/plugins/lens/public/drag_drop/readme.md @@ -30,7 +30,7 @@ In your child application, place a `ChildDragDropProvider` at the root of that, This enables your child application to share the same drag / drop context as the root application. -## Dragging +## DragDropIdentifier An item can be both draggable and droppable at the same time, but for simplicity's sake, we'll treat these two cases separately. @@ -88,7 +88,7 @@ The children `DragDrop` components must have props defined as in the example: droppable dragType="reorder" dropType="reorder" - itemsInGroup={fields.map((f) => f.id)} // consists ids of all reorderable elements in the group, eg. ['3', '5', '1'] + reorderableGroup={fields} // consists all reorderable elements in the group, eg. [{id:'3'}, {id:'5'}, {id:'1'}] value={{ id: f.id, }} diff --git a/x-pack/plugins/lens/public/editor_frame_service/editor_frame/config_panel/config_panel.test.tsx b/x-pack/plugins/lens/public/editor_frame_service/editor_frame/config_panel/config_panel.test.tsx index 70c4fb5567226..0ebcb5bb07482 100644 --- a/x-pack/plugins/lens/public/editor_frame_service/editor_frame/config_panel/config_panel.test.tsx +++ b/x-pack/plugins/lens/public/editor_frame_service/editor_frame/config_panel/config_panel.test.tsx @@ -92,6 +92,14 @@ describe('ConfigPanel', () => { mockDatasource = createMockDatasource('ds1'); }); + // in what case is this test needed? + it('should fail to render layerPanels if the public API is out of date', () => { + const props = getDefaultProps(); + props.framePublicAPI.datasourceLayers = {}; + const component = mountWithIntl(); + expect(component.find(LayerPanel).exists()).toBe(false); + }); + describe('focus behavior when adding or removing layers', () => { it('should focus the only layer when resetting the layer', () => { const component = mountWithIntl(); diff --git a/x-pack/plugins/lens/public/editor_frame_service/editor_frame/config_panel/config_panel.tsx b/x-pack/plugins/lens/public/editor_frame_service/editor_frame/config_panel/config_panel.tsx index ec1a5c226d351..67c8a6b5e4abc 100644 --- a/x-pack/plugins/lens/public/editor_frame_service/editor_frame/config_panel/config_panel.tsx +++ b/x-pack/plugins/lens/public/editor_frame_service/editor_frame/config_panel/config_panel.tsx @@ -134,37 +134,42 @@ export function LayerPanels( [dispatch] ); + const datasourcePublicAPIs = props.framePublicAPI.datasourceLayers; + return ( - {layerIds.map((layerId, index) => ( - { - dispatch({ - type: 'UPDATE_STATE', - subType: 'REMOVE_OR_CLEAR_LAYER', - updater: (state) => - removeLayer({ - activeVisualization, - layerId, - trackUiEvent, - datasourceMap, - state, - }), - }); - removeLayerRef(layerId); - }} - /> - ))} + {layerIds.map((layerId, layerIndex) => + datasourcePublicAPIs[layerId] ? ( + { + dispatch({ + type: 'UPDATE_STATE', + subType: 'REMOVE_OR_CLEAR_LAYER', + updater: (state) => + removeLayer({ + activeVisualization, + layerId, + trackUiEvent, + datasourceMap, + state, + }), + }); + removeLayerRef(layerId); + }} + /> + ) : null + )} {activeVisualization.appendLayer && visualizationState && ( + i18n.translate('xpack.lens.configure.editConfig', { + defaultMessage: 'Edit {label} configuration', + values: { label }, + }); + +export function DimensionButton({ + group, + children, + onClick, + onRemoveClick, + accessorConfig, + label, +}: { + group: VisualizationDimensionGroupConfig; + children: React.ReactElement; + onClick: (id: string) => void; + onRemoveClick: (id: string) => void; + accessorConfig: AccessorConfig; + label: string; +}) { + return ( + <> + onClick(accessorConfig.columnId)} + aria-label={triggerLinkA11yText(label)} + title={triggerLinkA11yText(label)} + > + {children} + + onRemoveClick(accessorConfig.columnId)} + /> + + + ); +} diff --git a/x-pack/plugins/lens/public/editor_frame_service/editor_frame/config_panel/draggable_dimension_button.tsx b/x-pack/plugins/lens/public/editor_frame_service/editor_frame/config_panel/draggable_dimension_button.tsx new file mode 100644 index 0000000000000..8de57cb43b16f --- /dev/null +++ b/x-pack/plugins/lens/public/editor_frame_service/editor_frame/config_panel/draggable_dimension_button.tsx @@ -0,0 +1,110 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License; + * you may not use this file except in compliance with the Elastic License. + */ + +import React, { useMemo } from 'react'; +import { DragDrop, DragDropIdentifier, DragContextState } from '../../../drag_drop'; +import { Datasource, VisualizationDimensionGroupConfig, isDraggedOperation } from '../../../types'; +import { LayerDatasourceDropProps } from './types'; + +const isFromTheSameGroup = (el1: DragDropIdentifier, el2?: DragDropIdentifier) => + el2 && isDraggedOperation(el2) && el1.groupId === el2.groupId && el1.columnId !== el2.columnId; + +const isSelf = (el1: DragDropIdentifier, el2?: DragDropIdentifier) => + isDraggedOperation(el2) && el1.columnId === el2.columnId; + +export function DraggableDimensionButton({ + layerId, + label, + accessorIndex, + groupIndex, + layerIndex, + columnId, + group, + onDrop, + children, + dragDropContext, + layerDatasourceDropProps, + layerDatasource, +}: { + dragDropContext: DragContextState; + layerId: string; + groupIndex: number; + layerIndex: number; + onDrop: (droppedItem: DragDropIdentifier, dropTarget: DragDropIdentifier) => void; + group: VisualizationDimensionGroupConfig; + label: string; + children: React.ReactElement; + layerDatasource: Datasource; + layerDatasourceDropProps: LayerDatasourceDropProps; + accessorIndex: number; + columnId: string; +}) { + const value = useMemo(() => { + return { + columnId, + groupId: group.groupId, + layerId, + id: columnId, + }; + }, [columnId, group.groupId, layerId]); + + const { dragging } = dragDropContext; + + const isCurrentGroup = group.groupId === dragging?.groupId; + const isOperationDragged = isDraggedOperation(dragging); + const canHandleDrop = + Boolean(dragDropContext.dragging) && + layerDatasource.canHandleDrop({ + ...layerDatasourceDropProps, + columnId, + filterOperations: group.filterOperations, + }); + + const dragType = isSelf(value, dragging) + ? 'move' + : isOperationDragged && isCurrentGroup + ? 'reorder' + : 'copy'; + + const dropType = isOperationDragged ? (!isCurrentGroup ? 'replace' : 'reorder') : 'add'; + + const isCompatibleFromOtherGroup = !isCurrentGroup && canHandleDrop; + + const isDroppable = isOperationDragged + ? dragType === 'reorder' + ? isFromTheSameGroup(value, dragging) + : isCompatibleFromOtherGroup + : canHandleDrop; + + const reorderableGroup = useMemo( + () => + group.accessors.map((a) => ({ + columnId: a.columnId, + id: a.columnId, + groupId: group.groupId, + layerId, + })), + [group, layerId] + ); + + return ( +
+ 1 ? reorderableGroup : undefined} + value={value} + label={label} + droppable={dragging && isDroppable} + onDrop={onDrop} + > + {children} + +
+ ); +} diff --git a/x-pack/plugins/lens/public/editor_frame_service/editor_frame/config_panel/empty_dimension_button.tsx b/x-pack/plugins/lens/public/editor_frame_service/editor_frame/config_panel/empty_dimension_button.tsx new file mode 100644 index 0000000000000..88e1663d0b49c --- /dev/null +++ b/x-pack/plugins/lens/public/editor_frame_service/editor_frame/config_panel/empty_dimension_button.tsx @@ -0,0 +1,97 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License; + * you may not use this file except in compliance with the Elastic License. + */ + +import React, { useMemo } from 'react'; +import { EuiButtonEmpty } from '@elastic/eui'; +import { FormattedMessage } from '@kbn/i18n/react'; +import { i18n } from '@kbn/i18n'; +import { generateId } from '../../../id_generator'; +import { DragDrop, DragDropIdentifier, DragContextState } from '../../../drag_drop'; +import { Datasource, VisualizationDimensionGroupConfig, isDraggedOperation } from '../../../types'; +import { LayerDatasourceDropProps } from './types'; + +export function EmptyDimensionButton({ + dragDropContext, + group, + layerDatasource, + layerDatasourceDropProps, + layerId, + groupIndex, + layerIndex, + onClick, + onDrop, +}: { + dragDropContext: DragContextState; + layerId: string; + groupIndex: number; + layerIndex: number; + onClick: (id: string) => void; + onDrop: (droppedItem: DragDropIdentifier, dropTarget: DragDropIdentifier) => void; + group: VisualizationDimensionGroupConfig; + + layerDatasource: Datasource; + layerDatasourceDropProps: LayerDatasourceDropProps; +}) { + const handleDrop = (droppedItem: DragDropIdentifier) => onDrop(droppedItem, value); + + const value = useMemo(() => { + const newId = generateId(); + return { + columnId: newId, + groupId: group.groupId, + layerId, + isNew: true, + id: newId, + }; + // eslint-disable-next-line react-hooks/exhaustive-deps + }, [group.accessors.length, group.groupId, layerId]); + + return ( +
+ +
+ { + onClick(value.columnId); + }} + > + + +
+ +
+ ); +} diff --git a/x-pack/plugins/lens/public/editor_frame_service/editor_frame/config_panel/layer_panel.scss b/x-pack/plugins/lens/public/editor_frame_service/editor_frame/config_panel/layer_panel.scss index 2ed91b962ff11..ec4c2adba8fd7 100644 --- a/x-pack/plugins/lens/public/editor_frame_service/editor_frame/config_panel/layer_panel.scss +++ b/x-pack/plugins/lens/public/editor_frame_service/editor_frame/config_panel/layer_panel.scss @@ -76,6 +76,7 @@ .lnsLayerPanel__dimensionContainer { margin: 0 $euiSizeS $euiSizeS; + position: relative; &:last-child { margin-bottom: 0; @@ -127,12 +128,13 @@ } } -.lnsLayerPanel__dimensionLink { +// Added .lnsLayerPanel__dimension specificity required for animation style override +.lnsLayerPanel__dimension .lnsLayerPanel__dimensionLink { width: 100%; &:focus { - background-color: transparent !important; // sass-lint:disable-line no-important - outline: none !important; // sass-lint:disable-line no-important + background-color: transparent; + animation: none !important; // sass-lint:disable-line no-important } &:focus .lnsLayerPanel__triggerTextLabel, diff --git a/x-pack/plugins/lens/public/editor_frame_service/editor_frame/config_panel/layer_panel.test.tsx b/x-pack/plugins/lens/public/editor_frame_service/editor_frame/config_panel/layer_panel.test.tsx index cab07150b6d56..d93cbbb58835e 100644 --- a/x-pack/plugins/lens/public/editor_frame_service/editor_frame/config_panel/layer_panel.test.tsx +++ b/x-pack/plugins/lens/public/editor_frame_service/editor_frame/config_panel/layer_panel.test.tsx @@ -12,7 +12,7 @@ import { createMockDatasource, DatasourceMock, } from '../../mocks'; -import { ChildDragDropProvider, DroppableEvent } from '../../../drag_drop'; +import { ChildDragDropProvider, DroppableEvent, DragDrop } from '../../../drag_drop'; import { EuiFormRow } from '@elastic/eui'; import { mountWithIntl } from '@kbn/test/jest'; import { Visualization } from '../../../types'; @@ -22,9 +22,20 @@ import { generateId } from '../../../id_generator'; jest.mock('../../../id_generator'); +const defaultContext = { + dragging: undefined, + setDragging: jest.fn(), + setActiveDropTarget: () => {}, + activeDropTarget: undefined, + keyboardMode: false, + setKeyboardMode: () => {}, + setA11yMessage: jest.fn(), +}; + describe('LayerPanel', () => { let mockVisualization: jest.Mocked; let mockVisualization2: jest.Mocked; + let mockDatasource: DatasourceMock; function getDefaultProps() { @@ -34,11 +45,7 @@ describe('LayerPanel', () => { }; return { layerId: 'first', - activeVisualizationId: 'vis1', - visualizationMap: { - vis1: mockVisualization, - vis2: mockVisualization2, - }, + activeVisualization: mockVisualization, activeDatasourceId: 'ds1', datasourceMap: { ds1: mockDatasource, @@ -58,7 +65,7 @@ describe('LayerPanel', () => { onRemoveLayer: jest.fn(), dispatch: jest.fn(), core: coreMock.createStart(), - index: 0, + layerIndex: 0, setLayerRef: jest.fn(), }; } @@ -92,20 +99,6 @@ describe('LayerPanel', () => { mockDatasource = createMockDatasource('ds1'); }); - it('should fail to render if the public API is out of date', () => { - const props = getDefaultProps(); - props.framePublicAPI.datasourceLayers = {}; - const component = mountWithIntl(); - expect(component.isEmptyRender()).toBe(true); - }); - - it('should fail to render if the active visualization is missing', () => { - const component = mountWithIntl( - - ); - expect(component.isEmptyRender()).toBe(true); - }); - describe('layer reset and remove', () => { it('should show the reset button when single layer', () => { const component = mountWithIntl(); @@ -147,8 +140,7 @@ describe('LayerPanel', () => { }); const component = mountWithIntl(); - - const group = component.find('DragDrop[data-test-subj="lnsGroup"]'); + const group = component.find('.lnsLayerPanel__dimensionContainer[data-test-subj="lnsGroup"]'); expect(group).toHaveLength(1); }); @@ -167,8 +159,7 @@ describe('LayerPanel', () => { }); const component = mountWithIntl(); - - const group = component.find('DragDrop[data-test-subj="lnsGroup"]'); + const group = component.find('.lnsLayerPanel__dimensionContainer[data-test-subj="lnsGroup"]'); expect(group).toHaveLength(1); }); @@ -231,50 +222,6 @@ describe('LayerPanel', () => { expect(panel.props.children).toHaveLength(2); }); - it('should keep the DimensionContainer open when configuring a new dimension', () => { - /** - * The ID generation system for new dimensions has been messy before, so - * this tests that the ID used in the first render is used to keep the container - * open in future renders - */ - (generateId as jest.Mock).mockReturnValueOnce(`newid`); - (generateId as jest.Mock).mockReturnValueOnce(`bad`); - mockVisualization.getConfiguration.mockReturnValueOnce({ - groups: [ - { - groupLabel: 'A', - groupId: 'a', - accessors: [], - filterOperations: () => true, - supportsMoreColumns: true, - dataTestSubj: 'lnsGroup', - }, - ], - }); - // Normally the configuration would change in response to a state update, - // but this test is updating it directly - mockVisualization.getConfiguration.mockReturnValueOnce({ - groups: [ - { - groupLabel: 'A', - groupId: 'a', - accessors: [{ columnId: 'newid' }], - filterOperations: () => true, - supportsMoreColumns: false, - dataTestSubj: 'lnsGroup', - }, - ], - }); - - const component = mountWithIntl(); - act(() => { - component.find('[data-test-subj="lns-empty-dimension"]').first().simulate('click'); - }); - component.update(); - - expect(component.find('EuiFlyoutHeader').exists()).toBe(true); - }); - it('should not update the visualization if the datasource is incomplete', () => { (generateId as jest.Mock).mockReturnValueOnce(`newid`); const updateAll = jest.fn(); @@ -338,6 +285,50 @@ describe('LayerPanel', () => { expect(updateAll).toHaveBeenCalled(); }); + it('should keep the DimensionContainer open when configuring a new dimension', () => { + /** + * The ID generation system for new dimensions has been messy before, so + * this tests that the ID used in the first render is used to keep the container + * open in future renders + */ + (generateId as jest.Mock).mockReturnValueOnce(`newid`); + (generateId as jest.Mock).mockReturnValueOnce(`bad`); + mockVisualization.getConfiguration.mockReturnValueOnce({ + groups: [ + { + groupLabel: 'A', + groupId: 'a', + accessors: [], + filterOperations: () => true, + supportsMoreColumns: true, + dataTestSubj: 'lnsGroup', + }, + ], + }); + // Normally the configuration would change in response to a state update, + // but this test is updating it directly + mockVisualization.getConfiguration.mockReturnValueOnce({ + groups: [ + { + groupLabel: 'A', + groupId: 'a', + accessors: [{ columnId: 'newid' }], + filterOperations: () => true, + supportsMoreColumns: false, + dataTestSubj: 'lnsGroup', + }, + ], + }); + + const component = mountWithIntl(); + act(() => { + component.find('[data-test-subj="lns-empty-dimension"]').first().simulate('click'); + }); + component.update(); + + expect(component.find('EuiFlyoutHeader').exists()).toBe(true); + }); + it('should close the DimensionContainer when the active visualization changes', () => { /** * The ID generation system for new dimensions has been messy before, so @@ -361,7 +352,7 @@ describe('LayerPanel', () => { }); // Normally the configuration would change in response to a state update, // but this test is updating it directly - mockVisualization.getConfiguration.mockReturnValueOnce({ + mockVisualization.getConfiguration.mockReturnValue({ groups: [ { groupLabel: 'A', @@ -382,7 +373,7 @@ describe('LayerPanel', () => { component.update(); expect(component.find('EuiFlyoutHeader').exists()).toBe(true); act(() => { - component.setProps({ activeVisualizationId: 'vis2' }); + component.setProps({ activeVisualization: mockVisualization2 }); }); component.update(); expect(component.find('EuiFlyoutHeader').exists()).toBe(false); @@ -452,7 +443,7 @@ describe('LayerPanel', () => { const draggingField = { field: { name: 'dragged' }, indexPatternId: 'a', id: '1' }; const component = mountWithIntl( - + ); @@ -465,7 +456,7 @@ describe('LayerPanel', () => { }) ); - component.find('DragDrop[data-test-subj="lnsGroup"]').first().simulate('drop'); + component.find('[data-test-subj="lnsGroup"] DragDrop').first().simulate('drop'); expect(mockDatasource.onDrop).toHaveBeenCalledWith( expect.objectContaining({ @@ -495,7 +486,7 @@ describe('LayerPanel', () => { const draggingField = { field: { name: 'dragged' }, indexPatternId: 'a', id: '1' }; const component = mountWithIntl( - + ); @@ -505,10 +496,14 @@ describe('LayerPanel', () => { ); expect( - component.find('DragDrop[data-test-subj="lnsGroup"]').first().prop('droppable') + component.find('[data-test-subj="lnsGroup"] DragDrop').first().prop('droppable') ).toEqual(false); - component.find('DragDrop[data-test-subj="lnsGroup"]').first().simulate('drop'); + component + .find('[data-test-subj="lnsGroup"] DragDrop') + .first() + .find('.lnsLayerPanel__dimension') + .simulate('drop'); expect(mockDatasource.onDrop).not.toHaveBeenCalled(); }); @@ -542,12 +537,11 @@ describe('LayerPanel', () => { const draggingOperation = { layerId: 'first', columnId: 'a', groupId: 'a', id: 'a' }; const component = mountWithIntl( - + ); - expect(mockDatasource.canHandleDrop).toHaveBeenCalledTimes(2); expect(mockDatasource.canHandleDrop).toHaveBeenCalledWith( expect.objectContaining({ dragDropContext: expect.objectContaining({ @@ -557,7 +551,7 @@ describe('LayerPanel', () => { ); // Simulate drop on the pre-populated dimension - component.find('DragDrop[data-test-subj="lnsGroupB"]').at(0).simulate('drop'); + component.find('[data-test-subj="lnsGroupB"] DragDrop').at(0).simulate('drop'); expect(mockDatasource.onDrop).toHaveBeenCalledWith( expect.objectContaining({ columnId: 'b', @@ -568,7 +562,7 @@ describe('LayerPanel', () => { ); // Simulate drop on the empty dimension - component.find('DragDrop[data-test-subj="lnsGroupB"]').at(1).simulate('drop'); + component.find('[data-test-subj="lnsGroupB"] DragDrop').at(1).simulate('drop'); expect(mockDatasource.onDrop).toHaveBeenCalledWith( expect.objectContaining({ columnId: 'newid', @@ -596,18 +590,55 @@ describe('LayerPanel', () => { const draggingOperation = { layerId: 'first', columnId: 'a', groupId: 'a', id: 'a' }; const component = mountWithIntl( - + + + + ); + + component.find(DragDrop).at(1).prop('onDrop')!(draggingOperation, { + layerId: 'first', + columnId: 'b', + groupId: 'a', + id: 'b', + }); + expect(mockDatasource.onDrop).toHaveBeenCalledWith( + expect.objectContaining({ + groupId: 'a', + droppedItem: draggingOperation, + }) + ); + }); + + it('should copy when dropping on empty slot in the same group', () => { + mockVisualization.getConfiguration.mockReturnValue({ + groups: [ + { + groupLabel: 'A', + groupId: 'a', + accessors: [{ columnId: 'a' }, { columnId: 'b' }], + filterOperations: () => true, + supportsMoreColumns: true, + dataTestSubj: 'lnsGroup', + }, + ], + }); + + const draggingOperation = { layerId: 'first', columnId: 'a', groupId: 'a', id: 'a' }; + + const component = mountWithIntl( + ); - expect(mockDatasource.canHandleDrop).not.toHaveBeenCalled(); - component.find('DragDrop[data-test-subj="lnsGroup"]').at(1).prop('onDrop')!( + component.find('[data-test-subj="lnsGroup"] DragDrop').at(2).prop('onDrop')!( (draggingOperation as unknown) as DroppableEvent ); expect(mockDatasource.onDrop).toHaveBeenCalledWith( expect.objectContaining({ - isReorder: true, + groupId: 'a', + droppedItem: draggingOperation, + isNew: true, }) ); }); diff --git a/x-pack/plugins/lens/public/editor_frame_service/editor_frame/config_panel/layer_panel.tsx b/x-pack/plugins/lens/public/editor_frame_service/editor_frame/config_panel/layer_panel.tsx index 999f75686b1cb..a1b13878851ee 100644 --- a/x-pack/plugins/lens/public/editor_frame_service/editor_frame/config_panel/layer_panel.tsx +++ b/x-pack/plugins/lens/public/editor_frame_service/editor_frame/config_panel/layer_panel.tsx @@ -5,66 +5,35 @@ */ import './layer_panel.scss'; -import React, { useContext, useState, useEffect } from 'react'; -import { - EuiPanel, - EuiSpacer, - EuiButtonIcon, - EuiFlexGroup, - EuiFlexItem, - EuiButtonEmpty, - EuiFormRow, - EuiLink, -} from '@elastic/eui'; +import React, { useContext, useState, useEffect, useMemo, useCallback } from 'react'; +import { EuiPanel, EuiSpacer, EuiFlexGroup, EuiFlexItem, EuiFormRow } from '@elastic/eui'; import { i18n } from '@kbn/i18n'; -import { FormattedMessage } from '@kbn/i18n/react'; import { NativeRenderer } from '../../../native_renderer'; -import { StateSetter, isDraggedOperation } from '../../../types'; -import { DragContext, DragDrop, ChildDragDropProvider, ReorderProvider } from '../../../drag_drop'; +import { StateSetter, Visualization } from '../../../types'; +import { + DragContext, + DragDropIdentifier, + ChildDragDropProvider, + ReorderProvider, +} from '../../../drag_drop'; import { LayerSettings } from './layer_settings'; import { trackUiEvent } from '../../../lens_ui_telemetry'; -import { generateId } from '../../../id_generator'; -import { ConfigPanelWrapperProps, ActiveDimensionState } from './types'; +import { LayerPanelProps, ActiveDimensionState } from './types'; import { DimensionContainer } from './dimension_container'; -import { ColorIndicator } from './color_indicator'; -import { PaletteIndicator } from './palette_indicator'; - -const triggerLinkA11yText = (label: string) => - i18n.translate('xpack.lens.configure.editConfig', { - defaultMessage: 'Click to edit configuration for {label} or drag to move', - values: { label }, - }); +import { RemoveLayerButton } from './remove_layer_button'; +import { EmptyDimensionButton } from './empty_dimension_button'; +import { DimensionButton } from './dimension_button'; +import { DraggableDimensionButton } from './draggable_dimension_button'; const initialActiveDimensionState = { isNew: false, }; -function isConfiguration( - value: unknown -): value is { columnId: string; groupId: string; layerId: string } { - return ( - Boolean(value) && - typeof value === 'object' && - 'columnId' in value! && - 'groupId' in value && - 'layerId' in value - ); -} - -function isSameConfiguration(config1: unknown, config2: unknown) { - return ( - isConfiguration(config1) && - isConfiguration(config2) && - config1.columnId === config2.columnId && - config1.groupId === config2.groupId && - config1.layerId === config2.layerId - ); -} - export function LayerPanel( - props: Exclude & { + props: Exclude & { + activeVisualization: Visualization; layerId: string; - index: number; + layerIndex: number; isOnlyLayer: boolean; updateVisualization: StateSetter; updateDatasource: (datasourceId: string, newState: unknown) => void; @@ -82,26 +51,25 @@ export function LayerPanel( initialActiveDimensionState ); - const { framePublicAPI, layerId, isOnlyLayer, onRemoveLayer, setLayerRef, index } = props; + const { + framePublicAPI, + layerId, + isOnlyLayer, + onRemoveLayer, + setLayerRef, + layerIndex, + activeVisualization, + updateVisualization, + updateDatasource, + } = props; const datasourcePublicAPI = framePublicAPI.datasourceLayers[layerId]; useEffect(() => { setActiveDimension(initialActiveDimensionState); - }, [props.activeVisualizationId]); + }, [activeVisualization.id]); - const setLayerRefMemoized = React.useCallback((el) => setLayerRef(layerId, el), [ - layerId, - setLayerRef, - ]); + const setLayerRefMemoized = useCallback((el) => setLayerRef(layerId, el), [layerId, setLayerRef]); - if ( - !datasourcePublicAPI || - !props.activeVisualizationId || - !props.visualizationMap[props.activeVisualizationId] - ) { - return null; - } - const activeVisualization = props.visualizationMap[props.activeVisualizationId]; const layerVisualizationConfigProps = { layerId, dragDropContext, @@ -110,18 +78,23 @@ export function LayerPanel( dateRange: props.framePublicAPI.dateRange, activeData: props.framePublicAPI.activeData, }; + const datasourceId = datasourcePublicAPI.datasourceId; const layerDatasourceState = props.datasourceStates[datasourceId].state; - const layerDatasource = props.datasourceMap[datasourceId]; - const layerDatasourceDropProps = { - layerId, - dragDropContext, - state: layerDatasourceState, - setState: (newState: unknown) => { - props.updateDatasource(datasourceId, newState); - }, - }; + const layerDatasourceDropProps = useMemo( + () => ({ + layerId, + dragDropContext, + state: layerDatasourceState, + setState: (newState: unknown) => { + updateDatasource(datasourceId, newState); + }, + }), + [layerId, dragDropContext, layerDatasourceState, datasourceId, updateDatasource] + ); + + const layerDatasource = props.datasourceMap[datasourceId]; const layerDatasourceConfigProps = { ...layerDatasourceDropProps, @@ -135,10 +108,68 @@ export function LayerPanel( const { activeId, activeGroup } = activeDimension; const columnLabelMap = layerDatasource.uniqueLabels(layerDatasourceConfigProps.state); + + const { setDimension, removeDimension } = activeVisualization; + const layerDatasourceOnDrop = layerDatasource.onDrop; + + const onDrop = useMemo(() => { + return (droppedItem: DragDropIdentifier, targetItem: DragDropIdentifier) => { + const { columnId, groupId, layerId: targetLayerId, isNew } = (targetItem as unknown) as { + groupId: string; + columnId: string; + layerId: string; + isNew?: boolean; + }; + + const filterOperations = + groups.find(({ groupId: gId }) => gId === targetItem.groupId)?.filterOperations || + (() => false); + + const dropResult = layerDatasourceOnDrop({ + ...layerDatasourceDropProps, + droppedItem, + columnId, + groupId, + layerId: targetLayerId, + isNew, + filterOperations, + }); + if (dropResult) { + updateVisualization( + setDimension({ + columnId, + groupId, + layerId: targetLayerId, + prevState: props.visualizationState, + }) + ); + + if (typeof dropResult === 'object') { + // When a column is moved, we delete the reference to the old + updateVisualization( + removeDimension({ + columnId: dropResult.deleted, + layerId: targetLayerId, + prevState: props.visualizationState, + }) + ); + } + } + }; + }, [ + groups, + layerDatasourceOnDrop, + props.visualizationState, + updateVisualization, + setDimension, + removeDimension, + layerDatasourceDropProps, + ]); + return (
- + - {groups.map((group) => { - const newId = generateId(); + {groups.map((group, groupIndex) => { const isMissing = !isEmptyLayer && group.required && group.accessors.length === 0; - return ( <> - - {group.accessors.map((accessorConfig) => { - const accessor = accessorConfig.columnId; - const { dragging } = dragDropContext; - const dragType = - isDraggedOperation(dragging) && accessor === dragging.columnId - ? 'move' - : isDraggedOperation(dragging) && group.groupId === dragging.groupId - ? 'reorder' - : 'copy'; - - const dropType = isDraggedOperation(dragging) - ? group.groupId !== dragging.groupId - ? 'replace' - : 'reorder' - : 'add'; - - const isFromCompatibleGroup = - dragging?.groupId !== group.groupId && - layerDatasource.canHandleDrop({ - ...layerDatasourceDropProps, - columnId: accessor, - filterOperations: group.filterOperations, - }); - - const isFromTheSameGroup = - isDraggedOperation(dragging) && - dragging.groupId === group.groupId && - dragging.columnId !== accessor; - - const isDroppable = isDraggedOperation(dragging) - ? dragType === 'reorder' - ? isFromTheSameGroup - : isFromCompatibleGroup - : layerDatasource.canHandleDrop({ - ...layerDatasourceDropProps, - columnId: accessor, - filterOperations: group.filterOperations, - }); + + {group.accessors.map((accessorConfig, accessorIndex) => { + const { columnId } = accessorConfig; return ( - - typeof a === 'string' ? a : a.columnId - )} - className={'lnsLayerPanel__dimensionContainer'} - value={{ - columnId: accessor, - groupId: group.groupId, - layerId, - id: accessor, - }} - isValueEqual={isSameConfiguration} - label={columnLabelMap[accessor]} - droppable={dragging && isDroppable} - dropTo={(dropTargetId: string) => { - layerDatasource.onDrop({ - isReorder: true, - ...layerDatasourceDropProps, - droppedItem: { - columnId: accessor, - groupId: group.groupId, - layerId, - id: accessor, - }, - columnId: dropTargetId, - filterOperations: group.filterOperations, - }); - }} - onDrop={(droppedItem) => { - const isReorder = - isDraggedOperation(droppedItem) && - droppedItem.groupId === group.groupId && - droppedItem.columnId !== accessor; - - const dropResult = layerDatasource.onDrop({ - isReorder, - ...layerDatasourceDropProps, - droppedItem, - columnId: accessor, - filterOperations: group.filterOperations, - }); - if (typeof dropResult === 'object') { - // When a column is moved, we delete the reference to the old - props.updateVisualization( - activeVisualization.removeDimension({ - layerId, - columnId: dropResult.deleted, - prevState: props.visualizationState, - }) - ); - } - }} +
- { - if (activeId) { - setActiveDimension(initialActiveDimensionState); - } else { - setActiveDimension({ - isNew: false, - activeGroup: group, - activeId: accessor, - }); - } + { + setActiveDimension({ + isNew: false, + activeGroup: group, + activeId: id, + }); }} - aria-label={triggerLinkA11yText(columnLabelMap[accessor])} - title={triggerLinkA11yText(columnLabelMap[accessor])} - > - - - - - { + onRemoveClick={(id: string) => { trackUiEvent('indexpattern_dimension_removed'); props.updateAll( datasourceId, layerDatasource.removeColumn({ layerId, - columnId: accessor, + columnId: id, prevState: layerDatasourceState, }), activeVisualization.removeDimension({ layerId, - columnId: accessor, + columnId: id, prevState: props.visualizationState, }) ); }} - /> - + > + +
- + ); })} {group.supportsMoreColumns ? ( -
- { - const dropResult = layerDatasource.onDrop({ - ...layerDatasourceDropProps, - droppedItem, - columnId: newId, - filterOperations: group.filterOperations, - }); - if (dropResult) { - props.updateVisualization( - activeVisualization.setDimension({ - layerId, - groupId: group.groupId, - columnId: newId, - prevState: props.visualizationState, - }) - ); - - if (typeof dropResult === 'object') { - // When a column is moved, we delete the reference to the old - props.updateVisualization( - activeVisualization.removeDimension({ - layerId, - columnId: dropResult.deleted, - prevState: props.visualizationState, - }) - ); - } - } - }} - > -
- { - if (activeId) { - setActiveDimension(initialActiveDimensionState); - } else { - setActiveDimension({ - isNew: true, - activeGroup: group, - activeId: newId, - }); - } - }} - > - - -
- -
+ { + setActiveDimension({ + activeGroup: group, + activeId: id, + isNew: true, + }); + }} + onDrop={onDrop} + /> ) : null} @@ -572,44 +426,11 @@ export function LayerPanel( - { - // If we don't blur the remove / clear button, it remains focused - // which is a strange UX in this case. e.target.blur doesn't work - // due to who knows what, but probably event re-writing. Additionally, - // activeElement does not have blur so, we need to do some casting + safeguards. - const el = (document.activeElement as unknown) as { blur: () => void }; - - if (el?.blur) { - el.blur(); - } - - onRemoveLayer(); - }} - > - {isOnlyLayer - ? i18n.translate('xpack.lens.resetLayer', { - defaultMessage: 'Reset layer', - }) - : i18n.translate('xpack.lens.deleteLayer', { - defaultMessage: `Delete layer`, - })} - + diff --git a/x-pack/plugins/lens/public/editor_frame_service/editor_frame/config_panel/remove_layer_button.tsx b/x-pack/plugins/lens/public/editor_frame_service/editor_frame/config_panel/remove_layer_button.tsx new file mode 100644 index 0000000000000..526e2fcefe19d --- /dev/null +++ b/x-pack/plugins/lens/public/editor_frame_service/editor_frame/config_panel/remove_layer_button.tsx @@ -0,0 +1,60 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License; + * you may not use this file except in compliance with the Elastic License. + */ + +import React from 'react'; +import { EuiButtonEmpty } from '@elastic/eui'; +import { i18n } from '@kbn/i18n'; + +export function RemoveLayerButton({ + onRemoveLayer, + layerIndex, + isOnlyLayer, +}: { + onRemoveLayer: () => void; + layerIndex: number; + isOnlyLayer: boolean; +}) { + return ( + { + // If we don't blur the remove / clear button, it remains focused + // which is a strange UX in this case. e.target.blur doesn't work + // due to who knows what, but probably event re-writing. Additionally, + // activeElement does not have blur so, we need to do some casting + safeguards. + const el = (document.activeElement as unknown) as { blur: () => void }; + + if (el?.blur) { + el.blur(); + } + + onRemoveLayer(); + }} + > + {isOnlyLayer + ? i18n.translate('xpack.lens.resetLayer', { + defaultMessage: 'Reset layer', + }) + : i18n.translate('xpack.lens.deleteLayer', { + defaultMessage: `Delete layer`, + })} + + ); +} diff --git a/x-pack/plugins/lens/public/editor_frame_service/editor_frame/config_panel/types.ts b/x-pack/plugins/lens/public/editor_frame_service/editor_frame/config_panel/types.ts index c172c6da6848c..0a53fc741c207 100644 --- a/x-pack/plugins/lens/public/editor_frame_service/editor_frame/config_panel/types.ts +++ b/x-pack/plugins/lens/public/editor_frame_service/editor_frame/config_panel/types.ts @@ -12,7 +12,7 @@ import { DatasourceDimensionEditorProps, VisualizationDimensionGroupConfig, } from '../../../types'; - +import { DragContextState } from '../../../drag_drop'; export interface ConfigPanelWrapperProps { activeDatasourceId: string; visualizationState: unknown; @@ -31,6 +31,30 @@ export interface ConfigPanelWrapperProps { core: DatasourceDimensionEditorProps['core']; } +export interface LayerPanelProps { + activeDatasourceId: string; + visualizationState: unknown; + datasourceMap: Record; + activeVisualization: Visualization; + dispatch: (action: Action) => void; + framePublicAPI: FramePublicAPI; + datasourceStates: Record< + string, + { + isLoading: boolean; + state: unknown; + } + >; + core: DatasourceDimensionEditorProps['core']; +} + +export interface LayerDatasourceDropProps { + layerId: string; + dragDropContext: DragContextState; + state: unknown; + setState: (newState: unknown) => void; +} + export interface ActiveDimensionState { isNew: boolean; activeId?: string; diff --git a/x-pack/plugins/lens/public/editor_frame_service/editor_frame/data_panel_wrapper.tsx b/x-pack/plugins/lens/public/editor_frame_service/editor_frame/data_panel_wrapper.tsx index 69bdff0151f6c..c45dc82a3aeb2 100644 --- a/x-pack/plugins/lens/public/editor_frame_service/editor_frame/data_panel_wrapper.tsx +++ b/x-pack/plugins/lens/public/editor_frame_service/editor_frame/data_panel_wrapper.tsx @@ -11,7 +11,7 @@ import { i18n } from '@kbn/i18n'; import { EuiPopover, EuiButtonIcon, EuiContextMenuPanel, EuiContextMenuItem } from '@elastic/eui'; import { NativeRenderer } from '../../native_renderer'; import { Action } from './state_management'; -import { DragContext, Dragging } from '../../drag_drop'; +import { DragContext, DragDropIdentifier } from '../../drag_drop'; import { StateSetter, FramePublicAPI, DatasourceDataPanelProps, Datasource } from '../../types'; import { Query, Filter } from '../../../../../../src/plugins/data/public'; @@ -26,8 +26,8 @@ interface DataPanelWrapperProps { query: Query; dateRange: FramePublicAPI['dateRange']; filters: Filter[]; - dropOntoWorkspace: (field: Dragging) => void; - hasSuggestionForField: (field: Dragging) => boolean; + dropOntoWorkspace: (field: DragDropIdentifier) => void; + hasSuggestionForField: (field: DragDropIdentifier) => boolean; } export const DataPanelWrapper = memo((props: DataPanelWrapperProps) => { diff --git a/x-pack/plugins/lens/public/editor_frame_service/editor_frame/editor_frame.test.tsx b/x-pack/plugins/lens/public/editor_frame_service/editor_frame/editor_frame.test.tsx index c0728bd030a0a..7daf1ebb17b97 100644 --- a/x-pack/plugins/lens/public/editor_frame_service/editor_frame/editor_frame.test.tsx +++ b/x-pack/plugins/lens/public/editor_frame_service/editor_frame/editor_frame.test.tsx @@ -1338,10 +1338,14 @@ describe('editor_frame', () => { instance.update(); act(() => { - instance.find(DragDrop).filter('[data-test-subj="mockVisA"]').prop('onDrop')!({ - indexPatternId: '1', - field: {}, - }); + instance.find('[data-test-subj="mockVisA"]').find(DragDrop).prop('onDrop')!( + { + indexPatternId: '1', + field: {}, + id: '1', + }, + { id: 'lnsWorkspace' } + ); }); expect(mockVisualization2.getConfiguration).toHaveBeenCalledWith( @@ -1435,10 +1439,14 @@ describe('editor_frame', () => { instance.update(); act(() => { - instance.find(DragDrop).filter('[data-test-subj="lnsWorkspace"]').prop('onDrop')!({ - indexPatternId: '1', - field: {}, - }); + instance.find(DragDrop).filter('[dataTestSubj="lnsWorkspace"]').prop('onDrop')!( + { + indexPatternId: '1', + field: {}, + id: '1', + }, + { id: 'lnsWorkspace' } + ); }); expect(mockVisualization3.getConfiguration).toHaveBeenCalledWith( diff --git a/x-pack/plugins/lens/public/editor_frame_service/editor_frame/editor_frame.tsx b/x-pack/plugins/lens/public/editor_frame_service/editor_frame/editor_frame.tsx index b6df0caa07577..c3412c32c2184 100644 --- a/x-pack/plugins/lens/public/editor_frame_service/editor_frame/editor_frame.tsx +++ b/x-pack/plugins/lens/public/editor_frame_service/editor_frame/editor_frame.tsx @@ -4,7 +4,7 @@ * you may not use this file except in compliance with the Elastic License. */ -import React, { useEffect, useReducer, useState } from 'react'; +import React, { useEffect, useReducer, useState, useCallback } from 'react'; import { CoreSetup, CoreStart } from 'kibana/public'; import { PaletteRegistry } from 'src/plugins/charts/public'; import { ReactExpressionRendererType } from '../../../../../../src/plugins/expressions/public'; @@ -16,7 +16,7 @@ import { FrameLayout } from './frame_layout'; import { SuggestionPanel } from './suggestion_panel'; import { WorkspacePanel } from './workspace_panel'; import { Document } from '../../persistence/saved_object_store'; -import { Dragging, RootDragDropProvider } from '../../drag_drop'; +import { DragDropIdentifier, RootDragDropProvider } from '../../drag_drop'; import { getSavedObjectFormat } from './save'; import { generateId } from '../../id_generator'; import { Filter, Query, SavedQuery } from '../../../../../../src/plugins/data/public'; @@ -260,7 +260,7 @@ export function EditorFrame(props: EditorFrameProps) { ); const getSuggestionForField = React.useCallback( - (field: Dragging) => { + (field: DragDropIdentifier) => { const { activeDatasourceId, datasourceStates } = state; const activeVisualizationId = state.visualization.activeId; const visualizationState = state.visualization.state; @@ -290,12 +290,12 @@ export function EditorFrame(props: EditorFrameProps) { ] ); - const hasSuggestionForField = React.useCallback( - (field: Dragging) => getSuggestionForField(field) !== undefined, + const hasSuggestionForField = useCallback( + (field: DragDropIdentifier) => getSuggestionForField(field) !== undefined, [getSuggestionForField] ); - const dropOntoWorkspace = React.useCallback( + const dropOntoWorkspace = useCallback( (field) => { const suggestion = getSuggestionForField(field); if (suggestion) { diff --git a/x-pack/plugins/lens/public/editor_frame_service/editor_frame/suggestion_helpers.ts b/x-pack/plugins/lens/public/editor_frame_service/editor_frame/suggestion_helpers.ts index 5cdc5ce592497..95dbf8264c588 100644 --- a/x-pack/plugins/lens/public/editor_frame_service/editor_frame/suggestion_helpers.ts +++ b/x-pack/plugins/lens/public/editor_frame_service/editor_frame/suggestion_helpers.ts @@ -19,7 +19,7 @@ import { DatasourcePublicAPI, } from '../../types'; import { Action } from './state_management'; -import { Dragging } from '../../drag_drop'; +import { DragDropIdentifier } from '../../drag_drop'; export interface Suggestion { visualizationId: string; @@ -231,7 +231,7 @@ export function getTopSuggestionForField( visualizationState: unknown, datasource: Datasource, datasourceStates: Record, - field: Dragging + field: DragDropIdentifier ) { const hasData = Object.values(datasourceLayers).some( (datasourceLayer) => datasourceLayer.getTableSpec().length > 0 diff --git a/x-pack/plugins/lens/public/editor_frame_service/editor_frame/workspace_panel/workspace_panel.test.tsx b/x-pack/plugins/lens/public/editor_frame_service/editor_frame/workspace_panel/workspace_panel.test.tsx index ddb2640d50d59..2f94d8e65dce6 100644 --- a/x-pack/plugins/lens/public/editor_frame_service/editor_frame/workspace_panel/workspace_panel.test.tsx +++ b/x-pack/plugins/lens/public/editor_frame_service/editor_frame/workspace_panel/workspace_panel.test.tsx @@ -784,7 +784,15 @@ describe('workspace_panel', () => { function initComponent(draggingContext = draggedField) { instance = mount( - {}}> + {}} + setActiveDropTarget={() => {}} + activeDropTarget={undefined} + keyboardMode={false} + setKeyboardMode={() => {}} + setA11yMessage={() => {}} + > { }); initComponent(); - instance.find(DragDrop).prop('onDrop')!(draggedField); + instance.find(DragDrop).prop('onDrop')!(draggedField, { id: 'lnsWorkspace' }); expect(mockDispatch).toHaveBeenCalledWith({ type: 'SWITCH_VISUALIZATION', diff --git a/x-pack/plugins/lens/public/editor_frame_service/editor_frame/workspace_panel/workspace_panel.tsx b/x-pack/plugins/lens/public/editor_frame_service/editor_frame/workspace_panel/workspace_panel.tsx index 5fc7b80a3d0ce..0c1fa932da09c 100644 --- a/x-pack/plugins/lens/public/editor_frame_service/editor_frame/workspace_panel/workspace_panel.tsx +++ b/x-pack/plugins/lens/public/editor_frame_service/editor_frame/workspace_panel/workspace_panel.tsx @@ -39,7 +39,7 @@ import { isLensFilterEvent, isLensEditEvent, } from '../../../types'; -import { DragDrop, DragContext, Dragging } from '../../../drag_drop'; +import { DragDrop, DragContext, DragDropIdentifier } from '../../../drag_drop'; import { Suggestion, switchToSuggestion } from '../suggestion_helpers'; import { buildExpression } from '../expression_helpers'; import { debouncedComponent } from '../../../debounced_component'; @@ -75,7 +75,7 @@ export interface WorkspacePanelProps { plugins: { uiActions?: UiActionsStart; data: DataPublicPluginStart }; title?: string; visualizeTriggerFieldContext?: VisualizeFieldContext; - getSuggestionForField: (field: Dragging) => Suggestion | undefined; + getSuggestionForField: (field: DragDropIdentifier) => Suggestion | undefined; } interface WorkspaceState { @@ -83,8 +83,10 @@ interface WorkspaceState { expandError: boolean; } +const workspaceDropValue = { id: 'lnsWorkspace' }; + // Exported for testing purposes only. -export function WorkspacePanel({ +export const WorkspacePanel = React.memo(function WorkspacePanel({ activeDatasourceId, activeVisualizationId, visualizationMap, @@ -102,7 +104,8 @@ export function WorkspacePanel({ }: WorkspacePanelProps) { const dragDropContext = useContext(DragContext); - const suggestionForDraggedField = getSuggestionForField(dragDropContext.dragging); + const suggestionForDraggedField = + dragDropContext.dragging && getSuggestionForField(dragDropContext.dragging); const [localState, setLocalState] = useState({ expressionBuildError: undefined, @@ -296,10 +299,11 @@ export function WorkspacePanel({ >
{renderVisualization()} @@ -308,7 +312,7 @@ export function WorkspacePanel({ ); -} +}); export const InnerVisualizationWrapper = ({ expression, diff --git a/x-pack/plugins/lens/public/indexpattern_datasource/datapanel.test.tsx b/x-pack/plugins/lens/public/indexpattern_datasource/datapanel.test.tsx index 8e41abf23e934..794ccd6936c90 100644 --- a/x-pack/plugins/lens/public/indexpattern_datasource/datapanel.test.tsx +++ b/x-pack/plugins/lens/public/indexpattern_datasource/datapanel.test.tsx @@ -278,7 +278,10 @@ describe('IndexPattern Data Panel', () => { {...defaultProps} state={state} setState={setStateSpy} - dragDropContext={{ dragging: { id: '1' }, setDragging: () => {} }} + dragDropContext={{ + ...createMockedDragDropContext(), + dragging: { id: '1' }, + }} /> ); @@ -297,7 +300,10 @@ describe('IndexPattern Data Panel', () => { indexPatterns: {}, }} setState={jest.fn()} - dragDropContext={{ dragging: { id: '1' }, setDragging: () => {} }} + dragDropContext={{ + ...createMockedDragDropContext(), + dragging: { id: '1' }, + }} changeIndexPattern={jest.fn()} /> ); @@ -329,7 +335,10 @@ describe('IndexPattern Data Panel', () => { ...defaultProps, changeIndexPattern: jest.fn(), setState, - dragDropContext: { dragging: { id: '1' }, setDragging: () => {} }, + dragDropContext: { + ...createMockedDragDropContext(), + dragging: { id: '1' }, + }, dateRange: { fromDate: '2019-01-01', toDate: '2020-01-01' }, state: { indexPatternRefs: [], diff --git a/x-pack/plugins/lens/public/indexpattern_datasource/datapanel.tsx b/x-pack/plugins/lens/public/indexpattern_datasource/datapanel.tsx index 4031cae548a10..c3dbcdc3e0573 100644 --- a/x-pack/plugins/lens/public/indexpattern_datasource/datapanel.tsx +++ b/x-pack/plugins/lens/public/indexpattern_datasource/datapanel.tsx @@ -426,6 +426,23 @@ export const InnerIndexPatternDataPanel = function InnerIndexPatternDataPanel({ ); }, [unfilteredFieldGroups, localState.nameFilter, localState.typeFilter]); + const checkFieldExists = useCallback( + (field) => + field.type === 'document' || + fieldExists(existingFields, currentIndexPattern.title, field.name), + [existingFields, currentIndexPattern.title] + ); + + const { nameFilter, typeFilter } = localState; + + const filter = useMemo( + () => ({ + nameFilter, + typeFilter, + }), + [nameFilter, typeFilter] + ); + const fieldProps = useMemo( () => ({ core, @@ -586,17 +603,11 @@ export const InnerIndexPatternDataPanel = function InnerIndexPatternDataPanel({ - field.type === 'document' || - fieldExists(existingFields, currentIndexPattern.title, field.name) - } + exists={checkFieldExists} fieldProps={fieldProps} fieldGroups={fieldGroups} hasSyncedExistingFields={!!hasSyncedExistingFields} - filter={{ - nameFilter: localState.nameFilter, - typeFilter: localState.typeFilter, - }} + filter={filter} currentIndexPatternId={currentIndexPatternId} existenceFetchFailed={existenceFetchFailed} existFieldsInIndex={!!allFields.length} diff --git a/x-pack/plugins/lens/public/indexpattern_datasource/dimension_panel/droppable.test.ts b/x-pack/plugins/lens/public/indexpattern_datasource/dimension_panel/droppable.test.ts index 6be03a92a445e..477f14848c08e 100644 --- a/x-pack/plugins/lens/public/indexpattern_datasource/dimension_panel/droppable.test.ts +++ b/x-pack/plugins/lens/public/indexpattern_datasource/dimension_panel/droppable.test.ts @@ -316,6 +316,7 @@ describe('IndexPatternDimensionEditorPanel', () => { droppedItem: dragging, columnId: 'col2', filterOperations: (op: OperationMetadata) => op.dataType === 'number', + groupId: '1', }); expect(setState).toBeCalledTimes(1); @@ -352,6 +353,7 @@ describe('IndexPatternDimensionEditorPanel', () => { droppedItem: dragging, columnId: 'col2', filterOperations: (op: OperationMetadata) => op.isBucketed, + groupId: '1', }); expect(setState).toBeCalledTimes(1); @@ -387,6 +389,7 @@ describe('IndexPatternDimensionEditorPanel', () => { }, droppedItem: dragging, filterOperations: (op: OperationMetadata) => op.dataType === 'number', + groupId: '1', }); expect(setState).toBeCalledTimes(1); @@ -438,6 +441,7 @@ describe('IndexPatternDimensionEditorPanel', () => { }, }, }, + groupId: '1', }); expect(setState).toBeCalledTimes(1); @@ -473,6 +477,7 @@ describe('IndexPatternDimensionEditorPanel', () => { }, droppedItem: dragging, columnId: 'col2', + groupId: '1', }); expect(setState).toBeCalledTimes(1); @@ -538,6 +543,7 @@ describe('IndexPatternDimensionEditorPanel', () => { }, droppedItem: dragging, state: testState, + groupId: '1', }); expect(setState).toBeCalledTimes(1); @@ -600,6 +606,7 @@ describe('IndexPatternDimensionEditorPanel', () => { droppedItem: dragging, state: testState, filterOperations: (op: OperationMetadata) => op.dataType === 'number', + groupId: 'a', }; const stateWithColumnOrder = (columnOrder: string[]) => { diff --git a/x-pack/plugins/lens/public/indexpattern_datasource/dimension_panel/droppable.ts b/x-pack/plugins/lens/public/indexpattern_datasource/dimension_panel/droppable.ts index e4eabafc6938e..0308d5e9103bf 100644 --- a/x-pack/plugins/lens/public/indexpattern_datasource/dimension_panel/droppable.ts +++ b/x-pack/plugins/lens/public/indexpattern_datasource/dimension_panel/droppable.ts @@ -8,6 +8,7 @@ import { DatasourceDimensionDropProps, DatasourceDimensionDropHandlerProps, isDraggedOperation, + DraggedOperation, } from '../../types'; import { IndexPatternColumn } from '../indexpattern'; import { insertOrReplaceColumn } from '../operations'; @@ -15,7 +16,15 @@ import { mergeLayer } from '../state_helpers'; import { hasField, isDraggedField } from '../utils'; import { IndexPatternPrivateState, IndexPatternField } from '../types'; import { trackUiEvent } from '../../lens_ui_telemetry'; -import { getOperationSupportMatrix } from './operation_support'; +import { getOperationSupportMatrix, OperationSupportMatrix } from './operation_support'; + +type DropHandlerProps = Pick< + DatasourceDimensionDropHandlerProps, + 'columnId' | 'setState' | 'state' | 'layerId' | 'droppedItem' +> & { + droppedItem: T; + operationSupportMatrix: OperationSupportMatrix; +}; export function canHandleDrop(props: DatasourceDimensionDropProps) { const operationSupportMatrix = getOperationSupportMatrix(props); @@ -29,11 +38,11 @@ export function canHandleDrop(props: DatasourceDimensionDropProps) { - const operationSupportMatrix = getOperationSupportMatrix(props); - const { setState, state, layerId, columnId, droppedItem } = props; - - if (isDraggedOperation(droppedItem) && props.isReorder) { - const dropEl = columnId; +const onReorderDrop = ({ columnId, setState, state, layerId, droppedItem }: DropHandlerProps) => { + setState( + mergeLayer({ + state, + layerId, + newLayer: { + columnOrder: reorderElements( + state.layers[layerId].columnOrder, + columnId, + droppedItem.columnId + ), + }, + }) + ); - setState( - mergeLayer({ - state, - layerId, - newLayer: { - columnOrder: reorderElements( - state.layers[layerId].columnOrder, - dropEl, - droppedItem.columnId - ), - }, - }) - ); - - return true; + return true; +}; + +const onMoveDropToCompatibleGroup = ({ + columnId, + setState, + state, + layerId, + droppedItem, +}: DropHandlerProps) => { + const layer = state.layers[layerId]; + const op = { ...layer.columns[droppedItem.columnId] }; + const newColumns = { ...layer.columns }; + delete newColumns[droppedItem.columnId]; + newColumns[columnId] = op; + + const newColumnOrder = [...layer.columnOrder]; + const oldIndex = newColumnOrder.findIndex((c) => c === droppedItem.columnId); + const newIndex = newColumnOrder.findIndex((c) => c === columnId); + + if (newIndex === -1) { + newColumnOrder[oldIndex] = columnId; + } else { + newColumnOrder.splice(oldIndex, 1); } + // Time to replace + setState( + mergeLayer({ + state, + layerId, + newLayer: { + columnOrder: newColumnOrder, + columns: newColumns, + }, + }) + ); + return { deleted: droppedItem.columnId }; +}; + +const onFieldDrop = ({ + columnId, + setState, + state, + layerId, + droppedItem, + operationSupportMatrix, +}: DropHandlerProps) => { function hasOperationForField(field: IndexPatternField) { return Boolean(operationSupportMatrix.operationByField[field.name]); } - if (isDraggedOperation(droppedItem) && droppedItem.layerId === layerId) { - const layer = state.layers[layerId]; - const op = { ...layer.columns[droppedItem.columnId] }; - if (!props.filterOperations(op)) { - return false; - } - - const newColumns = { ...layer.columns }; - delete newColumns[droppedItem.columnId]; - newColumns[columnId] = op; - - const newColumnOrder = [...layer.columnOrder]; - const oldIndex = newColumnOrder.findIndex((c) => c === droppedItem.columnId); - const newIndex = newColumnOrder.findIndex((c) => c === columnId); - - if (newIndex === -1) { - newColumnOrder[oldIndex] = columnId; - } else { - newColumnOrder.splice(oldIndex, 1); - } - - // Time to replace - setState( - mergeLayer({ - state, - layerId, - newLayer: { - columnOrder: newColumnOrder, - columns: newColumns, - }, - }) - ); - return { deleted: droppedItem.columnId }; - } - if (!isDraggedField(droppedItem) || !hasOperationForField(droppedItem.field)) { // TODO: What do we do if we couldn't find a column? return false; } + // dragged field, not operation + const operationsForNewField = operationSupportMatrix.operationByField[droppedItem.field.name]; if (!operationsForNewField || operationsForNewField.size === 0) { @@ -159,6 +174,56 @@ export function onDrop(props: DatasourceDimensionDropHandlerProps columns.length); trackUiEvent(hasData ? 'drop_non_empty' : 'drop_empty'); setState(mergeLayer({ state, layerId, newLayer })); - return true; +}; + +export function onDrop(props: DatasourceDimensionDropHandlerProps) { + const operationSupportMatrix = getOperationSupportMatrix(props); + const { setState, state, droppedItem, columnId, layerId, groupId, isNew } = props; + + if (!isDraggedOperation(droppedItem)) { + return onFieldDrop({ + columnId, + setState, + state, + layerId, + droppedItem, + operationSupportMatrix, + }); + } + const isExistingFromSameGroup = + droppedItem.groupId === groupId && droppedItem.columnId !== columnId && !isNew; + + // reorder in the same group + if (isExistingFromSameGroup) { + return onReorderDrop({ + columnId, + setState, + state, + layerId, + droppedItem, + operationSupportMatrix, + }); + } + + // replace or move to compatible group + const isFromOtherGroup = droppedItem.groupId !== groupId && droppedItem.layerId === layerId; + + if (isFromOtherGroup) { + const layer = state.layers[layerId]; + const op = { ...layer.columns[droppedItem.columnId] }; + + if (props.filterOperations(op)) { + return onMoveDropToCompatibleGroup({ + columnId, + setState, + state, + layerId, + droppedItem, + operationSupportMatrix, + }); + } + } + + return false; } diff --git a/x-pack/plugins/lens/public/indexpattern_datasource/field_item.test.tsx b/x-pack/plugins/lens/public/indexpattern_datasource/field_item.test.tsx index 1019b2c33e0e5..881e7a7228762 100644 --- a/x-pack/plugins/lens/public/indexpattern_datasource/field_item.test.tsx +++ b/x-pack/plugins/lens/public/indexpattern_datasource/field_item.test.tsx @@ -95,6 +95,8 @@ describe('IndexPattern Field Item', () => { }, exists: true, chartsThemeService, + groupIndex: 0, + itemIndex: 0, dropOntoWorkspace: () => {}, hasSuggestionForField: () => false, }; diff --git a/x-pack/plugins/lens/public/indexpattern_datasource/field_item.tsx b/x-pack/plugins/lens/public/indexpattern_datasource/field_item.tsx index 740b557b668b7..ff335a0da56ee 100644 --- a/x-pack/plugins/lens/public/indexpattern_datasource/field_item.tsx +++ b/x-pack/plugins/lens/public/indexpattern_datasource/field_item.tsx @@ -6,7 +6,7 @@ import './field_item.scss'; -import React, { useCallback, useState } from 'react'; +import React, { useCallback, useState, useMemo } from 'react'; import DateMath from '@elastic/datemath'; import { EuiButtonGroup, @@ -48,7 +48,7 @@ import { import { FieldButton } from '../../../../../src/plugins/kibana_react/public'; import { ChartsPluginSetup } from '../../../../../src/plugins/charts/public'; import { DraggedField } from './indexpattern'; -import { DragDrop, Dragging } from '../drag_drop'; +import { DragDrop, DragDropIdentifier } from '../drag_drop'; import { DatasourceDataPanelProps, DataType } from '../types'; import { BucketedAggregation, FieldStatsResponse } from '../../common'; import { IndexPattern, IndexPatternField } from './types'; @@ -69,6 +69,8 @@ export interface FieldItemProps { chartsThemeService: ChartsPluginSetup['theme']; filters: Filter[]; hideDetails?: boolean; + itemIndex: number; + groupIndex: number; dropOntoWorkspace: DatasourceDataPanelProps['dropOntoWorkspace']; hasSuggestionForField: DatasourceDataPanelProps['hasSuggestionForField']; } @@ -106,7 +108,7 @@ export const InnerFieldItem = function InnerFieldItem(props: FieldItemProps) { const [infoIsOpen, setOpen] = useState(false); const dropOntoWorkspaceAndClose = useCallback( - (droppedField: Dragging) => { + (droppedField: DragDropIdentifier) => { dropOntoWorkspace(droppedField); setOpen(false); }, @@ -163,10 +165,11 @@ export const InnerFieldItem = function InnerFieldItem(props: FieldItemProps) { } } - const value = React.useMemo( + const value = useMemo( () => ({ field, indexPatternId: indexPattern.id, id: field.name } as DraggedField), [field, indexPattern.id] ); + const lensFieldIcon = ; const lensInfoIcon = ( ('.application') || undefined} button={ allFieldCount + fields.length, 0); } -export function FieldList({ +export const FieldList = React.memo(function FieldList({ exists, fieldGroups, existenceFetchFailed, @@ -135,13 +135,15 @@ export function FieldList({ {Object.entries(fieldGroups) .filter(([, { showInAccordion }]) => !showInAccordion) .flatMap(([, { fields }]) => - fields.map((field) => ( + fields.map((field, index) => ( @@ -151,7 +153,7 @@ export function FieldList({ {Object.entries(fieldGroups) .filter(([, { showInAccordion }]) => showInAccordion) - .map(([key, fieldGroup]) => ( + .map(([key, fieldGroup], index) => ( { setAccordionState((s) => ({ ...s, @@ -198,4 +201,4 @@ export function FieldList({
); -} +}); diff --git a/x-pack/plugins/lens/public/indexpattern_datasource/fields_accordion.test.tsx b/x-pack/plugins/lens/public/indexpattern_datasource/fields_accordion.test.tsx index e2f615217bb4a..dca3de24014bc 100644 --- a/x-pack/plugins/lens/public/indexpattern_datasource/fields_accordion.test.tsx +++ b/x-pack/plugins/lens/public/indexpattern_datasource/fields_accordion.test.tsx @@ -72,6 +72,7 @@ describe('Fields Accordion', () => { fieldProps, renderCallout:
Callout
, exists: () => true, + groupIndex: 0, dropOntoWorkspace: () => {}, hasSuggestionForField: () => false, }; diff --git a/x-pack/plugins/lens/public/indexpattern_datasource/fields_accordion.tsx b/x-pack/plugins/lens/public/indexpattern_datasource/fields_accordion.tsx index 11adf1a128c1b..11710ffa18068 100644 --- a/x-pack/plugins/lens/public/indexpattern_datasource/fields_accordion.tsx +++ b/x-pack/plugins/lens/public/indexpattern_datasource/fields_accordion.tsx @@ -5,7 +5,7 @@ */ import './datapanel.scss'; -import React, { memo, useCallback } from 'react'; +import React, { memo, useCallback, useMemo } from 'react'; import { i18n } from '@kbn/i18n'; import { EuiText, @@ -50,11 +50,12 @@ export interface FieldsAccordionProps { exists: (field: IndexPatternField) => boolean; showExistenceFetchError?: boolean; hideDetails?: boolean; + groupIndex: number; dropOntoWorkspace: DatasourceDataPanelProps['dropOntoWorkspace']; hasSuggestionForField: DatasourceDataPanelProps['hasSuggestionForField']; } -export const InnerFieldsAccordion = function InnerFieldsAccordion({ +export const FieldsAccordion = memo(function InnerFieldsAccordion({ initialIsOpen, onToggle, id, @@ -69,28 +70,72 @@ export const InnerFieldsAccordion = function InnerFieldsAccordion({ exists, hideDetails, showExistenceFetchError, + groupIndex, dropOntoWorkspace, hasSuggestionForField, }: FieldsAccordionProps) { const renderField = useCallback( - (field: IndexPatternField) => ( + (field: IndexPatternField, index) => ( ), - [fieldProps, exists, hideDetails, dropOntoWorkspace, hasSuggestionForField] + [fieldProps, exists, hideDetails, dropOntoWorkspace, hasSuggestionForField, groupIndex] ); - const titleClassname = classNames({ - // eslint-disable-next-line @typescript-eslint/naming-convention - lnsInnerIndexPatternDataPanel__titleTooltip: !!helpTooltip, - }); + const renderButton = useMemo(() => { + const titleClassname = classNames({ + // eslint-disable-next-line @typescript-eslint/naming-convention + lnsInnerIndexPatternDataPanel__titleTooltip: !!helpTooltip, + }); + return ( + + {label} + {!!helpTooltip && ( + + )} + + ); + }, [label, helpTooltip]); + + const extraAction = useMemo(() => { + return showExistenceFetchError ? ( + + ) : hasLoaded ? ( + + {fieldsCount} + + ) : ( + + ); + }, [showExistenceFetchError, hasLoaded, isFiltered, fieldsCount]); return ( - {label} - {!!helpTooltip && ( - - )} - - } - extraAction={ - showExistenceFetchError ? ( - - ) : hasLoaded ? ( - - {fieldsCount} - - ) : ( - - ) - } + buttonContent={renderButton} + extraAction={extraAction} > {hasLoaded && @@ -148,6 +157,4 @@ export const InnerFieldsAccordion = function InnerFieldsAccordion({ ))} ); -}; - -export const FieldsAccordion = memo(InnerFieldsAccordion); +}); diff --git a/x-pack/plugins/lens/public/indexpattern_datasource/indexpattern.tsx b/x-pack/plugins/lens/public/indexpattern_datasource/indexpattern.tsx index e51cd36156d1b..7f77a7ce199b6 100644 --- a/x-pack/plugins/lens/public/indexpattern_datasource/indexpattern.tsx +++ b/x-pack/plugins/lens/public/indexpattern_datasource/indexpattern.tsx @@ -51,11 +51,11 @@ import { mergeLayer } from './state_helpers'; import { Datasource, StateSetter } from '../types'; import { ChartsPluginSetup } from '../../../../../src/plugins/charts/public'; import { deleteColumn, isReferenced } from './operations'; -import { Dragging } from '../drag_drop/providers'; +import { DragDropIdentifier } from '../drag_drop/providers'; export { OperationType, IndexPatternColumn, deleteColumn } from './operations'; -export type DraggedField = Dragging & { +export type DraggedField = DragDropIdentifier & { field: IndexPatternField; indexPatternId: string; }; @@ -167,7 +167,7 @@ export function getIndexPatternDatasource({ }); }, - toExpression, + toExpression: (state, layerId) => toExpression(state, layerId, uiSettings), renderDataPanel( domElement: Element, diff --git a/x-pack/plugins/lens/public/indexpattern_datasource/mocks.ts b/x-pack/plugins/lens/public/indexpattern_datasource/mocks.ts index 4aea9e8ac67a9..67ddbe8c45ab7 100644 --- a/x-pack/plugins/lens/public/indexpattern_datasource/mocks.ts +++ b/x-pack/plugins/lens/public/indexpattern_datasource/mocks.ts @@ -247,5 +247,10 @@ export function createMockedDragDropContext(): jest.Mocked { return { dragging: undefined, setDragging: jest.fn(), + activeDropTarget: undefined, + setActiveDropTarget: jest.fn(), + keyboardMode: false, + setKeyboardMode: jest.fn(), + setA11yMessage: jest.fn(), }; } diff --git a/x-pack/plugins/lens/public/indexpattern_datasource/operations/definitions/date_histogram.test.tsx b/x-pack/plugins/lens/public/indexpattern_datasource/operations/definitions/date_histogram.test.tsx index abd033c0db4cf..22275533b9554 100644 --- a/x-pack/plugins/lens/public/indexpattern_datasource/operations/definitions/date_histogram.test.tsx +++ b/x-pack/plugins/lens/public/indexpattern_datasource/operations/definitions/date_histogram.test.tsx @@ -83,9 +83,11 @@ const indexPattern2: IndexPattern = { ]), }; +const uiSettingsMock = {} as IUiSettingsClient; + const defaultOptions = { storage: {} as IStorageWrapper, - uiSettings: {} as IUiSettingsClient, + uiSettings: uiSettingsMock, savedObjectsClient: {} as SavedObjectsClientContract, dateRange: { fromDate: 'now-1y', @@ -200,7 +202,8 @@ describe('date_histogram', () => { layer.columns.col1 as DateHistogramIndexPatternColumn, 'col1', indexPattern1, - layer + layer, + uiSettingsMock ); expect(esAggsFn).toEqual( expect.objectContaining({ @@ -252,7 +255,8 @@ describe('date_histogram', () => { }, ]), }, - layer + layer, + uiSettingsMock ); expect(esAggsFn).toEqual( expect.objectContaining({ diff --git a/x-pack/plugins/lens/public/indexpattern_datasource/operations/definitions/filters/filters.test.tsx b/x-pack/plugins/lens/public/indexpattern_datasource/operations/definitions/filters/filters.test.tsx index 86767fbc8b469..3657013fa0bfa 100644 --- a/x-pack/plugins/lens/public/indexpattern_datasource/operations/definitions/filters/filters.test.tsx +++ b/x-pack/plugins/lens/public/indexpattern_datasource/operations/definitions/filters/filters.test.tsx @@ -16,9 +16,11 @@ import type { IndexPatternLayer } from '../../../types'; import { createMockedIndexPattern } from '../../../mocks'; import { FilterPopover } from './filter_popover'; +const uiSettingsMock = {} as IUiSettingsClient; + const defaultProps = { storage: {} as IStorageWrapper, - uiSettings: {} as IUiSettingsClient, + uiSettings: uiSettingsMock, savedObjectsClient: {} as SavedObjectsClientContract, dateRange: { fromDate: 'now-1d', toDate: 'now' }, data: dataPluginMock.createStartContract(), @@ -84,7 +86,8 @@ describe('filters', () => { layer.columns.col1 as FiltersIndexPatternColumn, 'col1', createMockedIndexPattern(), - layer + layer, + uiSettingsMock ); expect(esAggsFn).toEqual( expect.objectContaining({ diff --git a/x-pack/plugins/lens/public/indexpattern_datasource/operations/definitions/index.ts b/x-pack/plugins/lens/public/indexpattern_datasource/operations/definitions/index.ts index 1cdaff53c5458..0c0aa34bb40b3 100644 --- a/x-pack/plugins/lens/public/indexpattern_datasource/operations/definitions/index.ts +++ b/x-pack/plugins/lens/public/indexpattern_datasource/operations/definitions/index.ts @@ -239,7 +239,8 @@ interface FieldlessOperationDefinition { column: C, columnId: string, indexPattern: IndexPattern, - layer: IndexPatternLayer + layer: IndexPatternLayer, + uiSettings: IUiSettingsClient ) => ExpressionAstFunction; } @@ -283,7 +284,8 @@ interface FieldBasedOperationDefinition { column: C, columnId: string, indexPattern: IndexPattern, - layer: IndexPatternLayer + layer: IndexPatternLayer, + uiSettings: IUiSettingsClient ) => ExpressionAstFunction; /** * Optional function to return the suffix used for ES bucket paths and esaggs column id. diff --git a/x-pack/plugins/lens/public/indexpattern_datasource/operations/definitions/last_value.test.tsx b/x-pack/plugins/lens/public/indexpattern_datasource/operations/definitions/last_value.test.tsx index 96b12a714e613..8d5ab50770111 100644 --- a/x-pack/plugins/lens/public/indexpattern_datasource/operations/definitions/last_value.test.tsx +++ b/x-pack/plugins/lens/public/indexpattern_datasource/operations/definitions/last_value.test.tsx @@ -15,9 +15,11 @@ import { LastValueIndexPatternColumn } from './last_value'; import { lastValueOperation } from './index'; import type { IndexPattern, IndexPatternLayer } from '../../types'; +const uiSettingsMock = {} as IUiSettingsClient; + const defaultProps = { storage: {} as IStorageWrapper, - uiSettings: {} as IUiSettingsClient, + uiSettings: uiSettingsMock, savedObjectsClient: {} as SavedObjectsClientContract, dateRange: { fromDate: 'now-1d', toDate: 'now' }, data: dataPluginMock.createStartContract(), @@ -70,7 +72,8 @@ describe('last_value', () => { { ...lastValueColumn, params: { ...lastValueColumn.params } }, 'col1', {} as IndexPattern, - layer + layer, + uiSettingsMock ); expect(esAggsFn).toEqual( expect.objectContaining({ diff --git a/x-pack/plugins/lens/public/indexpattern_datasource/operations/definitions/percentile.test.tsx b/x-pack/plugins/lens/public/indexpattern_datasource/operations/definitions/percentile.test.tsx index c22eec62ea1ab..a340e17121e90 100644 --- a/x-pack/plugins/lens/public/indexpattern_datasource/operations/definitions/percentile.test.tsx +++ b/x-pack/plugins/lens/public/indexpattern_datasource/operations/definitions/percentile.test.tsx @@ -17,9 +17,11 @@ import { EuiFieldNumber } from '@elastic/eui'; import { act } from 'react-dom/test-utils'; import { EuiFormRow } from '@elastic/eui'; +const uiSettingsMock = {} as IUiSettingsClient; + const defaultProps = { storage: {} as IStorageWrapper, - uiSettings: {} as IUiSettingsClient, + uiSettings: uiSettingsMock, savedObjectsClient: {} as SavedObjectsClientContract, dateRange: { fromDate: 'now-1d', toDate: 'now' }, data: dataPluginMock.createStartContract(), @@ -72,7 +74,8 @@ describe('percentile', () => { percentileColumn, 'col1', {} as IndexPattern, - layer + layer, + uiSettingsMock ); expect(esAggsFn).toEqual( expect.objectContaining({ diff --git a/x-pack/plugins/lens/public/indexpattern_datasource/operations/definitions/ranges/range_editor.tsx b/x-pack/plugins/lens/public/indexpattern_datasource/operations/definitions/ranges/range_editor.tsx index ad5c146ff6624..d9698252177b0 100644 --- a/x-pack/plugins/lens/public/indexpattern_datasource/operations/definitions/ranges/range_editor.tsx +++ b/x-pack/plugins/lens/public/indexpattern_datasource/operations/definitions/ranges/range_editor.tsx @@ -4,7 +4,7 @@ * you may not use this file except in compliance with the Elastic License. */ -import React, { useEffect, useState } from 'react'; +import React, { useState } from 'react'; import { i18n } from '@kbn/i18n'; import { FormattedMessage } from '@kbn/i18n/react'; import { @@ -190,15 +190,6 @@ export const RangeEditor = ({ }) => { const [isAdvancedEditor, toggleAdvancedEditor] = useState(params.type === MODES.Range); - // if the maxBars in the params is set to auto refresh it with the default value only on bootstrap - useEffect(() => { - if (!isAdvancedEditor) { - if (params.maxBars !== maxBars) { - setParam('maxBars', maxBars); - } - } - }, [maxBars, params.maxBars, setParam, isAdvancedEditor]); - if (isAdvancedEditor) { return ( & React.MouseEvent; +// need this for MAX_HISTOGRAM value +const uiSettingsMock = ({ + get: jest.fn().mockReturnValue(100), +} as unknown) as IUiSettingsClient; + const sourceField = 'MyField'; const defaultOptions = { storage: {} as IStorageWrapper, - // need this for MAX_HISTOGRAM value - uiSettings: ({ - get: () => 100, - } as unknown) as IUiSettingsClient, + uiSettings: uiSettingsMock, savedObjectsClient: {} as SavedObjectsClientContract, dateRange: { fromDate: 'now-1y', @@ -143,7 +145,8 @@ describe('ranges', () => { layer.columns.col1 as RangeIndexPatternColumn, 'col1', {} as IndexPattern, - layer + layer, + uiSettingsMock ); expect(esAggsFn).toMatchInlineSnapshot(` Object { @@ -166,6 +169,9 @@ describe('ranges', () => { "interval": Array [ "auto", ], + "maxBars": Array [ + 49.5, + ], "min_doc_count": Array [ false, ], @@ -186,7 +192,8 @@ describe('ranges', () => { layer.columns.col1 as RangeIndexPatternColumn, 'col1', {} as IndexPattern, - layer + layer, + uiSettingsMock ); expect(esAggsFn).toEqual( @@ -206,7 +213,8 @@ describe('ranges', () => { layer.columns.col1 as RangeIndexPatternColumn, 'col1', {} as IndexPattern, - layer + layer, + uiSettingsMock ); expect(esAggsFn).toEqual( @@ -226,7 +234,8 @@ describe('ranges', () => { layer.columns.col1 as RangeIndexPatternColumn, 'col1', {} as IndexPattern, - layer + layer, + uiSettingsMock ); expect((esAggsFn as { arguments: unknown }).arguments).toEqual( @@ -275,7 +284,7 @@ describe('ranges', () => { it('should start update the state with the default maxBars value', () => { const updateLayerSpy = jest.fn(); - mount( + const instance = mount( { /> ); - expect(updateLayerSpy).toHaveBeenCalledWith({ - ...layer, - columns: { - ...layer.columns, - col1: { - ...layer.columns.col1, - params: { - ...layer.columns.col1.params, - maxBars: GRANULARITY_DEFAULT_VALUE, - }, - }, - }, - }); + expect(instance.find(EuiRange).prop('value')).toEqual(String(GRANULARITY_DEFAULT_VALUE)); }); it('should update state when changing Max bars number', () => { @@ -313,8 +310,6 @@ describe('ranges', () => { /> ); - // There's a useEffect in the component that updates the value on bootstrap - // because there's a debouncer, wait a bit before calling onChange act(() => { jest.advanceTimersByTime(TYPING_DEBOUNCE_TIME * 4); @@ -358,8 +353,6 @@ describe('ranges', () => { /> ); - // There's a useEffect in the component that updates the value on bootstrap - // because there's a debouncer, wait a bit before calling onChange act(() => { jest.advanceTimersByTime(TYPING_DEBOUNCE_TIME * 4); // minus button @@ -368,6 +361,7 @@ describe('ranges', () => { .find('button') .prop('onClick')!({} as ReactMouseEvent); jest.advanceTimersByTime(TYPING_DEBOUNCE_TIME * 4); + instance.update(); }); expect(updateLayerSpy).toHaveBeenCalledWith({ @@ -391,6 +385,7 @@ describe('ranges', () => { .find('button') .prop('onClick')!({} as ReactMouseEvent); jest.advanceTimersByTime(TYPING_DEBOUNCE_TIME * 4); + instance.update(); }); expect(updateLayerSpy).toHaveBeenCalledWith({ @@ -788,7 +783,7 @@ describe('ranges', () => { instance.find(EuiLink).first().prop('onClick')!({} as ReactMouseEvent); }); - expect(updateLayerSpy.mock.calls[1][0].columns.col1.params.format).toEqual({ + expect(updateLayerSpy.mock.calls[0][0].columns.col1.params.format).toEqual({ id: 'custom', params: { decimals: 3 }, }); diff --git a/x-pack/plugins/lens/public/indexpattern_datasource/operations/definitions/ranges/ranges.tsx b/x-pack/plugins/lens/public/indexpattern_datasource/operations/definitions/ranges/ranges.tsx index aa5cc8255a584..d8622a5aedf7d 100644 --- a/x-pack/plugins/lens/public/indexpattern_datasource/operations/definitions/ranges/ranges.tsx +++ b/x-pack/plugins/lens/public/indexpattern_datasource/operations/definitions/ranges/ranges.tsx @@ -132,7 +132,7 @@ export const rangeOperation: OperationDefinition { + toEsAggsFn: (column, columnId, indexPattern, layer, uiSettings) => { const { sourceField, params } = column; if (params.type === MODES.Range) { return buildExpressionFunction('aggRange', { @@ -158,13 +158,15 @@ export const rangeOperation: OperationDefinition('aggHistogram', { id: columnId, enabled: true, schema: 'segment', field: sourceField, - // fallback to 0 in case of empty string - maxBars: params.maxBars === AUTO_BARS ? undefined : params.maxBars, + maxBars: params.maxBars === AUTO_BARS ? maxBarsDefaultValue : params.maxBars, interval: 'auto', has_extended_bounds: false, min_doc_count: false, diff --git a/x-pack/plugins/lens/public/indexpattern_datasource/operations/definitions/terms/terms.test.tsx b/x-pack/plugins/lens/public/indexpattern_datasource/operations/definitions/terms/terms.test.tsx index d60992bda2e2a..3e25e127b37f4 100644 --- a/x-pack/plugins/lens/public/indexpattern_datasource/operations/definitions/terms/terms.test.tsx +++ b/x-pack/plugins/lens/public/indexpattern_datasource/operations/definitions/terms/terms.test.tsx @@ -17,9 +17,11 @@ import type { TermsIndexPatternColumn } from '.'; import { termsOperation } from '../index'; import { IndexPattern, IndexPatternLayer } from '../../../types'; +const uiSettingsMock = {} as IUiSettingsClient; + const defaultProps = { storage: {} as IStorageWrapper, - uiSettings: {} as IUiSettingsClient, + uiSettings: uiSettingsMock, savedObjectsClient: {} as SavedObjectsClientContract, dateRange: { fromDate: 'now-1d', toDate: 'now' }, data: dataPluginMock.createStartContract(), @@ -66,7 +68,8 @@ describe('terms', () => { { ...termsColumn, params: { ...termsColumn.params, otherBucket: true } }, 'col1', {} as IndexPattern, - layer + layer, + uiSettingsMock ); expect(esAggsFn).toEqual( expect.objectContaining({ @@ -89,7 +92,8 @@ describe('terms', () => { }, 'col1', {} as IndexPattern, - layer + layer, + uiSettingsMock ); expect(esAggsFn).toEqual( expect.objectContaining({ @@ -129,7 +133,8 @@ describe('terms', () => { }, }, }, - } + }, + uiSettingsMock ); expect(esAggsFn).toEqual( expect.objectContaining({ diff --git a/x-pack/plugins/lens/public/indexpattern_datasource/to_expression.ts b/x-pack/plugins/lens/public/indexpattern_datasource/to_expression.ts index 38f51f24aae7d..c9ee77a9f5e15 100644 --- a/x-pack/plugins/lens/public/indexpattern_datasource/to_expression.ts +++ b/x-pack/plugins/lens/public/indexpattern_datasource/to_expression.ts @@ -4,6 +4,7 @@ * you may not use this file except in compliance with the Elastic License. */ +import type { IUiSettingsClient } from 'kibana/public'; import { EsaggsExpressionFunctionDefinition, IndexPatternLoadExpressionFunctionDefinition, @@ -24,7 +25,8 @@ import { getEsAggsSuffix } from './operations/definitions/helpers'; function getExpressionForLayer( layer: IndexPatternLayer, - indexPattern: IndexPattern + indexPattern: IndexPattern, + uiSettings: IUiSettingsClient ): ExpressionAstExpression | null { const { columns, columnOrder } = layer; if (columnOrder.length === 0) { @@ -44,7 +46,7 @@ function getExpressionForLayer( aggs.push( buildExpression({ type: 'expression', - chain: [def.toEsAggsFn(col, colId, indexPattern, layer)], + chain: [def.toEsAggsFn(col, colId, indexPattern, layer, uiSettings)], }) ); } @@ -184,11 +186,16 @@ function getExpressionForLayer( return null; } -export function toExpression(state: IndexPatternPrivateState, layerId: string) { +export function toExpression( + state: IndexPatternPrivateState, + layerId: string, + uiSettings: IUiSettingsClient +) { if (state.layers[layerId]) { return getExpressionForLayer( state.layers[layerId], - state.indexPatterns[state.layers[layerId].indexPatternId] + state.indexPatterns[state.layers[layerId].indexPatternId], + uiSettings ); } diff --git a/x-pack/plugins/lens/public/types.ts b/x-pack/plugins/lens/public/types.ts index 907ef3a700ce6..8f202faeb9ee8 100644 --- a/x-pack/plugins/lens/public/types.ts +++ b/x-pack/plugins/lens/public/types.ts @@ -16,7 +16,7 @@ import { Datatable, SerializedFieldFormat, } from '../../../../src/plugins/expressions/public'; -import { DragContextState, Dragging } from './drag_drop'; +import { DragContextState, DragDropIdentifier } from './drag_drop'; import { Document } from './persistence'; import { DateRange } from '../common'; import { Query, Filter, SavedQuery, IFieldFormat } from '../../../../src/plugins/data/public'; @@ -226,8 +226,8 @@ export interface DatasourceDataPanelProps { query: Query; dateRange: DateRange; filters: Filter[]; - dropOntoWorkspace: (field: Dragging) => void; - hasSuggestionForField: (field: Dragging) => boolean; + dropOntoWorkspace: (field: DragDropIdentifier) => void; + hasSuggestionForField: (field: DragDropIdentifier) => boolean; } interface SharedDimensionProps { @@ -301,6 +301,8 @@ export type DatasourceDimensionDropProps = SharedDimensionProps & { export type DatasourceDimensionDropHandlerProps = DatasourceDimensionDropProps & { droppedItem: unknown; + groupId: string; + isNew?: boolean; }; export type DataType = 'document' | 'string' | 'number' | 'date' | 'boolean' | 'ip'; diff --git a/x-pack/plugins/lens/public/xy_visualization/xy_config_panel.tsx b/x-pack/plugins/lens/public/xy_visualization/xy_config_panel.tsx index b8bca09bb353c..91fa2f5921d2f 100644 --- a/x-pack/plugins/lens/public/xy_visualization/xy_config_panel.tsx +++ b/x-pack/plugins/lens/public/xy_visualization/xy_config_panel.tsx @@ -5,7 +5,7 @@ */ import './xy_config_panel.scss'; -import React, { useMemo, useState } from 'react'; +import React, { useMemo, useState, memo } from 'react'; import { i18n } from '@kbn/i18n'; import { Position } from '@elastic/charts'; import { debounce } from 'lodash'; @@ -179,8 +179,7 @@ function getValueLabelDisableReason({ defaultMessage: 'This setting cannot be changed on stacked or percentage bar charts', }); } - -export function XyToolbar(props: VisualizationToolbarProps) { +export const XyToolbar = memo(function XyToolbar(props: VisualizationToolbarProps) { const { state, setState, frame } = props; const hasNonBarSeries = state?.layers.some(({ seriesType }) => @@ -485,7 +484,8 @@ export function XyToolbar(props: VisualizationToolbarProps) { ); -} +}); + const idPrefix = htmlIdGenerator()(); export function DimensionEditor( @@ -653,7 +653,7 @@ const ColorPicker = ({ } }; - const updateColorInState: EuiColorPickerProps['onChange'] = React.useMemo( + const updateColorInState: EuiColorPickerProps['onChange'] = useMemo( () => debounce((text, output) => { const newYConfigs = [...(layer.yConfig || [])]; diff --git a/x-pack/plugins/maps/common/constants.ts b/x-pack/plugins/maps/common/constants.ts index b86d48bfccdab..c8db433a37235 100644 --- a/x-pack/plugins/maps/common/constants.ts +++ b/x-pack/plugins/maps/common/constants.ts @@ -85,6 +85,7 @@ export enum SOURCE_TYPES { REGIONMAP_FILE = 'REGIONMAP_FILE', GEOJSON_FILE = 'GEOJSON_FILE', MVT_SINGLE_LAYER = 'MVT_SINGLE_LAYER', + TABLE_SOURCE = 'TABLE_SOURCE', } export enum FIELD_ORIGIN { diff --git a/x-pack/plugins/maps/common/descriptor_types/layer_descriptor_types.ts b/x-pack/plugins/maps/common/descriptor_types/layer_descriptor_types.ts index b67f05cb169fd..65cc145e20c89 100644 --- a/x-pack/plugins/maps/common/descriptor_types/layer_descriptor_types.ts +++ b/x-pack/plugins/maps/common/descriptor_types/layer_descriptor_types.ts @@ -8,11 +8,11 @@ import { Query } from 'src/plugins/data/public'; import { StyleDescriptor, VectorStyleDescriptor } from './style_property_descriptor_types'; import { DataRequestDescriptor } from './data_request_descriptor_types'; -import { AbstractSourceDescriptor, ESTermSourceDescriptor } from './source_descriptor_types'; +import { AbstractSourceDescriptor, TermJoinSourceDescriptor } from './source_descriptor_types'; export type JoinDescriptor = { leftField?: string; - right: ESTermSourceDescriptor; + right: TermJoinSourceDescriptor; }; export type LayerDescriptor = { diff --git a/x-pack/plugins/maps/common/descriptor_types/source_descriptor_types.ts b/x-pack/plugins/maps/common/descriptor_types/source_descriptor_types.ts index b849b42429cf6..dca7ae766f375 100644 --- a/x-pack/plugins/maps/common/descriptor_types/source_descriptor_types.ts +++ b/x-pack/plugins/maps/common/descriptor_types/source_descriptor_types.ts @@ -8,7 +8,14 @@ import { FeatureCollection } from 'geojson'; import { Query } from 'src/plugins/data/public'; import { SortDirection } from 'src/plugins/data/common/search'; -import { AGG_TYPE, GRID_RESOLUTION, RENDER_AS, SCALING_TYPES, MVT_FIELD_TYPE } from '../constants'; +import { + AGG_TYPE, + GRID_RESOLUTION, + RENDER_AS, + SCALING_TYPES, + MVT_FIELD_TYPE, + SOURCE_TYPES, +} from '../constants'; export type AttributionDescriptor = { attributionText?: string; @@ -105,6 +112,7 @@ export type ESTermSourceDescriptor = AbstractESAggSourceDescriptor & { term: string; // term field name whereQuery?: Query; size?: number; + type: SOURCE_TYPES.ES_TERM_SOURCE; }; export type KibanaRegionmapSourceDescriptor = AbstractSourceDescriptor & { @@ -156,14 +164,24 @@ export type TiledSingleLayerVectorSourceDescriptor = AbstractSourceDescriptor & tooltipProperties: string[]; }; -export type GeoJsonFileFieldDescriptor = { +export type InlineFieldDescriptor = { name: string; type: 'string' | 'number'; }; export type GeojsonFileSourceDescriptor = { - __fields?: GeoJsonFileFieldDescriptor[]; + __fields?: InlineFieldDescriptor[]; __featureCollection: FeatureCollection; name: string; type: string; }; + +export type TableSourceDescriptor = { + id: string; + type: SOURCE_TYPES.TABLE_SOURCE; + __rows: Array<{ [key: string]: string | number }>; + __columns: InlineFieldDescriptor[]; + term: string; +}; + +export type TermJoinSourceDescriptor = ESTermSourceDescriptor | TableSourceDescriptor; diff --git a/x-pack/plugins/maps/common/migrations/add_type_to_termjoin.test.ts b/x-pack/plugins/maps/common/migrations/add_type_to_termjoin.test.ts new file mode 100644 index 0000000000000..c9ab4b00d8923 --- /dev/null +++ b/x-pack/plugins/maps/common/migrations/add_type_to_termjoin.test.ts @@ -0,0 +1,45 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License; + * you may not use this file except in compliance with the Elastic License. + */ + +import { addTypeToTermJoin } from './add_type_to_termjoin'; +import { LAYER_TYPE, SOURCE_TYPES } from '../constants'; +import { LayerDescriptor } from '../descriptor_types'; + +describe('addTypeToTermJoin', () => { + test('Should handle missing type attribute', () => { + const layerListJSON = JSON.stringify(([ + { + type: LAYER_TYPE.VECTOR, + joins: [ + { + right: {}, + }, + { + right: { + type: SOURCE_TYPES.TABLE_SOURCE, + }, + }, + { + right: { + type: SOURCE_TYPES.ES_TERM_SOURCE, + }, + }, + ], + }, + ] as unknown) as LayerDescriptor[]); + + const attributes = { + title: 'my map', + layerListJSON, + }; + + const { layerListJSON: migratedLayerListJSON } = addTypeToTermJoin({ attributes }); + const migratedLayerList = JSON.parse(migratedLayerListJSON!); + expect(migratedLayerList[0].joins[0].right.type).toEqual(SOURCE_TYPES.ES_TERM_SOURCE); + expect(migratedLayerList[0].joins[1].right.type).toEqual(SOURCE_TYPES.TABLE_SOURCE); + expect(migratedLayerList[0].joins[2].right.type).toEqual(SOURCE_TYPES.ES_TERM_SOURCE); + }); +}); diff --git a/x-pack/plugins/maps/common/migrations/add_type_to_termjoin.ts b/x-pack/plugins/maps/common/migrations/add_type_to_termjoin.ts new file mode 100644 index 0000000000000..84e13eb6c3947 --- /dev/null +++ b/x-pack/plugins/maps/common/migrations/add_type_to_termjoin.ts @@ -0,0 +1,48 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License; + * you may not use this file except in compliance with the Elastic License. + */ + +import { MapSavedObjectAttributes } from '../map_saved_object_type'; +import { JoinDescriptor, LayerDescriptor } from '../descriptor_types'; +import { LAYER_TYPE, SOURCE_TYPES } from '../constants'; + +// enforce type property on joins. It's possible older saved-objects do not have this correctly filled in +// e.g. sample-data was missing the right.type field. +// This is just to be safe. +export function addTypeToTermJoin({ + attributes, +}: { + attributes: MapSavedObjectAttributes; +}): MapSavedObjectAttributes { + if (!attributes || !attributes.layerListJSON) { + return attributes; + } + + const layerList: LayerDescriptor[] = JSON.parse(attributes.layerListJSON); + + layerList.forEach((layer: LayerDescriptor) => { + if (layer.type !== LAYER_TYPE.VECTOR) { + return; + } + + if (!layer.joins) { + return; + } + layer.joins.forEach((join: JoinDescriptor) => { + if (!join.right) { + return; + } + + if (typeof join.right.type === 'undefined') { + join.right.type = SOURCE_TYPES.ES_TERM_SOURCE; + } + }); + }); + + return { + ...attributes, + layerListJSON: JSON.stringify(layerList), + }; +} diff --git a/x-pack/plugins/maps/public/classes/fields/geojson_file_field.ts b/x-pack/plugins/maps/public/classes/fields/inline_field.ts similarity index 80% rename from x-pack/plugins/maps/public/classes/fields/geojson_file_field.ts rename to x-pack/plugins/maps/public/classes/fields/inline_field.ts index ae42b09d491c5..287edbd07cce8 100644 --- a/x-pack/plugins/maps/public/classes/fields/geojson_file_field.ts +++ b/x-pack/plugins/maps/public/classes/fields/inline_field.ts @@ -7,10 +7,9 @@ import { FIELD_ORIGIN } from '../../../common/constants'; import { IField, AbstractField } from './field'; import { IVectorSource } from '../sources/vector_source'; -import { GeoJsonFileSource } from '../sources/geojson_file_source'; -export class GeoJsonFileField extends AbstractField implements IField { - private readonly _source: GeoJsonFileSource; +export class InlineField extends AbstractField implements IField { + private readonly _source: T; private readonly _dataType: string; constructor({ @@ -20,7 +19,7 @@ export class GeoJsonFileField extends AbstractField implements IField { dataType, }: { fieldName: string; - source: GeoJsonFileSource; + source: T; origin: FIELD_ORIGIN; dataType: string; }) { diff --git a/x-pack/plugins/maps/public/classes/joins/inner_join.test.js b/x-pack/plugins/maps/public/classes/joins/inner_join.test.js index ca40ab1ea7db7..bca5954e73d7b 100644 --- a/x-pack/plugins/maps/public/classes/joins/inner_join.test.js +++ b/x-pack/plugins/maps/public/classes/joins/inner_join.test.js @@ -5,11 +5,13 @@ */ import { InnerJoin } from './inner_join'; +import { SOURCE_TYPES } from '../../../common/constants'; jest.mock('../../kibana_services', () => {}); jest.mock('../layers/vector_layer/vector_layer', () => {}); const rightSource = { + type: SOURCE_TYPES.ES_TERM_SOURCE, id: 'd3625663-5b34-4d50-a784-0d743f676a0c', indexPatternId: '90943e30-9a47-11e8-b64d-95841ca0b247', indexPatternTitle: 'kibana_sample_data_logs', diff --git a/x-pack/plugins/maps/public/classes/joins/inner_join.ts b/x-pack/plugins/maps/public/classes/joins/inner_join.ts index 32bd767aa94d8..95e163709dff9 100644 --- a/x-pack/plugins/maps/public/classes/joins/inner_join.ts +++ b/x-pack/plugins/maps/public/classes/joins/inner_join.ts @@ -9,29 +9,51 @@ import { Feature, GeoJsonProperties } from 'geojson'; import { ESTermSource } from '../sources/es_term_source'; import { getComputedFieldNamePrefix } from '../styles/vector/style_util'; import { - META_DATA_REQUEST_ID_SUFFIX, FORMATTERS_DATA_REQUEST_ID_SUFFIX, + META_DATA_REQUEST_ID_SUFFIX, + SOURCE_TYPES, } from '../../../common/constants'; -import { JoinDescriptor } from '../../../common/descriptor_types'; +import { + ESTermSourceDescriptor, + JoinDescriptor, + TableSourceDescriptor, + TermJoinSourceDescriptor, +} from '../../../common/descriptor_types'; import { IVectorSource } from '../sources/vector_source'; import { IField } from '../fields/field'; import { PropertiesMap } from '../../../common/elasticsearch_util'; +import { ITermJoinSource } from '../sources/term_join_source'; +import { TableSource } from '../sources/table_source'; +import { Adapters } from '../../../../../../src/plugins/inspector/common/adapters'; + +function createJoinTermSource( + descriptor: Partial | undefined, + inspectorAdapters: Adapters | undefined +): ITermJoinSource | undefined { + if (!descriptor) { + return; + } + + if ( + descriptor.type === SOURCE_TYPES.ES_TERM_SOURCE && + 'indexPatternId' in descriptor && + 'term' in descriptor + ) { + return new ESTermSource(descriptor as ESTermSourceDescriptor, inspectorAdapters); + } else if (descriptor.type === SOURCE_TYPES.TABLE_SOURCE) { + return new TableSource(descriptor as TableSourceDescriptor, inspectorAdapters); + } +} export class InnerJoin { private readonly _descriptor: JoinDescriptor; - private readonly _rightSource?: ESTermSource; + private readonly _rightSource?: ITermJoinSource; private readonly _leftField?: IField; constructor(joinDescriptor: JoinDescriptor, leftSource: IVectorSource) { this._descriptor = joinDescriptor; const inspectorAdapters = leftSource.getInspectorAdapters(); - if ( - joinDescriptor.right && - 'indexPatternId' in joinDescriptor.right && - 'term' in joinDescriptor.right - ) { - this._rightSource = new ESTermSource(joinDescriptor.right, inspectorAdapters); - } + this._rightSource = createJoinTermSource(this._descriptor.right, inspectorAdapters); this._leftField = joinDescriptor.leftField ? leftSource.createField({ fieldName: joinDescriptor.leftField }) : undefined; @@ -47,8 +69,8 @@ export class InnerJoin { return this._leftField && this._rightSource ? this._rightSource.hasCompleteConfig() : false; } - getJoinFields() { - return this._rightSource ? this._rightSource.getMetricFields() : []; + getJoinFields(): IField[] { + return this._rightSource ? this._rightSource.getRightFields() : []; } // Source request id must be static and unique because the re-fetch logic uses the id to locate the previous request. @@ -77,7 +99,7 @@ export class InnerJoin { if (!feature.properties || !this._leftField || !this._rightSource) { return false; } - const rightMetricFields = this._rightSource.getMetricFields(); + const rightMetricFields: IField[] = this._rightSource.getRightFields(); // delete feature properties added by previous join for (let j = 0; j < rightMetricFields.length; j++) { const metricPropertyKey = rightMetricFields[j].getName(); @@ -106,7 +128,7 @@ export class InnerJoin { } } - getRightJoinSource(): ESTermSource { + getRightJoinSource(): ITermJoinSource { if (!this._rightSource) { throw new Error('Cannot get rightSource from InnerJoin with incomplete config'); } diff --git a/x-pack/plugins/maps/public/classes/layers/layer.test.ts b/x-pack/plugins/maps/public/classes/layers/layer.test.ts index e669ddf13e5ac..d8e6a4906a63a 100644 --- a/x-pack/plugins/maps/public/classes/layers/layer.test.ts +++ b/x-pack/plugins/maps/public/classes/layers/layer.test.ts @@ -7,7 +7,13 @@ import { AbstractLayer } from './layer'; import { ISource } from '../sources/source'; -import { AGG_TYPE, FIELD_ORIGIN, LAYER_STYLE_TYPE, VECTOR_STYLES } from '../../../common/constants'; +import { + AGG_TYPE, + FIELD_ORIGIN, + LAYER_STYLE_TYPE, + SOURCE_TYPES, + VECTOR_STYLES, +} from '../../../common/constants'; import { ESTermSourceDescriptor, VectorStyleDescriptor } from '../../../common/descriptor_types'; import { getDefaultDynamicProperties } from '../styles/vector/vector_style_defaults'; @@ -73,7 +79,7 @@ describe('cloneDescriptor', () => { indexPatternTitle: 'logs-*', metrics: [{ type: AGG_TYPE.COUNT }], term: 'myTermField', - type: 'joinSource', + type: SOURCE_TYPES.ES_TERM_SOURCE, applyGlobalQuery: true, applyGlobalTime: true, }, diff --git a/x-pack/plugins/maps/public/classes/layers/layer.tsx b/x-pack/plugins/maps/public/classes/layers/layer.tsx index fe13e4f0ac2f6..1596c392e8d63 100644 --- a/x-pack/plugins/maps/public/classes/layers/layer.tsx +++ b/x-pack/plugins/maps/public/classes/layers/layer.tsx @@ -20,11 +20,13 @@ import { MB_SOURCE_ID_LAYER_ID_PREFIX_DELIMITER, MIN_ZOOM, SOURCE_DATA_REQUEST_ID, + SOURCE_TYPES, STYLE_TYPE, } from '../../../common/constants'; import { copyPersistentState } from '../../reducers/util'; import { AggDescriptor, + ESTermSourceDescriptor, JoinDescriptor, LayerDescriptor, MapExtent, @@ -158,6 +160,14 @@ export class AbstractLayer implements ILayer { if (clonedDescriptor.joins) { clonedDescriptor.joins.forEach((joinDescriptor: JoinDescriptor) => { + if (joinDescriptor.right && joinDescriptor.right.type === SOURCE_TYPES.TABLE_SOURCE) { + throw new Error( + 'Cannot clone table-source. Should only be used in MapEmbeddable, not in UX' + ); + } + const termSourceDescriptor: ESTermSourceDescriptor = joinDescriptor.right as ESTermSourceDescriptor; + + // todo: must tie this to generic thing const originalJoinId = joinDescriptor.right.id!; // right.id is uuid used to track requests in inspector @@ -166,8 +176,8 @@ export class AbstractLayer implements ILayer { // Update all data driven styling properties using join fields if (clonedDescriptor.style && 'properties' in clonedDescriptor.style) { const metrics = - joinDescriptor.right.metrics && joinDescriptor.right.metrics.length - ? joinDescriptor.right.metrics + termSourceDescriptor.metrics && termSourceDescriptor.metrics.length + ? termSourceDescriptor.metrics : [{ type: AGG_TYPE.COUNT }]; metrics.forEach((metricsDescriptor: AggDescriptor) => { const originalJoinKey = getJoinAggKey({ diff --git a/x-pack/plugins/maps/public/classes/layers/vector_layer/vector_layer.tsx b/x-pack/plugins/maps/public/classes/layers/vector_layer/vector_layer.tsx index 2304bb277da49..e3a80a4c9eb5d 100644 --- a/x-pack/plugins/maps/public/classes/layers/vector_layer/vector_layer.tsx +++ b/x-pack/plugins/maps/public/classes/layers/vector_layer/vector_layer.tsx @@ -63,6 +63,7 @@ import { ITooltipProperty } from '../../tooltips/tooltip_property'; import { IDynamicStyleProperty } from '../../styles/vector/properties/dynamic_style_property'; import { IESSource } from '../../sources/es_source'; import { PropertiesMap } from '../../../../common/elasticsearch_util'; +import { ITermJoinSource } from '../../sources/term_join_source'; interface SourceResult { refreshed: boolean; @@ -574,7 +575,7 @@ export class VectorLayer extends AbstractLayer { dynamicStyleProps: this.getCurrentStyle() .getDynamicPropertiesArray() .filter((dynamicStyleProp) => { - const matchingField = joinSource.getMetricFieldForName(dynamicStyleProp.getFieldName()); + const matchingField = joinSource.getFieldByName(dynamicStyleProp.getFieldName()); return ( dynamicStyleProp.getFieldOrigin() === FIELD_ORIGIN.JOIN && !!matchingField && @@ -599,7 +600,7 @@ export class VectorLayer extends AbstractLayer { }: { dataRequestId: string; dynamicStyleProps: Array>; - source: IVectorSource; + source: IVectorSource | ITermJoinSource; sourceQuery?: MapQuery; style: IVectorStyle; } & DataRequestContext) { @@ -679,7 +680,7 @@ export class VectorLayer extends AbstractLayer { fields: style .getDynamicPropertiesArray() .filter((dynamicStyleProp) => { - const matchingField = joinSource.getMetricFieldForName(dynamicStyleProp.getFieldName()); + const matchingField = joinSource.getFieldByName(dynamicStyleProp.getFieldName()); return dynamicStyleProp.getFieldOrigin() === FIELD_ORIGIN.JOIN && !!matchingField; }) .map((dynamicStyleProp) => { @@ -699,7 +700,7 @@ export class VectorLayer extends AbstractLayer { }: { dataRequestId: string; fields: IField[]; - source: IVectorSource; + source: IVectorSource | ITermJoinSource; } & DataRequestContext) { if (fields.length === 0) { return; diff --git a/x-pack/plugins/maps/public/classes/sources/es_agg_source/es_agg_source.ts b/x-pack/plugins/maps/public/classes/sources/es_agg_source/es_agg_source.ts index 77177dd47a166..5cb299ac33ff8 100644 --- a/x-pack/plugins/maps/public/classes/sources/es_agg_source/es_agg_source.ts +++ b/x-pack/plugins/maps/public/classes/sources/es_agg_source/es_agg_source.ts @@ -23,7 +23,6 @@ export interface IESAggSource extends IESSource { getAggKey(aggType: AGG_TYPE, fieldName: string): string; getAggLabel(aggType: AGG_TYPE, fieldLabel: string): string; getMetricFields(): IESAggField[]; - hasMatchingMetricField(fieldName: string): boolean; getMetricFieldForName(fieldName: string): IESAggField | null; getValueAggsDsl(indexPattern: IndexPattern): { [key: string]: unknown }; } @@ -74,11 +73,6 @@ export abstract class AbstractESAggSource extends AbstractESSource implements IE throw new Error('Cannot create a new field from just a fieldname for an es_agg_source.'); } - hasMatchingMetricField(fieldName: string): boolean { - const matchingField = this.getMetricFieldForName(fieldName); - return !!matchingField; - } - getMetricFieldForName(fieldName: string): IESAggField | null { const targetMetricField = this.getMetricFields().find((metricField: IESAggField) => { return metricField.getName() === fieldName; diff --git a/x-pack/plugins/maps/public/classes/sources/es_term_source/es_term_source.ts b/x-pack/plugins/maps/public/classes/sources/es_term_source/es_term_source.ts index 235e8e3a651ee..c7107964568c9 100644 --- a/x-pack/plugins/maps/public/classes/sources/es_term_source/es_term_source.ts +++ b/x-pack/plugins/maps/public/classes/sources/es_term_source/es_term_source.ts @@ -30,6 +30,8 @@ import { import { Adapters } from '../../../../../../../src/plugins/inspector/common/adapters'; import { PropertiesMap } from '../../../../common/elasticsearch_util'; import { isValidStringConfig } from '../../util/valid_string_config'; +import { ITermJoinSource } from '../term_join_source/term_join_source'; +import { IField } from '../../fields/field'; const TERMS_AGG_NAME = 'join'; const TERMS_BUCKET_KEYS_TO_IGNORE = ['key', 'doc_count']; @@ -47,7 +49,7 @@ export function extractPropertiesMap(rawEsData: any, countPropertyName: string): return propertiesMap; } -export class ESTermSource extends AbstractESAggSource { +export class ESTermSource extends AbstractESAggSource implements ITermJoinSource { static type = SOURCE_TYPES.ES_TERM_SOURCE; static createDescriptor(descriptor: Partial): ESTermSourceDescriptor { @@ -79,7 +81,7 @@ export class ESTermSource extends AbstractESAggSource { }); } - hasCompleteConfig() { + hasCompleteConfig(): boolean { return _.has(this._descriptor, 'indexPatternId') && _.has(this._descriptor, 'term'); } @@ -174,4 +176,8 @@ export class ESTermSource extends AbstractESAggSource { } : null; } + + getRightFields(): IField[] { + return this.getMetricFields(); + } } diff --git a/x-pack/plugins/maps/public/classes/sources/geojson_file_source/geojson_file_source.ts b/x-pack/plugins/maps/public/classes/sources/geojson_file_source/geojson_file_source.ts index 69d84dc65d382..35464b24185d0 100644 --- a/x-pack/plugins/maps/public/classes/sources/geojson_file_source/geojson_file_source.ts +++ b/x-pack/plugins/maps/public/classes/sources/geojson_file_source/geojson_file_source.ts @@ -8,15 +8,15 @@ import { Feature, FeatureCollection } from 'geojson'; import { AbstractVectorSource, BoundsFilters, GeoJsonWithMeta } from '../vector_source'; import { EMPTY_FEATURE_COLLECTION, FIELD_ORIGIN, SOURCE_TYPES } from '../../../../common/constants'; import { - GeoJsonFileFieldDescriptor, + InlineFieldDescriptor, GeojsonFileSourceDescriptor, MapExtent, } from '../../../../common/descriptor_types'; import { registerSource } from '../source_registry'; import { IField } from '../../fields/field'; import { getFeatureCollectionBounds } from '../../util/get_feature_collection_bounds'; -import { GeoJsonFileField } from '../../fields/geojson_file_field'; import { Adapters } from '../../../../../../../src/plugins/inspector/common/adapters'; +import { InlineField } from '../../fields/inline_field'; function getFeatureCollection( geoJson: Feature | FeatureCollection | null | undefined @@ -56,14 +56,14 @@ export class GeoJsonFileSource extends AbstractVectorSource { super(normalizedDescriptor, inspectorAdapters); } - _getFields(): GeoJsonFileFieldDescriptor[] { + _getFields(): InlineFieldDescriptor[] { const fields = (this._descriptor as GeojsonFileSourceDescriptor).__fields; return fields ? fields : []; } createField({ fieldName }: { fieldName: string }): IField { const fields = this._getFields(); - const descriptor: GeoJsonFileFieldDescriptor | undefined = fields.find((field) => { + const descriptor: InlineFieldDescriptor | undefined = fields.find((field) => { return field.name === fieldName; }); @@ -74,7 +74,7 @@ export class GeoJsonFileSource extends AbstractVectorSource { )} ` ); } - return new GeoJsonFileField({ + return new InlineField({ fieldName: descriptor.name, source: this, origin: FIELD_ORIGIN.SOURCE, @@ -84,8 +84,8 @@ export class GeoJsonFileSource extends AbstractVectorSource { async getFields(): Promise { const fields = this._getFields(); - return fields.map((field: GeoJsonFileFieldDescriptor) => { - return new GeoJsonFileField({ + return fields.map((field: InlineFieldDescriptor) => { + return new InlineField({ fieldName: field.name, source: this, origin: FIELD_ORIGIN.SOURCE, diff --git a/x-pack/plugins/maps/public/classes/sources/table_source/index.ts b/x-pack/plugins/maps/public/classes/sources/table_source/index.ts new file mode 100644 index 0000000000000..7258e6b464cd0 --- /dev/null +++ b/x-pack/plugins/maps/public/classes/sources/table_source/index.ts @@ -0,0 +1,7 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License; + * you may not use this file except in compliance with the Elastic License. + */ + +export { TableSource } from './table_source'; diff --git a/x-pack/plugins/maps/public/classes/sources/table_source/table_source.test.ts b/x-pack/plugins/maps/public/classes/sources/table_source/table_source.test.ts new file mode 100644 index 0000000000000..9409eefa4ae07 --- /dev/null +++ b/x-pack/plugins/maps/public/classes/sources/table_source/table_source.test.ts @@ -0,0 +1,200 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License; + * you may not use this file except in compliance with the Elastic License. + */ + +import { TableSource } from './table_source'; +import { FIELD_ORIGIN } from '../../../../common/constants'; +import { + MapFilters, + MapQuery, + VectorJoinSourceRequestMeta, + VectorSourceSyncMeta, +} from '../../../../common/descriptor_types'; + +describe('TableSource', () => { + describe('getName', () => { + it('should get default display name', async () => { + const tableSource = new TableSource({}); + expect((await tableSource.getDisplayName()).startsWith('table source')).toBe(true); + }); + }); + + describe('getPropertiesMap', () => { + it('should roll up results', async () => { + const tableSource = new TableSource({ + term: 'iso', + __rows: [ + { + iso: 'US', + population: 100, + }, + { + iso: 'CN', + population: 400, + foo: 'bar', // ignore this prop, not defined in `__columns` + }, + { + // ignore this row, cannot be joined + population: 400, + }, + { + // row ignored since it's not first row with key 'US' + iso: 'US', + population: -1, + }, + ], + __columns: [ + { + name: 'iso', + type: 'string', + }, + { + name: 'population', + type: 'number', + }, + ], + }); + + const propertiesMap = await tableSource.getPropertiesMap( + ({} as unknown) as VectorJoinSourceRequestMeta, + 'a', + 'b', + () => {} + ); + + expect(propertiesMap.size).toEqual(2); + expect(propertiesMap.get('US')).toEqual({ + population: 100, + }); + expect(propertiesMap.get('CN')).toEqual({ + population: 400, + }); + }); + }); + + describe('getTermField', () => { + it('should throw when no match', async () => { + const tableSource = new TableSource({ + term: 'foobar', + __columns: [ + { + name: 'iso', + type: 'string', + }, + { + name: 'population', + type: 'number', + }, + ], + }); + + expect(() => { + tableSource.getTermField(); + }).toThrow(); + }); + + it('should return field', async () => { + const tableSource = new TableSource({ + term: 'iso', + __columns: [ + { + name: 'iso', + type: 'string', + }, + { + name: 'population', + type: 'number', + }, + ], + }); + + const termField = tableSource.getTermField(); + expect(termField.getName()).toEqual('iso'); + expect(await termField.getDataType()).toEqual('string'); + }); + }); + + describe('getRightFields', () => { + it('should return columns', async () => { + const tableSource = new TableSource({ + term: 'foobar', + __columns: [ + { + name: 'iso', + type: 'string', + }, + { + name: 'population', + type: 'number', + }, + ], + }); + + const rightFields = tableSource.getRightFields(); + expect(rightFields[0].getName()).toEqual('iso'); + expect(await rightFields[0].getDataType()).toEqual('string'); + expect(rightFields[0].getOrigin()).toEqual(FIELD_ORIGIN.JOIN); + expect(rightFields[0].getSource()).toEqual(tableSource); + + expect(rightFields[1].getName()).toEqual('population'); + expect(await rightFields[1].getDataType()).toEqual('number'); + expect(rightFields[1].getOrigin()).toEqual(FIELD_ORIGIN.JOIN); + expect(rightFields[1].getSource()).toEqual(tableSource); + }); + }); + + describe('getFieldByName', () => { + it('should return columns', async () => { + const tableSource = new TableSource({ + term: 'foobar', + __columns: [ + { + name: 'iso', + type: 'string', + }, + { + name: 'population', + type: 'number', + }, + ], + }); + + const field = tableSource.getFieldByName('iso'); + expect(field!.getName()).toEqual('iso'); + expect(await field!.getDataType()).toEqual('string'); + expect(field!.getOrigin()).toEqual(FIELD_ORIGIN.JOIN); + expect(field!.getSource()).toEqual(tableSource); + }); + }); + + describe('getGeoJsonWithMeta', () => { + it('should throw - not implemented', async () => { + const tableSource = new TableSource({}); + + let didThrow = false; + try { + await tableSource.getGeoJsonWithMeta( + 'foobar', + ({} as unknown) as MapFilters & { + applyGlobalQuery: boolean; + applyGlobalTime: boolean; + fieldNames: string[]; + geogridPrecision?: number; + sourceQuery?: MapQuery; + sourceMeta: VectorSourceSyncMeta; + }, + () => {}, + () => { + return false; + } + ); + } catch (e) { + didThrow = true; + } finally { + expect(didThrow).toBe(true); + } + }); + }); +}); diff --git a/x-pack/plugins/maps/public/classes/sources/table_source/table_source.ts b/x-pack/plugins/maps/public/classes/sources/table_source/table_source.ts new file mode 100644 index 0000000000000..d157c4f5df60a --- /dev/null +++ b/x-pack/plugins/maps/public/classes/sources/table_source/table_source.ts @@ -0,0 +1,218 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License; + * you may not use this file except in compliance with the Elastic License. + */ + +import uuid from 'uuid'; +import { FIELD_ORIGIN, SOURCE_TYPES, VECTOR_SHAPE_TYPE } from '../../../../common/constants'; +import { + MapExtent, + MapFilters, + MapQuery, + TableSourceDescriptor, + VectorJoinSourceRequestMeta, + VectorSourceSyncMeta, +} from '../../../../common/descriptor_types'; +import { Adapters } from '../../../../../../../src/plugins/inspector/common/adapters'; +import { ITermJoinSource } from '../term_join_source'; +import { BucketProperties, PropertiesMap } from '../../../../common/elasticsearch_util'; +import { IField } from '../../fields/field'; +import { Query } from '../../../../../../../src/plugins/data/common/query'; +import { + AbstractVectorSource, + BoundsFilters, + GeoJsonWithMeta, + IVectorSource, + SourceTooltipConfig, +} from '../vector_source'; +import { DataRequest } from '../../util/data_request'; +import { InlineField } from '../../fields/inline_field'; + +export class TableSource extends AbstractVectorSource implements ITermJoinSource, IVectorSource { + static type = SOURCE_TYPES.TABLE_SOURCE; + + static createDescriptor(descriptor: Partial): TableSourceDescriptor { + return { + type: SOURCE_TYPES.TABLE_SOURCE, + __rows: descriptor.__rows || [], + __columns: descriptor.__columns || [], + term: descriptor.term || '', + id: descriptor.id || uuid(), + }; + } + + readonly _descriptor: TableSourceDescriptor; + + constructor(descriptor: Partial, inspectorAdapters?: Adapters) { + const sourceDescriptor = TableSource.createDescriptor(descriptor); + super(sourceDescriptor, inspectorAdapters); + this._descriptor = sourceDescriptor; + } + + async getDisplayName(): Promise { + // no need to localize. this is never rendered. + return `table source ${uuid()}`; + } + + getSyncMeta(): VectorSourceSyncMeta | null { + return null; + } + + async getPropertiesMap( + searchFilters: VectorJoinSourceRequestMeta, + leftSourceName: string, + leftFieldName: string, + registerCancelCallback: (callback: () => void) => void + ): Promise { + const propertiesMap: PropertiesMap = new Map(); + + const fieldNames = await this.getFieldNames(); + + for (let i = 0; i < this._descriptor.__rows.length; i++) { + const row: { [key: string]: string | number } = this._descriptor.__rows[i]; + let propKey: string | number | undefined; + const props: { [key: string]: string | number } = {}; + for (const key in row) { + if (row.hasOwnProperty(key)) { + if (key === this._descriptor.term && row[key]) { + propKey = row[key]; + } + if (fieldNames.indexOf(key) >= 0 && key !== this._descriptor.term) { + props[key] = row[key]; + } + } + } + if (propKey && !propertiesMap.has(propKey.toString())) { + // If propKey is not a primary key in the table, this will favor the first match + propertiesMap.set(propKey.toString(), props); + } + } + + return propertiesMap; + } + + getTermField(): IField { + const column = this._descriptor.__columns.find((c) => { + return c.name === this._descriptor.term; + }); + + if (!column) { + throw new Error( + `Cannot find column for ${this._descriptor.term} in ${JSON.stringify( + this._descriptor.__columns + )}` + ); + } + + return new InlineField({ + fieldName: column.name, + source: this, + origin: FIELD_ORIGIN.JOIN, + dataType: column.type, + }); + } + + getWhereQuery(): Query | undefined { + return undefined; + } + + hasCompleteConfig(): boolean { + return true; + } + + getId(): string { + return this._descriptor.id; + } + + getRightFields(): IField[] { + return this._descriptor.__columns.map((column) => { + return new InlineField({ + fieldName: column.name, + source: this, + origin: FIELD_ORIGIN.JOIN, + dataType: column.type, + }); + }); + } + + getFieldNames(): string[] { + return this._descriptor.__columns.map((column) => { + return column.name; + }); + } + + canFormatFeatureProperties(): boolean { + return false; + } + + createField({ fieldName }: { fieldName: string }): IField { + const field = this.getFieldByName(fieldName); + if (!field) { + throw new Error(`Cannot find field for ${fieldName}`); + } + return field; + } + + async getBoundsForFilters( + boundsFilters: BoundsFilters, + registerCancelCallback: (callback: () => void) => void + ): Promise { + return null; + } + + getFieldByName(fieldName: string): IField | null { + const column = this._descriptor.__columns.find((c) => { + return c.name === fieldName; + }); + + if (!column) { + return null; + } + + return new InlineField({ + fieldName: column.name, + source: this, + origin: FIELD_ORIGIN.JOIN, + dataType: column.type, + }); + } + + getFields(): Promise { + throw new Error('must implement'); + } + + // The below is the IVectorSource interface. + // Could be useful to implement, e.g. to preview raw csv data + async getGeoJsonWithMeta( + layerName: string, + searchFilters: MapFilters & { + applyGlobalQuery: boolean; + applyGlobalTime: boolean; + fieldNames: string[]; + geogridPrecision?: number; + sourceQuery?: MapQuery; + sourceMeta: VectorSourceSyncMeta; + }, + registerCancelCallback: (callback: () => void) => void, + isRequestStillActive: () => boolean + ): Promise { + throw new Error('TableSource cannot return GeoJson'); + } + + async getLeftJoinFields(): Promise { + throw new Error('TableSource cannot be used as a left-layer in a term join'); + } + + getSourceTooltipContent(sourceDataRequest?: DataRequest): SourceTooltipConfig { + throw new Error('must add tooltip content'); + } + + async getSupportedShapeTypes(): Promise { + return []; + } + + isBoundsAware(): boolean { + return false; + } +} diff --git a/x-pack/plugins/maps/public/classes/sources/term_join_source/index.ts b/x-pack/plugins/maps/public/classes/sources/term_join_source/index.ts new file mode 100644 index 0000000000000..1879d64d3b207 --- /dev/null +++ b/x-pack/plugins/maps/public/classes/sources/term_join_source/index.ts @@ -0,0 +1,7 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License; + * you may not use this file except in compliance with the Elastic License. + */ + +export { ITermJoinSource } from './term_join_source'; diff --git a/x-pack/plugins/maps/public/classes/sources/term_join_source/term_join_source.ts b/x-pack/plugins/maps/public/classes/sources/term_join_source/term_join_source.ts new file mode 100644 index 0000000000000..534ac9f200362 --- /dev/null +++ b/x-pack/plugins/maps/public/classes/sources/term_join_source/term_join_source.ts @@ -0,0 +1,33 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License; + * you may not use this file except in compliance with the Elastic License. + */ + +import { GeoJsonProperties } from 'geojson'; +import { IField } from '../../fields/field'; +import { Query } from '../../../../../../../src/plugins/data/common/query'; +import { + VectorJoinSourceRequestMeta, + VectorSourceSyncMeta, +} from '../../../../common/descriptor_types'; +import { PropertiesMap } from '../../../../common/elasticsearch_util'; +import { ITooltipProperty } from '../../tooltips/tooltip_property'; +import { ISource } from '../source'; + +export interface ITermJoinSource extends ISource { + hasCompleteConfig(): boolean; + getTermField(): IField; + getWhereQuery(): Query | undefined; + getPropertiesMap( + searchFilters: VectorJoinSourceRequestMeta, + leftSourceName: string, + leftFieldName: string, + registerCancelCallback: (callback: () => void) => void + ): Promise; + getSyncMeta(): VectorSourceSyncMeta | null; + getId(): string; + getRightFields(): IField[]; + getTooltipProperties(properties: GeoJsonProperties): Promise; + getFieldByName(fieldName: string): IField | null; +} diff --git a/x-pack/plugins/maps/public/classes/styles/vector/properties/dynamic_style_property.tsx b/x-pack/plugins/maps/public/classes/styles/vector/properties/dynamic_style_property.tsx index 882247e375ddc..96494a346e625 100644 --- a/x-pack/plugins/maps/public/classes/styles/vector/properties/dynamic_style_property.tsx +++ b/x-pack/plugins/maps/public/classes/styles/vector/properties/dynamic_style_property.tsx @@ -97,7 +97,7 @@ export class DynamicStyleProperty } const join = this._layer.getValidJoins().find((validJoin: InnerJoin) => { - return validJoin.getRightJoinSource().hasMatchingMetricField(fieldName); + return !!validJoin.getRightJoinSource().getFieldByName(fieldName); }); return join ? join.getSourceMetaDataRequestId() : null; } diff --git a/x-pack/plugins/maps/public/classes/styles/vector/vector_style.tsx b/x-pack/plugins/maps/public/classes/styles/vector/vector_style.tsx index 9bf4cafd66407..126f19b7012f8 100644 --- a/x-pack/plugins/maps/public/classes/styles/vector/vector_style.tsx +++ b/x-pack/plugins/maps/public/classes/styles/vector/vector_style.tsx @@ -620,7 +620,7 @@ export class VectorStyle implements IVectorStyle { dataRequestId = SOURCE_FORMATTERS_DATA_REQUEST_ID; } else { const targetJoin = this._layer.getValidJoins().find((join) => { - return join.getRightJoinSource().hasMatchingMetricField(fieldName); + return !!join.getRightJoinSource().getFieldByName(fieldName); }); if (targetJoin) { dataRequestId = targetJoin.getSourceFormattersDataRequestId(); @@ -841,7 +841,7 @@ export class VectorStyle implements IVectorStyle { this._iconOrientationProperty.syncIconRotationWithMb(symbolLayerId, mbMap); } - _makeField(fieldDescriptor?: StylePropertyField) { + _makeField(fieldDescriptor?: StylePropertyField): IField | null { if (!fieldDescriptor || !fieldDescriptor.name) { return null; } @@ -852,10 +852,10 @@ export class VectorStyle implements IVectorStyle { return this._source.getFieldByName(fieldDescriptor.name); } else if (fieldDescriptor.origin === FIELD_ORIGIN.JOIN) { const targetJoin = this._layer.getValidJoins().find((join) => { - return join.getRightJoinSource().hasMatchingMetricField(fieldDescriptor.name); + return !!join.getRightJoinSource().getFieldByName(fieldDescriptor.name); }); return targetJoin - ? targetJoin.getRightJoinSource().getMetricFieldForName(fieldDescriptor.name) + ? targetJoin.getRightJoinSource().getFieldByName(fieldDescriptor.name) : null; } else { throw new Error(`Unknown origin-type ${fieldDescriptor.origin}`); diff --git a/x-pack/plugins/maps/public/connected_components/layer_panel/join_editor/join_editor.tsx b/x-pack/plugins/maps/public/connected_components/layer_panel/join_editor/join_editor.tsx index d47f130d4ede3..ce5c0ed5fdcad 100644 --- a/x-pack/plugins/maps/public/connected_components/layer_panel/join_editor/join_editor.tsx +++ b/x-pack/plugins/maps/public/connected_components/layer_panel/join_editor/join_editor.tsx @@ -5,7 +5,6 @@ */ import React, { Fragment } from 'react'; -import _ from 'lodash'; import uuid from 'uuid/v4'; import { @@ -24,6 +23,7 @@ import { Join } from './resources/join'; import { ILayer } from '../../../classes/layers/layer'; import { JoinDescriptor } from '../../../../common/descriptor_types'; import { IField } from '../../../classes/fields/field'; +import { SOURCE_TYPES } from '../../../../common/constants'; export interface Props { joins: JoinDescriptor[]; @@ -44,19 +44,25 @@ export function JoinEditor({ joins, layer, onChange, leftJoinFields, layerDispla onChange(layer, [...joins.slice(0, index), ...joins.slice(index + 1)]); }; - return ( - - - - - ); + if (joinDescriptor.right.type === SOURCE_TYPES.TABLE_SOURCE) { + throw new Error( + 'PEBKAC - Table sources cannot be edited in the UX and should only be used in MapEmbeddable' + ); + } else { + return ( + + + + + ); + } }); }; diff --git a/x-pack/plugins/maps/public/connected_components/layer_panel/join_editor/resources/join.js b/x-pack/plugins/maps/public/connected_components/layer_panel/join_editor/resources/join.js index 507b32fa39fd8..a46b27b62a19e 100644 --- a/x-pack/plugins/maps/public/connected_components/layer_panel/join_editor/resources/join.js +++ b/x-pack/plugins/maps/public/connected_components/layer_panel/join_editor/resources/join.js @@ -17,6 +17,7 @@ import { GlobalTimeCheckbox } from '../../../../components/global_time_checkbox' import { indexPatterns } from '../../../../../../../../src/plugins/data/public'; import { getIndexPatternService } from '../../../../kibana_services'; +import { SOURCE_TYPES } from '../../../../../common/constants'; export class Join extends Component { state = { @@ -85,6 +86,7 @@ export class Join extends Component { ...restOfRight, indexPatternId, indexPatternTitle, + type: SOURCE_TYPES.ES_TERM_SOURCE, }, }); }; diff --git a/x-pack/plugins/maps/server/sample_data/ecommerce_saved_objects.js b/x-pack/plugins/maps/server/sample_data/ecommerce_saved_objects.js index a3dbf8b1438fa..d1aa044676e00 100644 --- a/x-pack/plugins/maps/server/sample_data/ecommerce_saved_objects.js +++ b/x-pack/plugins/maps/server/sample_data/ecommerce_saved_objects.js @@ -70,6 +70,7 @@ const layerList = [ { leftField: 'iso2', right: { + type: 'ES_TERM_SOURCE', id: '741db9c6-8ebb-4ea9-9885-b6b4ac019d14', indexPatternTitle: 'kibana_sample_data_ecommerce', term: 'geoip.country_iso_code', @@ -134,6 +135,7 @@ const layerList = [ { leftField: 'name', right: { + type: 'ES_TERM_SOURCE', id: '30a0ec24-49b6-476a-b4ed-6c1636333695', indexPatternTitle: 'kibana_sample_data_ecommerce', term: 'geoip.region_name', @@ -198,6 +200,7 @@ const layerList = [ { leftField: 'label_en', right: { + type: 'ES_TERM_SOURCE', id: 'e325c9da-73fa-4b3b-8b59-364b99370826', indexPatternTitle: 'kibana_sample_data_ecommerce', term: 'geoip.region_name', @@ -262,6 +265,7 @@ const layerList = [ { leftField: 'label_en', right: { + type: 'ES_TERM_SOURCE', id: '612d805d-8533-43a9-ac0e-cbf51fe63dcd', indexPatternTitle: 'kibana_sample_data_ecommerce', term: 'geoip.region_name', diff --git a/x-pack/plugins/maps/server/sample_data/web_logs_saved_objects.js b/x-pack/plugins/maps/server/sample_data/web_logs_saved_objects.js index ec445567de21c..010f06e00ca3f 100644 --- a/x-pack/plugins/maps/server/sample_data/web_logs_saved_objects.js +++ b/x-pack/plugins/maps/server/sample_data/web_logs_saved_objects.js @@ -70,6 +70,7 @@ const layerList = [ { leftField: 'iso2', right: { + type: 'ES_TERM_SOURCE', id: '673ff994-fc75-4c67-909b-69fcb0e1060e', indexPatternTitle: 'kibana_sample_data_logs', term: 'geo.src', diff --git a/x-pack/plugins/maps/server/saved_objects/migrations.js b/x-pack/plugins/maps/server/saved_objects/migrations.js index 653f07772ee58..346bc5eff1657 100644 --- a/x-pack/plugins/maps/server/saved_objects/migrations.js +++ b/x-pack/plugins/maps/server/saved_objects/migrations.js @@ -14,6 +14,7 @@ import { migrateUseTopHitsToScalingType } from '../../common/migrations/scaling_ import { migrateJoinAggKey } from '../../common/migrations/join_agg_key'; import { removeBoundsFromSavedObject } from '../../common/migrations/remove_bounds'; import { setDefaultAutoFitToBounds } from '../../common/migrations/set_default_auto_fit_to_bounds'; +import { addTypeToTermJoin } from '../../common/migrations/add_type_to_termjoin'; export const migrations = { map: { @@ -79,6 +80,14 @@ export const migrations = { '7.10.0': (doc) => { const attributes = setDefaultAutoFitToBounds(doc); + return { + ...doc, + attributes, + }; + }, + '7.12.0': (doc) => { + const attributes = addTypeToTermJoin(doc); + return { ...doc, attributes, diff --git a/x-pack/plugins/ml/common/types/anomaly_detection_jobs/combined_job.ts b/x-pack/plugins/ml/common/types/anomaly_detection_jobs/combined_job.ts index 42dee46e71fd6..dba7d006da282 100644 --- a/x-pack/plugins/ml/common/types/anomaly_detection_jobs/combined_job.ts +++ b/x-pack/plugins/ml/common/types/anomaly_detection_jobs/combined_job.ts @@ -4,7 +4,6 @@ * you may not use this file except in compliance with the Elastic License. */ -import { cloneDeep } from 'lodash'; import { Datafeed } from './datafeed'; import { DatafeedStats } from './datafeed_stats'; import { Job } from './job'; @@ -25,16 +24,6 @@ export interface CombinedJobWithStats extends JobWithStats { datafeed_config: DatafeedWithStats; } -export function expandCombinedJobConfig(combinedJob: CombinedJob) { - const combinedJobClone = cloneDeep(combinedJob); - const job = combinedJobClone; - const datafeed = combinedJobClone.datafeed_config; - // @ts-expect-error - delete job.datafeed_config; - - return { job, datafeed }; -} - export function isCombinedJobWithStats(arg: any): arg is CombinedJobWithStats { return typeof arg.job_id === 'string'; } diff --git a/x-pack/plugins/ml/public/application/components/scatterplot_matrix/index.ts b/x-pack/plugins/ml/public/application/components/scatterplot_matrix/index.ts index 4f564dde8cb43..903fe5b6ed985 100644 --- a/x-pack/plugins/ml/public/application/components/scatterplot_matrix/index.ts +++ b/x-pack/plugins/ml/public/application/components/scatterplot_matrix/index.ts @@ -4,5 +4,7 @@ * you may not use this file except in compliance with the Elastic License. */ +export { useScatterplotFieldOptions } from './use_scatterplot_field_options'; export { LEGEND_TYPES } from './scatterplot_matrix_vega_lite_spec'; export { ScatterplotMatrix } from './scatterplot_matrix'; +export type { ScatterplotMatrixViewProps as ScatterplotMatrixProps } from './scatterplot_matrix_view'; diff --git a/x-pack/plugins/ml/public/application/components/scatterplot_matrix/scatterplot_matrix.tsx b/x-pack/plugins/ml/public/application/components/scatterplot_matrix/scatterplot_matrix.tsx index b1ee9afb17788..a90fe924b91ac 100644 --- a/x-pack/plugins/ml/public/application/components/scatterplot_matrix/scatterplot_matrix.tsx +++ b/x-pack/plugins/ml/public/application/components/scatterplot_matrix/scatterplot_matrix.tsx @@ -4,316 +4,15 @@ * you may not use this file except in compliance with the Elastic License. */ -import React, { useMemo, useEffect, useState, FC } from 'react'; +import React, { FC, Suspense } from 'react'; -// There is still an issue with Vega Lite's typings with the strict mode Kibana is using. -// @ts-ignore -import { compile } from 'vega-lite/build-es5/vega-lite'; -import { parse, View, Warn } from 'vega'; -import { Handler } from 'vega-tooltip'; +import type { ScatterplotMatrixViewProps } from './scatterplot_matrix_view'; +import { ScatterplotMatrixLoading } from './scatterplot_matrix_loading'; -import { - htmlIdGenerator, - EuiComboBox, - EuiComboBoxOptionOption, - EuiFlexGroup, - EuiFlexItem, - EuiFormRow, - EuiLoadingSpinner, - EuiSelect, - EuiSpacer, - EuiSwitch, - EuiText, -} from '@elastic/eui'; +const ScatterplotMatrixLazy = React.lazy(() => import('./scatterplot_matrix_view')); -import { i18n } from '@kbn/i18n'; - -import type { SearchResponse7 } from '../../../../common/types/es_client'; - -import { useMlApiContext } from '../../contexts/kibana'; - -import { getProcessedFields } from '../data_grid'; -import { useCurrentEuiTheme } from '../color_range_legend'; - -import { - getScatterplotMatrixVegaLiteSpec, - LegendType, - OUTLIER_SCORE_FIELD, -} from './scatterplot_matrix_vega_lite_spec'; - -import './scatterplot_matrix.scss'; - -const SCATTERPLOT_MATRIX_DEFAULT_FIELDS = 4; -const SCATTERPLOT_MATRIX_DEFAULT_FETCH_SIZE = 1000; -const SCATTERPLOT_MATRIX_DEFAULT_FETCH_MIN_SIZE = 1; -const SCATTERPLOT_MATRIX_DEFAULT_FETCH_MAX_SIZE = 10000; - -const TOGGLE_ON = i18n.translate('xpack.ml.splom.toggleOn', { - defaultMessage: 'On', -}); -const TOGGLE_OFF = i18n.translate('xpack.ml.splom.toggleOff', { - defaultMessage: 'Off', -}); - -const sampleSizeOptions = [100, 1000, 10000].map((d) => ({ value: d, text: '' + d })); - -interface ScatterplotMatrixProps { - fields: string[]; - index: string; - resultsField?: string; - color?: string; - legendType?: LegendType; -} - -export const ScatterplotMatrix: FC = ({ - fields: allFields, - index, - resultsField, - color, - legendType, -}) => { - const { esSearch } = useMlApiContext(); - - // dynamicSize is optionally used for outlier charts where the scatterplot marks - // are sized according to outlier_score - const [dynamicSize, setDynamicSize] = useState(false); - - // used to give the use the option to customize the fields used for the matrix axes - const [fields, setFields] = useState([]); - - useEffect(() => { - const defaultFields = - allFields.length > SCATTERPLOT_MATRIX_DEFAULT_FIELDS - ? allFields.slice(0, SCATTERPLOT_MATRIX_DEFAULT_FIELDS) - : allFields; - setFields(defaultFields); - }, [allFields]); - - // the amount of documents to be fetched - const [fetchSize, setFetchSize] = useState(SCATTERPLOT_MATRIX_DEFAULT_FETCH_SIZE); - // flag to add a random score to the ES query to fetch documents - const [randomizeQuery, setRandomizeQuery] = useState(false); - - const [isLoading, setIsLoading] = useState(false); - - // contains the fetched documents and columns to be passed on to the Vega spec. - const [splom, setSplom] = useState<{ items: any[]; columns: string[] } | undefined>(); - - // formats the array of field names for EuiComboBox - const fieldOptions = useMemo( - () => - allFields.map((d) => ({ - label: d, - })), - [allFields] - ); - - const fieldsOnChange = (newFields: EuiComboBoxOptionOption[]) => { - setFields(newFields.map((d) => d.label)); - }; - - const fetchSizeOnChange = (e: React.ChangeEvent) => { - setFetchSize( - Math.min( - Math.max(parseInt(e.target.value, 10), SCATTERPLOT_MATRIX_DEFAULT_FETCH_MIN_SIZE), - SCATTERPLOT_MATRIX_DEFAULT_FETCH_MAX_SIZE - ) - ); - }; - - const randomizeQueryOnChange = () => { - setRandomizeQuery(!randomizeQuery); - }; - - const dynamicSizeOnChange = () => { - setDynamicSize(!dynamicSize); - }; - - const { euiTheme } = useCurrentEuiTheme(); - - useEffect(() => { - async function fetchSplom(options: { didCancel: boolean }) { - setIsLoading(true); - try { - const queryFields = [ - ...fields, - ...(color !== undefined ? [color] : []), - ...(legendType !== undefined ? [] : [`${resultsField}.${OUTLIER_SCORE_FIELD}`]), - ]; - - const query = randomizeQuery - ? { - function_score: { - random_score: { seed: 10, field: '_seq_no' }, - }, - } - : { match_all: {} }; - - const resp: SearchResponse7 = await esSearch({ - index, - body: { - fields: queryFields, - _source: false, - query, - from: 0, - size: fetchSize, - }, - }); - - if (!options.didCancel) { - const items = resp.hits.hits.map((d) => - getProcessedFields(d.fields, (key: string) => - key.startsWith(`${resultsField}.feature_importance`) - ) - ); - - setSplom({ columns: fields, items }); - setIsLoading(false); - } - } catch (e) { - // TODO error handling - setIsLoading(false); - } - } - - const options = { didCancel: false }; - fetchSplom(options); - return () => { - options.didCancel = true; - }; - // stringify the fields array, otherwise the comparator will trigger on new but identical instances. - }, [fetchSize, JSON.stringify(fields), index, randomizeQuery, resultsField]); - - const htmlId = useMemo(() => htmlIdGenerator()(), []); - - useEffect(() => { - if (splom === undefined) { - return; - } - - const { items, columns } = splom; - - const values = - resultsField !== undefined - ? items - : items.map((d) => { - d[`${resultsField}.${OUTLIER_SCORE_FIELD}`] = 0; - return d; - }); - - const vegaSpec = getScatterplotMatrixVegaLiteSpec( - values, - columns, - euiTheme, - resultsField, - color, - legendType, - dynamicSize - ); - - const vgSpec = compile(vegaSpec).spec; - - const view = new View(parse(vgSpec)) - .logLevel(Warn) - .renderer('canvas') - .tooltip(new Handler().call) - .initialize(`#${htmlId}`); - - view.runAsync(); // evaluate and render the view - }, [resultsField, splom, color, legendType, dynamicSize]); - - return ( - <> - {splom === undefined ? ( - - - - - - ) : ( - <> - - - - ({ - label: d, - }))} - onChange={fieldsOnChange} - isClearable={true} - data-test-subj="mlScatterplotMatrixFieldsComboBox" - /> - - - - - - - - - - - - - {resultsField !== undefined && legendType === undefined && ( - - - - - - )} - - -
- - )} - - ); -}; +export const ScatterplotMatrix: FC = (props) => ( + }> + + +); diff --git a/x-pack/plugins/ml/public/application/components/scatterplot_matrix/scatterplot_matrix_loading.tsx b/x-pack/plugins/ml/public/application/components/scatterplot_matrix/scatterplot_matrix_loading.tsx new file mode 100644 index 0000000000000..ccd4153769e9c --- /dev/null +++ b/x-pack/plugins/ml/public/application/components/scatterplot_matrix/scatterplot_matrix_loading.tsx @@ -0,0 +1,19 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License; + * you may not use this file except in compliance with the Elastic License. + */ + +import React from 'react'; + +import { EuiLoadingSpinner, EuiSpacer, EuiText } from '@elastic/eui'; + +export const ScatterplotMatrixLoading = () => { + return ( + + + + + + ); +}; diff --git a/x-pack/plugins/ml/public/application/components/scatterplot_matrix/scatterplot_matrix_vega_lite_spec.test.ts b/x-pack/plugins/ml/public/application/components/scatterplot_matrix/scatterplot_matrix_vega_lite_spec.test.ts index dd467161ff489..eada64b7a03ca 100644 --- a/x-pack/plugins/ml/public/application/components/scatterplot_matrix/scatterplot_matrix_vega_lite_spec.test.ts +++ b/x-pack/plugins/ml/public/application/components/scatterplot_matrix/scatterplot_matrix_vega_lite_spec.test.ts @@ -163,6 +163,7 @@ describe('getScatterplotMatrixVegaLiteSpec()', () => { type: 'nominal', }); expect(vegaLiteSpec.spec.encoding.tooltip).toEqual([ + { field: 'the-color-field', type: 'nominal' }, { field: 'x', type: 'quantitative' }, { field: 'y', type: 'quantitative' }, ]); diff --git a/x-pack/plugins/ml/public/application/components/scatterplot_matrix/scatterplot_matrix_vega_lite_spec.ts b/x-pack/plugins/ml/public/application/components/scatterplot_matrix/scatterplot_matrix_vega_lite_spec.ts index 9e0834dd8b922..c943e5d1b06e3 100644 --- a/x-pack/plugins/ml/public/application/components/scatterplot_matrix/scatterplot_matrix_vega_lite_spec.ts +++ b/x-pack/plugins/ml/public/application/components/scatterplot_matrix/scatterplot_matrix_vega_lite_spec.ts @@ -35,6 +35,8 @@ export const getColorSpec = ( color?: string, legendType?: LegendType ) => { + // For outlier detection result pages coloring is done based on a threshold. + // This returns a Vega spec using a conditional to return the color. if (outliers) { return { condition: { @@ -45,6 +47,8 @@ export const getColorSpec = ( }; } + // Based on the type of the color field, + // this returns either a continuous or categorical color spec. if (color !== undefined && legendType !== undefined) { return { field: color, @@ -80,6 +84,8 @@ export const getScatterplotMatrixVegaLiteSpec = ( }); } + const colorSpec = getColorSpec(euiTheme, outliers, color, legendType); + return { $schema: 'https://vega.github.io/schema/vega-lite/v4.17.0.json', background: 'transparent', @@ -115,10 +121,10 @@ export const getScatterplotMatrixVegaLiteSpec = ( : { type: 'circle', opacity: 0.75, size: 8 }), }, encoding: { - color: getColorSpec(euiTheme, outliers, color, legendType), + color: colorSpec, ...(dynamicSize ? { - stroke: getColorSpec(euiTheme, outliers, color, legendType), + stroke: colorSpec, opacity: { condition: { value: 1, @@ -163,6 +169,7 @@ export const getScatterplotMatrixVegaLiteSpec = ( scale: { zero: false }, }, tooltip: [ + ...(color !== undefined ? [{ type: colorSpec.type, field: color }] : []), ...columns.map((d) => ({ type: LEGEND_TYPES.QUANTITATIVE, field: d })), ...(outliers ? [{ type: LEGEND_TYPES.QUANTITATIVE, field: OUTLIER_SCORE_FIELD, format: '.3f' }] diff --git a/x-pack/plugins/ml/public/application/components/scatterplot_matrix/scatterplot_matrix.scss b/x-pack/plugins/ml/public/application/components/scatterplot_matrix/scatterplot_matrix_view.scss similarity index 100% rename from x-pack/plugins/ml/public/application/components/scatterplot_matrix/scatterplot_matrix.scss rename to x-pack/plugins/ml/public/application/components/scatterplot_matrix/scatterplot_matrix_view.scss diff --git a/x-pack/plugins/ml/public/application/components/scatterplot_matrix/scatterplot_matrix_view.tsx b/x-pack/plugins/ml/public/application/components/scatterplot_matrix/scatterplot_matrix_view.tsx new file mode 100644 index 0000000000000..0c065c1154a98 --- /dev/null +++ b/x-pack/plugins/ml/public/application/components/scatterplot_matrix/scatterplot_matrix_view.tsx @@ -0,0 +1,323 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License; + * you may not use this file except in compliance with the Elastic License. + */ + +import React, { useMemo, useEffect, useState, FC } from 'react'; + +// There is still an issue with Vega Lite's typings with the strict mode Kibana is using. +// @ts-ignore +import { compile } from 'vega-lite/build-es5/vega-lite'; +import { parse, View, Warn } from 'vega'; +import { Handler } from 'vega-tooltip'; + +import { + htmlIdGenerator, + EuiComboBox, + EuiComboBoxOptionOption, + EuiFlexGroup, + EuiFlexItem, + EuiFormRow, + EuiSelect, + EuiSwitch, +} from '@elastic/eui'; + +import { i18n } from '@kbn/i18n'; + +import type { SearchResponse7 } from '../../../../common/types/es_client'; +import type { ResultsSearchQuery } from '../../data_frame_analytics/common/analytics'; + +import { useMlApiContext } from '../../contexts/kibana'; + +import { getProcessedFields } from '../data_grid'; +import { useCurrentEuiTheme } from '../color_range_legend'; + +import { ScatterplotMatrixLoading } from './scatterplot_matrix_loading'; + +import { + getScatterplotMatrixVegaLiteSpec, + LegendType, + OUTLIER_SCORE_FIELD, +} from './scatterplot_matrix_vega_lite_spec'; + +import './scatterplot_matrix_view.scss'; + +const SCATTERPLOT_MATRIX_DEFAULT_FIELDS = 4; +const SCATTERPLOT_MATRIX_DEFAULT_FETCH_SIZE = 1000; +const SCATTERPLOT_MATRIX_DEFAULT_FETCH_MIN_SIZE = 1; +const SCATTERPLOT_MATRIX_DEFAULT_FETCH_MAX_SIZE = 10000; + +const TOGGLE_ON = i18n.translate('xpack.ml.splom.toggleOn', { + defaultMessage: 'On', +}); +const TOGGLE_OFF = i18n.translate('xpack.ml.splom.toggleOff', { + defaultMessage: 'Off', +}); + +const sampleSizeOptions = [100, 1000, 10000].map((d) => ({ value: d, text: '' + d })); + +export interface ScatterplotMatrixViewProps { + fields: string[]; + index: string; + resultsField?: string; + color?: string; + legendType?: LegendType; + searchQuery?: ResultsSearchQuery; +} + +export const ScatterplotMatrixView: FC = ({ + fields: allFields, + index, + resultsField, + color, + legendType, + searchQuery, +}) => { + const { esSearch } = useMlApiContext(); + + // dynamicSize is optionally used for outlier charts where the scatterplot marks + // are sized according to outlier_score + const [dynamicSize, setDynamicSize] = useState(false); + + // used to give the use the option to customize the fields used for the matrix axes + const [fields, setFields] = useState([]); + + useEffect(() => { + const defaultFields = + allFields.length > SCATTERPLOT_MATRIX_DEFAULT_FIELDS + ? allFields.slice(0, SCATTERPLOT_MATRIX_DEFAULT_FIELDS) + : allFields; + setFields(defaultFields); + }, [allFields]); + + // the amount of documents to be fetched + const [fetchSize, setFetchSize] = useState(SCATTERPLOT_MATRIX_DEFAULT_FETCH_SIZE); + // flag to add a random score to the ES query to fetch documents + const [randomizeQuery, setRandomizeQuery] = useState(false); + + const [isLoading, setIsLoading] = useState(false); + + // contains the fetched documents and columns to be passed on to the Vega spec. + const [splom, setSplom] = useState<{ items: any[]; columns: string[] } | undefined>(); + + // formats the array of field names for EuiComboBox + const fieldOptions = useMemo( + () => + allFields.map((d) => ({ + label: d, + })), + [allFields] + ); + + const fieldsOnChange = (newFields: EuiComboBoxOptionOption[]) => { + setFields(newFields.map((d) => d.label)); + }; + + const fetchSizeOnChange = (e: React.ChangeEvent) => { + setFetchSize( + Math.min( + Math.max(parseInt(e.target.value, 10), SCATTERPLOT_MATRIX_DEFAULT_FETCH_MIN_SIZE), + SCATTERPLOT_MATRIX_DEFAULT_FETCH_MAX_SIZE + ) + ); + }; + + const randomizeQueryOnChange = () => { + setRandomizeQuery(!randomizeQuery); + }; + + const dynamicSizeOnChange = () => { + setDynamicSize(!dynamicSize); + }; + + const { euiTheme } = useCurrentEuiTheme(); + + useEffect(() => { + async function fetchSplom(options: { didCancel: boolean }) { + setIsLoading(true); + try { + const queryFields = [ + ...fields, + ...(color !== undefined ? [color] : []), + ...(legendType !== undefined ? [] : [`${resultsField}.${OUTLIER_SCORE_FIELD}`]), + ]; + + const queryFallback = searchQuery !== undefined ? searchQuery : { match_all: {} }; + const query = randomizeQuery + ? { + function_score: { + query: queryFallback, + random_score: { seed: 10, field: '_seq_no' }, + }, + } + : queryFallback; + + const resp: SearchResponse7 = await esSearch({ + index, + body: { + fields: queryFields, + _source: false, + query, + from: 0, + size: fetchSize, + }, + }); + + if (!options.didCancel) { + const items = resp.hits.hits.map((d) => + getProcessedFields(d.fields, (key: string) => + key.startsWith(`${resultsField}.feature_importance`) + ) + ); + + setSplom({ columns: fields, items }); + setIsLoading(false); + } + } catch (e) { + // TODO error handling + setIsLoading(false); + } + } + + const options = { didCancel: false }; + fetchSplom(options); + return () => { + options.didCancel = true; + }; + // stringify the fields array and search, otherwise the comparator will trigger on new but identical instances. + }, [fetchSize, JSON.stringify({ fields, searchQuery }), index, randomizeQuery, resultsField]); + + const htmlId = useMemo(() => htmlIdGenerator()(), []); + + useEffect(() => { + if (splom === undefined) { + return; + } + + const { items, columns } = splom; + + const values = + resultsField !== undefined + ? items + : items.map((d) => { + d[`${resultsField}.${OUTLIER_SCORE_FIELD}`] = 0; + return d; + }); + + const vegaSpec = getScatterplotMatrixVegaLiteSpec( + values, + columns, + euiTheme, + resultsField, + color, + legendType, + dynamicSize + ); + + const vgSpec = compile(vegaSpec).spec; + + const view = new View(parse(vgSpec)) + .logLevel(Warn) + .renderer('canvas') + .tooltip(new Handler().call) + .initialize(`#${htmlId}`); + + view.runAsync(); // evaluate and render the view + }, [resultsField, splom, color, legendType, dynamicSize]); + + return ( + <> + {splom === undefined ? ( + + ) : ( + <> + + + + ({ + label: d, + }))} + onChange={fieldsOnChange} + isClearable={true} + data-test-subj="mlScatterplotMatrixFieldsComboBox" + /> + + + + + + + + + + + + + {resultsField !== undefined && legendType === undefined && ( + + + + + + )} + + +
+ + )} + + ); +}; + +// required for dynamic import using React.lazy() +// eslint-disable-next-line import/no-default-export +export default ScatterplotMatrixView; diff --git a/x-pack/plugins/ml/public/application/components/scatterplot_matrix/use_scatterplot_field_options.ts b/x-pack/plugins/ml/public/application/components/scatterplot_matrix/use_scatterplot_field_options.ts new file mode 100644 index 0000000000000..f5eedbc03951f --- /dev/null +++ b/x-pack/plugins/ml/public/application/components/scatterplot_matrix/use_scatterplot_field_options.ts @@ -0,0 +1,50 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License; + * you may not use this file except in compliance with the Elastic License. + */ + +import { useMemo } from 'react'; + +import type { IndexPattern } from '../../../../../../../src/plugins/data/public'; + +import { ML__INCREMENTAL_ID } from '../../data_frame_analytics/common/fields'; + +export const useScatterplotFieldOptions = ( + indexPattern?: IndexPattern, + includes?: string[], + excludes?: string[], + resultsField = '' +): string[] => { + return useMemo(() => { + const fields: string[] = []; + + if (indexPattern === undefined || includes === undefined) { + return fields; + } + + if (includes.length > 1) { + fields.push( + ...includes.filter((d) => + indexPattern.fields.some((f) => f.name === d && f.type === 'number') + ) + ); + } else { + fields.push( + ...indexPattern.fields + .filter( + (f) => + f.type === 'number' && + !indexPattern.metaFields.includes(f.name) && + !f.name.startsWith(`${resultsField}.`) && + f.name !== ML__INCREMENTAL_ID + ) + .map((f) => f.name) + ); + } + + return Array.isArray(excludes) && excludes.length > 0 + ? fields.filter((f) => !excludes.includes(f)) + : fields; + }, [indexPattern, includes, excludes]); +}; diff --git a/x-pack/plugins/ml/public/application/data_frame_analytics/common/get_scatterplot_matrix_legend_type.ts b/x-pack/plugins/ml/public/application/data_frame_analytics/common/get_scatterplot_matrix_legend_type.ts new file mode 100644 index 0000000000000..8850d42577bd9 --- /dev/null +++ b/x-pack/plugins/ml/public/application/data_frame_analytics/common/get_scatterplot_matrix_legend_type.ts @@ -0,0 +1,22 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License; + * you may not use this file except in compliance with the Elastic License. + */ + +import { ANALYSIS_CONFIG_TYPE } from './analytics'; + +import { AnalyticsJobType } from '../pages/analytics_management/hooks/use_create_analytics_form/state'; + +import { LEGEND_TYPES } from '../../components/scatterplot_matrix/scatterplot_matrix_vega_lite_spec'; + +export const getScatterplotMatrixLegendType = (jobType: AnalyticsJobType | 'unknown') => { + switch (jobType) { + case ANALYSIS_CONFIG_TYPE.CLASSIFICATION: + return LEGEND_TYPES.NOMINAL; + case ANALYSIS_CONFIG_TYPE.REGRESSION: + return LEGEND_TYPES.QUANTITATIVE; + default: + return undefined; + } +}; diff --git a/x-pack/plugins/ml/public/application/data_frame_analytics/common/index.ts b/x-pack/plugins/ml/public/application/data_frame_analytics/common/index.ts index 7ba3e910ddd32..d03f73ad13575 100644 --- a/x-pack/plugins/ml/public/application/data_frame_analytics/common/index.ts +++ b/x-pack/plugins/ml/public/application/data_frame_analytics/common/index.ts @@ -41,6 +41,7 @@ export { export { getIndexData } from './get_index_data'; export { getIndexFields } from './get_index_fields'; +export { getScatterplotMatrixLegendType } from './get_scatterplot_matrix_legend_type'; export { useResultsViewConfig } from './use_results_view_config'; export { DataFrameAnalyticsConfig } from '../../../../common/types/data_frame_analytics'; diff --git a/x-pack/plugins/ml/public/application/data_frame_analytics/common/use_results_view_config.ts b/x-pack/plugins/ml/public/application/data_frame_analytics/common/use_results_view_config.ts index 185513f75a12c..361a1262a59f8 100644 --- a/x-pack/plugins/ml/public/application/data_frame_analytics/common/use_results_view_config.ts +++ b/x-pack/plugins/ml/public/application/data_frame_analytics/common/use_results_view_config.ts @@ -102,6 +102,12 @@ export const useResultsViewConfig = (jobId: string) => { try { indexP = await mlContext.indexPatterns.get(destIndexPatternId); + + // Force refreshing the fields list here because a user directly coming + // from the job creation wizard might land on the page without the + // index pattern being fully initialized because it was created + // before the analytics job populated the destination index. + await mlContext.indexPatterns.refreshFields(indexP); } catch (e) { indexP = undefined; } diff --git a/x-pack/plugins/ml/public/application/data_frame_analytics/pages/analytics_creation/components/configuration_step/configuration_step_form.tsx b/x-pack/plugins/ml/public/application/data_frame_analytics/pages/analytics_creation/components/configuration_step/configuration_step_form.tsx index a5991f77e88e2..4b86f5ca12896 100644 --- a/x-pack/plugins/ml/public/application/data_frame_analytics/pages/analytics_creation/components/configuration_step/configuration_step_form.tsx +++ b/x-pack/plugins/ml/public/application/data_frame_analytics/pages/analytics_creation/components/configuration_step/configuration_step_form.tsx @@ -27,10 +27,10 @@ import { TRAINING_PERCENT_MAX, FieldSelectionItem, } from '../../../../common/analytics'; +import { getScatterplotMatrixLegendType } from '../../../../common/get_scatterplot_matrix_legend_type'; import { CreateAnalyticsStepProps } from '../../../analytics_management/hooks/use_create_analytics_form'; import { Messages } from '../shared'; import { - AnalyticsJobType, DEFAULT_MODEL_MEMORY_LIMIT, State, } from '../../../analytics_management/hooks/use_create_analytics_form/state'; @@ -51,18 +51,7 @@ import { SEARCH_QUERY_LANGUAGE } from '../../../../../../../common/constants/sea import { ExplorationQueryBarProps } from '../../../analytics_exploration/components/exploration_query_bar/exploration_query_bar'; import { Query } from '../../../../../../../../../../src/plugins/data/common/query'; -import { LEGEND_TYPES, ScatterplotMatrix } from '../../../../../components/scatterplot_matrix'; - -const getScatterplotMatrixLegendType = (jobType: AnalyticsJobType) => { - switch (jobType) { - case ANALYSIS_CONFIG_TYPE.CLASSIFICATION: - return LEGEND_TYPES.NOMINAL; - case ANALYSIS_CONFIG_TYPE.REGRESSION: - return LEGEND_TYPES.QUANTITATIVE; - default: - return undefined; - } -}; +import { ScatterplotMatrix } from '../../../../../components/scatterplot_matrix'; const requiredFieldsErrorText = i18n.translate( 'xpack.ml.dataframe.analytics.createWizard.requiredFieldsErrorMessage', @@ -498,6 +487,7 @@ export const ConfigurationStepForm: FC = ({ : undefined } legendType={getScatterplotMatrixLegendType(jobType)} + searchQuery={jobConfigQuery} /> diff --git a/x-pack/plugins/ml/public/application/data_frame_analytics/pages/analytics_creation/page.tsx b/x-pack/plugins/ml/public/application/data_frame_analytics/pages/analytics_creation/page.tsx index e72af6a0e30c2..2cd7223027566 100644 --- a/x-pack/plugins/ml/public/application/data_frame_analytics/pages/analytics_creation/page.tsx +++ b/x-pack/plugins/ml/public/application/data_frame_analytics/pages/analytics_creation/page.tsx @@ -62,7 +62,7 @@ export const Page: FC = ({ jobId }) => { if (currentIndexPattern) { (async function () { if (jobId !== undefined) { - const analyticsConfigs = await ml.dataFrameAnalytics.getDataFrameAnalytics(jobId); + const analyticsConfigs = await ml.dataFrameAnalytics.getDataFrameAnalytics(jobId, true); if ( Array.isArray(analyticsConfigs.data_frame_analytics) && analyticsConfigs.data_frame_analytics.length > 0 diff --git a/x-pack/plugins/ml/public/application/data_frame_analytics/pages/analytics_exploration/components/expandable_section/expandable_section_splom.tsx b/x-pack/plugins/ml/public/application/data_frame_analytics/pages/analytics_exploration/components/expandable_section/expandable_section_splom.tsx index 5ec8963e0fc25..8c51c95d7fd63 100644 --- a/x-pack/plugins/ml/public/application/data_frame_analytics/pages/analytics_exploration/components/expandable_section/expandable_section_splom.tsx +++ b/x-pack/plugins/ml/public/application/data_frame_analytics/pages/analytics_exploration/components/expandable_section/expandable_section_splom.tsx @@ -12,17 +12,14 @@ import { FormattedMessage } from '@kbn/i18n/react'; import { EuiHorizontalRule, EuiSpacer } from '@elastic/eui'; -import { ScatterplotMatrix } from '../../../../../components/scatterplot_matrix'; +import { + ScatterplotMatrix, + ScatterplotMatrixProps, +} from '../../../../../components/scatterplot_matrix'; import { ExpandableSection } from './expandable_section'; -interface ExpandableSectionSplomProps { - fields: string[]; - index: string; - resultsField?: string; -} - -export const ExpandableSectionSplom: FC = (props) => { +export const ExpandableSectionSplom: FC = (props) => { const splomSectionHeaderItems = undefined; const splomSectionContent = ( <> diff --git a/x-pack/plugins/ml/public/application/data_frame_analytics/pages/analytics_exploration/components/exploration_page_wrapper/exploration_page_wrapper.tsx b/x-pack/plugins/ml/public/application/data_frame_analytics/pages/analytics_exploration/components/exploration_page_wrapper/exploration_page_wrapper.tsx index 1329644322f33..46715af0ef0cb 100644 --- a/x-pack/plugins/ml/public/application/data_frame_analytics/pages/analytics_exploration/components/exploration_page_wrapper/exploration_page_wrapper.tsx +++ b/x-pack/plugins/ml/public/application/data_frame_analytics/pages/analytics_exploration/components/exploration_page_wrapper/exploration_page_wrapper.tsx @@ -9,16 +9,21 @@ import React, { FC, useCallback, useState } from 'react'; import { EuiCallOut, EuiFlexGroup, EuiFlexItem, EuiPanel, EuiSpacer, EuiText } from '@elastic/eui'; import { i18n } from '@kbn/i18n'; +import { getAnalysisType, getDependentVar } from '../../../../../../../common/util/analytics_utils'; + +import { useScatterplotFieldOptions } from '../../../../../components/scatterplot_matrix'; + import { defaultSearchQuery, + getScatterplotMatrixLegendType, useResultsViewConfig, DataFrameAnalyticsConfig, } from '../../../../common'; -import { ResultsSearchQuery } from '../../../../common/analytics'; +import { ResultsSearchQuery, ANALYSIS_CONFIG_TYPE } from '../../../../common/analytics'; import { DataFrameTaskStateType } from '../../../analytics_management/components/analytics_list/common'; -import { ExpandableSectionAnalytics } from '../expandable_section'; +import { ExpandableSectionAnalytics, ExpandableSectionSplom } from '../expandable_section'; import { ExplorationResultsTable } from '../exploration_results_table'; import { ExplorationQueryBar } from '../exploration_query_bar'; import { JobConfigErrorCallout } from '../job_config_error_callout'; @@ -99,6 +104,14 @@ export const ExplorationPageWrapper: FC = ({ language: pageUrlState.queryLanguage, }; + const resultsField = jobConfig?.dest.results_field ?? ''; + const scatterplotFieldOptions = useScatterplotFieldOptions( + indexPattern, + jobConfig?.analyzed_fields.includes, + jobConfig?.analyzed_fields.excludes, + resultsField + ); + if (indexPatternErrorMessage !== undefined) { return ( @@ -125,6 +138,9 @@ export const ExplorationPageWrapper: FC = ({ ); } + const jobType = + jobConfig && jobConfig.analysis ? getAnalysisType(jobConfig?.analysis) : undefined; + return ( <> {typeof jobConfig?.description !== 'undefined' && ( @@ -179,6 +195,27 @@ export const ExplorationPageWrapper: FC = ({ )} + {isLoadingJobConfig === true && jobConfig === undefined && } + {isLoadingJobConfig === false && + jobConfig !== undefined && + isInitialized === true && + typeof jobConfig?.id === 'string' && + scatterplotFieldOptions.length > 1 && + typeof jobConfig?.analysis !== 'undefined' && ( + + )} + {isLoadingJobConfig === true && jobConfig === undefined && } {isLoadingJobConfig === false && jobConfig !== undefined && diff --git a/x-pack/plugins/ml/public/application/data_frame_analytics/pages/analytics_exploration/components/outlier_exploration/outlier_exploration.tsx b/x-pack/plugins/ml/public/application/data_frame_analytics/pages/analytics_exploration/components/outlier_exploration/outlier_exploration.tsx index 26eee9bc95d73..7e11e0bd97015 100644 --- a/x-pack/plugins/ml/public/application/data_frame_analytics/pages/analytics_exploration/components/outlier_exploration/outlier_exploration.tsx +++ b/x-pack/plugins/ml/public/application/data_frame_analytics/pages/analytics_exploration/components/outlier_exploration/outlier_exploration.tsx @@ -4,7 +4,7 @@ * you may not use this file except in compliance with the Elastic License. */ -import React, { useState, FC, useCallback } from 'react'; +import React, { useCallback, useState, FC } from 'react'; import { EuiCallOut, EuiPanel, EuiSpacer, EuiText } from '@elastic/eui'; @@ -15,6 +15,7 @@ import { COLOR_RANGE, COLOR_RANGE_SCALE, } from '../../../../../components/color_range_legend'; +import { useScatterplotFieldOptions } from '../../../../../components/scatterplot_matrix'; import { SavedSearchQuery } from '../../../../../contexts/ml'; import { defaultSearchQuery, isOutlierAnalysis, useResultsViewConfig } from '../../../../common'; @@ -90,6 +91,13 @@ export const OutlierExploration: FC = React.memo(({ jobId }) = (d) => d.id === `${resultsField}.${FEATURE_INFLUENCE}.feature_name` ) === -1; + const scatterplotFieldOptions = useScatterplotFieldOptions( + indexPattern, + jobConfig?.analyzed_fields.includes, + jobConfig?.analyzed_fields.excludes, + resultsField + ); + if (indexPatternErrorMessage !== undefined) { return ( @@ -126,11 +134,12 @@ export const OutlierExploration: FC = React.memo(({ jobId }) = )} {typeof jobConfig?.id === 'string' && } - {typeof jobConfig?.id === 'string' && jobConfig?.analyzed_fields.includes.length > 1 && ( + {typeof jobConfig?.id === 'string' && scatterplotFieldOptions.length > 1 && ( )} {showLegacyFeatureInfluenceFormatCallout && ( diff --git a/x-pack/plugins/ml/public/application/data_frame_analytics/pages/analytics_management/components/action_clone/clone_action_name.tsx b/x-pack/plugins/ml/public/application/data_frame_analytics/pages/analytics_management/components/action_clone/clone_action_name.tsx index b1592d51874c7..72c6f5a9eca99 100644 --- a/x-pack/plugins/ml/public/application/data_frame_analytics/pages/analytics_management/components/action_clone/clone_action_name.tsx +++ b/x-pack/plugins/ml/public/application/data_frame_analytics/pages/analytics_management/components/action_clone/clone_action_name.tsx @@ -310,9 +310,6 @@ export type CloneDataFrameAnalyticsConfig = Omit< */ export function extractCloningConfig({ id, - version, - // eslint-disable-next-line @typescript-eslint/naming-convention - create_time, ...configToClone }: DeepReadonly): CloneDataFrameAnalyticsConfig { return (cloneDeep({ diff --git a/x-pack/plugins/ml/public/application/jobs/jobs_list/components/utils.js b/x-pack/plugins/ml/public/application/jobs/jobs_list/components/utils.js index 338222e3ac4a2..0e2fee70c748a 100644 --- a/x-pack/plugins/ml/public/application/jobs/jobs_list/components/utils.js +++ b/x-pack/plugins/ml/public/application/jobs/jobs_list/components/utils.js @@ -36,6 +36,23 @@ export function loadFullJob(jobId) { }); } +export function loadJobForCloning(jobId) { + return new Promise((resolve, reject) => { + ml.jobs + .jobForCloning(jobId) + .then((resp) => { + if (resp) { + resolve(resp); + } else { + throw new Error(`Could not find job ${jobId}`); + } + }) + .catch((error) => { + reject(error); + }); + }); +} + export function isStartable(jobs) { return jobs.some( (j) => j.datafeedState === DATAFEED_STATE.STOPPED && j.jobState !== JOB_STATE.CLOSING @@ -180,31 +197,38 @@ function showResults(resp, action) { export async function cloneJob(jobId) { try { - const job = await loadFullJob(jobId); - if (job.custom_settings && job.custom_settings.created_by) { + const [{ job: cloneableJob, datafeed }, originalJob] = await Promise.all([ + loadJobForCloning(jobId), + loadFullJob(jobId, false), + ]); + if (cloneableJob !== undefined && originalJob?.custom_settings?.created_by !== undefined) { // if the job is from a wizards, i.e. contains a created_by property // use tempJobCloningObjects to temporarily store the job - mlJobService.tempJobCloningObjects.job = job; + mlJobService.tempJobCloningObjects.createdBy = originalJob?.custom_settings?.created_by; + mlJobService.tempJobCloningObjects.job = cloneableJob; if ( - job.data_counts.earliest_record_timestamp !== undefined && - job.data_counts.latest_record_timestamp !== undefined && - job.data_counts.latest_bucket_timestamp !== undefined + originalJob.data_counts.earliest_record_timestamp !== undefined && + originalJob.data_counts.latest_record_timestamp !== undefined && + originalJob.data_counts.latest_bucket_timestamp !== undefined ) { // if the job has run before, use the earliest and latest record timestamp // as the cloned job's time range - let start = job.data_counts.earliest_record_timestamp; - let end = job.data_counts.latest_record_timestamp; + let start = originalJob.data_counts.earliest_record_timestamp; + let end = originalJob.data_counts.latest_record_timestamp; - if (job.datafeed_config.aggregations !== undefined) { + if (originalJob.datafeed_config.aggregations !== undefined) { // if the datafeed uses aggregations the earliest and latest record timestamps may not be the same // as the start and end of the data in the index. - const bucketSpanMs = parseInterval(job.analysis_config.bucket_span).asMilliseconds(); + const bucketSpanMs = parseInterval( + originalJob.analysis_config.bucket_span + ).asMilliseconds(); // round down to the start of the nearest bucket start = - Math.floor(job.data_counts.earliest_record_timestamp / bucketSpanMs) * bucketSpanMs; + Math.floor(originalJob.data_counts.earliest_record_timestamp / bucketSpanMs) * + bucketSpanMs; // use latest_bucket_timestamp and add two bucket spans minus one ms - end = job.data_counts.latest_bucket_timestamp + bucketSpanMs * 2 - 1; + end = originalJob.data_counts.latest_bucket_timestamp + bucketSpanMs * 2 - 1; } mlJobService.tempJobCloningObjects.start = start; @@ -212,12 +236,17 @@ export async function cloneJob(jobId) { } } else { // otherwise use the tempJobCloningObjects - mlJobService.tempJobCloningObjects.job = job; + mlJobService.tempJobCloningObjects.job = cloneableJob; + // resets the createdBy field in case it still retains previous settings + mlJobService.tempJobCloningObjects.createdBy = undefined; + } + if (datafeed !== undefined) { + mlJobService.tempJobCloningObjects.datafeed = datafeed; } - if (job.calendars) { + if (originalJob.calendars) { mlJobService.tempJobCloningObjects.calendars = await mlCalendarService.fetchCalendarsByIds( - job.calendars + originalJob.calendars ); } diff --git a/x-pack/plugins/ml/public/application/jobs/new_job/pages/index_or_search/preconfigured_job_redirect.ts b/x-pack/plugins/ml/public/application/jobs/new_job/pages/index_or_search/preconfigured_job_redirect.ts index cedaaa3b5dfaa..18992e5cbf5d8 100644 --- a/x-pack/plugins/ml/public/application/jobs/new_job/pages/index_or_search/preconfigured_job_redirect.ts +++ b/x-pack/plugins/ml/public/application/jobs/new_job/pages/index_or_search/preconfigured_job_redirect.ts @@ -8,7 +8,7 @@ import { ApplicationStart } from 'kibana/public'; import { IndexPatternsContract } from '../../../../../../../../../src/plugins/data/public'; import { mlJobService } from '../../../../services/job_service'; import { loadIndexPatterns, getIndexPatternIdFromName } from '../../../../util/index_utils'; -import { CombinedJob } from '../../../../../../common/types/anomaly_detection_jobs'; +import { Datafeed, Job } from '../../../../../../common/types/anomaly_detection_jobs'; import { CREATED_BY_LABEL, JOB_TYPE } from '../../../../../../common/constants/new_job'; export async function preConfiguredJobRedirect( @@ -16,11 +16,11 @@ export async function preConfiguredJobRedirect( basePath: string, navigateToUrl: ApplicationStart['navigateToUrl'] ) { - const { job } = mlJobService.tempJobCloningObjects; - if (job) { + const { createdBy, job, datafeed } = mlJobService.tempJobCloningObjects; + if (job && datafeed) { try { await loadIndexPatterns(indexPatterns); - const redirectUrl = getWizardUrlFromCloningJob(job); + const redirectUrl = getWizardUrlFromCloningJob(createdBy, job, datafeed); await navigateToUrl(`${basePath}/app/ml/${redirectUrl}`); return Promise.reject(); } catch (error) { @@ -33,8 +33,8 @@ export async function preConfiguredJobRedirect( } } -function getWizardUrlFromCloningJob(job: CombinedJob) { - const created = job?.custom_settings?.created_by; +function getWizardUrlFromCloningJob(createdBy: string | undefined, job: Job, datafeed: Datafeed) { + const created = createdBy; let page = ''; switch (created) { @@ -55,7 +55,7 @@ function getWizardUrlFromCloningJob(job: CombinedJob) { break; } - const indexPatternId = getIndexPatternIdFromName(job.datafeed_config.indices.join()); + const indexPatternId = getIndexPatternIdFromName(datafeed.indices.join()); return `jobs/new_job/${page}?index=${indexPatternId}&_g=()`; } diff --git a/x-pack/plugins/ml/public/application/jobs/new_job/pages/new_job/page.tsx b/x-pack/plugins/ml/public/application/jobs/new_job/pages/new_job/page.tsx index 8f7f93763fdd6..a196934610855 100644 --- a/x-pack/plugins/ml/public/application/jobs/new_job/pages/new_job/page.tsx +++ b/x-pack/plugins/ml/public/application/jobs/new_job/pages/new_job/page.tsx @@ -37,7 +37,6 @@ import { useMlContext } from '../../../../contexts/ml'; import { getTimeFilterRange } from '../../../../components/full_time_range_selector'; import { getTimeBucketsFromCache } from '../../../../util/time_buckets'; import { ExistingJobsAndGroups, mlJobService } from '../../../../services/job_service'; -import { expandCombinedJobConfig } from '../../../../../../common/types/anomaly_detection_jobs'; import { newJobCapsService } from '../../../../services/new_job_capabilities_service'; import { EVENT_RATE_FIELD_ID } from '../../../../../../common/types/fields'; import { getNewJobDefaults } from '../../../../services/ml_server_info'; @@ -74,10 +73,11 @@ export const Page: FC = ({ existingJobsAndGroups, jobType }) => { if (mlJobService.tempJobCloningObjects.job !== undefined) { // cloning a job - const clonedJob = mlJobService.cloneJob(mlJobService.tempJobCloningObjects.job); - const { job, datafeed } = expandCombinedJobConfig(clonedJob); + const clonedJob = mlJobService.tempJobCloningObjects.job; + const clonedDatafeed = mlJobService.cloneDatafeed(mlJobService.tempJobCloningObjects.datafeed); + initCategorizationSettings(); - jobCreator.cloneFromExistingJob(job, datafeed); + jobCreator.cloneFromExistingJob(clonedJob, clonedDatafeed); // if we're not skipping the time range, this is a standard job clone, so wipe the jobId if (mlJobService.tempJobCloningObjects.skipTimeRangeStep === false) { diff --git a/x-pack/plugins/ml/public/application/services/job_service.d.ts b/x-pack/plugins/ml/public/application/services/job_service.d.ts index 30b2ec044285a..0bf40bc0dad77 100644 --- a/x-pack/plugins/ml/public/application/services/job_service.d.ts +++ b/x-pack/plugins/ml/public/application/services/job_service.d.ts @@ -6,7 +6,7 @@ import { SearchResponse } from 'elasticsearch'; import { TimeRange } from 'src/plugins/data/common/query/timefilter/types'; -import { CombinedJob } from '../../../common/types/anomaly_detection_jobs'; +import { CombinedJob, Datafeed } from '../../../common/types/anomaly_detection_jobs'; import { Calendar } from '../../../common/types/calendars'; export interface ExistingJobsAndGroups { @@ -18,6 +18,8 @@ declare interface JobService { jobs: CombinedJob[]; createResultsUrlForJobs: (jobs: any[], target: string, timeRange?: TimeRange) => string; tempJobCloningObjects: { + createdBy?: string; + datafeed?: Datafeed; job: any; skipTimeRangeStep: boolean; start?: number; @@ -26,7 +28,7 @@ declare interface JobService { }; skipTimeRangeStep: boolean; saveNewJob(job: any): Promise; - cloneJob(job: any): any; + cloneDatafeed(datafeed: any): Datafeed; openJob(jobId: string): Promise; saveNewDatafeed(datafeedConfig: any, jobId: string): Promise; startDatafeed( diff --git a/x-pack/plugins/ml/public/application/services/job_service.js b/x-pack/plugins/ml/public/application/services/job_service.js index 5f504e4665500..06f1aea3e6e1e 100644 --- a/x-pack/plugins/ml/public/application/services/job_service.js +++ b/x-pack/plugins/ml/public/application/services/job_service.js @@ -28,6 +28,8 @@ class JobService { // if populated when loading the job management page, the start datafeed modal // is automatically opened. this.tempJobCloningObjects = { + createdBy: undefined, + datafeed: undefined, job: undefined, skipTimeRangeStep: false, start: undefined, @@ -325,67 +327,15 @@ class JobService { return ml.addJob({ jobId: job.job_id, job }).then(func).catch(func); } - cloneJob(job) { - // create a deep copy of a job object - // also remove items from the job which are set by the server and not needed - // in the future this formatting could be optional - const tempJob = cloneDeep(job); - - // remove all of the items which should not be copied - // such as counts, state and times - delete tempJob.state; - delete tempJob.job_version; - delete tempJob.data_counts; - delete tempJob.create_time; - delete tempJob.finished_time; - delete tempJob.last_data_time; - delete tempJob.model_size_stats; - delete tempJob.node; - delete tempJob.average_bucket_processing_time_ms; - delete tempJob.model_snapshot_id; - delete tempJob.open_time; - delete tempJob.established_model_memory; - delete tempJob.calendars; - delete tempJob.timing_stats; - delete tempJob.forecasts_stats; - delete tempJob.assignment_explanation; - - delete tempJob.analysis_config.use_per_partition_normalization; - - each(tempJob.analysis_config.detectors, (d) => { - delete d.detector_index; - }); + cloneDatafeed(datafeed) { + const tempDatafeed = cloneDeep(datafeed); // remove parts of the datafeed config which should not be copied - if (tempJob.datafeed_config) { - delete tempJob.datafeed_config.datafeed_id; - delete tempJob.datafeed_config.job_id; - delete tempJob.datafeed_config.state; - delete tempJob.datafeed_config.node; - delete tempJob.datafeed_config.timing_stats; - delete tempJob.datafeed_config.assignment_explanation; - - // remove query_delay if it's between 60s and 120s - // the back-end produces a random value between 60 and 120 and so - // by deleting it, the back-end will produce a new random value - if (tempJob.datafeed_config.query_delay) { - const interval = parseInterval(tempJob.datafeed_config.query_delay); - if (interval !== null) { - const queryDelay = interval.asSeconds(); - if (queryDelay > 60 && queryDelay < 120) { - delete tempJob.datafeed_config.query_delay; - } - } - } + if (tempDatafeed) { + delete tempDatafeed.datafeed_id; + delete tempDatafeed.job_id; } - - // when jumping from a wizard to the advanced job creation, - // the wizard's created_by information should be stripped. - if (tempJob.custom_settings && tempJob.custom_settings.created_by) { - delete tempJob.custom_settings.created_by; - } - - return tempJob; + return tempDatafeed; } // find a job based on the id diff --git a/x-pack/plugins/ml/public/application/services/ml_api_service/data_frame_analytics.ts b/x-pack/plugins/ml/public/application/services/ml_api_service/data_frame_analytics.ts index 7b246e557d7a5..98a8e4c9cbf2d 100644 --- a/x-pack/plugins/ml/public/application/services/ml_api_service/data_frame_analytics.ts +++ b/x-pack/plugins/ml/public/application/services/ml_api_service/data_frame_analytics.ts @@ -52,11 +52,12 @@ interface JobsExistsResponse { } export const dataFrameAnalytics = { - getDataFrameAnalytics(analyticsId?: string) { + getDataFrameAnalytics(analyticsId?: string, excludeGenerated?: boolean) { const analyticsIdString = analyticsId !== undefined ? `/${analyticsId}` : ''; return http({ path: `${basePath()}/data_frame/analytics${analyticsIdString}`, method: 'GET', + ...(excludeGenerated ? { query: { excludeGenerated } } : {}), }); }, getDataFrameAnalyticsStats(analyticsId?: string) { diff --git a/x-pack/plugins/ml/public/application/services/ml_api_service/jobs.ts b/x-pack/plugins/ml/public/application/services/ml_api_service/jobs.ts index 10e035103dbec..67aaf6b557168 100644 --- a/x-pack/plugins/ml/public/application/services/ml_api_service/jobs.ts +++ b/x-pack/plugins/ml/public/application/services/ml_api_service/jobs.ts @@ -13,6 +13,8 @@ import { MlJobWithTimeRange, MlSummaryJobs, CombinedJobWithStats, + Job, + Datafeed, } from '../../../../common/types/anomaly_detection_jobs'; import { JobMessage } from '../../../../common/types/audit_message'; import { AggFieldNamePair } from '../../../../common/types/fields'; @@ -48,6 +50,15 @@ export const jobsApiProvider = (httpService: HttpService) => ({ }); }, + jobForCloning(jobId: string) { + const body = JSON.stringify({ jobId }); + return httpService.http<{ job?: Job; datafeed?: Datafeed } | undefined>({ + path: `${basePath()}/jobs/job_for_cloning`, + method: 'POST', + body, + }); + }, + jobs(jobIds: string[]) { const body = JSON.stringify({ jobIds }); return httpService.http({ diff --git a/x-pack/plugins/ml/server/models/job_service/datafeeds.ts b/x-pack/plugins/ml/server/models/job_service/datafeeds.ts index e144da0ae0800..b0c942647227c 100644 --- a/x-pack/plugins/ml/server/models/job_service/datafeeds.ts +++ b/x-pack/plugins/ml/server/models/job_service/datafeeds.ts @@ -160,11 +160,55 @@ export function datafeedsProvider(mlClient: MlClient) { }, {} as { [id: string]: string }); } + async function getDatafeedByJobId( + jobId: string, + excludeGenerated?: boolean + ): Promise { + async function findDatafeed() { + // if the job was doesn't use the standard datafeedId format + // get all the datafeeds and match it with the jobId + const { + body: { datafeeds }, + } = await mlClient.getDatafeeds( + excludeGenerated ? { exclude_generated: true } : {} + ); + for (const result of datafeeds) { + if (result.job_id === jobId) { + return result; + } + } + } + // if the job was created by the wizard, + // then we can assume it uses the standard format of the datafeedId + const assumedDefaultDatafeedId = `datafeed-${jobId}`; + try { + const { + body: { datafeeds: datafeedsResults }, + } = await mlClient.getDatafeeds({ + datafeed_id: assumedDefaultDatafeedId, + ...(excludeGenerated ? { exclude_generated: true } : {}), + }); + if ( + Array.isArray(datafeedsResults) && + datafeedsResults.length === 1 && + datafeedsResults[0].job_id === jobId + ) { + return datafeedsResults[0]; + } else { + return await findDatafeed(); + } + } catch (e) { + // if assumedDefaultDatafeedId does not exist, ES will throw an error + return await findDatafeed(); + } + } + return { forceStartDatafeeds, stopDatafeeds, forceDeleteDatafeed, getDatafeedIdsByJobId, getJobIdsByDatafeedId, + getDatafeedByJobId, }; } diff --git a/x-pack/plugins/ml/server/models/job_service/jobs.ts b/x-pack/plugins/ml/server/models/job_service/jobs.ts index d47a1d4b4892d..6ab4af63004b4 100644 --- a/x-pack/plugins/ml/server/models/job_service/jobs.ts +++ b/x-pack/plugins/ml/server/models/job_service/jobs.ts @@ -18,6 +18,8 @@ import { AuditMessage, DatafeedWithStats, CombinedJobWithStats, + Datafeed, + Job, } from '../../../common/types/anomaly_detection_jobs'; import { MlJobsResponse, @@ -47,7 +49,9 @@ interface Results { export function jobsProvider(client: IScopedClusterClient, mlClient: MlClient) { const { asInternalUser } = client; - const { forceDeleteDatafeed, getDatafeedIdsByJobId } = datafeedsProvider(mlClient); + const { forceDeleteDatafeed, getDatafeedIdsByJobId, getDatafeedByJobId } = datafeedsProvider( + mlClient + ); const { getAuditMessagesSummary } = jobAuditMessagesProvider(client, mlClient); const { getLatestBucketTimestampByJob } = resultsServiceProvider(mlClient); const calMngr = new CalendarManager(mlClient); @@ -257,6 +261,25 @@ export function jobsProvider(client: IScopedClusterClient, mlClient: MlClient) { return { jobs, jobsMap }; } + async function getJobForCloning(jobId: string) { + const [{ body: jobResults }, datafeedResult] = await Promise.all([ + mlClient.getJobs({ job_id: jobId, exclude_generated: true }), + getDatafeedByJobId(jobId, true), + ]); + const result: { datafeed?: Datafeed; job?: Job } = { job: undefined, datafeed: undefined }; + if (datafeedResult && datafeedResult.job_id === jobId) { + result.datafeed = datafeedResult; + } + + if (jobResults && jobResults.jobs) { + const job = jobResults.jobs.find((j) => j.job_id === jobId); + if (job) { + result.job = job; + } + } + return result; + } + async function createFullJobsList(jobIds: string[] = []) { const jobs: CombinedJobWithStats[] = []; const groups: { [jobId: string]: string[] } = {}; @@ -265,6 +288,7 @@ export function jobsProvider(client: IScopedClusterClient, mlClient: MlClient) { const globalCalendars: string[] = []; const jobIdsString = jobIds.join(); + const [ { body: jobResults }, { body: jobStatsResults }, @@ -502,6 +526,7 @@ export function jobsProvider(client: IScopedClusterClient, mlClient: MlClient) { forceStopAndCloseJob, jobsSummary, jobsWithTimerange, + getJobForCloning, createFullJobsList, deletingJobTasks, jobsExist, diff --git a/x-pack/plugins/ml/server/routes/apidoc.json b/x-pack/plugins/ml/server/routes/apidoc.json index 015ec6e4ec9c0..5dc9a3107af86 100644 --- a/x-pack/plugins/ml/server/routes/apidoc.json +++ b/x-pack/plugins/ml/server/routes/apidoc.json @@ -73,6 +73,7 @@ "CloseJobs", "JobsSummary", "JobsWithTimeRange", + "GetJobForCloning", "CreateFullJobsList", "GetAllGroups", "JobsExist", diff --git a/x-pack/plugins/ml/server/routes/data_frame_analytics.ts b/x-pack/plugins/ml/server/routes/data_frame_analytics.ts index 0abba7a429aea..4d504f4f2ef20 100644 --- a/x-pack/plugins/ml/server/routes/data_frame_analytics.ts +++ b/x-pack/plugins/ml/server/routes/data_frame_analytics.ts @@ -19,6 +19,7 @@ import { stopsDataFrameAnalyticsJobQuerySchema, deleteDataFrameAnalyticsJobSchema, jobsExistSchema, + analyticsQuerySchema, } from './schemas/data_analytics_schema'; import { GetAnalyticsMapArgs, ExtendAnalyticsMapArgs } from '../models/data_frame_analytics/types'; import { IndexPatternHandler } from '../models/data_frame_analytics/index_patterns'; @@ -102,7 +103,9 @@ export function dataFrameAnalyticsRoutes({ router, mlLicense, routeGuard }: Rout }, routeGuard.fullLicenseAPIGuard(async ({ mlClient, response }) => { try { - const { body } = await mlClient.getDataFrameAnalytics({ size: 1000 }); + const { body } = await mlClient.getDataFrameAnalytics({ + size: 1000, + }); return response.ok({ body, }); @@ -126,6 +129,7 @@ export function dataFrameAnalyticsRoutes({ router, mlLicense, routeGuard }: Rout path: '/api/ml/data_frame/analytics/{analyticsId}', validate: { params: analyticsIdSchema, + query: analyticsQuerySchema, }, options: { tags: ['access:ml:canGetDataFrameAnalytics'], @@ -134,8 +138,11 @@ export function dataFrameAnalyticsRoutes({ router, mlLicense, routeGuard }: Rout routeGuard.fullLicenseAPIGuard(async ({ mlClient, request, response }) => { try { const { analyticsId } = request.params; + const { excludeGenerated } = request.query; + const { body } = await mlClient.getDataFrameAnalytics({ id: analyticsId, + ...(excludeGenerated ? { exclude_generated: true } : {}), }); return response.ok({ body, diff --git a/x-pack/plugins/ml/server/routes/job_service.ts b/x-pack/plugins/ml/server/routes/job_service.ts index c067d9ce0abbc..a72e942e987aa 100644 --- a/x-pack/plugins/ml/server/routes/job_service.ts +++ b/x-pack/plugins/ml/server/routes/job_service.ts @@ -272,6 +272,40 @@ export function jobServiceRoutes({ router, routeGuard }: RouteInitialization) { }) ); + /** + * @apiGroup JobService + * + * @api {post} /api/ml/jobs/job_for_cloning Get job for cloning + * @apiName GetJobForCloning + * @apiDescription Get the job configuration with auto generated fields excluded for cloning + * + * @apiSchema (body) jobIdSchema + */ + router.post( + { + path: '/api/ml/jobs/job_for_cloning', + validate: { + body: jobIdSchema, + }, + options: { + tags: ['access:ml:canGetJobs'], + }, + }, + routeGuard.fullLicenseAPIGuard(async ({ client, mlClient, request, response }) => { + try { + const { getJobForCloning } = jobServiceProvider(client, mlClient); + const { jobId } = request.body; + + const resp = await getJobForCloning(jobId); + return response.ok({ + body: resp, + }); + } catch (e) { + return response.customError(wrapError(e)); + } + }) + ); + /** * @apiGroup JobService * diff --git a/x-pack/plugins/ml/server/routes/schemas/data_analytics_schema.ts b/x-pack/plugins/ml/server/routes/schemas/data_analytics_schema.ts index cf52d1cb27433..0f965cf500b86 100644 --- a/x-pack/plugins/ml/server/routes/schemas/data_analytics_schema.ts +++ b/x-pack/plugins/ml/server/routes/schemas/data_analytics_schema.ts @@ -64,6 +64,13 @@ export const analyticsIdSchema = schema.object({ analyticsId: schema.string(), }); +export const analyticsQuerySchema = schema.object({ + /** + * Analytics Query + */ + excludeGenerated: schema.maybe(schema.boolean()), +}); + export const deleteDataFrameAnalyticsJobSchema = schema.object({ /** * Analytics Destination Index diff --git a/x-pack/plugins/ml/server/routes/schemas/job_service_schema.ts b/x-pack/plugins/ml/server/routes/schemas/job_service_schema.ts index 583c9c41727ea..56094a4950a0c 100644 --- a/x-pack/plugins/ml/server/routes/schemas/job_service_schema.ts +++ b/x-pack/plugins/ml/server/routes/schemas/job_service_schema.ts @@ -39,6 +39,11 @@ export const forceStartDatafeedSchema = schema.object({ end: schema.maybe(schema.number()), }); +export const jobIdSchema = schema.object({ + /** Optional list of job IDs. */ + jobIds: schema.maybe(schema.string()), +}); + export const jobIdsSchema = schema.object({ /** Optional list of job IDs. */ jobIds: schema.maybe(schema.arrayOf(schema.maybe(schema.string()))), diff --git a/x-pack/plugins/ml/server/saved_objects/authorization.ts b/x-pack/plugins/ml/server/saved_objects/authorization.ts index 958ee2091f11e..9afc479c32d7d 100644 --- a/x-pack/plugins/ml/server/saved_objects/authorization.ts +++ b/x-pack/plugins/ml/server/saved_objects/authorization.ts @@ -10,6 +10,15 @@ import { ML_SAVED_OBJECT_TYPE } from '../../common/types/saved_objects'; export function authorizationProvider(authorization: SecurityPluginSetup['authz']) { async function authorizationCheck(request: KibanaRequest) { + const shouldAuthorizeRequest = authorization?.mode.useRbacForRequest(request) ?? false; + + if (shouldAuthorizeRequest === false) { + return { + canCreateGlobally: true, + canCreateAtSpace: true, + }; + } + const checkPrivilegesWithRequest = authorization.checkPrivilegesWithRequest(request); // Checking privileges "dynamically" will check against the current space, if spaces are enabled. // If spaces are disabled, then this will check privileges globally instead. diff --git a/x-pack/plugins/monitoring/server/es_client/instantiate_client.ts b/x-pack/plugins/monitoring/server/es_client/instantiate_client.ts index 734caa7374686..3336e65da2b11 100644 --- a/x-pack/plugins/monitoring/server/es_client/instantiate_client.ts +++ b/x-pack/plugins/monitoring/server/es_client/instantiate_client.ts @@ -30,7 +30,6 @@ export function instantiateClient( const cluster = createClient('monitoring', { ...(isMonitoringCluster ? elasticsearchConfig : {}), plugins: [monitoringBulk, monitoringEndpointDisableWatches], - logQueries: Boolean(elasticsearchConfig.logQueries), } as ESClusterConfig); const configSource = isMonitoringCluster ? 'monitoring' : 'production'; diff --git a/x-pack/plugins/observability/public/components/shared/action_menu/index.tsx b/x-pack/plugins/observability/public/components/shared/action_menu/index.tsx index 4819a0760d88a..af61f618a89b2 100644 --- a/x-pack/plugins/observability/public/components/shared/action_menu/index.tsx +++ b/x-pack/plugins/observability/public/components/shared/action_menu/index.tsx @@ -26,7 +26,7 @@ export function SectionTitle({ children }: { children?: ReactNode }) {
{children}
 - + ); } @@ -55,7 +55,7 @@ export function SectionSpacer() { } export const Section = styled.div` - margin-bottom: 24px; + margin-bottom: 16px; &:last-of-type { margin-bottom: 0; } @@ -63,7 +63,7 @@ export const Section = styled.div` export type SectionLinkProps = EuiListGroupItemProps; export function SectionLink(props: SectionLinkProps) { - return ; + return ; } export function ActionMenuDivider() { diff --git a/x-pack/plugins/observability/public/index.ts b/x-pack/plugins/observability/public/index.ts index c052541956c13..49dc298d9a9b0 100644 --- a/x-pack/plugins/observability/public/index.ts +++ b/x-pack/plugins/observability/public/index.ts @@ -33,3 +33,4 @@ export * from './typings'; export { useChartTheme } from './hooks/use_chart_theme'; export { useTheme } from './hooks/use_theme'; +export { getApmTraceUrl } from './utils/get_apm_trace_url'; diff --git a/x-pack/plugins/observability/public/utils/get_apm_trace_url.test.ts b/x-pack/plugins/observability/public/utils/get_apm_trace_url.test.ts new file mode 100644 index 0000000000000..14ede0d3a114e --- /dev/null +++ b/x-pack/plugins/observability/public/utils/get_apm_trace_url.test.ts @@ -0,0 +1,14 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License; + * you may not use this file except in compliance with the Elastic License. + */ +import { getApmTraceUrl } from './get_apm_trace_url'; + +describe('getApmTraceUrl', () => { + it('returns a trace url', () => { + expect(getApmTraceUrl({ traceId: 'foo', rangeFrom: '123', rangeTo: '456' })).toEqual( + '/link-to/trace/foo?rangeFrom=123&rangeTo=456' + ); + }); +}); diff --git a/x-pack/plugins/apm/public/components/shared/Links/apm/ExternalLinks.ts b/x-pack/plugins/observability/public/utils/get_apm_trace_url.ts similarity index 69% rename from x-pack/plugins/apm/public/components/shared/Links/apm/ExternalLinks.ts rename to x-pack/plugins/observability/public/utils/get_apm_trace_url.ts index 819c6eafe80b4..e60ec10e45a86 100644 --- a/x-pack/plugins/apm/public/components/shared/Links/apm/ExternalLinks.ts +++ b/x-pack/plugins/observability/public/utils/get_apm_trace_url.ts @@ -4,7 +4,7 @@ * you may not use this file except in compliance with the Elastic License. */ -export const getTraceUrl = ({ +export function getApmTraceUrl({ traceId, rangeFrom, rangeTo, @@ -12,9 +12,6 @@ export const getTraceUrl = ({ traceId: string; rangeFrom: string; rangeTo: string; -}) => { - return ( - `/link-to/trace/${traceId}?` + - new URLSearchParams({ rangeFrom, rangeTo }).toString() - ); -}; +}) { + return `/link-to/trace/${traceId}?` + new URLSearchParams({ rangeFrom, rangeTo }).toString(); +} diff --git a/x-pack/plugins/reporting/common/constants.ts b/x-pack/plugins/reporting/common/constants.ts index 16e40bab65a46..882387184ba9c 100644 --- a/x-pack/plugins/reporting/common/constants.ts +++ b/x-pack/plugins/reporting/common/constants.ts @@ -56,14 +56,19 @@ export const LAYOUT_TYPES = { }; // Export Type Definitions -export const CSV_REPORT_TYPE = 'CSV'; export const PDF_REPORT_TYPE = 'printablePdf'; -export const PNG_REPORT_TYPE = 'PNG'; - export const PDF_JOB_TYPE = 'printable_pdf'; + +export const PNG_REPORT_TYPE = 'PNG'; export const PNG_JOB_TYPE = 'PNG'; -export const CSV_JOB_TYPE = 'csv'; + export const CSV_FROM_SAVEDOBJECT_JOB_TYPE = 'csv_from_savedobject'; + +// This is deprecated because it lacks support for runtime fields +// but the extension points are still needed for pre-existing scripted automation, until 8.0 +export const CSV_REPORT_TYPE_DEPRECATED = 'CSV'; +export const CSV_JOB_TYPE_DEPRECATED = 'csv'; + export const USES_HEADLESS_JOB_TYPES = [PDF_JOB_TYPE, PNG_JOB_TYPE]; // Licenses diff --git a/x-pack/plugins/reporting/public/components/reporting_panel_content.tsx b/x-pack/plugins/reporting/public/components/reporting_panel_content.tsx index bbdc2e1aebe77..bafb5d7a68630 100644 --- a/x-pack/plugins/reporting/public/components/reporting_panel_content.tsx +++ b/x-pack/plugins/reporting/public/components/reporting_panel_content.tsx @@ -10,7 +10,11 @@ import React, { Component, ReactElement } from 'react'; import { ToastsSetup } from 'src/core/public'; import url from 'url'; import { toMountPoint } from '../../../../../src/plugins/kibana_react/public'; -import { CSV_REPORT_TYPE, PDF_REPORT_TYPE, PNG_REPORT_TYPE } from '../../common/constants'; +import { + CSV_REPORT_TYPE_DEPRECATED, + PDF_REPORT_TYPE, + PNG_REPORT_TYPE, +} from '../../common/constants'; import { BaseParams } from '../../common/types'; import { ReportingAPIClient } from '../lib/reporting_api_client'; @@ -173,7 +177,7 @@ class ReportingPanelContentUi extends Component { case PDF_REPORT_TYPE: return 'PDF'; case 'csv': - return CSV_REPORT_TYPE; + return CSV_REPORT_TYPE_DEPRECATED; case 'png': return PNG_REPORT_TYPE; default: diff --git a/x-pack/plugins/reporting/public/panel_actions/get_csv_panel_action.tsx b/x-pack/plugins/reporting/public/panel_actions/get_csv_panel_action.tsx index 9a4832b114e40..49c0eaaa2960d 100644 --- a/x-pack/plugins/reporting/public/panel_actions/get_csv_panel_action.tsx +++ b/x-pack/plugins/reporting/public/panel_actions/get_csv_panel_action.tsx @@ -103,7 +103,6 @@ export class GetCsvReportPanelAction implements ActionDefinition const kibanaTimezone = this.core.uiSettings.get('dateFormat:tz'); const id = `search:${embeddable.getSavedSearch().id}`; - const filename = embeddable.getSavedSearch().title; const timezone = kibanaTimezone === 'Browser' ? moment.tz.guess() : kibanaTimezone; const fromTime = dateMath.parse(from); const toTime = dateMath.parse(to, { roundUp: true }); @@ -140,7 +139,7 @@ export class GetCsvReportPanelAction implements ActionDefinition .then((rawResponse: string) => { this.isDownloading = false; - const download = `${filename}.csv`; + const download = `${embeddable.getSavedSearch().title}.csv`; const blob = new Blob([rawResponse], { type: 'text/csv;charset=utf-8;' }); // Hack for IE11 Support diff --git a/x-pack/plugins/reporting/public/share_context_menu/register_csv_reporting.tsx b/x-pack/plugins/reporting/public/share_context_menu/register_csv_reporting.tsx index 7126762c0f4ee..4659952eef720 100644 --- a/x-pack/plugins/reporting/public/share_context_menu/register_csv_reporting.tsx +++ b/x-pack/plugins/reporting/public/share_context_menu/register_csv_reporting.tsx @@ -10,7 +10,10 @@ import React from 'react'; import { IUiSettingsClient, ToastsSetup } from 'src/core/public'; import { ShareContext } from '../../../../../src/plugins/share/public'; import { LicensingPluginSetup } from '../../../licensing/public'; -import { JobParamsCSV, SearchRequest } from '../../server/export_types/csv/types'; +import { + JobParamsDeprecatedCSV, + SearchRequestDeprecatedCSV, +} from '../../server/export_types/csv/types'; import { ReportingPanelContent } from '../components/reporting_panel_content_lazy'; import { checkLicense } from '../lib/license_check'; import { ReportingAPIClient } from '../lib/reporting_api_client'; @@ -59,12 +62,12 @@ export const csvReportingProvider = ({ return []; } - const jobParams: JobParamsCSV = { + const jobParams: JobParamsDeprecatedCSV = { browserTimezone, objectType, title: sharingData.title as string, indexPatternId: sharingData.indexPatternId as string, - searchRequest: sharingData.searchRequest as SearchRequest, + searchRequest: sharingData.searchRequest as SearchRequestDeprecatedCSV, fields: sharingData.fields as string[], metaFields: sharingData.metaFields as string[], conflictedTypesFields: sharingData.conflictedTypesFields as string[], diff --git a/x-pack/plugins/reporting/server/export_types/csv/create_job.ts b/x-pack/plugins/reporting/server/export_types/csv/create_job.ts index f0f72a0bc9965..e704f9650b7a8 100644 --- a/x-pack/plugins/reporting/server/export_types/csv/create_job.ts +++ b/x-pack/plugins/reporting/server/export_types/csv/create_job.ts @@ -4,15 +4,19 @@ * you may not use this file except in compliance with the Elastic License. */ -import { CSV_JOB_TYPE } from '../../../common/constants'; +import { CSV_JOB_TYPE_DEPRECATED } from '../../../common/constants'; import { cryptoFactory } from '../../lib'; import { CreateJobFn, CreateJobFnFactory } from '../../types'; -import { IndexPatternSavedObject, JobParamsCSV, TaskPayloadCSV } from './types'; +import { + IndexPatternSavedObjectDeprecatedCSV, + JobParamsDeprecatedCSV, + TaskPayloadDeprecatedCSV, +} from './types'; export const createJobFnFactory: CreateJobFnFactory< - CreateJobFn + CreateJobFn > = function createJobFactoryFn(reporting, parentLogger) { - const logger = parentLogger.clone([CSV_JOB_TYPE, 'create-job']); + const logger = parentLogger.clone([CSV_JOB_TYPE_DEPRECATED, 'create-job']); const config = reporting.getConfig(); const crypto = cryptoFactory(config.get('encryptionKey')); @@ -24,7 +28,7 @@ export const createJobFnFactory: CreateJobFnFactory< const indexPatternSavedObject = ((await savedObjectsClient.get( 'index-pattern', jobParams.indexPatternId - )) as unknown) as IndexPatternSavedObject; // FIXME + )) as unknown) as IndexPatternSavedObjectDeprecatedCSV; return { headers: serializedEncryptedHeaders, diff --git a/x-pack/plugins/reporting/server/export_types/csv/execute_job.test.ts b/x-pack/plugins/reporting/server/export_types/csv/execute_job.test.ts index ea65262c090ee..098a90959f8a7 100644 --- a/x-pack/plugins/reporting/server/export_types/csv/execute_job.test.ts +++ b/x-pack/plugins/reporting/server/export_types/csv/execute_job.test.ts @@ -22,7 +22,7 @@ import { LevelLogger } from '../../lib'; import { setFieldFormats } from '../../services'; import { createMockReportingCore } from '../../test_helpers'; import { runTaskFnFactory } from './execute_job'; -import { TaskPayloadCSV } from './types'; +import { TaskPayloadDeprecatedCSV } from './types'; const delay = (ms: number) => new Promise((resolve) => setTimeout(() => resolve(), ms)); @@ -31,7 +31,7 @@ const getRandomScrollId = () => { return puid.generate(); }; -const getBasePayload = (baseObj: any) => baseObj as TaskPayloadCSV; +const getBasePayload = (baseObj: any) => baseObj as TaskPayloadDeprecatedCSV; describe('CSV Execute Job', function () { const encryptionKey = 'testEncryptionKey'; diff --git a/x-pack/plugins/reporting/server/export_types/csv/execute_job.ts b/x-pack/plugins/reporting/server/export_types/csv/execute_job.ts index 6b4dd48583efe..cb321b7573701 100644 --- a/x-pack/plugins/reporting/server/export_types/csv/execute_job.ts +++ b/x-pack/plugins/reporting/server/export_types/csv/execute_job.ts @@ -4,20 +4,20 @@ * you may not use this file except in compliance with the Elastic License. */ -import { CONTENT_TYPE_CSV, CSV_JOB_TYPE } from '../../../common/constants'; +import { CONTENT_TYPE_CSV, CSV_JOB_TYPE_DEPRECATED } from '../../../common/constants'; import { RunTaskFn, RunTaskFnFactory } from '../../types'; import { decryptJobHeaders } from '../common'; import { createGenerateCsv } from './generate_csv'; -import { TaskPayloadCSV } from './types'; +import { TaskPayloadDeprecatedCSV } from './types'; export const runTaskFnFactory: RunTaskFnFactory< - RunTaskFn + RunTaskFn > = function executeJobFactoryFn(reporting, parentLogger) { const config = reporting.getConfig(); return async function runTask(jobId, job, cancellationToken) { const elasticsearch = reporting.getElasticsearchService(); - const logger = parentLogger.clone([CSV_JOB_TYPE, 'execute-job', jobId]); + const logger = parentLogger.clone([CSV_JOB_TYPE_DEPRECATED, 'execute-job', jobId]); const generateCsv = createGenerateCsv(logger); const encryptionKey = config.get('encryptionKey'); diff --git a/x-pack/plugins/reporting/server/export_types/csv/generate_csv/field_format_map.test.ts b/x-pack/plugins/reporting/server/export_types/csv/generate_csv/field_format_map.test.ts index 4cb8de5810584..0c74e3aa54b0e 100644 --- a/x-pack/plugins/reporting/server/export_types/csv/generate_csv/field_format_map.test.ts +++ b/x-pack/plugins/reporting/server/export_types/csv/generate_csv/field_format_map.test.ts @@ -6,13 +6,13 @@ import expect from '@kbn/expect'; import { fieldFormats, FieldFormatsGetConfigFn, UI_SETTINGS } from 'src/plugins/data/server'; -import { IndexPatternSavedObject } from '../types'; +import { IndexPatternSavedObjectDeprecatedCSV } from '../types'; import { fieldFormatMapFactory } from './field_format_map'; type ConfigValue = { number: { id: string; params: {} } } | string; describe('field format map', function () { - const indexPatternSavedObject: IndexPatternSavedObject = { + const indexPatternSavedObject: IndexPatternSavedObjectDeprecatedCSV = { timeFieldName: '@timestamp', title: 'logstash-*', attributes: { diff --git a/x-pack/plugins/reporting/server/export_types/csv/generate_csv/field_format_map.ts b/x-pack/plugins/reporting/server/export_types/csv/generate_csv/field_format_map.ts index e01fee530fc65..c05dc7d3fd75f 100644 --- a/x-pack/plugins/reporting/server/export_types/csv/generate_csv/field_format_map.ts +++ b/x-pack/plugins/reporting/server/export_types/csv/generate_csv/field_format_map.ts @@ -7,7 +7,7 @@ import _ from 'lodash'; import { FieldFormat } from 'src/plugins/data/common'; import { FieldFormatConfig, IFieldFormatsRegistry } from 'src/plugins/data/server'; -import { IndexPatternSavedObject } from '../types'; +import { IndexPatternSavedObjectDeprecatedCSV } from '../types'; /** * Create a map of FieldFormat instances for index pattern fields @@ -17,7 +17,7 @@ import { IndexPatternSavedObject } from '../types'; * @return {Map} key: field name, value: FieldFormat instance */ export function fieldFormatMapFactory( - indexPatternSavedObject: IndexPatternSavedObject, + indexPatternSavedObject: IndexPatternSavedObjectDeprecatedCSV, fieldFormatsRegistry: IFieldFormatsRegistry, timezone: string | undefined ) { diff --git a/x-pack/plugins/reporting/server/export_types/csv/generate_csv/index.ts b/x-pack/plugins/reporting/server/export_types/csv/generate_csv/index.ts index 2f6df9cd67a75..ee09f3904678c 100644 --- a/x-pack/plugins/reporting/server/export_types/csv/generate_csv/index.ts +++ b/x-pack/plugins/reporting/server/export_types/csv/generate_csv/index.ts @@ -12,7 +12,7 @@ import { CSV_BOM_CHARS } from '../../../../common/constants'; import { byteSizeValueToNumber } from '../../../../common/schema_utils'; import { LevelLogger } from '../../../lib'; import { getFieldFormats } from '../../../services'; -import { IndexPatternSavedObject, SavedSearchGeneratorResult } from '../types'; +import { IndexPatternSavedObjectDeprecatedCSV, SavedSearchGeneratorResult } from '../types'; import { checkIfRowsHaveFormulas } from './check_cells_for_formulas'; import { createEscapeValue } from './escape_value'; import { fieldFormatMapFactory } from './field_format_map'; @@ -39,7 +39,7 @@ interface SearchRequest { export interface GenerateCsvParams { browserTimezone?: string; searchRequest: SearchRequest; - indexPatternSavedObject: IndexPatternSavedObject; + indexPatternSavedObject: IndexPatternSavedObjectDeprecatedCSV; fields: string[]; metaFields: string[]; conflictedTypesFields: string[]; diff --git a/x-pack/plugins/reporting/server/export_types/csv/index.ts b/x-pack/plugins/reporting/server/export_types/csv/index.ts index f7b7ff5709fe6..23f4b879eb140 100644 --- a/x-pack/plugins/reporting/server/export_types/csv/index.ts +++ b/x-pack/plugins/reporting/server/export_types/csv/index.ts @@ -5,7 +5,7 @@ */ import { - CSV_JOB_TYPE as jobType, + CSV_JOB_TYPE_DEPRECATED as jobType, LICENSE_TYPE_BASIC, LICENSE_TYPE_ENTERPRISE, LICENSE_TYPE_GOLD, @@ -17,11 +17,11 @@ import { CreateJobFn, ExportTypeDefinition, RunTaskFn } from '../../types'; import { createJobFnFactory } from './create_job'; import { runTaskFnFactory } from './execute_job'; import { metadata } from './metadata'; -import { JobParamsCSV, TaskPayloadCSV } from './types'; +import { JobParamsDeprecatedCSV, TaskPayloadDeprecatedCSV } from './types'; export const getExportType = (): ExportTypeDefinition< - CreateJobFn, - RunTaskFn + CreateJobFn, + RunTaskFn > => ({ ...metadata, jobType, diff --git a/x-pack/plugins/reporting/server/export_types/csv/types.d.ts b/x-pack/plugins/reporting/server/export_types/csv/types.d.ts index 78615a0e7b72c..dd0b37a17a2ff 100644 --- a/x-pack/plugins/reporting/server/export_types/csv/types.d.ts +++ b/x-pack/plugins/reporting/server/export_types/csv/types.d.ts @@ -8,7 +8,7 @@ import { BaseParams, BasePayload } from '../../types'; export type RawValue = string | object | null | undefined; -export interface IndexPatternSavedObject { +export interface IndexPatternSavedObjectDeprecatedCSV { title: string; timeFieldName: string; fields?: any[]; @@ -18,25 +18,25 @@ export interface IndexPatternSavedObject { }; } -interface BaseParamsCSV { - searchRequest: SearchRequest; +interface BaseParamsDeprecatedCSV { + searchRequest: SearchRequestDeprecatedCSV; fields: string[]; metaFields: string[]; conflictedTypesFields: string[]; } -export type JobParamsCSV = BaseParamsCSV & +export type JobParamsDeprecatedCSV = BaseParamsDeprecatedCSV & BaseParams & { indexPatternId: string; }; // CSV create job method converts indexPatternID to indexPatternSavedObject -export type TaskPayloadCSV = BaseParamsCSV & +export type TaskPayloadDeprecatedCSV = BaseParamsDeprecatedCSV & BasePayload & { - indexPatternSavedObject: IndexPatternSavedObject; + indexPatternSavedObject: IndexPatternSavedObjectDeprecatedCSV; }; -export interface SearchRequest { +export interface SearchRequestDeprecatedCSV { index: string; body: | { @@ -66,7 +66,7 @@ export interface SearchRequest { | any; } -type FormatsMap = Map< +type FormatsMapDeprecatedCSV = Map< string, { id: string; diff --git a/x-pack/plugins/reporting/server/export_types/csv_from_savedobject/lib/get_data_source.ts b/x-pack/plugins/reporting/server/export_types/csv_from_savedobject/lib/get_data_source.ts index e3631b9c89724..fa983c5af639c 100644 --- a/x-pack/plugins/reporting/server/export_types/csv_from_savedobject/lib/get_data_source.ts +++ b/x-pack/plugins/reporting/server/export_types/csv_from_savedobject/lib/get_data_source.ts @@ -4,7 +4,7 @@ * you may not use this file except in compliance with the Elastic License. */ -import { IndexPatternSavedObject } from '../../csv/types'; +import { IndexPatternSavedObjectDeprecatedCSV } from '../../csv/types'; import { SavedObjectReference, SavedSearchObjectAttributesJSON, SearchSource } from '../types'; export async function getDataSource( @@ -12,10 +12,10 @@ export async function getDataSource( indexPatternId?: string, savedSearchObjectId?: string ): Promise<{ - indexPatternSavedObject: IndexPatternSavedObject; + indexPatternSavedObject: IndexPatternSavedObjectDeprecatedCSV; searchSource: SearchSource | null; }> { - let indexPatternSavedObject: IndexPatternSavedObject; + let indexPatternSavedObject: IndexPatternSavedObjectDeprecatedCSV; let searchSource: SearchSource | null = null; if (savedSearchObjectId) { diff --git a/x-pack/plugins/reporting/server/routes/lib/get_document_payload.ts b/x-pack/plugins/reporting/server/routes/lib/get_document_payload.ts index 7706aa9d650c7..641ce6e48a1f3 100644 --- a/x-pack/plugins/reporting/server/routes/lib/get_document_payload.ts +++ b/x-pack/plugins/reporting/server/routes/lib/get_document_payload.ts @@ -7,7 +7,7 @@ // @ts-ignore import contentDisposition from 'content-disposition'; import { get } from 'lodash'; -import { CSV_JOB_TYPE } from '../../../common/constants'; +import { CSV_JOB_TYPE_DEPRECATED } from '../../../common/constants'; import { ExportTypesRegistry, statuses } from '../../lib'; import { ReportDocument } from '../../lib/store'; import { TaskRunResult } from '../../lib/tasks'; @@ -33,7 +33,7 @@ const getTitle = (exportType: ExportTypeDefinition, title?: string): string => const getReportingHeaders = (output: TaskRunResult, exportType: ExportTypeDefinition) => { const metaDataHeaders: Record = {}; - if (exportType.jobType === CSV_JOB_TYPE) { + if (exportType.jobType === CSV_JOB_TYPE_DEPRECATED) { const csvContainsFormulas = get(output, 'csv_contains_formulas', false); const maxSizedReach = get(output, 'max_size_reached', false); diff --git a/x-pack/plugins/reporting/server/usage/decorate_range_stats.ts b/x-pack/plugins/reporting/server/usage/decorate_range_stats.ts index 30befcf291a54..8d69d75f66212 100644 --- a/x-pack/plugins/reporting/server/usage/decorate_range_stats.ts +++ b/x-pack/plugins/reporting/server/usage/decorate_range_stats.ts @@ -5,7 +5,7 @@ */ import { uniq } from 'lodash'; -import { CSV_JOB_TYPE, PDF_JOB_TYPE, PNG_JOB_TYPE } from '../../common/constants'; +import { CSV_JOB_TYPE_DEPRECATED, PDF_JOB_TYPE, PNG_JOB_TYPE } from '../../common/constants'; import { AvailableTotal, ExportType, FeatureAvailabilityMap, RangeStats } from './types'; function getForFeature( @@ -54,7 +54,7 @@ export const decorateRangeStats = ( // combine the known types with any unknown type found in reporting data const keysBasic = uniq([ - CSV_JOB_TYPE, + CSV_JOB_TYPE_DEPRECATED, PNG_JOB_TYPE, ...Object.keys(rangeStatsBasic), ]) as ExportType[]; diff --git a/x-pack/plugins/security/public/authentication/login/components/login_form/__snapshots__/login_form.test.tsx.snap b/x-pack/plugins/security/public/authentication/login/components/login_form/__snapshots__/login_form.test.tsx.snap index 99780542b97f4..d6eb4c20b8003 100644 --- a/x-pack/plugins/security/public/authentication/login/components/login_form/__snapshots__/login_form.test.tsx.snap +++ b/x-pack/plugins/security/public/authentication/login/components/login_form/__snapshots__/login_form.test.tsx.snap @@ -128,6 +128,7 @@ exports[`LoginForm renders as expected 1`] = ` > { {...this.validator.validateUsername(this.state.username)} > { wrappingComponent: TestProviders, }); expect(wrapper.find('[data-test-subj="field-mapping-desc"]').first().text()).toBe( - 'Field mappings require an established connection to ServiceNow. Please check your connection credentials.' + 'Field mappings require an established connection to ServiceNow ITSM. Please check your connection credentials.' ); }); }); diff --git a/x-pack/plugins/security_solution/public/cases/components/connectors/config.ts b/x-pack/plugins/security_solution/public/cases/components/connectors/config.ts index 3aca186378820..a29531d89b405 100644 --- a/x-pack/plugins/security_solution/public/cases/components/connectors/config.ts +++ b/x-pack/plugins/security_solution/public/cases/components/connectors/config.ts @@ -7,14 +7,14 @@ /* eslint-disable @kbn/eslint/no-restricted-paths */ import { - ServiceNowConnectorConfiguration, + ServiceNowITSMConnectorConfiguration, JiraConnectorConfiguration, ResilientConnectorConfiguration, } from '../../../../../triggers_actions_ui/public/common'; import { ConnectorConfiguration } from './types'; export const connectorsConfiguration: Record = { - '.servicenow': ServiceNowConnectorConfiguration as ConnectorConfiguration, + '.servicenow': ServiceNowITSMConnectorConfiguration as ConnectorConfiguration, '.jira': JiraConnectorConfiguration as ConnectorConfiguration, '.resilient': ResilientConnectorConfiguration as ConnectorConfiguration, }; diff --git a/x-pack/plugins/security_solution/public/cases/components/connectors/types.ts b/x-pack/plugins/security_solution/public/cases/components/connectors/types.ts index 5d83c226bfeca..00bc01b2ec0a7 100644 --- a/x-pack/plugins/security_solution/public/cases/components/connectors/types.ts +++ b/x-pack/plugins/security_solution/public/cases/components/connectors/types.ts @@ -22,5 +22,4 @@ export interface ThirdPartyField { export interface ConnectorConfiguration extends ActionType { logo: string; - fields: Record; } diff --git a/x-pack/plugins/security_solution/public/cases/components/create/form_context.test.tsx b/x-pack/plugins/security_solution/public/cases/components/create/form_context.test.tsx index f3b47f756bce9..1b4df7730cc8b 100644 --- a/x-pack/plugins/security_solution/public/cases/components/create/form_context.test.tsx +++ b/x-pack/plugins/security_solution/public/cases/components/create/form_context.test.tsx @@ -33,8 +33,13 @@ import { import { FormContext } from './form_context'; import { CreateCaseForm } from './form'; import { SubmitCaseButton } from './submit_button'; +import { usePostPushToService } from '../../containers/use_post_push_to_service'; +import { noop } from 'lodash/fp'; + +const sampleId = 'case-id'; jest.mock('../../containers/use_post_case'); +jest.mock('../../containers/use_post_push_to_service'); jest.mock('../../containers/use_get_tags'); jest.mock('../../containers/configure/use_connectors'); jest.mock('../../containers/configure/use_configure'); @@ -48,19 +53,28 @@ jest.mock('../settings/jira/use_get_issues'); const useConnectorsMock = useConnectors as jest.Mock; const useCaseConfigureMock = useCaseConfigure as jest.Mock; const usePostCaseMock = usePostCase as jest.Mock; +const usePostPushToServiceMock = usePostPushToService as jest.Mock; const useGetIncidentTypesMock = useGetIncidentTypes as jest.Mock; const useGetSeverityMock = useGetSeverity as jest.Mock; const useGetIssueTypesMock = useGetIssueTypes as jest.Mock; const useGetFieldsByIssueTypeMock = useGetFieldsByIssueType as jest.Mock; const postCase = jest.fn(); +const postPushToService = jest.fn(); const defaultPostCase = { isLoading: false, isError: false, - caseData: null, postCase, }; +const defaultPostPushToService = { + serviceData: null, + pushedCaseData: null, + isLoading: false, + isError: false, + postPushToService, +}; + const fillForm = (wrapper: ReactWrapper) => { wrapper .find(`[data-test-subj="caseTitle"] input`) @@ -85,7 +99,12 @@ describe('Create case', () => { beforeEach(() => { jest.resetAllMocks(); + postCase.mockResolvedValue({ + id: sampleId, + ...sampleData, + }); usePostCaseMock.mockImplementation(() => defaultPostCase); + usePostPushToServiceMock.mockImplementation(() => defaultPostPushToService); useConnectorsMock.mockReturnValue(sampleConnectorData); useCaseConfigureMock.mockImplementation(() => useCaseConfigureResponse); useGetIncidentTypesMock.mockReturnValue(useGetIncidentTypesResponse); @@ -163,25 +182,6 @@ describe('Create case', () => { ); }); - it('should redirect to new case when caseData is there', async () => { - const sampleId = 'case-id'; - usePostCaseMock.mockImplementation(() => ({ - ...defaultPostCase, - caseData: { id: sampleId }, - })); - - mount( - - - - - - - ); - - await waitFor(() => expect(onFormSubmitSuccess).toHaveBeenCalledWith({ id: 'case-id' })); - }); - it('it should select the default connector set in the configuration', async () => { useCaseConfigureMock.mockImplementation(() => ({ ...useCaseConfigureResponse, @@ -258,12 +258,15 @@ describe('Create case', () => { fillForm(wrapper); wrapper.find(`[data-test-subj="create-case-submit"]`).first().simulate('click'); - await waitFor(() => expect(postCase).toBeCalledWith(sampleData)); + await waitFor(() => { + expect(postCase).toBeCalledWith(sampleData); + expect(postPushToService).not.toHaveBeenCalled(); + }); }); }); describe('Step 2 - Connector Fields', () => { - it(`it should submit a Jira connector`, async () => { + it(`it should submit and push to Jira connector`, async () => { useConnectorsMock.mockReturnValue({ ...sampleConnectorData, connectors: connectorsMock, @@ -304,7 +307,7 @@ describe('Create case', () => { wrapper.find(`[data-test-subj="create-case-submit"]`).first().simulate('click'); - await waitFor(() => + await waitFor(() => { expect(postCase).toBeCalledWith({ ...sampleData, connector: { @@ -313,11 +316,27 @@ describe('Create case', () => { type: '.jira', fields: { issueType: '10007', parent: null, priority: '2' }, }, - }) - ); + }); + expect(postPushToService).toHaveBeenCalledWith({ + caseId: sampleId, + caseServices: {}, + connector: { + id: 'jira-1', + name: 'Jira', + type: '.jira', + fields: { issueType: '10007', parent: null, priority: '2' }, + }, + alerts: {}, + updateCase: noop, + }); + expect(onFormSubmitSuccess).toHaveBeenCalledWith({ + id: sampleId, + ...sampleData, + }); + }); }); - it(`it should submit a resilient connector`, async () => { + it(`it should submit and push to resilient connector`, async () => { useConnectorsMock.mockReturnValue({ ...sampleConnectorData, connectors: connectorsMock, @@ -359,7 +378,7 @@ describe('Create case', () => { wrapper.find(`[data-test-subj="create-case-submit"]`).first().simulate('click'); - await waitFor(() => + await waitFor(() => { expect(postCase).toBeCalledWith({ ...sampleData, connector: { @@ -368,11 +387,29 @@ describe('Create case', () => { type: '.resilient', fields: { incidentTypes: ['19'], severityCode: '4' }, }, - }) - ); + }); + + expect(postPushToService).toHaveBeenCalledWith({ + caseId: sampleId, + caseServices: {}, + connector: { + id: 'resilient-2', + name: 'My Connector 2', + type: '.resilient', + fields: { incidentTypes: ['19'], severityCode: '4' }, + }, + alerts: {}, + updateCase: noop, + }); + + expect(onFormSubmitSuccess).toHaveBeenCalledWith({ + id: sampleId, + ...sampleData, + }); + }); }); - it(`it should submit a servicenow connector`, async () => { + it(`it should submit and push to servicenow connector`, async () => { useConnectorsMock.mockReturnValue({ ...sampleConnectorData, connectors: connectorsMock, @@ -404,7 +441,7 @@ describe('Create case', () => { wrapper.find(`[data-test-subj="create-case-submit"]`).first().simulate('click'); - await waitFor(() => + await waitFor(() => { expect(postCase).toBeCalledWith({ ...sampleData, connector: { @@ -413,8 +450,26 @@ describe('Create case', () => { type: '.servicenow', fields: { impact: '2', severity: '2', urgency: '2' }, }, - }) - ); + }); + + expect(postPushToService).toHaveBeenCalledWith({ + caseId: sampleId, + caseServices: {}, + connector: { + id: 'servicenow-1', + name: 'My Connector', + type: '.servicenow', + fields: { impact: '2', severity: '2', urgency: '2' }, + }, + alerts: {}, + updateCase: noop, + }); + + expect(onFormSubmitSuccess).toHaveBeenCalledWith({ + id: sampleId, + ...sampleData, + }); + }); }); }); }); diff --git a/x-pack/plugins/security_solution/public/cases/components/create/form_context.tsx b/x-pack/plugins/security_solution/public/cases/components/create/form_context.tsx index 4315011ac8df1..03e03d853878c 100644 --- a/x-pack/plugins/security_solution/public/cases/components/create/form_context.tsx +++ b/x-pack/plugins/security_solution/public/cases/components/create/form_context.tsx @@ -3,9 +3,8 @@ * or more contributor license agreements. Licensed under the Elastic License; * you may not use this file except in compliance with the Elastic License. */ - import React, { useCallback, useEffect, useMemo } from 'react'; - +import { noop } from 'lodash/fp'; import { schema, FormProps } from './schema'; import { Form, useForm } from '../../../shared_imports'; import { @@ -14,6 +13,8 @@ import { normalizeActionConnector, } from '../configure_cases/utils'; import { usePostCase } from '../../containers/use_post_case'; +import { usePostPushToService } from '../../containers/use_post_push_to_service'; + import { useConnectors } from '../../containers/configure/use_connectors'; import { useCaseConfigure } from '../../containers/configure/use_configure'; import { Case } from '../../containers/types'; @@ -34,7 +35,9 @@ interface Props { export const FormContext: React.FC = ({ children, onSuccess }) => { const { connectors } = useConnectors(); const { connector: configurationConnector } = useCaseConfigure(); - const { caseData, postCase } = usePostCase(); + const { postCase } = usePostCase(); + const { postPushToService } = usePostPushToService(); + const connectorId = useMemo( () => connectors.some((connector) => connector.id === configurationConnector.id) @@ -50,18 +53,33 @@ export const FormContext: React.FC = ({ children, onSuccess }) => { ) => { if (isValid) { const caseConnector = getConnectorById(dataConnectorId, connectors); + const connectorToUpdate = caseConnector ? normalizeActionConnector(caseConnector, fields) : getNoneConnector(); - await postCase({ + const updatedCase = await postCase({ ...dataWithoutConnectorId, connector: connectorToUpdate, settings: { syncAlerts }, }); + + if (updatedCase?.id && dataConnectorId !== 'none') { + await postPushToService({ + caseId: updatedCase.id, + caseServices: {}, + connector: connectorToUpdate, + alerts: {}, + updateCase: noop, + }); + } + + if (onSuccess && updatedCase) { + onSuccess(updatedCase); + } } }, - [postCase, connectors] + [connectors, postCase, onSuccess, postPushToService] ); const { form } = useForm({ @@ -70,18 +88,10 @@ export const FormContext: React.FC = ({ children, onSuccess }) => { schema, onSubmit: submitCase, }); - const { setFieldValue } = form; - // Set the selected connector to the configuration connector useEffect(() => setFieldValue('connectorId', connectorId), [connectorId, setFieldValue]); - useEffect(() => { - if (caseData && onSuccess) { - onSuccess(caseData); - } - }, [caseData, onSuccess]); - return
{children}
; }; diff --git a/x-pack/plugins/security_solution/public/cases/containers/use_post_case.test.tsx b/x-pack/plugins/security_solution/public/cases/containers/use_post_case.test.tsx index 8e8432d0d190c..bd57f57713e08 100644 --- a/x-pack/plugins/security_solution/public/cases/containers/use_post_case.test.tsx +++ b/x-pack/plugins/security_solution/public/cases/containers/use_post_case.test.tsx @@ -6,9 +6,9 @@ import { renderHook, act } from '@testing-library/react-hooks'; import { usePostCase, UsePostCase } from './use_post_case'; -import { basicCasePost } from './mock'; import * as api from './api'; import { ConnectorTypes } from '../../../../case/common/api/connectors'; +import { basicCasePost } from './mock'; jest.mock('./api'); @@ -40,7 +40,6 @@ describe('usePostCase', () => { expect(result.current).toEqual({ isLoading: false, isError: false, - caseData: null, postCase: result.current.postCase, }); }); @@ -59,6 +58,16 @@ describe('usePostCase', () => { }); }); + it('calls postCase with correct result', async () => { + await act(async () => { + const { result, waitForNextUpdate } = renderHook(() => usePostCase()); + await waitForNextUpdate(); + + const postData = await result.current.postCase(samplePost); + expect(postData).toEqual(basicCasePost); + }); + }); + it('post case', async () => { await act(async () => { const { result, waitForNextUpdate } = renderHook(() => usePostCase()); @@ -66,7 +75,6 @@ describe('usePostCase', () => { result.current.postCase(samplePost); await waitForNextUpdate(); expect(result.current).toEqual({ - caseData: basicCasePost, isLoading: false, isError: false, postCase: result.current.postCase, @@ -96,7 +104,6 @@ describe('usePostCase', () => { result.current.postCase(samplePost); expect(result.current).toEqual({ - caseData: null, isLoading: false, isError: true, postCase: result.current.postCase, diff --git a/x-pack/plugins/security_solution/public/cases/containers/use_post_case.tsx b/x-pack/plugins/security_solution/public/cases/containers/use_post_case.tsx index 3ca78dfe75c80..c98446effe47d 100644 --- a/x-pack/plugins/security_solution/public/cases/containers/use_post_case.tsx +++ b/x-pack/plugins/security_solution/public/cases/containers/use_post_case.tsx @@ -3,25 +3,17 @@ * or more contributor license agreements. Licensed under the Elastic License; * you may not use this file except in compliance with the Elastic License. */ - -import { useReducer, useCallback } from 'react'; - +import { useReducer, useCallback, useRef, useEffect } from 'react'; import { CasePostRequest } from '../../../../case/common/api'; import { errorToToaster, useStateToaster } from '../../common/components/toasters'; import { postCase } from './api'; import * as i18n from './translations'; import { Case } from './types'; - interface NewCaseState { - caseData: Case | null; isLoading: boolean; isError: boolean; } -type Action = - | { type: 'FETCH_INIT' } - | { type: 'FETCH_SUCCESS'; payload: Case } - | { type: 'FETCH_FAILURE' }; - +type Action = { type: 'FETCH_INIT' } | { type: 'FETCH_SUCCESS' } | { type: 'FETCH_FAILURE' }; const dataFetchReducer = (state: NewCaseState, action: Action): NewCaseState => { switch (action.type) { case 'FETCH_INIT': @@ -35,7 +27,6 @@ const dataFetchReducer = (state: NewCaseState, action: Action): NewCaseState => ...state, isLoading: false, isError: false, - caseData: action.payload ?? null, }; case 'FETCH_FAILURE': return { @@ -47,47 +38,47 @@ const dataFetchReducer = (state: NewCaseState, action: Action): NewCaseState => return state; } }; - export interface UsePostCase extends NewCaseState { - postCase: (data: CasePostRequest) => Promise<() => void>; + postCase: (data: CasePostRequest) => Promise; } export const usePostCase = (): UsePostCase => { const [state, dispatch] = useReducer(dataFetchReducer, { isLoading: false, isError: false, - caseData: null, }); const [, dispatchToaster] = useStateToaster(); - - const postMyCase = useCallback(async (data: CasePostRequest) => { - let cancel = false; - const abortCtrl = new AbortController(); - - try { - dispatch({ type: 'FETCH_INIT' }); - const response = await postCase(data, abortCtrl.signal); - if (!cancel) { - dispatch({ - type: 'FETCH_SUCCESS', - payload: response, - }); + const cancel = useRef(false); + const abortCtrl = useRef(new AbortController()); + const postMyCase = useCallback( + async (data: CasePostRequest) => { + try { + dispatch({ type: 'FETCH_INIT' }); + abortCtrl.current.abort(); + cancel.current = false; + abortCtrl.current = new AbortController(); + const response = await postCase(data, abortCtrl.current.signal); + if (!cancel.current) { + dispatch({ type: 'FETCH_SUCCESS' }); + } + return response; + } catch (error) { + if (!cancel.current) { + errorToToaster({ + title: i18n.ERROR_TITLE, + error: error.body && error.body.message ? new Error(error.body.message) : error, + dispatchToaster, + }); + dispatch({ type: 'FETCH_FAILURE' }); + } } - } catch (error) { - if (!cancel) { - errorToToaster({ - title: i18n.ERROR_TITLE, - error: error.body && error.body.message ? new Error(error.body.message) : error, - dispatchToaster, - }); - dispatch({ type: 'FETCH_FAILURE' }); - } - } + }, + [dispatchToaster] + ); + useEffect(() => { return () => { - abortCtrl.abort(); - cancel = true; + abortCtrl.current.abort(); + cancel.current = true; }; - // eslint-disable-next-line react-hooks/exhaustive-deps }, []); - return { ...state, postCase: postMyCase }; }; diff --git a/x-pack/plugins/security_solution/public/detections/containers/detection_engine/alerts/use_privilege_user.tsx b/x-pack/plugins/security_solution/public/detections/containers/detection_engine/alerts/use_privilege_user.tsx index 191c3955caa9b..c444702312ffb 100644 --- a/x-pack/plugins/security_solution/public/detections/containers/detection_engine/alerts/use_privilege_user.tsx +++ b/x-pack/plugins/security_solution/public/detections/containers/detection_engine/alerts/use_privilege_user.tsx @@ -62,7 +62,7 @@ export const usePrivilegeUser = (): ReturnPrivilegeUser => { setPrivilegeUser({ isAuthenticated: privilege.is_authenticated, hasEncryptionKey: privilege.has_encryption_key, - hasIndexManage: privilege.index[indexName].manage, + hasIndexManage: privilege.index[indexName].manage && privilege.cluster.manage, hasIndexMaintenance: privilege.index[indexName].maintenance, hasIndexWrite: privilege.index[indexName].create || diff --git a/x-pack/plugins/security_solution/public/timelines/components/notes/add_note/index.tsx b/x-pack/plugins/security_solution/public/timelines/components/notes/add_note/index.tsx index 92e0ac757cc39..10e8ca42f35ae 100644 --- a/x-pack/plugins/security_solution/public/timelines/components/notes/add_note/index.tsx +++ b/x-pack/plugins/security_solution/public/timelines/components/notes/add_note/index.tsx @@ -9,7 +9,6 @@ import { EuiButtonEmpty, EuiFlexGroup, EuiFlexItem, - EuiFocusTrap, EuiScreenReaderOnly, } from '@elastic/eui'; import React, { useCallback } from 'react'; @@ -83,31 +82,29 @@ export const AddNote = React.memo<{ return ( - -
- -

{i18n.YOU_ARE_EDITING_A_NOTE}

- - - - {onCancelAddNote != null ? ( - - - - ) : null} +
+ +

{i18n.YOU_ARE_EDITING_A_NOTE}

+ + + + {onCancelAddNote != null ? ( - - {i18n.ADD_NOTE} - + - -
- + ) : null} + + + {i18n.ADD_NOTE} + + + +
); }); diff --git a/x-pack/plugins/security_solution/scripts/endpoint/resolver_generator_script.ts b/x-pack/plugins/security_solution/scripts/endpoint/resolver_generator_script.ts index 9ad094086b632..de0cec3c06033 100644 --- a/x-pack/plugins/security_solution/scripts/endpoint/resolver_generator_script.ts +++ b/x-pack/plugins/security_solution/scripts/endpoint/resolver_generator_script.ts @@ -5,9 +5,10 @@ */ /* eslint-disable no-console */ import yargs from 'yargs'; +import fs from 'fs'; import { Client, ClientOptions } from '@elastic/elasticsearch'; import { ResponseError } from '@elastic/elasticsearch/lib/errors'; -import { KbnClient, ToolingLog } from '@kbn/dev-utils'; +import { KbnClient, ToolingLog, CA_CERT_PATH } from '@kbn/dev-utils'; import { AxiosResponse } from 'axios'; import { indexHostsAndAlerts } from '../../common/endpoint/index_data'; import { ANCESTRY_LIMIT, EndpointDocGenerator } from '../../common/endpoint/generate_data'; @@ -202,15 +203,41 @@ async function main() { type: 'boolean', default: false, }, + ssl: { + alias: 'ssl', + describe: 'Use https for elasticsearch and kbn clients', + type: 'boolean', + default: false, + }, }).argv; + let ca: Buffer; + let kbnClient: KbnClientWithApiKeySupport; + let clientOptions: ClientOptions; - const kbnClient = new KbnClientWithApiKeySupport({ - log: new ToolingLog({ - level: 'info', - writeTo: process.stdout, - }), - url: argv.kibana, - }); + if (argv.ssl) { + ca = fs.readFileSync(CA_CERT_PATH); + const url = argv.kibana.replace('http:', 'https:'); + const node = argv.node.replace('http:', 'https:'); + kbnClient = new KbnClientWithApiKeySupport({ + log: new ToolingLog({ + level: 'info', + writeTo: process.stdout, + }), + url, + certificateAuthorities: [ca], + }); + clientOptions = { node, ssl: { ca: [ca] } }; + } else { + kbnClient = new KbnClientWithApiKeySupport({ + log: new ToolingLog({ + level: 'info', + writeTo: process.stdout, + }), + url: argv.kibana, + }); + clientOptions = { node: argv.node }; + } + const client = new Client(clientOptions); try { await doIngestSetup(kbnClient); @@ -219,9 +246,6 @@ async function main() { process.exit(1); } - const clientOptions: ClientOptions = { node: argv.node }; - const client = new Client(clientOptions); - if (argv.delete) { await deleteIndices( [argv.eventIndex, argv.metadataIndex, argv.policyIndex, argv.alertIndex], diff --git a/x-pack/plugins/security_solution/server/lib/detection_engine/signals/search_after_bulk_create.ts b/x-pack/plugins/security_solution/server/lib/detection_engine/signals/search_after_bulk_create.ts index 3030bd8c52c70..2aa8981cc618b 100644 --- a/x-pack/plugins/security_solution/server/lib/detection_engine/signals/search_after_bulk_create.ts +++ b/x-pack/plugins/security_solution/server/lib/detection_engine/signals/search_after_bulk_create.ts @@ -70,6 +70,7 @@ export const searchAfterAndBulkCreate = async ({ interval, buildRuleMessage, }); + const tuplesToBeLogged = [...totalToFromTuples]; logger.debug(buildRuleMessage(`totalToFromTuples: ${totalToFromTuples.length}`)); while (totalToFromTuples.length > 0) { @@ -294,5 +295,6 @@ export const searchAfterAndBulkCreate = async ({ } } logger.debug(buildRuleMessage(`[+] completed bulk index of ${toReturn.createdSignalsCount}`)); + toReturn.totalToFromTuples = tuplesToBeLogged; return toReturn; }; diff --git a/x-pack/plugins/security_solution/server/lib/detection_engine/signals/signal_rule_alert_type.ts b/x-pack/plugins/security_solution/server/lib/detection_engine/signals/signal_rule_alert_type.ts index d08ab66af5683..2b0abdfdfa090 100644 --- a/x-pack/plugins/security_solution/server/lib/detection_engine/signals/signal_rule_alert_type.ts +++ b/x-pack/plugins/security_solution/server/lib/detection_engine/signals/signal_rule_alert_type.ts @@ -670,6 +670,21 @@ export const signalRulesAlertType = ({ lastLookBackDate: result.lastLookBackDate?.toISOString(), }); } + + // adding this log line so we can get some information from cloud + logger.info( + buildRuleMessage( + `[+] Finished indexing ${result.createdSignalsCount} ${ + !isEmpty(result.totalToFromTuples) + ? `signals searched between date ranges ${JSON.stringify( + result.totalToFromTuples, + null, + 2 + )}` + : '' + }` + ) + ); } else { const errorMessage = buildRuleMessage( 'Bulk Indexing of signals failed:', diff --git a/x-pack/plugins/security_solution/server/lib/detection_engine/signals/types.ts b/x-pack/plugins/security_solution/server/lib/detection_engine/signals/types.ts index 5ae411678aa03..cb955673a7ea6 100644 --- a/x-pack/plugins/security_solution/server/lib/detection_engine/signals/types.ts +++ b/x-pack/plugins/security_solution/server/lib/detection_engine/signals/types.ts @@ -5,7 +5,7 @@ */ import { DslQuery, Filter } from 'src/plugins/data/common'; -import moment from 'moment'; +import moment, { Moment } from 'moment'; import { Status } from '../../../../common/detection_engine/schemas/common/schemas'; import { RulesSchema } from '../../../../common/detection_engine/schemas/response/rules_schema'; import { @@ -263,6 +263,11 @@ export interface SearchAfterAndBulkCreateReturnType { createdSignalsCount: number; createdSignals: SignalHit[]; errors: string[]; + totalToFromTuples?: Array<{ + to: Moment | undefined; + from: Moment | undefined; + maxSignals: number; + }>; } export interface ThresholdAggregationBucket extends TermAggregationBucket { diff --git a/x-pack/plugins/security_solution/server/search_strategy/security_solution/index.ts b/x-pack/plugins/security_solution/server/search_strategy/security_solution/index.ts index 8b2cce01cf07a..d25f1aaccc5e7 100644 --- a/x-pack/plugins/security_solution/server/search_strategy/security_solution/index.ts +++ b/x-pack/plugins/security_solution/server/search_strategy/security_solution/index.ts @@ -37,7 +37,7 @@ export const securitySolutionSearchStrategyProvider = { }) ).toEqual([0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]); }); + + test('supports histogram buckets that begin in the past when tasks are overdue', async () => { + expect( + padBuckets(20, 3000, { + key: '2021-02-02T10:08:32.161Z-2021-02-02T10:09:32.161Z', + from: 1612260512161, + from_as_string: '2021-02-02T10:08:32.161Z', + to: 1612260572161, + to_as_string: '2021-02-02T10:09:32.161Z', + doc_count: 2, + histogram: { + buckets: [ + { + key_as_string: '2021-02-02T10:08:30.000Z', + key: 1612260510000, + doc_count: 1, + interval: { + doc_count_error_upper_bound: 0, + sum_other_doc_count: 0, + buckets: [ + { + key: '2s', + doc_count: 1, + }, + ], + }, + }, + { + key_as_string: '2021-02-02T10:08:33.000Z', + key: 1612260513000, + doc_count: 0, + interval: { + doc_count_error_upper_bound: 0, + sum_other_doc_count: 0, + buckets: [], + }, + }, + { + key_as_string: '2021-02-02T10:08:36.000Z', + key: 1612260516000, + doc_count: 0, + interval: { + doc_count_error_upper_bound: 0, + sum_other_doc_count: 0, + buckets: [], + }, + }, + { + key_as_string: '2021-02-02T10:08:39.000Z', + key: 1612260519000, + doc_count: 0, + interval: { + doc_count_error_upper_bound: 0, + sum_other_doc_count: 0, + buckets: [], + }, + }, + { + key_as_string: '2021-02-02T10:08:42.000Z', + key: 1612260522000, + doc_count: 0, + interval: { + doc_count_error_upper_bound: 0, + sum_other_doc_count: 0, + buckets: [], + }, + }, + { + key_as_string: '2021-02-02T10:08:45.000Z', + key: 1612260525000, + doc_count: 0, + interval: { + doc_count_error_upper_bound: 0, + sum_other_doc_count: 0, + buckets: [], + }, + }, + { + key_as_string: '2021-02-02T10:08:48.000Z', + key: 1612260528000, + doc_count: 0, + interval: { + doc_count_error_upper_bound: 0, + sum_other_doc_count: 0, + buckets: [], + }, + }, + { + key_as_string: '2021-02-02T10:08:51.000Z', + key: 1612260531000, + doc_count: 0, + interval: { + doc_count_error_upper_bound: 0, + sum_other_doc_count: 0, + buckets: [], + }, + }, + { + key_as_string: '2021-02-02T10:08:54.000Z', + key: 1612260534000, + doc_count: 1, + interval: { + doc_count_error_upper_bound: 0, + sum_other_doc_count: 0, + buckets: [ + { + key: '60s', + doc_count: 1, + }, + ], + }, + }, + ], + }, + }).length + // we need to ensure overdue buckets don't cause us to over pad the timeline by adding additional + // buckets before and after the reported timeline + ).toEqual(20); + }); }); function setTaskTypeCount( diff --git a/x-pack/plugins/task_manager/server/monitoring/workload_statistics.ts b/x-pack/plugins/task_manager/server/monitoring/workload_statistics.ts index 8002ee44d01ff..8bd22bd88cf41 100644 --- a/x-pack/plugins/task_manager/server/monitoring/workload_statistics.ts +++ b/x-pack/plugins/task_manager/server/monitoring/workload_statistics.ts @@ -244,10 +244,19 @@ export function padBuckets( const firstBucket = histogram.buckets[0].key; const lastBucket = histogram.buckets[histogram.buckets.length - 1].key; - const bucketsToPadBeforeFirstBucket = calculateBucketsBetween(firstBucket, from, pollInterval); + // detect when the first bucket is before the `from` so that we can take that into + // account by begining the timeline earlier + // This can happen when you have overdue tasks and Elasticsearch returns their bucket + // as begining before the `from` + const firstBucketStartsInThePast = firstBucket - from < 0; + + const bucketsToPadBeforeFirstBucket = firstBucketStartsInThePast + ? [] + : calculateBucketsBetween(firstBucket, from, pollInterval); + const bucketsToPadAfterLast = calculateBucketsBetween( lastBucket + pollInterval, - to, + firstBucketStartsInThePast ? to - pollInterval : to, pollInterval ); diff --git a/x-pack/plugins/translations/translations/ja-JP.json b/x-pack/plugins/translations/translations/ja-JP.json index d0634d6cd87a2..fdf1c74f20512 100644 --- a/x-pack/plugins/translations/translations/ja-JP.json +++ b/x-pack/plugins/translations/translations/ja-JP.json @@ -4784,7 +4784,6 @@ "xpack.actions.builtin.pagerdutyTitle": "PagerDuty", "xpack.actions.builtin.serverLog.errorLoggingErrorMessage": "メッセージのロギングエラー", "xpack.actions.builtin.serverLogTitle": "サーバーログ", - "xpack.actions.builtin.servicenowTitle": "ServiceNow", "xpack.actions.builtin.slack.errorPostingErrorMessage": "slack メッセージの投稿エラー", "xpack.actions.builtin.slack.errorPostingRetryDateErrorMessage": "slack メッセージの投稿エラー、 {retryString} で再試行", "xpack.actions.builtin.slack.errorPostingRetryLaterErrorMessage": "slack メッセージの投稿エラー、後ほど再試行", @@ -11188,7 +11187,6 @@ "xpack.lens.discover.visualizeFieldLegend": "Visualize フィールド", "xpack.lens.dragDrop.elementLifted": "位置 {position} のアイテム {itemLabel} を持ち上げました。", "xpack.lens.dragDrop.elementMoved": "位置 {prevPosition} から位置 {position} までアイテム {itemLabel} を移動しました", - "xpack.lens.dragDrop.reorderInstructions": "スペースバーを押すと、ドラッグを開始します。ドラッグするときには、矢印キーで並べ替えることができます。もう一度スペースバーを押すと終了します。", "xpack.lens.editLayerSettings": "レイヤー設定を編集", "xpack.lens.editLayerSettingsChartType": "レイヤー設定を編集、{chartType}", "xpack.lens.editorFrame.buildExpressionError": "グラフの準備中に予期しないエラーが発生しました", @@ -14210,8 +14208,8 @@ "xpack.ml.splom.dynamicSizeLabel": "動的サイズ", "xpack.ml.splom.fieldSelectionLabel": "フィールド", "xpack.ml.splom.fieldSelectionPlaceholder": "フィールドを選択", - "xpack.ml.splom.RandomScoringLabel": "ランダムスコアリング", - "xpack.ml.splom.SampleSizeLabel": "サンプルサイズ", + "xpack.ml.splom.randomScoringLabel": "ランダムスコアリング", + "xpack.ml.splom.sampleSizeLabel": "サンプルサイズ", "xpack.ml.splom.toggleOff": "オフ", "xpack.ml.splom.toggleOn": "オン", "xpack.ml.splomSpec.outlierScoreThresholdName": "異常スコアしきい値：", @@ -21162,7 +21160,6 @@ "xpack.triggersActionsUI.common.expressionItems.threshold.popoverTitle": "タイミング", "xpack.triggersActionsUI.components.addMessageVariables.addVariablePopoverButton": "変数を追加", "xpack.triggersActionsUI.components.addMessageVariables.addVariableTitle": "アラート変数を追加", - "xpack.triggersActionsUI.components.builtinActionTypes.common.requiredDescriptionTextField": "説明が必要です。", "xpack.triggersActionsUI.components.builtinActionTypes.common.requiredShortDescTextField": "短い説明が必要です。", "xpack.triggersActionsUI.components.builtinActionTypes.emailAction.actionTypeTitle": "メールに送信", "xpack.triggersActionsUI.components.builtinActionTypes.emailAction.configureAccountsHelpLabel": "電子メールアカウントの構成", @@ -21293,35 +21290,16 @@ "xpack.triggersActionsUI.components.builtinActionTypes.serverLogAction.logLevelFieldLabel": "レベル", "xpack.triggersActionsUI.components.builtinActionTypes.serverLogAction.logMessageFieldLabel": "メッセージ", "xpack.triggersActionsUI.components.builtinActionTypes.serverLogAction.selectMessageText": "Kibana ログにメッセージを追加します。", - "xpack.triggersActionsUI.components.builtinActionTypes.servicenow.actionTypeTitle": "ServiceNow", - "xpack.triggersActionsUI.components.builtinActionTypes.servicenow.apiTokenTextFieldLabel": "APIトークン", "xpack.triggersActionsUI.components.builtinActionTypes.servicenow.apiUrlTextFieldLabel": "URL", "xpack.triggersActionsUI.components.builtinActionTypes.servicenow.authenticationLabel": "認証", - "xpack.triggersActionsUI.components.builtinActionTypes.serviceNow.commentsTextAreaFieldLabel": "追加のコメント", - "xpack.triggersActionsUI.components.builtinActionTypes.serviceNow.descriptionTextAreaFieldLabel": "説明", - "xpack.triggersActionsUI.components.builtinActionTypes.servicenow.emailTextFieldLabel": "メール", - "xpack.triggersActionsUI.components.builtinActionTypes.serviceNow.impactSelectFieldLabel": "インパクト", "xpack.triggersActionsUI.components.builtinActionTypes.servicenow.invalidApiUrlTextField": "URL が無効です。", - "xpack.triggersActionsUI.components.builtinActionTypes.servicenow.mappingFieldComments": "コメント", - "xpack.triggersActionsUI.components.builtinActionTypes.servicenow.mappingFieldDescription": "説明", - "xpack.triggersActionsUI.components.builtinActionTypes.servicenow.mappingFieldShortDescription": "短い説明", "xpack.triggersActionsUI.components.builtinActionTypes.servicenow.passwordTextFieldLabel": "パスワード", "xpack.triggersActionsUI.components.builtinActionTypes.servicenow.reenterValuesLabel": "ユーザー名とパスワードは暗号化されます。これらのフィールドの値を再入力してください。", "xpack.triggersActionsUI.components.builtinActionTypes.servicenow.rememberValuesLabel": "これらの値を覚えておいてください。コネクターを編集するたびに再入力する必要があります。", - "xpack.triggersActionsUI.components.builtinActionTypes.servicenow.requiredApiTokenTextField": "API トークンが必要です。", "xpack.triggersActionsUI.components.builtinActionTypes.servicenow.requiredApiUrlTextField": "URL が必要です。", - "xpack.triggersActionsUI.components.builtinActionTypes.servicenow.requiredEmailTextField": "電子メールが必要です。", "xpack.triggersActionsUI.components.builtinActionTypes.servicenow.requiredPasswordTextField": "パスワードが必要です。", "xpack.triggersActionsUI.components.builtinActionTypes.servicenow.requiredUsernameTextField": "ユーザー名が必要です。", "xpack.triggersActionsUI.components.builtinActionTypes.servicenow.requireHttpsApiUrlTextField": "URL は https:// から始める必要があります。", - "xpack.triggersActionsUI.components.builtinActionTypes.servicenow.selectMessageText": "ServiceNow でインシデントを作成します。", - "xpack.triggersActionsUI.components.builtinActionTypes.serviceNow.severitySelectFieldLabel": "深刻度", - "xpack.triggersActionsUI.components.builtinActionTypes.servicenow.severitySelectHighOptionLabel": "高", - "xpack.triggersActionsUI.components.builtinActionTypes.servicenow.severitySelectLawOptionLabel": "低", - "xpack.triggersActionsUI.components.builtinActionTypes.servicenow.severitySelectMediumOptionLabel": "中", - "xpack.triggersActionsUI.components.builtinActionTypes.serviceNow.title": "インシデント", - "xpack.triggersActionsUI.components.builtinActionTypes.serviceNow.titleFieldLabel": "短い説明（必須）", - "xpack.triggersActionsUI.components.builtinActionTypes.serviceNow.urgencySelectFieldLabel": "緊急", "xpack.triggersActionsUI.components.builtinActionTypes.servicenow.usernameTextFieldLabel": "ユーザー名", "xpack.triggersActionsUI.components.builtinActionTypes.serviceNowAction.apiUrlHelpLabel": "Personal Developer Instance の構成", "xpack.triggersActionsUI.components.builtinActionTypes.slackAction.actionTypeTitle": "Slack に送信", diff --git a/x-pack/plugins/translations/translations/zh-CN.json b/x-pack/plugins/translations/translations/zh-CN.json index 4ca6d11aa8940..b3fefd55ce55a 100644 --- a/x-pack/plugins/translations/translations/zh-CN.json +++ b/x-pack/plugins/translations/translations/zh-CN.json @@ -4789,7 +4789,6 @@ "xpack.actions.builtin.pagerdutyTitle": "PagerDuty", "xpack.actions.builtin.serverLog.errorLoggingErrorMessage": "记录消息时出错", "xpack.actions.builtin.serverLogTitle": "服务器日志", - "xpack.actions.builtin.servicenowTitle": "ServiceNow", "xpack.actions.builtin.slack.errorPostingErrorMessage": "发布 slack 消息时出错", "xpack.actions.builtin.slack.errorPostingRetryDateErrorMessage": "发布 Slack 消息时出错，在 {retryString} 重试", "xpack.actions.builtin.slack.errorPostingRetryLaterErrorMessage": "发布 slack 消息时出错，稍后重试", @@ -11217,7 +11216,6 @@ "xpack.lens.discover.visualizeFieldLegend": "可视化字段", "xpack.lens.dragDrop.elementLifted": "您已将项目 {itemLabel} 提升到位置 {position}", "xpack.lens.dragDrop.elementMoved": "您已将项目 {itemLabel} 从位置 {prevPosition} 移到位置 {position}", - "xpack.lens.dragDrop.reorderInstructions": "按空格键开始拖动。拖动时，使用方向键重新排序。再次按空格键结束操作。", "xpack.lens.editLayerSettings": "编辑图层设置", "xpack.lens.editLayerSettingsChartType": "编辑图层设置 {chartType}", "xpack.lens.editorFrame.buildExpressionError": "准备图表时发生意外错误", @@ -14249,8 +14247,8 @@ "xpack.ml.splom.dynamicSizeLabel": "动态大小", "xpack.ml.splom.fieldSelectionLabel": "字段", "xpack.ml.splom.fieldSelectionPlaceholder": "选择字段", - "xpack.ml.splom.RandomScoringLabel": "随机评分", - "xpack.ml.splom.SampleSizeLabel": "样例大小", + "xpack.ml.splom.randomScoringLabel": "随机评分", + "xpack.ml.splom.sampleSizeLabel": "样例大小", "xpack.ml.splom.toggleOff": "关闭", "xpack.ml.splom.toggleOn": "开启", "xpack.ml.splomSpec.outlierScoreThresholdName": "离群值分数阈值：", @@ -21213,7 +21211,6 @@ "xpack.triggersActionsUI.common.expressionItems.threshold.popoverTitle": "当", "xpack.triggersActionsUI.components.addMessageVariables.addVariablePopoverButton": "添加变量", "xpack.triggersActionsUI.components.addMessageVariables.addVariableTitle": "添加告警变量", - "xpack.triggersActionsUI.components.builtinActionTypes.common.requiredDescriptionTextField": "“描述”必填。", "xpack.triggersActionsUI.components.builtinActionTypes.common.requiredShortDescTextField": "“简短描述”必填。", "xpack.triggersActionsUI.components.builtinActionTypes.emailAction.actionTypeTitle": "发送到电子邮件", "xpack.triggersActionsUI.components.builtinActionTypes.emailAction.configureAccountsHelpLabel": "配置电子邮件帐户", @@ -21344,35 +21341,16 @@ "xpack.triggersActionsUI.components.builtinActionTypes.serverLogAction.logLevelFieldLabel": "级别", "xpack.triggersActionsUI.components.builtinActionTypes.serverLogAction.logMessageFieldLabel": "消息", "xpack.triggersActionsUI.components.builtinActionTypes.serverLogAction.selectMessageText": "将消息添加到 Kibana 日志。", - "xpack.triggersActionsUI.components.builtinActionTypes.servicenow.actionTypeTitle": "ServiceNow", - "xpack.triggersActionsUI.components.builtinActionTypes.servicenow.apiTokenTextFieldLabel": "Api 令牌", "xpack.triggersActionsUI.components.builtinActionTypes.servicenow.apiUrlTextFieldLabel": "URL", "xpack.triggersActionsUI.components.builtinActionTypes.servicenow.authenticationLabel": "身份验证", - "xpack.triggersActionsUI.components.builtinActionTypes.serviceNow.commentsTextAreaFieldLabel": "其他注释", - "xpack.triggersActionsUI.components.builtinActionTypes.serviceNow.descriptionTextAreaFieldLabel": "描述", - "xpack.triggersActionsUI.components.builtinActionTypes.servicenow.emailTextFieldLabel": "电子邮件", - "xpack.triggersActionsUI.components.builtinActionTypes.serviceNow.impactSelectFieldLabel": "影响", "xpack.triggersActionsUI.components.builtinActionTypes.servicenow.invalidApiUrlTextField": "URL 无效。", - "xpack.triggersActionsUI.components.builtinActionTypes.servicenow.mappingFieldComments": "注释", - "xpack.triggersActionsUI.components.builtinActionTypes.servicenow.mappingFieldDescription": "描述", - "xpack.triggersActionsUI.components.builtinActionTypes.servicenow.mappingFieldShortDescription": "简短描述", "xpack.triggersActionsUI.components.builtinActionTypes.servicenow.passwordTextFieldLabel": "密码", "xpack.triggersActionsUI.components.builtinActionTypes.servicenow.reenterValuesLabel": "用户名和密码已加密。请为这些字段重新输入值。", "xpack.triggersActionsUI.components.builtinActionTypes.servicenow.rememberValuesLabel": "请记住这些值。每次编辑连接器时都必须重新输入。", - "xpack.triggersActionsUI.components.builtinActionTypes.servicenow.requiredApiTokenTextField": "“Api 令牌”必填。", "xpack.triggersActionsUI.components.builtinActionTypes.servicenow.requiredApiUrlTextField": "“URL”必填。", - "xpack.triggersActionsUI.components.builtinActionTypes.servicenow.requiredEmailTextField": "“电子邮件”必填。", "xpack.triggersActionsUI.components.builtinActionTypes.servicenow.requiredPasswordTextField": "“密码”必填。", "xpack.triggersActionsUI.components.builtinActionTypes.servicenow.requiredUsernameTextField": "“用户名”必填。", "xpack.triggersActionsUI.components.builtinActionTypes.servicenow.requireHttpsApiUrlTextField": "URL 必须以 https:// 开头。", - "xpack.triggersActionsUI.components.builtinActionTypes.servicenow.selectMessageText": "在 ServiceNow 中创建事件。", - "xpack.triggersActionsUI.components.builtinActionTypes.serviceNow.severitySelectFieldLabel": "严重性", - "xpack.triggersActionsUI.components.builtinActionTypes.servicenow.severitySelectHighOptionLabel": "高", - "xpack.triggersActionsUI.components.builtinActionTypes.servicenow.severitySelectLawOptionLabel": "低", - "xpack.triggersActionsUI.components.builtinActionTypes.servicenow.severitySelectMediumOptionLabel": "中", - "xpack.triggersActionsUI.components.builtinActionTypes.serviceNow.title": "事件", - "xpack.triggersActionsUI.components.builtinActionTypes.serviceNow.titleFieldLabel": "简短描述（必填）", - "xpack.triggersActionsUI.components.builtinActionTypes.serviceNow.urgencySelectFieldLabel": "紧急性", "xpack.triggersActionsUI.components.builtinActionTypes.servicenow.usernameTextFieldLabel": "用户名", "xpack.triggersActionsUI.components.builtinActionTypes.serviceNowAction.apiUrlHelpLabel": "配置个人开发者实例", "xpack.triggersActionsUI.components.builtinActionTypes.slackAction.actionTypeTitle": "发送到 Slack", diff --git a/x-pack/plugins/triggers_actions_ui/public/application/components/builtin_action_types/index.ts b/x-pack/plugins/triggers_actions_ui/public/application/components/builtin_action_types/index.ts index b8514a06dc253..003b2c5eedb10 100644 --- a/x-pack/plugins/triggers_actions_ui/public/application/components/builtin_action_types/index.ts +++ b/x-pack/plugins/triggers_actions_ui/public/application/components/builtin_action_types/index.ts @@ -12,7 +12,7 @@ import { getPagerDutyActionType } from './pagerduty'; import { getWebhookActionType } from './webhook'; import { TypeRegistry } from '../../type_registry'; import { ActionTypeModel } from '../../../types'; -import { getServiceNowActionType } from './servicenow'; +import { getServiceNowITSMActionType, getServiceNowSIRActionType } from './servicenow'; import { getJiraActionType } from './jira'; import { getResilientActionType } from './resilient'; import { getTeamsActionType } from './teams'; @@ -28,7 +28,8 @@ export function registerBuiltInActionTypes({ actionTypeRegistry.register(getIndexActionType()); actionTypeRegistry.register(getPagerDutyActionType()); actionTypeRegistry.register(getWebhookActionType()); - actionTypeRegistry.register(getServiceNowActionType()); + actionTypeRegistry.register(getServiceNowITSMActionType()); + actionTypeRegistry.register(getServiceNowSIRActionType()); actionTypeRegistry.register(getJiraActionType()); actionTypeRegistry.register(getResilientActionType()); actionTypeRegistry.register(getTeamsActionType()); diff --git a/x-pack/plugins/triggers_actions_ui/public/application/components/builtin_action_types/jira/api.test.ts b/x-pack/plugins/triggers_actions_ui/public/application/components/builtin_action_types/jira/api.test.ts index d5474aaceaa48..4759eecf3ef0e 100644 --- a/x-pack/plugins/triggers_actions_ui/public/application/components/builtin_action_types/jira/api.test.ts +++ b/x-pack/plugins/triggers_actions_ui/public/application/components/builtin_action_types/jira/api.test.ts @@ -8,6 +8,7 @@ import { httpServiceMock } from '../../../../../../../../src/core/public/mocks'; import { getIssueTypes, getFieldsByIssueType, getIssues, getIssue } from './api'; const issueTypesResponse = { + status: 'ok', data: { projects: [ { @@ -24,9 +25,11 @@ const issueTypesResponse = { }, ], }, + actionId: 'test', }; const fieldsResponse = { + status: 'ok', data: { projects: [ { @@ -70,13 +73,18 @@ const fieldsResponse = { ], }, ], + actionId: 'test', }, }; const issueResponse = { - id: '10267', - key: 'RJ-107', - fields: { summary: 'Test title' }, + status: 'ok', + data: { + id: '10267', + key: 'RJ-107', + fields: { summary: 'Test title' }, + }, + actionId: 'test', }; const issuesResponse = [issueResponse]; diff --git a/x-pack/plugins/triggers_actions_ui/public/application/components/builtin_action_types/jira/config.ts b/x-pack/plugins/triggers_actions_ui/public/application/components/builtin_action_types/jira/config.ts index 628600ee91c8e..d05bf78a5106e 100644 --- a/x-pack/plugins/triggers_actions_ui/public/application/components/builtin_action_types/jira/config.ts +++ b/x-pack/plugins/triggers_actions_ui/public/application/components/builtin_action_types/jira/config.ts @@ -15,24 +15,4 @@ export const connectorConfiguration = { enabledInConfig: true, enabledInLicense: true, minimumLicenseRequired: 'gold', - fields: { - summary: { - label: i18n.MAPPING_FIELD_SUMMARY, - validSourceFields: ['title', 'description'], - defaultSourceField: 'title', - defaultActionType: 'overwrite', - }, - description: { - label: i18n.MAPPING_FIELD_DESC, - validSourceFields: ['title', 'description'], - defaultSourceField: 'description', - defaultActionType: 'overwrite', - }, - comments: { - label: i18n.MAPPING_FIELD_COMMENTS, - validSourceFields: ['comments'], - defaultSourceField: 'comments', - defaultActionType: 'append', - }, - }, }; diff --git a/x-pack/plugins/triggers_actions_ui/public/application/components/builtin_action_types/resilient/api.test.ts b/x-pack/plugins/triggers_actions_ui/public/application/components/builtin_action_types/resilient/api.test.ts new file mode 100644 index 0000000000000..24c7f7687da69 --- /dev/null +++ b/x-pack/plugins/triggers_actions_ui/public/application/components/builtin_action_types/resilient/api.test.ts @@ -0,0 +1,87 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License; + * you may not use this file except in compliance with the Elastic License. + */ + +import { httpServiceMock } from '../../../../../../../../src/core/public/mocks'; +import { getIncidentTypes, getSeverity } from './api'; + +const incidentTypesResponse = { + status: 'ok', + data: [ + { id: 17, name: 'Communication error (fax; email)' }, + { id: 1001, name: 'Custom type' }, + { id: 21, name: 'Denial of Service' }, + { id: 6, name: 'Improper disposal: digital asset(s)' }, + { id: 7, name: 'Improper disposal: documents / files' }, + { id: 4, name: 'Lost documents / files / records' }, + { id: 3, name: 'Lost PC / laptop / tablet' }, + { id: 1, name: 'Lost PDA / smartphone' }, + { id: 8, name: 'Lost storage device / media' }, + { id: 19, name: 'Malware' }, + { id: 23, name: 'Not an Issue' }, + { id: 18, name: 'Other' }, + { id: 22, name: 'Phishing' }, + { id: 11, name: 'Stolen documents / files / records' }, + { id: 12, name: 'Stolen PC / laptop / tablet' }, + { id: 13, name: 'Stolen PDA / smartphone' }, + { id: 14, name: 'Stolen storage device / media' }, + { id: 20, name: 'System Intrusion' }, + { id: 16, name: 'TBD / Unknown' }, + { id: 15, name: 'Vendor / 3rd party error' }, + ], + actionId: 'test', +}; + +const severityResponse = { + status: 'ok', + data: [ + { id: 4, name: 'Low' }, + { id: 5, name: 'Medium' }, + { id: 6, name: 'High' }, + ], + actionId: 'test', +}; + +describe('Resilient API', () => { + const http = httpServiceMock.createStartContract(); + + beforeEach(() => jest.resetAllMocks()); + + describe('getIncidentTypes', () => { + test('should call get choices API', async () => { + const abortCtrl = new AbortController(); + http.post.mockResolvedValueOnce(incidentTypesResponse); + const res = await getIncidentTypes({ + http, + signal: abortCtrl.signal, + connectorId: 'test', + }); + + expect(res).toEqual(incidentTypesResponse); + expect(http.post).toHaveBeenCalledWith('/api/actions/action/test/_execute', { + body: '{"params":{"subAction":"incidentTypes","subActionParams":{}}}', + signal: abortCtrl.signal, + }); + }); + }); + + describe('getSeverity', () => { + test('should call get choices API', async () => { + const abortCtrl = new AbortController(); + http.post.mockResolvedValueOnce(severityResponse); + const res = await getSeverity({ + http, + signal: abortCtrl.signal, + connectorId: 'test', + }); + + expect(res).toEqual(severityResponse); + expect(http.post).toHaveBeenCalledWith('/api/actions/action/test/_execute', { + body: '{"params":{"subAction":"severity","subActionParams":{}}}', + signal: abortCtrl.signal, + }); + }); + }); +}); diff --git a/x-pack/plugins/triggers_actions_ui/public/application/components/builtin_action_types/resilient/config.ts b/x-pack/plugins/triggers_actions_ui/public/application/components/builtin_action_types/resilient/config.ts index a2054585c19b8..9717d594b20ec 100644 --- a/x-pack/plugins/triggers_actions_ui/public/application/components/builtin_action_types/resilient/config.ts +++ b/x-pack/plugins/triggers_actions_ui/public/application/components/builtin_action_types/resilient/config.ts @@ -15,24 +15,4 @@ export const connectorConfiguration = { enabledInConfig: true, enabledInLicense: true, minimumLicenseRequired: 'platinum', - fields: { - name: { - label: i18n.MAPPING_FIELD_NAME, - validSourceFields: ['title', 'description'], - defaultSourceField: 'title', - defaultActionType: 'overwrite', - }, - description: { - label: i18n.MAPPING_FIELD_DESC, - validSourceFields: ['title', 'description'], - defaultSourceField: 'description', - defaultActionType: 'overwrite', - }, - comments: { - label: i18n.MAPPING_FIELD_COMMENTS, - validSourceFields: ['comments'], - defaultSourceField: 'comments', - defaultActionType: 'append', - }, - }, }; diff --git a/x-pack/plugins/triggers_actions_ui/public/application/components/builtin_action_types/servicenow/api.test.ts b/x-pack/plugins/triggers_actions_ui/public/application/components/builtin_action_types/servicenow/api.test.ts new file mode 100644 index 0000000000000..e87b84439f6f8 --- /dev/null +++ b/x-pack/plugins/triggers_actions_ui/public/application/components/builtin_action_types/servicenow/api.test.ts @@ -0,0 +1,69 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License; + * you may not use this file except in compliance with the Elastic License. + */ + +import { httpServiceMock } from '../../../../../../../../src/core/public/mocks'; +import { getChoices } from './api'; + +const choicesResponse = { + status: 'ok', + data: [ + { + dependent_value: '', + label: '1 - Critical', + value: '1', + element: 'priority', + }, + { + dependent_value: '', + label: '2 - High', + value: '2', + element: 'priority', + }, + { + dependent_value: '', + label: '3 - Moderate', + value: '3', + element: 'priority', + }, + { + dependent_value: '', + label: '4 - Low', + value: '4', + element: 'priority', + }, + { + dependent_value: '', + label: '5 - Planning', + value: '5', + element: 'priority', + }, + ], +}; + +describe('ServiceNow API', () => { + const http = httpServiceMock.createStartContract(); + + beforeEach(() => jest.resetAllMocks()); + + describe('getChoices', () => { + test('should call get choices API', async () => { + const abortCtrl = new AbortController(); + http.post.mockResolvedValueOnce(choicesResponse); + const res = await getChoices({ + http, + signal: abortCtrl.signal, + connectorId: 'test', + fields: ['priority'], + }); + + expect(res).toEqual(choicesResponse); + expect(http.post).toHaveBeenCalledWith('/api/actions/action/test/_execute', { + body: '{"params":{"subAction":"getChoices","subActionParams":{"fields":["priority"]}}}', + signal: abortCtrl.signal, + }); + }); + }); +}); diff --git a/x-pack/plugins/triggers_actions_ui/public/application/components/builtin_action_types/servicenow/api.ts b/x-pack/plugins/triggers_actions_ui/public/application/components/builtin_action_types/servicenow/api.ts new file mode 100644 index 0000000000000..ecfc66f1b0391 --- /dev/null +++ b/x-pack/plugins/triggers_actions_ui/public/application/components/builtin_action_types/servicenow/api.ts @@ -0,0 +1,27 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License; + * you may not use this file except in compliance with the Elastic License. + */ + +import { HttpSetup } from 'kibana/public'; +import { BASE_ACTION_API_PATH } from '../../../constants'; + +export async function getChoices({ + http, + signal, + connectorId, + fields, +}: { + http: HttpSetup; + signal: AbortSignal; + connectorId: string; + fields: string[]; +}): Promise> { + return await http.post(`${BASE_ACTION_API_PATH}/action/${connectorId}/_execute`, { + body: JSON.stringify({ + params: { subAction: 'getChoices', subActionParams: { fields } }, + }), + signal, + }); +} diff --git a/x-pack/plugins/triggers_actions_ui/public/application/components/builtin_action_types/servicenow/config.ts b/x-pack/plugins/triggers_actions_ui/public/application/components/builtin_action_types/servicenow/config.ts index 7f810cf5eb38f..6920ee71144a5 100644 --- a/x-pack/plugins/triggers_actions_ui/public/application/components/builtin_action_types/servicenow/config.ts +++ b/x-pack/plugins/triggers_actions_ui/public/application/components/builtin_action_types/servicenow/config.ts @@ -7,32 +7,24 @@ import * as i18n from './translations'; import logo from './logo.svg'; -export const connectorConfiguration = { +export const serviceNowITSMConfiguration = { id: '.servicenow', - name: i18n.SERVICENOW_TITLE, + name: i18n.SERVICENOW_ITSM_TITLE, + desc: i18n.SERVICENOW_ITSM_DESC, + logo, + enabled: true, + enabledInConfig: true, + enabledInLicense: true, + minimumLicenseRequired: 'platinum', +}; + +export const serviceNowSIRConfiguration = { + id: '.servicenow-sir', + name: i18n.SERVICENOW_SIR_TITLE, + desc: i18n.SERVICENOW_SIR_DESC, logo, enabled: true, enabledInConfig: true, enabledInLicense: true, minimumLicenseRequired: 'platinum', - fields: { - short_description: { - label: i18n.MAPPING_FIELD_SHORT_DESC, - validSourceFields: ['title', 'description'], - defaultSourceField: 'title', - defaultActionType: 'overwrite', - }, - description: { - label: i18n.MAPPING_FIELD_DESC, - validSourceFields: ['title', 'description'], - defaultSourceField: 'description', - defaultActionType: 'overwrite', - }, - comments: { - label: i18n.MAPPING_FIELD_COMMENTS, - validSourceFields: ['comments'], - defaultSourceField: 'comments', - defaultActionType: 'append', - }, - }, }; diff --git a/x-pack/plugins/triggers_actions_ui/public/application/components/builtin_action_types/servicenow/index.ts b/x-pack/plugins/triggers_actions_ui/public/application/components/builtin_action_types/servicenow/index.ts index 65bb3ae4f5a37..e1f66e506ed8e 100644 --- a/x-pack/plugins/triggers_actions_ui/public/application/components/builtin_action_types/servicenow/index.ts +++ b/x-pack/plugins/triggers_actions_ui/public/application/components/builtin_action_types/servicenow/index.ts @@ -4,4 +4,4 @@ * you may not use this file except in compliance with the Elastic License. */ -export { getActionType as getServiceNowActionType } from './servicenow'; +export { getServiceNowITSMActionType, getServiceNowSIRActionType } from './servicenow'; diff --git a/x-pack/plugins/triggers_actions_ui/public/application/components/builtin_action_types/servicenow/servicenow.test.tsx b/x-pack/plugins/triggers_actions_ui/public/application/components/builtin_action_types/servicenow/servicenow.test.tsx index dfa9bf56cc7a9..ce69f428e10a1 100644 --- a/x-pack/plugins/triggers_actions_ui/public/application/components/builtin_action_types/servicenow/servicenow.test.tsx +++ b/x-pack/plugins/triggers_actions_ui/public/application/components/builtin_action_types/servicenow/servicenow.test.tsx @@ -8,102 +8,110 @@ import { registerBuiltInActionTypes } from '.././index'; import { ActionTypeModel } from '../../../../types'; import { ServiceNowActionConnector } from './types'; -const ACTION_TYPE_ID = '.servicenow'; -let actionTypeModel: ActionTypeModel; +const SERVICENOW_ITSM_ACTION_TYPE_ID = '.servicenow'; +const SERVICENOW_SIR_ACTION_TYPE_ID = '.servicenow-sir'; +let actionTypeRegistry: TypeRegistry; beforeAll(() => { - const actionTypeRegistry = new TypeRegistry(); + actionTypeRegistry = new TypeRegistry(); registerBuiltInActionTypes({ actionTypeRegistry }); - const getResult = actionTypeRegistry.get(ACTION_TYPE_ID); - if (getResult !== null) { - actionTypeModel = getResult; - } }); describe('actionTypeRegistry.get() works', () => { - test('action type static data is as expected', () => { - expect(actionTypeModel.id).toEqual(ACTION_TYPE_ID); + [SERVICENOW_ITSM_ACTION_TYPE_ID, SERVICENOW_SIR_ACTION_TYPE_ID].forEach((id) => { + test(`${id}: action type static data is as expected`, () => { + const actionTypeModel = actionTypeRegistry.get(id); + expect(actionTypeModel.id).toEqual(id); + }); }); }); describe('servicenow connector validation', () => { - test('connector validation succeeds when connector config is valid', () => { - const actionConnector = { - secrets: { - username: 'user', - password: 'pass', - }, - id: 'test', - actionTypeId: '.servicenow', - name: 'ServiceNow', - isPreconfigured: false, - config: { - apiUrl: 'https://dev94428.service-now.com/', - }, - } as ServiceNowActionConnector; + [SERVICENOW_ITSM_ACTION_TYPE_ID, SERVICENOW_SIR_ACTION_TYPE_ID].forEach((id) => { + test(`${id}: connector validation succeeds when connector config is valid`, () => { + const actionTypeModel = actionTypeRegistry.get(id); + const actionConnector = { + secrets: { + username: 'user', + password: 'pass', + }, + id: 'test', + actionTypeId: id, + name: 'ServiceNow', + isPreconfigured: false, + config: { + apiUrl: 'https://dev94428.service-now.com/', + }, + } as ServiceNowActionConnector; - expect(actionTypeModel.validateConnector(actionConnector)).toEqual({ - config: { - errors: { - apiUrl: [], + expect(actionTypeModel.validateConnector(actionConnector)).toEqual({ + config: { + errors: { + apiUrl: [], + }, }, - }, - secrets: { - errors: { - username: [], - password: [], + secrets: { + errors: { + username: [], + password: [], + }, }, - }, + }); }); - }); - test('connector validation fails when connector config is not valid', () => { - const actionConnector = ({ - secrets: { - username: 'user', - }, - id: '.servicenow', - actionTypeId: '.servicenow', - name: 'servicenow', - config: {}, - } as unknown) as ServiceNowActionConnector; + test(`${id}: connector validation fails when connector config is not valid`, () => { + const actionTypeModel = actionTypeRegistry.get(id); + const actionConnector = ({ + secrets: { + username: 'user', + }, + id, + actionTypeId: id, + name: 'servicenow', + config: {}, + } as unknown) as ServiceNowActionConnector; - expect(actionTypeModel.validateConnector(actionConnector)).toEqual({ - config: { - errors: { - apiUrl: ['URL is required.'], + expect(actionTypeModel.validateConnector(actionConnector)).toEqual({ + config: { + errors: { + apiUrl: ['URL is required.'], + }, }, - }, - secrets: { - errors: { - username: [], - password: ['Password is required.'], + secrets: { + errors: { + username: [], + password: ['Password is required.'], + }, }, - }, + }); }); }); }); describe('servicenow action params validation', () => { - test('action params validation succeeds when action params is valid', () => { - const actionParams = { - subActionParams: { incident: { short_description: 'some title {{test}}' }, comments: [] }, - }; + [SERVICENOW_ITSM_ACTION_TYPE_ID, SERVICENOW_SIR_ACTION_TYPE_ID].forEach((id) => { + test(`${id}: action params validation succeeds when action params is valid`, () => { + const actionTypeModel = actionTypeRegistry.get(id); + const actionParams = { + subActionParams: { incident: { short_description: 'some title {{test}}' }, comments: [] }, + }; - expect(actionTypeModel.validateParams(actionParams)).toEqual({ - errors: { ['subActionParams.incident.short_description']: [] }, + expect(actionTypeModel.validateParams(actionParams)).toEqual({ + errors: { ['subActionParams.incident.short_description']: [] }, + }); }); - }); - test('params validation fails when body is not valid', () => { - const actionParams = { - subActionParams: { incident: { short_description: '' }, comments: [] }, - }; + test(`${id}: params validation fails when body is not valid`, () => { + const actionTypeModel = actionTypeRegistry.get(id); + const actionParams = { + subActionParams: { incident: { short_description: '' }, comments: [] }, + }; - expect(actionTypeModel.validateParams(actionParams)).toEqual({ - errors: { - ['subActionParams.incident.short_description']: ['Short description is required.'], - }, + expect(actionTypeModel.validateParams(actionParams)).toEqual({ + errors: { + ['subActionParams.incident.short_description']: ['Short description is required.'], + }, + }); }); }); }); diff --git a/x-pack/plugins/triggers_actions_ui/public/application/components/builtin_action_types/servicenow/servicenow.tsx b/x-pack/plugins/triggers_actions_ui/public/application/components/builtin_action_types/servicenow/servicenow.tsx index 4389abff72fcd..1b968cfff5d01 100644 --- a/x-pack/plugins/triggers_actions_ui/public/application/components/builtin_action_types/servicenow/servicenow.tsx +++ b/x-pack/plugins/triggers_actions_ui/public/application/components/builtin_action_types/servicenow/servicenow.tsx @@ -10,13 +10,14 @@ import { ActionTypeModel, ConnectorValidationResult, } from '../../../../types'; -import { connectorConfiguration } from './config'; +import { serviceNowITSMConfiguration, serviceNowSIRConfiguration } from './config'; import logo from './logo.svg'; import { ServiceNowActionConnector, ServiceNowConfig, ServiceNowSecrets, - ServiceNowActionParams, + ServiceNowITSMActionParams, + ServiceNowSIRActionParams, } from './types'; import * as i18n from './translations'; import { isValidUrl } from '../../../lib/value_validators'; @@ -60,19 +61,21 @@ const validateConnector = ( return validationResult; }; -export function getActionType(): ActionTypeModel< +export function getServiceNowITSMActionType(): ActionTypeModel< ServiceNowConfig, ServiceNowSecrets, - ServiceNowActionParams + ServiceNowITSMActionParams > { return { - id: connectorConfiguration.id, + id: serviceNowITSMConfiguration.id, iconClass: logo, - selectMessage: i18n.SERVICENOW_DESC, - actionTypeTitle: connectorConfiguration.name, + selectMessage: serviceNowITSMConfiguration.desc, + actionTypeTitle: serviceNowITSMConfiguration.name, validateConnector, actionConnectorFields: lazy(() => import('./servicenow_connectors')), - validateParams: (actionParams: ServiceNowActionParams): GenericValidationResult => { + validateParams: ( + actionParams: ServiceNowITSMActionParams + ): GenericValidationResult => { const errors = { // eslint-disable-next-line @typescript-eslint/naming-convention 'subActionParams.incident.short_description': new Array(), @@ -89,6 +92,39 @@ export function getActionType(): ActionTypeModel< } return validationResult; }, - actionParamsFields: lazy(() => import('./servicenow_params')), + actionParamsFields: lazy(() => import('./servicenow_itsm_params')), + }; +} + +export function getServiceNowSIRActionType(): ActionTypeModel< + ServiceNowConfig, + ServiceNowSecrets, + ServiceNowSIRActionParams +> { + return { + id: serviceNowSIRConfiguration.id, + iconClass: logo, + selectMessage: serviceNowSIRConfiguration.desc, + actionTypeTitle: serviceNowSIRConfiguration.name, + validateConnector, + actionConnectorFields: lazy(() => import('./servicenow_connectors')), + validateParams: (actionParams: ServiceNowSIRActionParams): GenericValidationResult => { + const errors = { + // eslint-disable-next-line @typescript-eslint/naming-convention + 'subActionParams.incident.short_description': new Array(), + }; + const validationResult = { + errors, + }; + if ( + actionParams.subActionParams && + actionParams.subActionParams.incident && + !actionParams.subActionParams.incident.short_description?.length + ) { + errors['subActionParams.incident.short_description'].push(i18n.TITLE_REQUIRED); + } + return validationResult; + }, + actionParamsFields: lazy(() => import('./servicenow_sir_params')), }; } diff --git a/x-pack/plugins/triggers_actions_ui/public/application/components/builtin_action_types/servicenow/servicenow_params.test.tsx b/x-pack/plugins/triggers_actions_ui/public/application/components/builtin_action_types/servicenow/servicenow_itsm_params.test.tsx similarity index 53% rename from x-pack/plugins/triggers_actions_ui/public/application/components/builtin_action_types/servicenow/servicenow_params.test.tsx rename to x-pack/plugins/triggers_actions_ui/public/application/components/builtin_action_types/servicenow/servicenow_itsm_params.test.tsx index 5519d7498a85e..51318e14a2cfd 100644 --- a/x-pack/plugins/triggers_actions_ui/public/application/components/builtin_action_types/servicenow/servicenow_params.test.tsx +++ b/x-pack/plugins/triggers_actions_ui/public/application/components/builtin_action_types/servicenow/servicenow_itsm_params.test.tsx @@ -5,8 +5,18 @@ */ import React from 'react'; import { mount } from 'enzyme'; -import ServiceNowParamsFields from './servicenow_params'; +import { act } from '@testing-library/react'; + import { ActionConnector } from '../../../../types'; +import { useGetChoices } from './use_get_choices'; +import ServiceNowITSMParamsFields from './servicenow_itsm_params'; +import { Choice } from './types'; + +jest.mock('./use_get_choices'); +jest.mock('../../../../common/lib/kibana'); + +const useGetChoicesMock = useGetChoices as jest.Mock; + const actionParams = { subAction: 'pushToService', subActionParams: { @@ -16,7 +26,6 @@ const actionParams = { severity: '1', urgency: '2', impact: '3', - savedObjectId: '123', externalId: null, }, comments: [], @@ -31,6 +40,7 @@ const connector: ActionConnector = { name: 'Test', isPreconfigured: false, }; + const editAction = jest.fn(); const defaultProps = { actionConnector: connector, @@ -40,31 +50,71 @@ const defaultProps = { index: 0, messageVariables: [], }; -describe('ServiceNowParamsFields renders', () => { + +const useGetChoicesResponse = { + isLoading: false, + choices: ['severity', 'urgency', 'impact'] + .map((element) => [ + { + dependent_value: '', + label: '1 - Critical', + value: '1', + element, + }, + { + dependent_value: '', + label: '2 - High', + value: '2', + element, + }, + { + dependent_value: '', + label: '3 - Moderate', + value: '3', + element, + }, + { + dependent_value: '', + label: '4 - Low', + value: '4', + element, + }, + ]) + .flat(), +}; + +describe('ServiceNowITSMParamsFields renders', () => { + let onChoices = (choices: Choice[]) => {}; + beforeEach(() => { jest.clearAllMocks(); + useGetChoicesMock.mockImplementation((args) => { + onChoices = args.onSuccess; + return useGetChoicesResponse; + }); }); + test('all params fields is rendered', () => { - const wrapper = mount(); - expect(wrapper.find('[data-test-subj="urgencySelect"]').length > 0).toBeTruthy(); - expect(wrapper.find('[data-test-subj="severitySelect"]').first().prop('value')).toStrictEqual( - '1' - ); - expect(wrapper.find('[data-test-subj="impactSelect"]').length > 0).toBeTruthy(); - expect(wrapper.find('[data-test-subj="short_descriptionInput"]').length > 0).toBeTruthy(); - expect(wrapper.find('[data-test-subj="descriptionTextArea"]').length > 0).toBeTruthy(); - expect(wrapper.find('[data-test-subj="commentsTextArea"]').length > 0).toBeTruthy(); + const wrapper = mount(); + expect(wrapper.find('[data-test-subj="urgencySelect"]').exists()).toBeTruthy(); + expect(wrapper.find('[data-test-subj="severitySelect"]').exists()).toBeTruthy(); + expect(wrapper.find('[data-test-subj="impactSelect"]').exists()).toBeTruthy(); + expect(wrapper.find('[data-test-subj="short_descriptionInput"]').exists()).toBeTruthy(); + expect(wrapper.find('[data-test-subj="descriptionTextArea"]').exists()).toBeTruthy(); + expect(wrapper.find('[data-test-subj="commentsTextArea"]').exists()).toBeTruthy(); }); + test('If short_description has errors, form row is invalid', () => { const newProps = { ...defaultProps, // eslint-disable-next-line @typescript-eslint/naming-convention errors: { 'subActionParams.incident.short_description': ['error'] }, }; - const wrapper = mount(); + const wrapper = mount(); const title = wrapper.find('[data-test-subj="short_descriptionInput"]').first(); expect(title.prop('isInvalid')).toBeTruthy(); }); + test('When subActionParams is undefined, set to default', () => { const { subActionParams, ...newParams } = actionParams; @@ -72,12 +122,13 @@ describe('ServiceNowParamsFields renders', () => { ...defaultProps, actionParams: newParams, }; - mount(); + mount(); expect(editAction.mock.calls[0][1]).toEqual({ incident: {}, comments: [], }); }); + test('When subAction is undefined, set to default', () => { const { subAction, ...newParams } = actionParams; @@ -85,11 +136,12 @@ describe('ServiceNowParamsFields renders', () => { ...defaultProps, actionParams: newParams, }; - mount(); + mount(); expect(editAction.mock.calls[0][1]).toEqual('pushToService'); }); + test('Resets fields when connector changes', () => { - const wrapper = mount(); + const wrapper = mount(); expect(editAction.mock.calls.length).toEqual(0); wrapper.setProps({ actionConnector: { ...connector, id: '1234' } }); expect(editAction.mock.calls.length).toEqual(1); @@ -98,6 +150,52 @@ describe('ServiceNowParamsFields renders', () => { comments: [], }); }); + + test('it transforms the urgencies to options correctly', async () => { + const wrapper = mount(); + act(() => { + onChoices(useGetChoicesResponse.choices); + }); + + wrapper.update(); + expect(wrapper.find('[data-test-subj="urgencySelect"]').first().prop('options')).toEqual([ + { value: '1', text: '1 - Critical' }, + { value: '2', text: '2 - High' }, + { value: '3', text: '3 - Moderate' }, + { value: '4', text: '4 - Low' }, + ]); + }); + + test('it transforms the severities to options correctly', async () => { + const wrapper = mount(); + act(() => { + onChoices(useGetChoicesResponse.choices); + }); + + wrapper.update(); + expect(wrapper.find('[data-test-subj="severitySelect"]').first().prop('options')).toEqual([ + { value: '1', text: '1 - Critical' }, + { value: '2', text: '2 - High' }, + { value: '3', text: '3 - Moderate' }, + { value: '4', text: '4 - Low' }, + ]); + }); + + test('it transforms the impacts to options correctly', async () => { + const wrapper = mount(); + act(() => { + onChoices(useGetChoicesResponse.choices); + }); + + wrapper.update(); + expect(wrapper.find('[data-test-subj="impactSelect"]').first().prop('options')).toEqual([ + { value: '1', text: '1 - Critical' }, + { value: '2', text: '2 - High' }, + { value: '3', text: '3 - Moderate' }, + { value: '4', text: '4 - Low' }, + ]); + }); + describe('UI updates', () => { const changeEvent = { target: { value: 'Bug' } } as React.ChangeEvent; const simpleFields = [ @@ -107,22 +205,25 @@ describe('ServiceNowParamsFields renders', () => { { dataTestSubj: '[data-test-subj="severitySelect"]', key: 'severity' }, { dataTestSubj: '[data-test-subj="impactSelect"]', key: 'impact' }, ]; + simpleFields.forEach((field) => test(`${field.key} update triggers editAction :D`, () => { - const wrapper = mount(); + const wrapper = mount(); const theField = wrapper.find(field.dataTestSubj).first(); theField.prop('onChange')!(changeEvent); expect(editAction.mock.calls[0][1].incident[field.key]).toEqual(changeEvent.target.value); }) ); + test('A comment triggers editAction', () => { - const wrapper = mount(); + const wrapper = mount(); const comments = wrapper.find('textarea[data-test-subj="commentsTextArea"]'); expect(comments.simulate('change', changeEvent)); expect(editAction.mock.calls[0][1].comments.length).toEqual(1); }); + test('An empty comment does not trigger editAction', () => { - const wrapper = mount(); + const wrapper = mount(); const emptyComment = { target: { value: '' } }; const comments = wrapper.find('[data-test-subj="commentsTextArea"] textarea'); expect(comments.simulate('change', emptyComment)); diff --git a/x-pack/plugins/triggers_actions_ui/public/application/components/builtin_action_types/servicenow/servicenow_params.tsx b/x-pack/plugins/triggers_actions_ui/public/application/components/builtin_action_types/servicenow/servicenow_itsm_params.tsx similarity index 64% rename from x-pack/plugins/triggers_actions_ui/public/application/components/builtin_action_types/servicenow/servicenow_params.tsx rename to x-pack/plugins/triggers_actions_ui/public/application/components/builtin_action_types/servicenow/servicenow_itsm_params.tsx index 3e6b443d790a9..658b964f8b91d 100644 --- a/x-pack/plugins/triggers_actions_ui/public/application/components/builtin_action_types/servicenow/servicenow_params.tsx +++ b/x-pack/plugins/triggers_actions_ui/public/application/components/builtin_action_types/servicenow/servicenow_itsm_params.tsx @@ -4,8 +4,7 @@ * you may not use this file except in compliance with the Elastic License. */ -import React, { Fragment, useCallback, useEffect, useMemo, useRef } from 'react'; -import { i18n } from '@kbn/i18n'; +import React, { Fragment, useCallback, useEffect, useMemo, useRef, useState } from 'react'; import { EuiFormRow, EuiSelect, @@ -14,38 +13,29 @@ import { EuiSpacer, EuiTitle, } from '@elastic/eui'; +import { useKibana } from '../../../../common/lib/kibana'; import { ActionParamsProps } from '../../../../types'; -import { ServiceNowActionParams } from './types'; +import { ServiceNowITSMActionParams, Choice, Options } from './types'; import { TextAreaWithMessageVariables } from '../../text_area_with_message_variables'; import { TextFieldWithMessageVariables } from '../../text_field_with_message_variables'; +import { useGetChoices } from './use_get_choices'; +import * as i18n from './translations'; -const selectOptions = [ - { - value: '1', - text: i18n.translate( - 'xpack.triggersActionsUI.components.builtinActionTypes.servicenow.severitySelectHighOptionLabel', - { defaultMessage: 'High' } - ), - }, - { - value: '2', - text: i18n.translate( - 'xpack.triggersActionsUI.components.builtinActionTypes.servicenow.severitySelectMediumOptionLabel', - { defaultMessage: 'Medium' } - ), - }, - { - value: '3', - text: i18n.translate( - 'xpack.triggersActionsUI.components.builtinActionTypes.servicenow.severitySelectLawOptionLabel', - { defaultMessage: 'Low' } - ), - }, -]; +const useGetChoicesFields = ['urgency', 'severity', 'impact']; +const defaultOptions: Options = { + urgency: [], + severity: [], + impact: [], +}; const ServiceNowParamsFields: React.FunctionComponent< - ActionParamsProps + ActionParamsProps > = ({ actionConnector, actionParams, editAction, index, errors, messageVariables }) => { + const { + http, + notifications: { toasts }, + } = useKibana().services; + const actionConnectorRef = useRef(actionConnector?.id ?? ''); const { incident, comments } = useMemo( () => @@ -53,10 +43,12 @@ const ServiceNowParamsFields: React.FunctionComponent< (({ incident: {}, comments: [], - } as unknown) as ServiceNowActionParams['subActionParams']), + } as unknown) as ServiceNowITSMActionParams['subActionParams']), [actionParams.subActionParams] ); + const [options, setOptions] = useState(defaultOptions); + const editSubActionProperty = useCallback( (key: string, value: any) => { const newProps = @@ -80,6 +72,28 @@ const ServiceNowParamsFields: React.FunctionComponent< [editSubActionProperty] ); + const onChoicesSuccess = (choices: Choice[]) => + setOptions( + choices.reduce( + (acc, choice) => ({ + ...acc, + [choice.element]: [ + ...(acc[choice.element] != null ? acc[choice.element] : []), + { value: choice.value, text: choice.label }, + ], + }), + defaultOptions + ) + ); + + const { isLoading: isLoadingChoices } = useGetChoices({ + http, + toastNotifications: toasts, + actionConnector, + fields: useGetChoicesFields, + onSuccess: onChoicesSuccess, + }); + useEffect(() => { if (actionConnector != null && actionConnectorRef.current !== actionConnector.id) { actionConnectorRef.current = actionConnector.id; @@ -94,6 +108,7 @@ const ServiceNowParamsFields: React.FunctionComponent< } // eslint-disable-next-line react-hooks/exhaustive-deps }, [actionConnector]); + useEffect(() => { if (!actionParams.subAction) { editAction('subAction', 'pushToService', index); @@ -114,64 +129,47 @@ const ServiceNowParamsFields: React.FunctionComponent< return ( -

- {i18n.translate( - 'xpack.triggersActionsUI.components.builtinActionTypes.serviceNow.title', - { defaultMessage: 'Incident' } - )} -

+

{i18n.INCIDENT}

 - + editSubActionProperty('urgency', e.target.value)} /> - + editSubActionProperty('severity', e.target.value)} /> - + editSubActionProperty('impact', e.target.value)} /> @@ -185,10 +183,7 @@ const ServiceNowParamsFields: React.FunctionComponent< errors['subActionParams.incident.short_description'].length > 0 && incident.short_description !== undefined } - label={i18n.translate( - 'xpack.triggersActionsUI.components.builtinActionTypes.serviceNow.titleFieldLabel', - { defaultMessage: 'Short description (required)' } - )} + label={i18n.SHORT_DESCRIPTION_LABEL} > 0 ? comments[0].comment : undefined} - label={i18n.translate( - 'xpack.triggersActionsUI.components.builtinActionTypes.serviceNow.commentsTextAreaFieldLabel', - { defaultMessage: 'Additional comments' } - )} + label={i18n.COMMENTS_LABEL} /> ); diff --git a/x-pack/plugins/triggers_actions_ui/public/application/components/builtin_action_types/servicenow/servicenow_sir_params.test.tsx b/x-pack/plugins/triggers_actions_ui/public/application/components/builtin_action_types/servicenow/servicenow_sir_params.test.tsx new file mode 100644 index 0000000000000..72dfd63da3d4e --- /dev/null +++ b/x-pack/plugins/triggers_actions_ui/public/application/components/builtin_action_types/servicenow/servicenow_sir_params.test.tsx @@ -0,0 +1,311 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License; + * you may not use this file except in compliance with the Elastic License. + */ +import React from 'react'; +import { mount } from 'enzyme'; +import { act } from '@testing-library/react'; + +import { ActionConnector } from '../../../../types'; +import { useGetChoices } from './use_get_choices'; +import ServiceNowSIRParamsFields from './servicenow_sir_params'; +import { Choice } from './types'; + +jest.mock('./use_get_choices'); +jest.mock('../../../../common/lib/kibana'); + +const useGetChoicesMock = useGetChoices as jest.Mock; + +const actionParams = { + subAction: 'pushToService', + subActionParams: { + incident: { + short_description: 'sn title', + description: 'some description', + category: 'Denial of Service', + dest_ip: '192.168.1.1', + source_ip: '192.168.1.2', + malware_hash: '098f6bcd4621d373cade4e832627b4f6', + malware_url: 'https://attack.com', + priority: '1', + subcategory: '20', + externalId: null, + }, + comments: [], + }, +}; + +const connector: ActionConnector = { + secrets: {}, + config: {}, + id: 'test', + actionTypeId: '.test', + name: 'Test', + isPreconfigured: false, +}; + +const editAction = jest.fn(); +const defaultProps = { + actionConnector: connector, + actionParams, + errors: { ['subActionParams.incident.short_description']: [] }, + editAction, + index: 0, + messageVariables: [], +}; + +const choicesResponse = { + isLoading: false, + choices: [ + { + dependent_value: '', + label: 'Priviledge Escalation', + value: 'Priviledge Escalation', + element: 'category', + }, + { + dependent_value: '', + label: 'Criminal activity/investigation', + value: 'Criminal activity/investigation', + element: 'category', + }, + { + dependent_value: '', + label: 'Denial of Service', + value: 'Denial of Service', + element: 'category', + }, + { + dependent_value: 'Denial of Service', + label: 'Inbound or outbound', + value: '12', + element: 'subcategory', + }, + { + dependent_value: 'Denial of Service', + label: 'Single or distributed (DoS or DDoS)', + value: '26', + element: 'subcategory', + }, + { + dependent_value: 'Denial of Service', + label: 'Inbound DDos', + value: 'inbound_ddos', + element: 'subcategory', + }, + { + dependent_value: '', + label: '1 - Critical', + value: '1', + element: 'priority', + }, + { + dependent_value: '', + label: '2 - High', + value: '2', + element: 'priority', + }, + { + dependent_value: '', + label: '3 - Moderate', + value: '3', + element: 'priority', + }, + { + dependent_value: '', + label: '4 - Low', + value: '4', + element: 'priority', + }, + { + dependent_value: '', + label: '5 - Planning', + value: '5', + element: 'priority', + }, + ], +}; + +describe('ServiceNowSIRParamsFields renders', () => { + let onChoicesSuccess = (choices: Choice[]) => {}; + + beforeEach(() => { + jest.clearAllMocks(); + useGetChoicesMock.mockImplementation((args) => { + onChoicesSuccess = args.onSuccess; + return choicesResponse; + }); + }); + + test('all params fields is rendered', () => { + const wrapper = mount(); + expect(wrapper.find('[data-test-subj="short_descriptionInput"]').exists()).toBeTruthy(); + expect(wrapper.find('[data-test-subj="source_ipInput"]').exists()).toBeTruthy(); + expect(wrapper.find('[data-test-subj="dest_ipInput"]').exists()).toBeTruthy(); + expect(wrapper.find('[data-test-subj="malware_urlInput"]').exists()).toBeTruthy(); + expect(wrapper.find('[data-test-subj="malware_hashInput"]').exists()).toBeTruthy(); + expect(wrapper.find('[data-test-subj="prioritySelect"]').exists()).toBeTruthy(); + expect(wrapper.find('[data-test-subj="categorySelect"]').exists()).toBeTruthy(); + expect(wrapper.find('[data-test-subj="subcategorySelect"]').exists()).toBeTruthy(); + expect(wrapper.find('[data-test-subj="descriptionTextArea"]').exists()).toBeTruthy(); + expect(wrapper.find('[data-test-subj="commentsTextArea"]').exists()).toBeTruthy(); + }); + + test('If short_description has errors, form row is invalid', () => { + const newProps = { + ...defaultProps, + // eslint-disable-next-line @typescript-eslint/naming-convention + errors: { 'subActionParams.incident.short_description': ['error'] }, + }; + const wrapper = mount(); + const title = wrapper.find('[data-test-subj="short_descriptionInput"]').first(); + expect(title.prop('isInvalid')).toBeTruthy(); + }); + + test('When subActionParams is undefined, set to default', () => { + const { subActionParams, ...newParams } = actionParams; + + const newProps = { + ...defaultProps, + actionParams: newParams, + }; + mount(); + expect(editAction.mock.calls[0][1]).toEqual({ + incident: {}, + comments: [], + }); + }); + + test('When subAction is undefined, set to default', () => { + const { subAction, ...newParams } = actionParams; + + const newProps = { + ...defaultProps, + actionParams: newParams, + }; + mount(); + expect(editAction.mock.calls[0][1]).toEqual('pushToService'); + }); + + test('Resets fields when connector changes', () => { + const wrapper = mount(); + expect(editAction.mock.calls.length).toEqual(0); + wrapper.setProps({ actionConnector: { ...connector, id: '1234' } }); + expect(editAction.mock.calls.length).toEqual(1); + expect(editAction.mock.calls[0][1]).toEqual({ + incident: {}, + comments: [], + }); + }); + + test('it transforms the categories to options correctly', async () => { + const wrapper = mount(); + act(() => { + onChoicesSuccess(choicesResponse.choices); + }); + + wrapper.update(); + expect(wrapper.find('[data-test-subj="categorySelect"]').first().prop('options')).toEqual([ + { value: 'Priviledge Escalation', text: 'Priviledge Escalation' }, + { + value: 'Criminal activity/investigation', + text: 'Criminal activity/investigation', + }, + { value: 'Denial of Service', text: 'Denial of Service' }, + ]); + }); + + test('it transforms the subcategories to options correctly', async () => { + const wrapper = mount(); + act(() => { + onChoicesSuccess(choicesResponse.choices); + }); + + wrapper.update(); + expect(wrapper.find('[data-test-subj="subcategorySelect"]').first().prop('options')).toEqual([ + { + text: 'Inbound or outbound', + value: '12', + }, + { + text: 'Single or distributed (DoS or DDoS)', + value: '26', + }, + { + text: 'Inbound DDos', + value: 'inbound_ddos', + }, + ]); + }); + + test('it transforms the priorities to options correctly', async () => { + const wrapper = mount(); + act(() => { + onChoicesSuccess(choicesResponse.choices); + }); + + wrapper.update(); + expect(wrapper.find('[data-test-subj="prioritySelect"]').first().prop('options')).toEqual([ + { + text: '1 - Critical', + value: '1', + }, + { + text: '2 - High', + value: '2', + }, + { + text: '3 - Moderate', + value: '3', + }, + { + text: '4 - Low', + value: '4', + }, + { + text: '5 - Planning', + value: '5', + }, + ]); + }); + + describe('UI updates', () => { + const changeEvent = { target: { value: 'Bug' } } as React.ChangeEvent; + const simpleFields = [ + { dataTestSubj: 'input[data-test-subj="short_descriptionInput"]', key: 'short_description' }, + { dataTestSubj: 'textarea[data-test-subj="descriptionTextArea"]', key: 'description' }, + { dataTestSubj: '[data-test-subj="source_ipInput"]', key: 'source_ip' }, + { dataTestSubj: '[data-test-subj="dest_ipInput"]', key: 'dest_ip' }, + { dataTestSubj: '[data-test-subj="malware_urlInput"]', key: 'malware_url' }, + { dataTestSubj: '[data-test-subj="malware_hashInput"]', key: 'malware_hash' }, + { dataTestSubj: '[data-test-subj="prioritySelect"]', key: 'priority' }, + { dataTestSubj: '[data-test-subj="categorySelect"]', key: 'category' }, + { dataTestSubj: '[data-test-subj="subcategorySelect"]', key: 'subcategory' }, + ]; + + simpleFields.forEach((field) => + test(`${field.key} update triggers editAction :D`, () => { + const wrapper = mount(); + const theField = wrapper.find(field.dataTestSubj).first(); + theField.prop('onChange')!(changeEvent); + expect(editAction.mock.calls[0][1].incident[field.key]).toEqual(changeEvent.target.value); + }) + ); + + test('A comment triggers editAction', () => { + const wrapper = mount(); + const comments = wrapper.find('textarea[data-test-subj="commentsTextArea"]'); + expect(comments.simulate('change', changeEvent)); + expect(editAction.mock.calls[0][1].comments.length).toEqual(1); + }); + + test('An empty comment does not trigger editAction', () => { + const wrapper = mount(); + const emptyComment = { target: { value: '' } }; + const comments = wrapper.find('[data-test-subj="commentsTextArea"] textarea'); + expect(comments.simulate('change', emptyComment)); + expect(editAction.mock.calls.length).toEqual(0); + }); + }); +}); diff --git a/x-pack/plugins/triggers_actions_ui/public/application/components/builtin_action_types/servicenow/servicenow_sir_params.tsx b/x-pack/plugins/triggers_actions_ui/public/application/components/builtin_action_types/servicenow/servicenow_sir_params.tsx new file mode 100644 index 0000000000000..26957d828f5e0 --- /dev/null +++ b/x-pack/plugins/triggers_actions_ui/public/application/components/builtin_action_types/servicenow/servicenow_sir_params.tsx @@ -0,0 +1,295 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License; + * you may not use this file except in compliance with the Elastic License. + */ + +import React, { Fragment, useCallback, useEffect, useMemo, useRef, useState } from 'react'; +import { + EuiFormRow, + EuiSelect, + EuiFlexGroup, + EuiFlexItem, + EuiSpacer, + EuiTitle, + EuiSelectOption, +} from '@elastic/eui'; +import { useKibana } from '../../../../common/lib/kibana'; +import { ActionParamsProps } from '../../../../types'; +import { TextAreaWithMessageVariables } from '../../text_area_with_message_variables'; +import { TextFieldWithMessageVariables } from '../../text_field_with_message_variables'; + +import * as i18n from './translations'; +import { useGetChoices } from './use_get_choices'; +import { ServiceNowSIRActionParams, Fields, Choice } from './types'; + +const useGetChoicesFields = ['category', 'subcategory', 'priority']; +const defaultFields: Fields = { + category: [], + subcategory: [], + priority: [], +}; + +const choicesToEuiOptions = (choices: Choice[]): EuiSelectOption[] => + choices.map((choice) => ({ value: choice.value, text: choice.label })); + +const ServiceNowSIRParamsFields: React.FunctionComponent< + ActionParamsProps +> = ({ actionConnector, actionParams, editAction, index, errors, messageVariables }) => { + const { + http, + notifications: { toasts }, + } = useKibana().services; + + const actionConnectorRef = useRef(actionConnector?.id ?? ''); + const { incident, comments } = useMemo( + () => + actionParams.subActionParams ?? + (({ + incident: {}, + comments: [], + } as unknown) as ServiceNowSIRActionParams['subActionParams']), + [actionParams.subActionParams] + ); + + const [choices, setChoices] = useState(defaultFields); + + const editSubActionProperty = useCallback( + (key: string, value: any) => { + const newProps = + key !== 'comments' + ? { + incident: { ...incident, [key]: value }, + comments, + } + : { incident, [key]: value }; + editAction('subActionParams', newProps, index); + }, + [comments, editAction, incident, index] + ); + + const editComment = useCallback( + (key, value) => { + if (value.length > 0) { + editSubActionProperty(key, [{ commentId: '1', comment: value }]); + } + }, + [editSubActionProperty] + ); + + const onChoicesSuccess = useCallback((values: Choice[]) => { + setChoices( + values.reduce( + (acc, value) => ({ + ...acc, + [value.element]: [...(acc[value.element] != null ? acc[value.element] : []), value], + }), + defaultFields + ) + ); + }, []); + + const { isLoading: isLoadingChoices } = useGetChoices({ + http, + toastNotifications: toasts, + actionConnector, + // Not having a memoized fields variable will cause infinitive API calls. + fields: useGetChoicesFields, + onSuccess: onChoicesSuccess, + }); + + const categoryOptions = useMemo(() => choicesToEuiOptions(choices.category), [choices.category]); + const priorityOptions = useMemo(() => choicesToEuiOptions(choices.priority), [choices.priority]); + + const subcategoryOptions = useMemo( + () => + choicesToEuiOptions( + choices.subcategory.filter( + (subcategory) => subcategory.dependent_value === incident.category + ) + ), + [choices.subcategory, incident.category] + ); + + useEffect(() => { + if (actionConnector != null && actionConnectorRef.current !== actionConnector.id) { + actionConnectorRef.current = actionConnector.id; + editAction( + 'subActionParams', + { + incident: {}, + comments: [], + }, + index + ); + } + // eslint-disable-next-line react-hooks/exhaustive-deps + }, [actionConnector]); + + useEffect(() => { + if (!actionParams.subAction) { + editAction('subAction', 'pushToService', index); + } + if (!actionParams.subActionParams) { + editAction( + 'subActionParams', + { + incident: {}, + comments: [], + }, + index + ); + } + // eslint-disable-next-line react-hooks/exhaustive-deps + }, [actionParams]); + + return ( + + +

{i18n.INCIDENT}

+ + + 0 && + incident.short_description !== undefined + } + label={i18n.SHORT_DESCRIPTION_LABEL} + > + + + + + + + + + + + + + + + + + + + + + { + editAction( + 'subActionParams', + { + incident: { ...incident, priority: e.target.value }, + comments, + }, + index + ); + }} + /> + + + + + + { + editAction( + 'subActionParams', + { + incident: { ...incident, category: e.target.value, subcategory: null }, + comments, + }, + index + ); + }} + /> + + + + + editSubActionProperty('subcategory', e.target.value)} + /> + + + + + + 0 ? comments[0].comment : undefined} + label={i18n.COMMENTS_LABEL} + /> + + ); +}; + +// eslint-disable-next-line import/no-default-export +export { ServiceNowSIRParamsFields as default }; diff --git a/x-pack/plugins/triggers_actions_ui/public/application/components/builtin_action_types/servicenow/translations.ts b/x-pack/plugins/triggers_actions_ui/public/application/components/builtin_action_types/servicenow/translations.ts index c84a916c0fef4..c8bc2f427bde2 100644 --- a/x-pack/plugins/triggers_actions_ui/public/application/components/builtin_action_types/servicenow/translations.ts +++ b/x-pack/plugins/triggers_actions_ui/public/application/components/builtin_action_types/servicenow/translations.ts @@ -6,17 +6,31 @@ import { i18n } from '@kbn/i18n'; -export const SERVICENOW_DESC = i18n.translate( - 'xpack.triggersActionsUI.components.builtinActionTypes.servicenow.selectMessageText', +export const SERVICENOW_ITSM_DESC = i18n.translate( + 'xpack.triggersActionsUI.components.builtinActionTypes.serviceNowITSM.selectMessageText', { - defaultMessage: 'Create an incident in ServiceNow.', + defaultMessage: 'Create an incident in ServiceNow ITSM.', } ); -export const SERVICENOW_TITLE = i18n.translate( - 'xpack.triggersActionsUI.components.builtinActionTypes.servicenow.actionTypeTitle', +export const SERVICENOW_SIR_DESC = i18n.translate( + 'xpack.triggersActionsUI.components.builtinActionTypes.serviceNowSIR.selectMessageText', { - defaultMessage: 'ServiceNow', + defaultMessage: 'Create an incident in ServiceNow SIR.', + } +); + +export const SERVICENOW_ITSM_TITLE = i18n.translate( + 'xpack.triggersActionsUI.components.builtinActionTypes.serviceNowITSM.actionTypeTitle', + { + defaultMessage: 'ServiceNow ITSM', + } +); + +export const SERVICENOW_SIR_TITLE = i18n.translate( + 'xpack.triggersActionsUI.components.builtinActionTypes.serviceNowSIR.actionTypeTitle', + { + defaultMessage: 'ServiceNow SIR', } ); @@ -98,65 +112,114 @@ export const PASSWORD_REQUIRED = i18n.translate( } ); -export const API_TOKEN_LABEL = i18n.translate( - 'xpack.triggersActionsUI.components.builtinActionTypes.servicenow.apiTokenTextFieldLabel', +export const TITLE_REQUIRED = i18n.translate( + 'xpack.triggersActionsUI.components.builtinActionTypes.common.requiredShortDescTextField', { - defaultMessage: 'Api token', + defaultMessage: 'Short description is required.', } ); -export const API_TOKEN_REQUIRED = i18n.translate( - 'xpack.triggersActionsUI.components.builtinActionTypes.servicenow.requiredApiTokenTextField', +export const SOURCE_IP_LABEL = i18n.translate( + 'xpack.triggersActionsUI.components.builtinActionTypes.servicenow.sourceIPTitle', { - defaultMessage: 'Api token is required.', + defaultMessage: 'Source IP', } ); -export const EMAIL_LABEL = i18n.translate( - 'xpack.triggersActionsUI.components.builtinActionTypes.servicenow.emailTextFieldLabel', +export const DEST_IP_LABEL = i18n.translate( + 'xpack.triggersActionsUI.components.builtinActionTypes.servicenow.destinationIPTitle', { - defaultMessage: 'Email', + defaultMessage: 'Destination IP', } ); -export const EMAIL_REQUIRED = i18n.translate( - 'xpack.triggersActionsUI.components.builtinActionTypes.servicenow.requiredEmailTextField', +export const INCIDENT = i18n.translate( + 'xpack.triggersActionsUI.components.builtinActionTypes.servicenow.title', { - defaultMessage: 'Email is required.', + defaultMessage: 'Incident', } ); -export const MAPPING_FIELD_SHORT_DESC = i18n.translate( - 'xpack.triggersActionsUI.components.builtinActionTypes.servicenow.mappingFieldShortDescription', +export const SHORT_DESCRIPTION_LABEL = i18n.translate( + 'xpack.triggersActionsUI.components.builtinActionTypes.servicenow.titleFieldLabel', { - defaultMessage: 'Short Description', + defaultMessage: 'Short description (required)', } ); -export const MAPPING_FIELD_DESC = i18n.translate( - 'xpack.triggersActionsUI.components.builtinActionTypes.servicenow.mappingFieldDescription', +export const DESCRIPTION_LABEL = i18n.translate( + 'xpack.triggersActionsUI.components.builtinActionTypes.servicenow.descriptionTextAreaFieldLabel', { defaultMessage: 'Description', } ); -export const MAPPING_FIELD_COMMENTS = i18n.translate( - 'xpack.triggersActionsUI.components.builtinActionTypes.servicenow.mappingFieldComments', +export const COMMENTS_LABEL = i18n.translate( + 'xpack.triggersActionsUI.components.builtinActionTypes.servicenow.commentsTextAreaFieldLabel', { - defaultMessage: 'Comments', + defaultMessage: 'Additional comments', } ); -export const DESCRIPTION_REQUIRED = i18n.translate( - 'xpack.triggersActionsUI.components.builtinActionTypes.common.requiredDescriptionTextField', +export const MALWARE_URL_LABEL = i18n.translate( + 'xpack.triggersActionsUI.components.builtinActionTypes.servicenow.malwareURLTitle', { - defaultMessage: 'Description is required.', + defaultMessage: 'Malware URL', } ); -export const TITLE_REQUIRED = i18n.translate( - 'xpack.triggersActionsUI.components.builtinActionTypes.common.requiredShortDescTextField', +export const MALWARE_HASH_LABEL = i18n.translate( + 'xpack.triggersActionsUI.components.builtinActionTypes.servicenow.malwareHashTitle', { - defaultMessage: 'Short description is required.', + defaultMessage: 'Malware hash', + } +); + +export const CHOICES_API_ERROR = i18n.translate( + 'xpack.triggersActionsUI.components.builtinActionTypes.servicenow.unableToGetChoicesMessage', + { + defaultMessage: 'Unable to get choices', + } +); + +export const CATEGORY_LABEL = i18n.translate( + 'xpack.triggersActionsUI.components.builtinActionTypes.servicenow.categoryTitle', + { + defaultMessage: 'Category', + } +); + +export const SUBCATEGORY_LABEL = i18n.translate( + 'xpack.triggersActionsUI.components.builtinActionTypes.servicenow.subcategoryTitle', + { + defaultMessage: 'Subcategory', + } +); + +export const URGENCY_LABEL = i18n.translate( + 'xpack.triggersActionsUI.components.builtinActionTypes.servicenow.urgencySelectFieldLabel', + { + defaultMessage: 'Urgency', + } +); + +export const SEVERITY_LABEL = i18n.translate( + 'xpack.triggersActionsUI.components.builtinActionTypes.servicenow.severitySelectFieldLabel', + { + defaultMessage: 'Severity', + } +); + +export const IMPACT_LABEL = i18n.translate( + 'xpack.triggersActionsUI.components.builtinActionTypes.servicenow.impactSelectFieldLabel', + { + defaultMessage: 'Impact', + } +); + +export const PRIORITY_LABEL = i18n.translate( + 'xpack.triggersActionsUI.components.builtinActionTypes.servicenow.prioritySelectFieldLabel', + { + defaultMessage: 'Priority', } ); diff --git a/x-pack/plugins/triggers_actions_ui/public/application/components/builtin_action_types/servicenow/types.ts b/x-pack/plugins/triggers_actions_ui/public/application/components/builtin_action_types/servicenow/types.ts index ae03680a80534..be9a7c634af8a 100644 --- a/x-pack/plugins/triggers_actions_ui/public/application/components/builtin_action_types/servicenow/types.ts +++ b/x-pack/plugins/triggers_actions_ui/public/application/components/builtin_action_types/servicenow/types.ts @@ -4,18 +4,27 @@ * you may not use this file except in compliance with the Elastic License. */ +import { EuiSelectOption } from '@elastic/eui'; import { UserConfiguredActionConnector } from '../../../../types'; -// eslint-disable-next-line @kbn/eslint/no-restricted-paths -import { ExecutorSubActionPushParams } from '../../../../../../actions/server/builtin_action_types/servicenow/types'; +import { + ExecutorSubActionPushParamsITSM, + ExecutorSubActionPushParamsSIR, + // eslint-disable-next-line @kbn/eslint/no-restricted-paths +} from '../../../../../../actions/server/builtin_action_types/servicenow/types'; export type ServiceNowActionConnector = UserConfiguredActionConnector< ServiceNowConfig, ServiceNowSecrets >; -export interface ServiceNowActionParams { +export interface ServiceNowITSMActionParams { subAction: string; - subActionParams: ExecutorSubActionPushParams; + subActionParams: ExecutorSubActionPushParamsITSM; +} + +export interface ServiceNowSIRActionParams { + subAction: string; + subActionParams: ExecutorSubActionPushParamsSIR; } export interface ServiceNowConfig { @@ -26,3 +35,13 @@ export interface ServiceNowSecrets { username: string; password: string; } + +export interface Choice { + value: string; + label: string; + element: string; + dependent_value: string; +} + +export type Fields = Record; +export type Options = Record; diff --git a/x-pack/plugins/triggers_actions_ui/public/application/components/builtin_action_types/servicenow/use_get_choices.test.tsx b/x-pack/plugins/triggers_actions_ui/public/application/components/builtin_action_types/servicenow/use_get_choices.test.tsx new file mode 100644 index 0000000000000..4e8061ebaa6e9 --- /dev/null +++ b/x-pack/plugins/triggers_actions_ui/public/application/components/builtin_action_types/servicenow/use_get_choices.test.tsx @@ -0,0 +1,164 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License; + * you may not use this file except in compliance with the Elastic License. + */ + +import { renderHook } from '@testing-library/react-hooks'; + +import { useKibana } from '../../../../common/lib/kibana'; +import { ActionConnector } from '../../../../types'; +import { useGetChoices, UseGetChoices, UseGetChoicesProps } from './use_get_choices'; +import { getChoices } from './api'; + +jest.mock('./api'); +jest.mock('../../../../common/lib/kibana'); + +const useKibanaMock = useKibana as jest.Mocked; +const getChoicesMock = getChoices as jest.Mock; +const onSuccess = jest.fn(); + +const actionConnector = { + secrets: { + username: 'user', + password: 'pass', + }, + id: 'test', + actionTypeId: '.servicenow', + name: 'ServiceNow ITSM', + isPreconfigured: false, + config: { + apiUrl: 'https://dev94428.service-now.com/', + }, +} as ActionConnector; + +const getChoicesResponse = [ + { + dependent_value: '', + label: 'Priviledge Escalation', + value: 'Priviledge Escalation', + element: 'category', + }, + { + dependent_value: '', + label: 'Criminal activity/investigation', + value: 'Criminal activity/investigation', + element: 'category', + }, + { + dependent_value: '', + label: 'Denial of Service', + value: 'Denial of Service', + element: 'category', + }, +]; + +describe('useGetChoices', () => { + const { services } = useKibanaMock(); + getChoicesMock.mockResolvedValue({ + data: getChoicesResponse, + }); + + beforeEach(() => { + jest.clearAllMocks(); + }); + + const fields = ['priority']; + + it('init', async () => { + const { result, waitForNextUpdate } = renderHook(() => + useGetChoices({ + http: services.http, + actionConnector, + toastNotifications: services.notifications.toasts, + fields, + onSuccess, + }) + ); + + await waitForNextUpdate(); + + expect(result.current).toEqual({ + isLoading: false, + choices: getChoicesResponse, + }); + }); + + it('returns an empty array when connector is not presented', async () => { + const { result } = renderHook(() => + useGetChoices({ + http: services.http, + actionConnector: undefined, + toastNotifications: services.notifications.toasts, + fields, + onSuccess, + }) + ); + + expect(result.current).toEqual({ + isLoading: false, + choices: [], + }); + }); + + it('it calls onSuccess', async () => { + const { waitForNextUpdate } = renderHook(() => + useGetChoices({ + http: services.http, + actionConnector, + toastNotifications: services.notifications.toasts, + fields, + onSuccess, + }) + ); + + await waitForNextUpdate(); + + expect(onSuccess).toHaveBeenCalledWith(getChoicesResponse); + }); + + it('it displays an error when service fails', async () => { + getChoicesMock.mockResolvedValue({ + status: 'error', + serviceMessage: 'An error occurred', + }); + + const { waitForNextUpdate } = renderHook(() => + useGetChoices({ + http: services.http, + actionConnector, + toastNotifications: services.notifications.toasts, + fields, + onSuccess, + }) + ); + + await waitForNextUpdate(); + + expect(services.notifications.toasts.addDanger).toHaveBeenCalledWith({ + text: 'An error occurred', + title: 'Unable to get choices', + }); + }); + + it('it displays an error when http throws an error', async () => { + getChoicesMock.mockImplementation(() => { + throw new Error('An error occurred'); + }); + + renderHook(() => + useGetChoices({ + http: services.http, + actionConnector, + toastNotifications: services.notifications.toasts, + fields, + onSuccess, + }) + ); + + expect(services.notifications.toasts.addDanger).toHaveBeenCalledWith({ + text: 'An error occurred', + title: 'Unable to get choices', + }); + }); +}); diff --git a/x-pack/plugins/triggers_actions_ui/public/application/components/builtin_action_types/servicenow/use_get_choices.tsx b/x-pack/plugins/triggers_actions_ui/public/application/components/builtin_action_types/servicenow/use_get_choices.tsx new file mode 100644 index 0000000000000..0e4338cec0e18 --- /dev/null +++ b/x-pack/plugins/triggers_actions_ui/public/application/components/builtin_action_types/servicenow/use_get_choices.tsx @@ -0,0 +1,99 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License; + * you may not use this file except in compliance with the Elastic License. + */ + +import { useState, useEffect, useRef, useCallback } from 'react'; +import { HttpSetup, ToastsApi } from 'kibana/public'; +import { ActionConnector } from '../../../../types'; +import { getChoices } from './api'; +import { Choice } from './types'; +import * as i18n from './translations'; + +export interface UseGetChoicesProps { + http: HttpSetup; + toastNotifications: Pick< + ToastsApi, + 'get$' | 'add' | 'remove' | 'addSuccess' | 'addWarning' | 'addDanger' | 'addError' + >; + actionConnector?: ActionConnector; + fields: string[]; + onSuccess?: (choices: Choice[]) => void; +} + +export interface UseGetChoices { + choices: Choice[]; + isLoading: boolean; +} + +export const useGetChoices = ({ + http, + actionConnector, + toastNotifications, + fields, + onSuccess, +}: UseGetChoicesProps): UseGetChoices => { + const [isLoading, setIsLoading] = useState(false); + const [choices, setChoices] = useState([]); + const didCancel = useRef(false); + const abortCtrl = useRef(new AbortController()); + + const fetchData = useCallback(async () => { + if (!actionConnector) { + setIsLoading(false); + return; + } + + try { + didCancel.current = false; + abortCtrl.current.abort(); + abortCtrl.current = new AbortController(); + setIsLoading(true); + + const res = await getChoices({ + http, + signal: abortCtrl.current.signal, + connectorId: actionConnector.id, + fields, + }); + + if (!didCancel.current) { + setIsLoading(false); + setChoices(res.data ?? []); + if (res.status && res.status === 'error') { + toastNotifications.addDanger({ + title: i18n.CHOICES_API_ERROR, + text: `${res.serviceMessage ?? res.message}`, + }); + } else if (onSuccess) { + onSuccess(res.data ?? []); + } + } + } catch (error) { + if (!didCancel.current) { + setIsLoading(false); + toastNotifications.addDanger({ + title: i18n.CHOICES_API_ERROR, + text: error.message, + }); + } + } + }, [actionConnector, http, fields, onSuccess, toastNotifications]); + + useEffect(() => { + fetchData(); + + return () => { + didCancel.current = true; + abortCtrl.current.abort(); + setIsLoading(false); + }; + // eslint-disable-next-line react-hooks/exhaustive-deps + }, []); + + return { + choices, + isLoading, + }; +}; diff --git a/x-pack/plugins/triggers_actions_ui/public/application/sections/action_connector_form/action_form.tsx b/x-pack/plugins/triggers_actions_ui/public/application/sections/action_connector_form/action_form.tsx index 2145443ba044c..d0621e44aebc4 100644 --- a/x-pack/plugins/triggers_actions_ui/public/application/sections/action_connector_form/action_form.tsx +++ b/x-pack/plugins/triggers_actions_ui/public/application/sections/action_connector_form/action_form.tsx @@ -34,7 +34,11 @@ import { ActionTypeForm, ActionTypeFormProps } from './action_type_form'; import { AddConnectorInline } from './connector_add_inline'; import { actionTypeCompare } from '../../lib/action_type_compare'; import { checkActionFormActionTypeEnabled } from '../../lib/check_action_type_enabled'; -import { VIEW_LICENSE_OPTIONS_LINK, DEFAULT_HIDDEN_ACTION_TYPES } from '../../../common/constants'; +import { + VIEW_LICENSE_OPTIONS_LINK, + DEFAULT_HIDDEN_ACTION_TYPES, + DEFAULT_HIDDEN_ONLY_ON_ALERTS_ACTION_TYPES, +} from '../../../common/constants'; import { ActionGroup, AlertActionParam } from '../../../../../alerts/common'; import { useKibana } from '../../../common/lib/kibana'; import { DefaultActionParamsGetter } from '../../lib/get_defaults_for_action_params'; @@ -230,9 +234,15 @@ export const ActionForm = ({ .list() /** * TODO: Remove when cases connector is available across Kibana. Issue: https://github.com/elastic/kibana/issues/82502. + * TODO: Need to decide about ServiceNow SIR connector. * If actionTypes are set, hidden connectors are filtered out. Otherwise, they are not. */ - .filter(({ id }) => actionTypes ?? !DEFAULT_HIDDEN_ACTION_TYPES.includes(id)) + .filter( + ({ id }) => + actionTypes ?? + (!DEFAULT_HIDDEN_ACTION_TYPES.includes(id) && + !DEFAULT_HIDDEN_ONLY_ON_ALERTS_ACTION_TYPES.includes(id)) + ) .filter((item) => actionTypesIndex[item.id]) .filter((item) => !!item.actionParamsFields) .sort((a, b) => diff --git a/x-pack/plugins/triggers_actions_ui/public/common/constants/index.ts b/x-pack/plugins/triggers_actions_ui/public/common/constants/index.ts index 833ed915fad59..8832f8b826eab 100644 --- a/x-pack/plugins/triggers_actions_ui/public/common/constants/index.ts +++ b/x-pack/plugins/triggers_actions_ui/public/common/constants/index.ts @@ -11,3 +11,5 @@ export { builtInGroupByTypes } from './group_by_types'; export const VIEW_LICENSE_OPTIONS_LINK = 'https://www.elastic.co/subscriptions'; // TODO: Remove when cases connector is available across Kibana. Issue: https://github.com/elastic/kibana/issues/82502. export const DEFAULT_HIDDEN_ACTION_TYPES = ['.case']; +// Action types included in this array will be hidden only from the alert's action type node list +export const DEFAULT_HIDDEN_ONLY_ON_ALERTS_ACTION_TYPES = ['.servicenow-sir']; diff --git a/x-pack/plugins/triggers_actions_ui/public/common/index.ts b/x-pack/plugins/triggers_actions_ui/public/common/index.ts index 86c33a373753f..c7f43fedb6f03 100644 --- a/x-pack/plugins/triggers_actions_ui/public/common/index.ts +++ b/x-pack/plugins/triggers_actions_ui/public/common/index.ts @@ -10,6 +10,6 @@ export * from './index_controls'; export * from './lib'; export * from './types'; -export { connectorConfiguration as ServiceNowConnectorConfiguration } from '../application/components/builtin_action_types/servicenow/config'; +export { serviceNowITSMConfiguration as ServiceNowITSMConnectorConfiguration } from '../application/components/builtin_action_types/servicenow/config'; export { connectorConfiguration as JiraConnectorConfiguration } from '../application/components/builtin_action_types/jira/config'; export { connectorConfiguration as ResilientConnectorConfiguration } from '../application/components/builtin_action_types/resilient/config'; diff --git a/x-pack/plugins/upgrade_assistant/common/version.ts b/x-pack/plugins/upgrade_assistant/common/version.ts deleted file mode 100644 index 231614dc38c6d..0000000000000 --- a/x-pack/plugins/upgrade_assistant/common/version.ts +++ /dev/null @@ -1,13 +0,0 @@ -/* - * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one - * or more contributor license agreements. Licensed under the Elastic License; - * you may not use this file except in compliance with the Elastic License. - */ - -import SemVer from 'semver/classes/semver'; -import pkg from '../../../../package.json'; - -export const CURRENT_VERSION = new SemVer(pkg.version as string); -export const CURRENT_MAJOR_VERSION = CURRENT_VERSION.major; -export const NEXT_MAJOR_VERSION = CURRENT_VERSION.major + 1; -export const PREV_MAJOR_VERSION = CURRENT_VERSION.major - 1; diff --git a/x-pack/plugins/upgrade_assistant/public/application/app.tsx b/x-pack/plugins/upgrade_assistant/public/application/app.tsx index 2b245bceceb6c..38e8b9d31d773 100644 --- a/x-pack/plugins/upgrade_assistant/public/application/app.tsx +++ b/x-pack/plugins/upgrade_assistant/public/application/app.tsx @@ -8,7 +8,6 @@ import React from 'react'; import { I18nStart } from 'src/core/public'; import { EuiPageHeader, EuiPageHeaderSection, EuiTitle } from '@elastic/eui'; import { FormattedMessage } from '@kbn/i18n/react'; -import { NEXT_MAJOR_VERSION } from '../../common/version'; import { UpgradeAssistantTabs } from './components/tabs'; import { AppContextProvider, ContextValue, AppContext } from './app_context'; @@ -17,6 +16,7 @@ export interface AppDependencies extends ContextValue { } export const RootComponent = ({ i18n, ...contextValue }: AppDependencies) => { + const { nextMajor } = contextValue.kibanaVersionInfo; return ( @@ -28,7 +28,7 @@ export const RootComponent = ({ i18n, ...contextValue }: AppDependencies) => { diff --git a/x-pack/plugins/upgrade_assistant/public/application/app_context.tsx b/x-pack/plugins/upgrade_assistant/public/application/app_context.tsx index 11c88a52ea24e..865f134713779 100644 --- a/x-pack/plugins/upgrade_assistant/public/application/app_context.tsx +++ b/x-pack/plugins/upgrade_assistant/public/application/app_context.tsx @@ -6,10 +6,17 @@ import { DocLinksStart, HttpSetup } from 'src/core/public'; import React, { createContext, useContext } from 'react'; +export interface KibanaVersionContext { + currentMajor: number; + prevMajor: number; + nextMajor: number; +} + export interface ContextValue { http: HttpSetup; isCloudEnabled: boolean; docLinks: DocLinksStart; + kibanaVersionInfo: KibanaVersionContext; } export const AppContext = createContext({} as any); diff --git a/x-pack/plugins/upgrade_assistant/public/application/components/latest_minor_banner.tsx b/x-pack/plugins/upgrade_assistant/public/application/components/latest_minor_banner.tsx index d9ec183231739..5d0df54dd532c 100644 --- a/x-pack/plugins/upgrade_assistant/public/application/components/latest_minor_banner.tsx +++ b/x-pack/plugins/upgrade_assistant/public/application/components/latest_minor_banner.tsx @@ -9,15 +9,16 @@ import React from 'react'; import { EuiCallOut, EuiLink } from '@elastic/eui'; import { FormattedMessage } from '@kbn/i18n/react'; -import { CURRENT_MAJOR_VERSION, NEXT_MAJOR_VERSION } from '../../../common/version'; import { useAppContext } from '../app_context'; export const LatestMinorBanner: React.FunctionComponent = () => { - const { docLinks } = useAppContext(); + const { docLinks, kibanaVersionInfo } = useAppContext(); const { ELASTIC_WEBSITE_URL } = docLinks; const esDocBasePath = `${ELASTIC_WEBSITE_URL}guide/en/elasticsearch/reference`; + const { currentMajor, nextMajor } = kibanaVersionInfo; + return ( { /> ), - nextEsVersion: `${NEXT_MAJOR_VERSION}.x`, - currentEsVersion: `${CURRENT_MAJOR_VERSION}.x`, + nextEsVersion: `${nextMajor}.x`, + currentEsVersion: `${currentMajor}.x`, }} />

diff --git a/x-pack/plugins/upgrade_assistant/public/application/components/tabs.test.tsx b/x-pack/plugins/upgrade_assistant/public/application/components/tabs.test.tsx index 6a99bd24ef26b..600e764afd32b 100644 --- a/x-pack/plugins/upgrade_assistant/public/application/components/tabs.test.tsx +++ b/x-pack/plugins/upgrade_assistant/public/application/components/tabs.test.tsx @@ -5,6 +5,7 @@ */ import React from 'react'; +import SemVer from 'semver/classes/semver'; import { mountWithIntl } from '@kbn/test/jest'; import { httpServiceMock } from 'src/core/public/mocks'; import { UpgradeAssistantTabs } from './tabs'; @@ -16,6 +17,7 @@ import { OverviewTab } from './tabs/overview'; const promisesToResolve = () => new Promise((resolve) => setTimeout(resolve, 0)); const mockHttp = httpServiceMock.createSetupContract(); +const mockKibanaVersion = new SemVer('8.0.0'); jest.mock('../app_context', () => { return { @@ -25,6 +27,11 @@ jest.mock('../app_context', () => { DOC_LINK_VERSION: 'current', ELASTIC_WEBSITE_URL: 'https://www.elastic.co/', }, + kibanaVersionInfo: { + currentMajor: mockKibanaVersion.major, + prevMajor: mockKibanaVersion.major - 1, + nextMajor: mockKibanaVersion.major + 1, + }, }; }, }; diff --git a/x-pack/plugins/upgrade_assistant/public/application/components/tabs/checkup/checkup_tab.test.tsx b/x-pack/plugins/upgrade_assistant/public/application/components/tabs/checkup/checkup_tab.test.tsx index 3a1e042a3aa5f..65dc9c25dacbd 100644 --- a/x-pack/plugins/upgrade_assistant/public/application/components/tabs/checkup/checkup_tab.test.tsx +++ b/x-pack/plugins/upgrade_assistant/public/application/components/tabs/checkup/checkup_tab.test.tsx @@ -6,6 +6,7 @@ import { shallow } from 'enzyme'; import React from 'react'; +import SemVer from 'semver/classes/semver'; import { LoadingState } from '../../types'; import AssistanceData from '../__fixtures__/checkup_api_response.json'; @@ -20,6 +21,8 @@ const defaultProps = { setSelectedTabIndex: jest.fn(), }; +const mockKibanaVersion = new SemVer('8.0.0'); + jest.mock('../../../app_context', () => { return { useAppContext: () => { @@ -28,6 +31,11 @@ jest.mock('../../../app_context', () => { DOC_LINK_VERSION: 'current', ELASTIC_WEBSITE_URL: 'https://www.elastic.co/', }, + kibanaVersionInfo: { + currentMajor: mockKibanaVersion.major, + prevMajor: mockKibanaVersion.major - 1, + nextMajor: mockKibanaVersion.major + 1, + }, }; }, }; diff --git a/x-pack/plugins/upgrade_assistant/public/application/components/tabs/checkup/checkup_tab.tsx b/x-pack/plugins/upgrade_assistant/public/application/components/tabs/checkup/checkup_tab.tsx index 02cbc87483e55..4fa4dafb55ff5 100644 --- a/x-pack/plugins/upgrade_assistant/public/application/components/tabs/checkup/checkup_tab.tsx +++ b/x-pack/plugins/upgrade_assistant/public/application/components/tabs/checkup/checkup_tab.tsx @@ -18,7 +18,6 @@ import { } from '@elastic/eui'; import { FormattedMessage } from '@kbn/i18n/react'; -import { NEXT_MAJOR_VERSION } from '../../../../../common/version'; import { LoadingErrorBanner } from '../../error_banner'; import { useAppContext } from '../../../app_context'; import { @@ -53,11 +52,13 @@ export const CheckupTab: FunctionComponent = ({ const [search, setSearch] = useState(''); const [currentGroupBy, setCurrentGroupBy] = useState(GroupByOption.message); - const { docLinks } = useAppContext(); + const { docLinks, kibanaVersionInfo } = useAppContext(); const { DOC_LINK_VERSION, ELASTIC_WEBSITE_URL } = docLinks; const esDocBasePath = `${ELASTIC_WEBSITE_URL}guide/en/elasticsearch/reference/${DOC_LINK_VERSION}`; + const { nextMajor } = kibanaVersionInfo; + const changeFilter = (filter: LevelFilterOption) => { setCurrentFilter(filter); }; @@ -99,7 +100,7 @@ export const CheckupTab: FunctionComponent = ({ defaultMessage="These {strongCheckupLabel} issues need your attention. Resolve them before upgrading to Elasticsearch {nextEsVersion}." values={{ strongCheckupLabel: {checkupLabel}, - nextEsVersion: `${NEXT_MAJOR_VERSION}.x`, + nextEsVersion: `${nextMajor}.x`, }} />

diff --git a/x-pack/plugins/upgrade_assistant/public/application/components/tabs/overview/index.tsx b/x-pack/plugins/upgrade_assistant/public/application/components/tabs/overview/index.tsx index 8789927167766..d7a30bf2e6a5e 100644 --- a/x-pack/plugins/upgrade_assistant/public/application/components/tabs/overview/index.tsx +++ b/x-pack/plugins/upgrade_assistant/public/application/components/tabs/overview/index.tsx @@ -4,7 +4,7 @@ * you may not use this file except in compliance with the Elastic License. */ -import React, { Fragment, FunctionComponent } from 'react'; +import React, { FunctionComponent } from 'react'; import { EuiFlexGroup, @@ -17,54 +17,59 @@ import { } from '@elastic/eui'; import { FormattedMessage } from '@kbn/i18n/react'; -import { NEXT_MAJOR_VERSION } from '../../../../../common/version'; +import { useAppContext } from '../../../app_context'; import { LoadingErrorBanner } from '../../error_banner'; import { LoadingState, UpgradeAssistantTabProps } from '../../types'; import { Steps } from './steps'; -export const OverviewTab: FunctionComponent = (props) => ( - - +export const OverviewTab: FunctionComponent = (props) => { + const { kibanaVersionInfo } = useAppContext(); + const { nextMajor } = kibanaVersionInfo; - -

- -

- + values={{ + nextEsVersion: `${nextMajor}.x`, + }} + /> +

+ - + - {props.alertBanner && ( - - {props.alertBanner} + {props.alertBanner && ( + <> + {props.alertBanner} - - - )} + + + )} - - - {props.loadingState === LoadingState.Success && } + + + {props.loadingState === LoadingState.Success && } - {props.loadingState === LoadingState.Loading && ( - - - - - - )} + {props.loadingState === LoadingState.Loading && ( + + + + + + )} - {props.loadingState === LoadingState.Error && ( - - )} - - - -); + {props.loadingState === LoadingState.Error && ( + + )} + + + + ); +}; diff --git a/x-pack/plugins/upgrade_assistant/public/application/components/tabs/overview/steps.tsx b/x-pack/plugins/upgrade_assistant/public/application/components/tabs/overview/steps.tsx index dd392f6d1b294..d81e709768065 100644 --- a/x-pack/plugins/upgrade_assistant/public/application/components/tabs/overview/steps.tsx +++ b/x-pack/plugins/upgrade_assistant/public/application/components/tabs/overview/steps.tsx @@ -19,22 +19,21 @@ import { import { i18n } from '@kbn/i18n'; import { FormattedMessage } from '@kbn/i18n/react'; -import { CURRENT_MAJOR_VERSION, NEXT_MAJOR_VERSION } from '../../../../../common/version'; import { UpgradeAssistantTabProps } from '../../types'; import { DeprecationLoggingToggle } from './deprecation_logging_toggle'; import { useAppContext } from '../../../app_context'; // Leaving these here even if unused so they are picked up for i18n static analysis // Keep this until last minor release (when next major is also released). -const WAIT_FOR_RELEASE_STEP = { +const WAIT_FOR_RELEASE_STEP = (majorVersion: number, nextMajorVersion: number) => ({ title: i18n.translate('xpack.upgradeAssistant.overviewTab.steps.waitForReleaseStep.stepTitle', { defaultMessage: 'Wait for the Elasticsearch {nextEsVersion} release', values: { - nextEsVersion: `${NEXT_MAJOR_VERSION}.0`, + nextEsVersion: `${nextMajorVersion}.0`, }, }), children: ( - + <>

- + ), -}; +}); // Swap in this step for the one above it on the last minor release. // @ts-ignore @@ -100,11 +99,13 @@ export const Steps: FunctionComponent = ({ }, {} as { [checkupType: string]: number }); // Uncomment when START_UPGRADE_STEP is in use! - const { docLinks, http /* , isCloudEnabled */ } = useAppContext(); + const { kibanaVersionInfo, docLinks, http /* , isCloudEnabled */ } = useAppContext(); const { DOC_LINK_VERSION, ELASTIC_WEBSITE_URL } = docLinks; const esDocBasePath = `${ELASTIC_WEBSITE_URL}guide/en/elasticsearch/reference/${DOC_LINK_VERSION}`; + const { currentMajor, nextMajor } = kibanaVersionInfo; + return ( = ({ /> ), - nextEsVersion: `${NEXT_MAJOR_VERSION}.0`, + nextEsVersion: `${nextMajor}.0`, }} />

@@ -278,7 +279,7 @@ export const Steps: FunctionComponent = ({ }, // Swap in START_UPGRADE_STEP on the last minor release. - WAIT_FOR_RELEASE_STEP, + WAIT_FOR_RELEASE_STEP(currentMajor, nextMajor), // START_UPGRADE_STEP(isCloudEnabled, esDocBasePath), ]} /> diff --git a/x-pack/plugins/upgrade_assistant/public/application/mount_management_section.ts b/x-pack/plugins/upgrade_assistant/public/application/mount_management_section.ts index c0124d52e45d7..906bf8f6c0709 100644 --- a/x-pack/plugins/upgrade_assistant/public/application/mount_management_section.ts +++ b/x-pack/plugins/upgrade_assistant/public/application/mount_management_section.ts @@ -6,11 +6,13 @@ import { CoreSetup } from 'src/core/public'; import { ManagementAppMountParams } from '../../../../../src/plugins/management/public'; import { renderApp } from './render_app'; +import { KibanaVersionContext } from './app_context'; export async function mountManagementSection( coreSetup: CoreSetup, isCloudEnabled: boolean, - params: ManagementAppMountParams + params: ManagementAppMountParams, + kibanaVersionInfo: KibanaVersionContext ) { const [{ i18n, docLinks }] = await coreSetup.getStartServices(); return renderApp({ @@ -19,5 +21,6 @@ export async function mountManagementSection( http: coreSetup.http, i18n, docLinks, + kibanaVersionInfo, }); } diff --git a/x-pack/plugins/upgrade_assistant/public/plugin.ts b/x-pack/plugins/upgrade_assistant/public/plugin.ts index 98f1b8351b88b..4bbb79cdd83b8 100644 --- a/x-pack/plugins/upgrade_assistant/public/plugin.ts +++ b/x-pack/plugins/upgrade_assistant/public/plugin.ts @@ -4,13 +4,13 @@ * you may not use this file except in compliance with the Elastic License. */ +import SemVer from 'semver/classes/semver'; import { i18n } from '@kbn/i18n'; import { Plugin, CoreSetup, PluginInitializerContext } from 'src/core/public'; import { CloudSetup } from '../../cloud/public'; import { ManagementSetup } from '../../../../src/plugins/management/public'; -import { NEXT_MAJOR_VERSION } from '../common/version'; import { Config } from '../common/config'; interface Dependencies { @@ -29,10 +29,17 @@ export class UpgradeAssistantUIPlugin implements Plugin { const appRegistrar = management.sections.section.stack; const isCloudEnabled = Boolean(cloud?.isCloudEnabled); + const kibanaVersion = new SemVer(this.ctx.env.packageInfo.version); + + const kibanaVersionInfo = { + currentMajor: kibanaVersion.major, + prevMajor: kibanaVersion.major - 1, + nextMajor: kibanaVersion.major + 1, + }; const pluginName = i18n.translate('xpack.upgradeAssistant.appTitle', { defaultMessage: '{version} Upgrade Assistant', - values: { version: `${NEXT_MAJOR_VERSION}.0` }, + values: { version: `${kibanaVersionInfo.nextMajor}.0` }, }); appRegistrar.registerApp({ @@ -47,8 +54,14 @@ export class UpgradeAssistantUIPlugin implements Plugin { } = coreStart; docTitle.change(pluginName); + const { mountManagementSection } = await import('./application/mount_management_section'); - const unmountAppCallback = await mountManagementSection(coreSetup, isCloudEnabled, params); + const unmountAppCallback = await mountManagementSection( + coreSetup, + isCloudEnabled, + params, + kibanaVersionInfo + ); return () => { docTitle.reset(); diff --git a/x-pack/plugins/upgrade_assistant/server/lib/__fixtures__/version.ts b/x-pack/plugins/upgrade_assistant/server/lib/__fixtures__/version.ts new file mode 100644 index 0000000000000..f08f449bbdae9 --- /dev/null +++ b/x-pack/plugins/upgrade_assistant/server/lib/__fixtures__/version.ts @@ -0,0 +1,20 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License; + * you may not use this file except in compliance with the Elastic License. + */ +import { SemVer } from 'semver'; + +export const MOCK_VERSION_STRING = '8.0.0'; + +export const getMockVersionInfo = (versionString = MOCK_VERSION_STRING) => { + const currentVersion = new SemVer(versionString); + const currentMajor = currentVersion.major; + + return { + currentVersion, + currentMajor, + prevMajor: currentMajor - 1, + nextMajor: currentMajor + 1, + }; +}; diff --git a/x-pack/plugins/upgrade_assistant/server/lib/es_version_precheck.test.ts b/x-pack/plugins/upgrade_assistant/server/lib/es_version_precheck.test.ts index 2310f993ce27d..74ec268d71e84 100644 --- a/x-pack/plugins/upgrade_assistant/server/lib/es_version_precheck.test.ts +++ b/x-pack/plugins/upgrade_assistant/server/lib/es_version_precheck.test.ts @@ -7,12 +7,16 @@ import { SemVer } from 'semver'; import { IScopedClusterClient, kibanaResponseFactory } from 'src/core/server'; import { xpackMocks } from '../../../../mocks'; -import { CURRENT_VERSION } from '../../common/version'; +import { MOCK_VERSION_STRING, getMockVersionInfo } from './__fixtures__/version'; + import { esVersionCheck, getAllNodeVersions, verifyAllMatchKibanaVersion, } from './es_version_precheck'; +import { versionService } from './version'; + +const { currentMajor, currentVersion } = getMockVersionInfo(); describe('getAllNodeVersions', () => { it('returns a list of unique node versions', async () => { @@ -41,25 +45,25 @@ describe('getAllNodeVersions', () => { describe('verifyAllMatchKibanaVersion', () => { it('detects higher version nodes', () => { - const result = verifyAllMatchKibanaVersion([new SemVer('99999.0.0')]); + const result = verifyAllMatchKibanaVersion([new SemVer('99999.0.0')], currentMajor); expect(result.allNodesMatch).toBe(false); expect(result.allNodesUpgraded).toBe(true); }); it('detects lower version nodes', () => { - const result = verifyAllMatchKibanaVersion([new SemVer('0.0.0')]); + const result = verifyAllMatchKibanaVersion([new SemVer('0.0.0')], currentMajor); expect(result.allNodesMatch).toBe(false); expect(result.allNodesUpgraded).toBe(true); }); it('detects if all are on same major correctly', () => { const versions = [ - CURRENT_VERSION, - CURRENT_VERSION.inc('minor'), - CURRENT_VERSION.inc('minor').inc('minor'), + currentVersion, + currentVersion.inc('minor'), + currentVersion.inc('minor').inc('minor'), ]; - const result = verifyAllMatchKibanaVersion(versions); + const result = verifyAllMatchKibanaVersion(versions, currentMajor); expect(result.allNodesMatch).toBe(true); expect(result.allNodesUpgraded).toBe(false); }); @@ -67,17 +71,21 @@ describe('verifyAllMatchKibanaVersion', () => { it('detects partial matches', () => { const versions = [ new SemVer('0.0.0'), - CURRENT_VERSION.inc('minor'), - CURRENT_VERSION.inc('minor').inc('minor'), + currentVersion.inc('minor'), + currentVersion.inc('minor').inc('minor'), ]; - const result = verifyAllMatchKibanaVersion(versions); + const result = verifyAllMatchKibanaVersion(versions, currentMajor); expect(result.allNodesMatch).toBe(false); expect(result.allNodesUpgraded).toBe(false); }); }); describe('EsVersionPrecheck', () => { + beforeEach(() => { + versionService.setup(MOCK_VERSION_STRING); + }); + it('returns a 403 when callCluster fails with a 403', async () => { const fakeCall = jest.fn().mockRejectedValue({ statusCode: 403 }); @@ -107,8 +115,8 @@ describe('EsVersionPrecheck', () => { info: jest.fn().mockResolvedValue({ body: { nodes: { - node1: { version: CURRENT_VERSION.raw }, - node2: { version: new SemVer(CURRENT_VERSION.raw).inc('major').raw }, + node1: { version: currentVersion.raw }, + node2: { version: new SemVer(currentVersion.raw).inc('major').raw }, }, }, }), @@ -132,8 +140,8 @@ describe('EsVersionPrecheck', () => { info: jest.fn().mockResolvedValue({ body: { nodes: { - node1: { version: new SemVer(CURRENT_VERSION.raw).inc('major').raw }, - node2: { version: new SemVer(CURRENT_VERSION.raw).inc('major').raw }, + node1: { version: new SemVer(currentVersion.raw).inc('major').raw }, + node2: { version: new SemVer(currentVersion.raw).inc('major').raw }, }, }, }), @@ -157,8 +165,8 @@ describe('EsVersionPrecheck', () => { info: jest.fn().mockResolvedValue({ body: { nodes: { - node1: { version: CURRENT_VERSION.raw }, - node2: { version: CURRENT_VERSION.raw }, + node1: { version: currentVersion.raw }, + node2: { version: currentVersion.raw }, }, }, }), diff --git a/x-pack/plugins/upgrade_assistant/server/lib/es_version_precheck.ts b/x-pack/plugins/upgrade_assistant/server/lib/es_version_precheck.ts index be6c4f5ff0230..c308334c6cb08 100644 --- a/x-pack/plugins/upgrade_assistant/server/lib/es_version_precheck.ts +++ b/x-pack/plugins/upgrade_assistant/server/lib/es_version_precheck.ts @@ -13,7 +13,7 @@ import { RequestHandler, RequestHandlerContext, } from 'src/core/server'; -import { CURRENT_VERSION } from '../../common/version'; +import { versionService } from './version'; interface Nodes { nodes: { @@ -39,14 +39,14 @@ export const getAllNodeVersions = async (adminClient: IScopedClusterClient) => { .map((version) => new SemVer(version)); }; -export const verifyAllMatchKibanaVersion = (allNodeVersions: SemVer[]) => { +export const verifyAllMatchKibanaVersion = (allNodeVersions: SemVer[], majorVersion: number) => { // Determine if all nodes in the cluster are running the same major version as Kibana. const numDifferentVersion = allNodeVersions.filter( - (esNodeVersion) => esNodeVersion.major !== CURRENT_VERSION.major + (esNodeVersion) => esNodeVersion.major !== majorVersion ).length; const numSameVersion = allNodeVersions.filter( - (esNodeVersion) => esNodeVersion.major === CURRENT_VERSION.major + (esNodeVersion) => esNodeVersion.major === majorVersion ).length; if (numDifferentVersion) { @@ -83,7 +83,9 @@ export const esVersionCheck = async ( throw e; } - const result = verifyAllMatchKibanaVersion(allNodeVersions); + const majorVersion = versionService.getMajorVersion(); + + const result = verifyAllMatchKibanaVersion(allNodeVersions, majorVersion); if (!result.allNodesMatch) { return response.customError({ // 426 means "Upgrade Required" and is used when semver compatibility is not met. diff --git a/x-pack/plugins/upgrade_assistant/server/lib/reindexing/index_settings.test.ts b/x-pack/plugins/upgrade_assistant/server/lib/reindexing/index_settings.test.ts index 9ec06b72f02e2..2111b77422f3e 100644 --- a/x-pack/plugins/upgrade_assistant/server/lib/reindexing/index_settings.test.ts +++ b/x-pack/plugins/upgrade_assistant/server/lib/reindexing/index_settings.test.ts @@ -4,7 +4,9 @@ * you may not use this file except in compliance with the Elastic License. */ -import { CURRENT_MAJOR_VERSION, PREV_MAJOR_VERSION } from '../../../common/version'; +import { versionService } from '../version'; +import { MOCK_VERSION_STRING, getMockVersionInfo } from '../__fixtures__/version'; + import { generateNewIndexName, getReindexWarnings, @@ -12,6 +14,8 @@ import { transformFlatSettings, } from './index_settings'; +const { currentMajor, prevMajor } = getMockVersionInfo(); + describe('transformFlatSettings', () => { it('does not blow up for empty mappings', () => { expect( @@ -56,6 +60,10 @@ describe('transformFlatSettings', () => { }); describe('sourceNameForIndex', () => { + beforeEach(() => { + versionService.setup(MOCK_VERSION_STRING); + }); + it('parses internal indices', () => { expect(sourceNameForIndex('.myInternalIndex')).toEqual('.myInternalIndex'); }); @@ -69,42 +77,46 @@ describe('sourceNameForIndex', () => { expect(sourceNameForIndex('.myInternalIndex-reindexed-v5')).toEqual('.myInternalIndex'); }); - it('replaces reindexed-v${PREV_MAJOR_VERSION} with reindexed-v${CURRENT_MAJOR_VERSION} in newIndexName', () => { - expect(sourceNameForIndex(`reindexed-v${PREV_MAJOR_VERSION}-myIndex`)).toEqual('myIndex'); - expect(sourceNameForIndex(`.reindexed-v${PREV_MAJOR_VERSION}-myInternalIndex`)).toEqual( + it(`replaces reindexed-v${prevMajor} with reindexed-v${currentMajor} in newIndexName`, () => { + expect(sourceNameForIndex(`reindexed-v${prevMajor}-myIndex`)).toEqual('myIndex'); + expect(sourceNameForIndex(`.reindexed-v${prevMajor}-myInternalIndex`)).toEqual( '.myInternalIndex' ); }); }); describe('generateNewIndexName', () => { + beforeEach(() => { + versionService.setup(MOCK_VERSION_STRING); + }); + it('parses internal indices', () => { expect(generateNewIndexName('.myInternalIndex')).toEqual( - `.reindexed-v${CURRENT_MAJOR_VERSION}-myInternalIndex` + `.reindexed-v${currentMajor}-myInternalIndex` ); }); it('parses non-internal indices', () => { - expect(generateNewIndexName('myIndex')).toEqual(`reindexed-v${CURRENT_MAJOR_VERSION}-myIndex`); + expect(generateNewIndexName('myIndex')).toEqual(`reindexed-v${currentMajor}-myIndex`); }); it('excludes appended v5 reindexing string from generateNewIndexName', () => { expect(generateNewIndexName('myIndex-reindexed-v5')).toEqual( - `reindexed-v${CURRENT_MAJOR_VERSION}-myIndex` + `reindexed-v${currentMajor}-myIndex` ); expect(generateNewIndexName('.myInternalIndex-reindexed-v5')).toEqual( - `.reindexed-v${CURRENT_MAJOR_VERSION}-myInternalIndex` + `.reindexed-v${currentMajor}-myInternalIndex` ); }); - it('replaces reindexed-v${PREV_MAJOR_VERSION} with reindexed-v${CURRENT_MAJOR_VERSION} in generateNewIndexName', () => { - expect(generateNewIndexName(`reindexed-v${PREV_MAJOR_VERSION}-myIndex`)).toEqual( - `reindexed-v${CURRENT_MAJOR_VERSION}-myIndex` + it(`replaces reindexed-v${prevMajor} with reindexed-v${currentMajor} in generateNewIndexName`, () => { + expect(generateNewIndexName(`reindexed-v${prevMajor}-myIndex`)).toEqual( + `reindexed-v${currentMajor}-myIndex` ); - expect(generateNewIndexName(`.reindexed-v${PREV_MAJOR_VERSION}-myInternalIndex`)).toEqual( - `.reindexed-v${CURRENT_MAJOR_VERSION}-myInternalIndex` + expect(generateNewIndexName(`.reindexed-v${prevMajor}-myInternalIndex`)).toEqual( + `.reindexed-v${currentMajor}-myInternalIndex` ); }); }); diff --git a/x-pack/plugins/upgrade_assistant/server/lib/reindexing/index_settings.ts b/x-pack/plugins/upgrade_assistant/server/lib/reindexing/index_settings.ts index 5722a6c29b68f..b632bbfa1faec 100644 --- a/x-pack/plugins/upgrade_assistant/server/lib/reindexing/index_settings.ts +++ b/x-pack/plugins/upgrade_assistant/server/lib/reindexing/index_settings.ts @@ -6,7 +6,7 @@ import { flow, omit } from 'lodash'; import { ReindexWarning } from '../../../common/types'; -import { CURRENT_MAJOR_VERSION, PREV_MAJOR_VERSION } from '../../../common/version'; +import { versionService } from '../version'; import { FlatSettings } from './types'; export interface ParsedIndexName { @@ -44,7 +44,10 @@ export const sourceNameForIndex = (indexName: string): string => { // in 5.6 the upgrade assistant appended to the index, in 6.7+ we prepend to // avoid conflicts with index patterns/templates/etc - const reindexedMatcher = new RegExp(`(-reindexed-v5$|reindexed-v${PREV_MAJOR_VERSION}-)`, 'g'); + const reindexedMatcher = new RegExp( + `(-reindexed-v5$|reindexed-v${versionService.getPrevMajorVersion()}-)`, + 'g' + ); const cleanBaseName = baseName.replace(reindexedMatcher, ''); return `${internal}${cleanBaseName}`; @@ -58,7 +61,7 @@ export const sourceNameForIndex = (indexName: string): string => { */ export const generateNewIndexName = (indexName: string): string => { const sourceName = sourceNameForIndex(indexName); - const currentVersion = `reindexed-v${CURRENT_MAJOR_VERSION}`; + const currentVersion = `reindexed-v${versionService.getMajorVersion()}`; return indexName.startsWith('.') ? `.${currentVersion}-${sourceName.substr(1)}` diff --git a/x-pack/plugins/upgrade_assistant/server/lib/reindexing/reindex_actions.test.ts b/x-pack/plugins/upgrade_assistant/server/lib/reindexing/reindex_actions.test.ts index d059c03bcecb1..9a6ac4030e051 100644 --- a/x-pack/plugins/upgrade_assistant/server/lib/reindexing/reindex_actions.test.ts +++ b/x-pack/plugins/upgrade_assistant/server/lib/reindexing/reindex_actions.test.ts @@ -17,8 +17,11 @@ import { ReindexStatus, ReindexStep, } from '../../../common/types'; -import { CURRENT_MAJOR_VERSION, PREV_MAJOR_VERSION } from '../../../common/version'; +import { versionService } from '../version'; import { LOCK_WINDOW, ReindexActions, reindexActionsFactory } from './reindex_actions'; +import { MOCK_VERSION_STRING, getMockVersionInfo } from '../__fixtures__/version'; + +const { currentMajor, prevMajor } = getMockVersionInfo(); describe('ReindexActions', () => { let client: jest.Mocked; @@ -47,13 +50,16 @@ describe('ReindexActions', () => { }); describe('createReindexOp', () => { - beforeEach(() => client.create.mockResolvedValue()); + beforeEach(() => { + versionService.setup(MOCK_VERSION_STRING); + client.create.mockResolvedValue(); + }); - it(`prepends reindexed-v${CURRENT_MAJOR_VERSION} to new name`, async () => { + it(`prepends reindexed-v${currentMajor} to new name`, async () => { await actions.createReindexOp('myIndex'); expect(client.create).toHaveBeenCalledWith(REINDEX_OP_TYPE, { indexName: 'myIndex', - newIndexName: `reindexed-v${CURRENT_MAJOR_VERSION}-myIndex`, + newIndexName: `reindexed-v${currentMajor}-myIndex`, reindexOptions: undefined, status: ReindexStatus.inProgress, lastCompletedStep: ReindexStep.created, @@ -65,11 +71,11 @@ describe('ReindexActions', () => { }); }); - it(`prepends reindexed-v${CURRENT_MAJOR_VERSION} to new name, preserving leading period`, async () => { + it(`prepends reindexed-v${currentMajor} to new name, preserving leading period`, async () => { await actions.createReindexOp('.internalIndex'); expect(client.create).toHaveBeenCalledWith(REINDEX_OP_TYPE, { indexName: '.internalIndex', - newIndexName: `.reindexed-v${CURRENT_MAJOR_VERSION}-internalIndex`, + newIndexName: `.reindexed-v${currentMajor}-internalIndex`, reindexOptions: undefined, status: ReindexStatus.inProgress, lastCompletedStep: ReindexStep.created, @@ -82,12 +88,12 @@ describe('ReindexActions', () => { }); // in v5.6, the upgrade assistant appended to the index name instead of prepending - it(`prepends reindexed-v${CURRENT_MAJOR_VERSION}- and removes reindex appended in v5`, async () => { + it(`prepends reindexed-v${currentMajor}- and removes reindex appended in v5`, async () => { const indexName = 'myIndex-reindexed-v5'; await actions.createReindexOp(indexName); expect(client.create).toHaveBeenCalledWith(REINDEX_OP_TYPE, { indexName, - newIndexName: `reindexed-v${CURRENT_MAJOR_VERSION}-myIndex`, + newIndexName: `reindexed-v${currentMajor}-myIndex`, reindexOptions: undefined, status: ReindexStatus.inProgress, lastCompletedStep: ReindexStep.created, @@ -99,11 +105,11 @@ describe('ReindexActions', () => { }); }); - it(`replaces reindexed-v${PREV_MAJOR_VERSION} with reindexed-v${CURRENT_MAJOR_VERSION}`, async () => { - await actions.createReindexOp(`reindexed-v${PREV_MAJOR_VERSION}-myIndex`); + it(`replaces reindexed-v${prevMajor} with reindexed-v${currentMajor}`, async () => { + await actions.createReindexOp(`reindexed-v${prevMajor}-myIndex`); expect(client.create).toHaveBeenCalledWith(REINDEX_OP_TYPE, { - indexName: `reindexed-v${PREV_MAJOR_VERSION}-myIndex`, - newIndexName: `reindexed-v${CURRENT_MAJOR_VERSION}-myIndex`, + indexName: `reindexed-v${prevMajor}-myIndex`, + newIndexName: `reindexed-v${currentMajor}-myIndex`, reindexOptions: undefined, status: ReindexStatus.inProgress, lastCompletedStep: ReindexStep.created, @@ -291,7 +297,7 @@ describe('ReindexActions', () => { } as RequestEvent); it('returns flat settings', async () => { - clusterClient.asCurrentUser.indices.getSettings.mockResolvedValueOnce( + clusterClient.asCurrentUser.indices.get.mockResolvedValueOnce( asApiResponse({ myIndex: { settings: { 'index.mySetting': '1' }, @@ -306,7 +312,7 @@ describe('ReindexActions', () => { }); it('returns null if index does not exist', async () => { - clusterClient.asCurrentUser.indices.getSettings.mockResolvedValueOnce(asApiResponse({})); + clusterClient.asCurrentUser.indices.get.mockResolvedValueOnce(asApiResponse({})); await expect(actions.getFlatSettings('myIndex')).resolves.toBeNull(); }); }); diff --git a/x-pack/plugins/upgrade_assistant/server/lib/reindexing/reindex_actions.ts b/x-pack/plugins/upgrade_assistant/server/lib/reindexing/reindex_actions.ts index 611ab3c92b72b..653bf8336255b 100644 --- a/x-pack/plugins/upgrade_assistant/server/lib/reindexing/reindex_actions.ts +++ b/x-pack/plugins/upgrade_assistant/server/lib/reindexing/reindex_actions.ts @@ -236,7 +236,7 @@ export const reindexActionsFactory = ( }, async getFlatSettings(indexName: string) { - const { body: flatSettings } = await esClient.indices.getSettings<{ + const { body: flatSettings } = await esClient.indices.get<{ [indexName: string]: FlatSettings; }>({ index: indexName, diff --git a/x-pack/plugins/upgrade_assistant/server/lib/reindexing/reindex_service.test.ts b/x-pack/plugins/upgrade_assistant/server/lib/reindexing/reindex_service.test.ts index 8a7033c1594da..29c8207a5f284 100644 --- a/x-pack/plugins/upgrade_assistant/server/lib/reindexing/reindex_service.test.ts +++ b/x-pack/plugins/upgrade_assistant/server/lib/reindexing/reindex_service.test.ts @@ -18,11 +18,12 @@ import { ReindexStatus, ReindexStep, } from '../../../common/types'; -import { CURRENT_MAJOR_VERSION, PREV_MAJOR_VERSION } from '../../../common/version'; import { licensingMock } from '../../../../licensing/server/mocks'; import { LicensingPluginSetup } from '../../../../licensing/server'; +import { MOCK_VERSION_STRING, getMockVersionInfo } from '../__fixtures__/version'; import { esIndicesStateCheck } from '../es_indices_state_check'; +import { versionService } from '../version'; import { isMlIndex, @@ -36,6 +37,8 @@ const asApiResponse = (body: T): RequestEvent => body, } as RequestEvent); +const { currentMajor, prevMajor } = getMockVersionInfo(); + describe('reindexService', () => { let actions: jest.Mocked; let clusterClient: ScopedClusterClientMock; @@ -82,6 +85,8 @@ describe('reindexService', () => { log, licensingPluginSetup ); + + versionService.setup(MOCK_VERSION_STRING); }); describe('hasRequiredPrivileges', () => { @@ -107,7 +112,7 @@ describe('reindexService', () => { cluster: ['manage'], index: [ { - names: ['anIndex', `reindexed-v${CURRENT_MAJOR_VERSION}-anIndex`], + names: ['anIndex', `reindexed-v${currentMajor}-anIndex`], allow_restricted_indices: true, privileges: ['all'], }, @@ -131,7 +136,7 @@ describe('reindexService', () => { cluster: ['manage', 'manage_ml'], index: [ { - names: ['.ml-anomalies', `.reindexed-v${CURRENT_MAJOR_VERSION}-ml-anomalies`], + names: ['.ml-anomalies', `.reindexed-v${currentMajor}-ml-anomalies`], allow_restricted_indices: true, privileges: ['all'], }, @@ -149,9 +154,7 @@ describe('reindexService', () => { asApiResponse({ has_all_requested: true }) ); - const hasRequired = await service.hasRequiredPrivileges( - `reindexed-v${PREV_MAJOR_VERSION}-anIndex` - ); + const hasRequired = await service.hasRequiredPrivileges(`reindexed-v${prevMajor}-anIndex`); expect(hasRequired).toBe(true); expect(clusterClient.asCurrentUser.security.hasPrivileges).toHaveBeenCalledWith({ body: { @@ -159,8 +162,8 @@ describe('reindexService', () => { index: [ { names: [ - `reindexed-v${PREV_MAJOR_VERSION}-anIndex`, - `reindexed-v${CURRENT_MAJOR_VERSION}-anIndex`, + `reindexed-v${prevMajor}-anIndex`, + `reindexed-v${currentMajor}-anIndex`, 'anIndex', ], allow_restricted_indices: true, @@ -188,7 +191,7 @@ describe('reindexService', () => { cluster: ['manage', 'manage_watcher'], index: [ { - names: ['.watches', `.reindexed-v${CURRENT_MAJOR_VERSION}-watches`], + names: ['.watches', `.reindexed-v${currentMajor}-watches`], allow_restricted_indices: true, privileges: ['all'], }, @@ -497,9 +500,9 @@ describe('reindexService', () => { }); it('is true for ML re-indexed indices', () => { - expect(isMlIndex(`.reindexed-v${PREV_MAJOR_VERSION}-ml-state`)).toBe(true); - expect(isMlIndex(`.reindexed-v${PREV_MAJOR_VERSION}-ml-anomalies`)).toBe(true); - expect(isMlIndex(`.reindexed-v${PREV_MAJOR_VERSION}-ml-config`)).toBe(true); + expect(isMlIndex(`.reindexed-v${prevMajor}-ml-state`)).toBe(true); + expect(isMlIndex(`.reindexed-v${prevMajor}-ml-anomalies`)).toBe(true); + expect(isMlIndex(`.reindexed-v${prevMajor}-ml-config`)).toBe(true); }); }); @@ -514,8 +517,8 @@ describe('reindexService', () => { }); it('is true for watcher re-indexed indices', () => { - expect(isWatcherIndex(`.reindexed-v${PREV_MAJOR_VERSION}-watches`)).toBe(true); - expect(isWatcherIndex(`.reindexed-v${PREV_MAJOR_VERSION}-triggered-watches`)).toBe(true); + expect(isWatcherIndex(`.reindexed-v${prevMajor}-watches`)).toBe(true); + expect(isWatcherIndex(`.reindexed-v${prevMajor}-triggered-watches`)).toBe(true); }); }); @@ -829,7 +832,7 @@ describe('reindexService', () => { }); it('fails if create index is not acknowledged', async () => { - clusterClient.asCurrentUser.indices.getSettings.mockResolvedValueOnce( + clusterClient.asCurrentUser.indices.get.mockResolvedValueOnce( asApiResponse({ myIndex: settingsMappings }) ); @@ -844,7 +847,7 @@ describe('reindexService', () => { }); it('fails if create index fails', async () => { - clusterClient.asCurrentUser.indices.getSettings.mockResolvedValueOnce( + clusterClient.asCurrentUser.indices.get.mockResolvedValueOnce( asApiResponse({ myIndex: settingsMappings }) ); diff --git a/x-pack/plugins/upgrade_assistant/server/lib/version.ts b/x-pack/plugins/upgrade_assistant/server/lib/version.ts new file mode 100644 index 0000000000000..f33200d215638 --- /dev/null +++ b/x-pack/plugins/upgrade_assistant/server/lib/version.ts @@ -0,0 +1,33 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License; + * you may not use this file except in compliance with the Elastic License. + */ + +import SemVer from 'semver/classes/semver'; + +export class Version { + private version!: SemVer; + + public setup(version: string) { + this.version = new SemVer(version); + } + + public getCurrentVersion() { + return this.version; + } + + public getMajorVersion() { + return this.version?.major; + } + + public getNextMajorVersion() { + return this.version?.major + 1; + } + + public getPrevMajorVersion() { + return this.version?.major - 1; + } +} + +export const versionService = new Version(); diff --git a/x-pack/plugins/upgrade_assistant/server/plugin.ts b/x-pack/plugins/upgrade_assistant/server/plugin.ts index 9ef0f250da8ef..ea3677d01423b 100644 --- a/x-pack/plugins/upgrade_assistant/server/plugin.ts +++ b/x-pack/plugins/upgrade_assistant/server/plugin.ts @@ -22,6 +22,7 @@ import { LicensingPluginSetup } from '../../licensing/server'; import { CredentialStore, credentialStoreFactory } from './lib/reindexing/credential_store'; import { ReindexWorker } from './lib/reindexing'; import { registerUpgradeAssistantUsageCollector } from './lib/telemetry'; +import { versionService } from './lib/version'; import { registerClusterCheckupRoutes } from './routes/cluster_checkup'; import { registerDeprecationLoggingRoutes } from './routes/deprecation_logging'; import { registerReindexIndicesRoutes, createReindexWorker } from './routes/reindex_indices'; @@ -40,6 +41,7 @@ interface PluginsSetup { export class UpgradeAssistantServerPlugin implements Plugin { private readonly logger: Logger; private readonly credentialStore: CredentialStore; + private readonly kibanaVersion: string; // Properties set at setup private licensing?: LicensingPluginSetup; @@ -48,9 +50,10 @@ export class UpgradeAssistantServerPlugin implements Plugin { private savedObjectsServiceStart?: SavedObjectsServiceStart; private worker?: ReindexWorker; - constructor({ logger }: PluginInitializerContext) { + constructor({ logger, env }: PluginInitializerContext) { this.logger = logger.get(); this.credentialStore = credentialStoreFactory(); + this.kibanaVersion = env.packageInfo.version; } private getWorker() { @@ -98,6 +101,9 @@ export class UpgradeAssistantServerPlugin implements Plugin { licensing, }; + // Initialize version service with current kibana version + versionService.setup(this.kibanaVersion); + registerClusterCheckupRoutes(dependencies); registerDeprecationLoggingRoutes(dependencies); registerReindexIndicesRoutes(dependencies, this.getWorker.bind(this)); diff --git a/x-pack/plugins/uptime/public/components/monitor/ping_list/columns/ping_timestamp.tsx b/x-pack/plugins/uptime/public/components/monitor/ping_list/columns/ping_timestamp.tsx deleted file mode 100644 index be4f0fc62271d..0000000000000 --- a/x-pack/plugins/uptime/public/components/monitor/ping_list/columns/ping_timestamp.tsx +++ /dev/null @@ -1,219 +0,0 @@ -/* - * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one - * or more contributor license agreements. Licensed under the Elastic License; - * you may not use this file except in compliance with the Elastic License. - */ - -import React, { useContext, useEffect, useState } from 'react'; -import { - EuiButtonIcon, - EuiFlexGroup, - EuiFlexItem, - EuiImage, - EuiSpacer, - EuiText, - EuiLoadingSpinner, -} from '@elastic/eui'; -import useIntersection from 'react-use/lib/useIntersection'; -import moment from 'moment'; -import styled from 'styled-components'; -import { FormattedMessage } from '@kbn/i18n/react'; -import { Ping } from '../../../../../common/runtime_types/ping'; -import { getShortTimeStamp } from '../../../overview/monitor_list/columns/monitor_status_column'; -import { euiStyled } from '../../../../../../../../src/plugins/kibana_react/common'; -import { useFetcher, FETCH_STATUS } from '../../../../../../observability/public'; -import { getJourneyScreenshot } from '../../../../state/api/journey'; -import { UptimeSettingsContext } from '../../../../contexts'; - -const StepImage = styled(EuiImage)` - &&& { - display: flex; - figcaption { - white-space: nowrap; - align-self: center; - margin-left: 8px; - margin-top: 8px; - text-decoration: none !important; - } - } -`; - -const StepDiv = styled.div` - figure.euiImage { - div.stepArrowsFullScreen { - display: none; - } - } - - figure.euiImage-isFullScreen { - div.stepArrowsFullScreen { - display: flex; - } - } - position: relative; - div.stepArrows { - display: none; - } - :hover { - div.stepArrows { - display: flex; - } - } -`; - -interface Props { - timestamp: string; - ping: Ping; -} - -export const PingTimestamp = ({ timestamp, ping }: Props) => { - const [stepNo, setStepNo] = useState(1); - - const [stepImages, setStepImages] = useState([]); - - const intersectionRef = React.useRef(null); - - const { basePath } = useContext(UptimeSettingsContext); - - const imgPath = basePath + `/api/uptime/journey/screenshot/${ping.monitor.check_group}/${stepNo}`; - - const intersection = useIntersection(intersectionRef, { - root: null, - rootMargin: '0px', - threshold: 1, - }); - - const { data, status } = useFetcher(() => { - if (intersection && intersection.intersectionRatio === 1 && !stepImages[stepNo - 1]) - return getJourneyScreenshot(imgPath); - }, [intersection?.intersectionRatio, stepNo]); - - useEffect(() => { - if (data) { - setStepImages((prevState) => [...prevState, data?.src]); - } - }, [data]); - - const imgSrc = stepImages[stepNo] || data?.src; - - const isLoading = status === FETCH_STATUS.LOADING; - const isPending = status === FETCH_STATUS.PENDING; - - const captionContent = `Step:${stepNo} ${data?.stepName}`; - - const ImageCaption = ( - <> -
- {imgSrc && ( - - - { - setStepNo(stepNo - 1); - }} - iconType="arrowLeft" - aria-label="Next" - /> - - - {captionContent} - - - { - setStepNo(stepNo + 1); - }} - iconType="arrowRight" - aria-label="Next" - /> - - - )} -
- {/* TODO: Add link to details page once it's available */} - {getShortTimeStamp(moment(timestamp))} - - - ); - - return ( - - {imgSrc ? ( - - ) : ( - - - {isLoading || isPending ? ( - - ) : ( - - )} - - {ImageCaption} - - )} - - - { - setStepNo(stepNo - 1); - }} - iconType="arrowLeft" - aria-label="Next" - /> - - - { - setStepNo(stepNo + 1); - }} - iconType="arrowRight" - aria-label="Next" - /> - - - - ); -}; - -const BorderedText = euiStyled(EuiText)` - width: 120px; - text-align: center; - border: 1px solid ${(props) => props.theme.eui.euiColorLightShade}; -`; - -export const NoImageAvailable = () => { - return ( - - - - - - ); -}; diff --git a/x-pack/plugins/uptime/public/components/monitor/ping_list/columns/ping_timestamp/index.ts b/x-pack/plugins/uptime/public/components/monitor/ping_list/columns/ping_timestamp/index.ts new file mode 100644 index 0000000000000..db9c18e30cfc1 --- /dev/null +++ b/x-pack/plugins/uptime/public/components/monitor/ping_list/columns/ping_timestamp/index.ts @@ -0,0 +1,7 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License; + * you may not use this file except in compliance with the Elastic License. + */ + +export { PingTimestamp } from './ping_timestamp'; diff --git a/x-pack/plugins/uptime/public/components/monitor/ping_list/columns/ping_timestamp/nav_buttons.test.tsx b/x-pack/plugins/uptime/public/components/monitor/ping_list/columns/ping_timestamp/nav_buttons.test.tsx new file mode 100644 index 0000000000000..c8acfd48a9913 --- /dev/null +++ b/x-pack/plugins/uptime/public/components/monitor/ping_list/columns/ping_timestamp/nav_buttons.test.tsx @@ -0,0 +1,88 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License; + * you may not use this file except in compliance with the Elastic License. + */ + +import { fireEvent, waitFor } from '@testing-library/react'; +import React from 'react'; +import { render } from '../../../../../lib/helper/rtl_helpers'; +import { NavButtons, NavButtonsProps } from './nav_buttons'; + +describe('NavButtons', () => { + let defaultProps: NavButtonsProps; + + beforeEach(() => { + defaultProps = { + maxSteps: 3, + stepNumber: 2, + setStepNumber: jest.fn(), + setIsImagePopoverOpen: jest.fn(), + }; + }); + + it('labels prev and next buttons', () => { + const { getByLabelText } = render(); + + expect(getByLabelText('Previous step')); + expect(getByLabelText('Next step')); + }); + + it('increments step number on next click', async () => { + const { getByLabelText } = render(); + + const nextButton = getByLabelText('Next step'); + + fireEvent.click(nextButton); + + await waitFor(() => { + expect(defaultProps.setStepNumber).toHaveBeenCalledTimes(1); + expect(defaultProps.setStepNumber).toHaveBeenCalledWith(3); + }); + }); + + it('decrements step number on prev click', async () => { + const { getByLabelText } = render(); + + const nextButton = getByLabelText('Previous step'); + + fireEvent.click(nextButton); + + await waitFor(() => { + expect(defaultProps.setStepNumber).toHaveBeenCalledTimes(1); + expect(defaultProps.setStepNumber).toHaveBeenCalledWith(1); + }); + }); + + it('disables `next` button on final step', () => { + defaultProps.stepNumber = 3; + + const { getByLabelText } = render(); + + // getByLabelText('Next step'); + expect(getByLabelText('Next step')).toHaveAttribute('disabled'); + expect(getByLabelText('Previous step')).not.toHaveAttribute('disabled'); + }); + + it('disables `prev` button on final step', () => { + defaultProps.stepNumber = 1; + + const { getByLabelText } = render(); + + expect(getByLabelText('Next step')).not.toHaveAttribute('disabled'); + expect(getByLabelText('Previous step')).toHaveAttribute('disabled'); + }); + + it('opens popover when mouse enters', async () => { + const { getByLabelText } = render(); + + const nextButton = getByLabelText('Next step'); + + fireEvent.mouseEnter(nextButton); + + await waitFor(() => { + expect(defaultProps.setIsImagePopoverOpen).toHaveBeenCalledTimes(1); + expect(defaultProps.setIsImagePopoverOpen).toHaveBeenCalledWith(true); + }); + }); +}); diff --git a/x-pack/plugins/uptime/public/components/monitor/ping_list/columns/ping_timestamp/nav_buttons.tsx b/x-pack/plugins/uptime/public/components/monitor/ping_list/columns/ping_timestamp/nav_buttons.tsx new file mode 100644 index 0000000000000..1c24caba6a917 --- /dev/null +++ b/x-pack/plugins/uptime/public/components/monitor/ping_list/columns/ping_timestamp/nav_buttons.tsx @@ -0,0 +1,56 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License; + * you may not use this file except in compliance with the Elastic License. + */ + +import { EuiButtonIcon, EuiFlexItem, EuiFlexGroup } from '@elastic/eui'; +import React from 'react'; +import { nextAriaLabel, prevAriaLabel } from './translations'; + +export interface NavButtonsProps { + maxSteps?: number; + setIsImagePopoverOpen: React.Dispatch>; + setStepNumber: React.Dispatch>; + stepNumber: number; +} + +export const NavButtons: React.FC = ({ + maxSteps, + setIsImagePopoverOpen, + setStepNumber, + stepNumber, +}) => ( + setIsImagePopoverOpen(true)} + style={{ position: 'absolute', bottom: 0, left: 30 }} + > + + { + setStepNumber(stepNumber - 1); + }} + iconType="arrowLeft" + aria-label={prevAriaLabel} + /> + + + { + setStepNumber(stepNumber + 1); + }} + iconType="arrowRight" + aria-label={nextAriaLabel} + /> + + +); diff --git a/x-pack/plugins/uptime/public/components/monitor/ping_list/columns/ping_timestamp/no_image_available.test.tsx b/x-pack/plugins/uptime/public/components/monitor/ping_list/columns/ping_timestamp/no_image_available.test.tsx new file mode 100644 index 0000000000000..17e679846a66d --- /dev/null +++ b/x-pack/plugins/uptime/public/components/monitor/ping_list/columns/ping_timestamp/no_image_available.test.tsx @@ -0,0 +1,17 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License; + * you may not use this file except in compliance with the Elastic License. + */ + +import React from 'react'; +import { render } from '../../../../../lib/helper/rtl_helpers'; +import { NoImageAvailable } from './no_image_available'; + +describe('NoImageAvailable', () => { + it('renders expected text', () => { + const { getByText } = render(); + + expect(getByText('No image available')); + }); +}); diff --git a/x-pack/plugins/uptime/public/components/monitor/ping_list/columns/ping_timestamp/no_image_available.tsx b/x-pack/plugins/uptime/public/components/monitor/ping_list/columns/ping_timestamp/no_image_available.tsx new file mode 100644 index 0000000000000..2498e07969f11 --- /dev/null +++ b/x-pack/plugins/uptime/public/components/monitor/ping_list/columns/ping_timestamp/no_image_available.tsx @@ -0,0 +1,29 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License; + * you may not use this file except in compliance with the Elastic License. + */ + +import { EuiText } from '@elastic/eui'; +import { FormattedMessage } from '@kbn/i18n/react'; +import React from 'react'; +import { euiStyled } from '../../../../../../../../../src/plugins/kibana_react/common'; + +const BorderedText = euiStyled(EuiText)` + width: 120px; + text-align: center; + border: 1px solid ${(props) => props.theme.eui.euiColorLightShade}; +`; + +export const NoImageAvailable = () => { + return ( + + + + + + ); +}; diff --git a/x-pack/plugins/uptime/public/components/monitor/ping_list/columns/ping_timestamp/no_image_display.test.tsx b/x-pack/plugins/uptime/public/components/monitor/ping_list/columns/ping_timestamp/no_image_display.test.tsx new file mode 100644 index 0000000000000..24080e2f4061d --- /dev/null +++ b/x-pack/plugins/uptime/public/components/monitor/ping_list/columns/ping_timestamp/no_image_display.test.tsx @@ -0,0 +1,44 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License; + * you may not use this file except in compliance with the Elastic License. + */ + +import React from 'react'; +import { render } from '../../../../../lib/helper/rtl_helpers'; +import { NoImageDisplay, NoImageDisplayProps } from './no_image_display'; +import { imageLoadingSpinnerAriaLabel } from './translations'; + +describe('NoImageDisplay', () => { + let defaultProps: NoImageDisplayProps; + beforeEach(() => { + defaultProps = { + imageCaption:
test caption
, + isLoading: false, + isPending: false, + }; + }); + + it('renders a loading spinner for loading state', () => { + defaultProps.isLoading = true; + const { getByText, getByLabelText } = render(); + + expect(getByLabelText(imageLoadingSpinnerAriaLabel)); + expect(getByText('test caption')); + }); + + it('renders a loading spinner for pending state', () => { + defaultProps.isPending = true; + const { getByText, getByLabelText } = render(); + + expect(getByLabelText(imageLoadingSpinnerAriaLabel)); + expect(getByText('test caption')); + }); + + it('renders no image available when not loading or pending', () => { + const { getByText } = render(); + + expect(getByText('No image available')); + expect(getByText('test caption')); + }); +}); diff --git a/x-pack/plugins/uptime/public/components/monitor/ping_list/columns/ping_timestamp/no_image_display.tsx b/x-pack/plugins/uptime/public/components/monitor/ping_list/columns/ping_timestamp/no_image_display.tsx new file mode 100644 index 0000000000000..185f488d5acd2 --- /dev/null +++ b/x-pack/plugins/uptime/public/components/monitor/ping_list/columns/ping_timestamp/no_image_display.tsx @@ -0,0 +1,39 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License; + * you may not use this file except in compliance with the Elastic License. + */ + +import { EuiFlexItem, EuiFlexGroup, EuiLoadingSpinner } from '@elastic/eui'; +import React from 'react'; +import { NoImageAvailable } from './no_image_available'; +import { imageLoadingSpinnerAriaLabel } from './translations'; + +export interface NoImageDisplayProps { + imageCaption: JSX.Element; + isLoading: boolean; + isPending: boolean; +} + +export const NoImageDisplay: React.FC = ({ + imageCaption, + isLoading, + isPending, +}) => { + return ( + + + {isLoading || isPending ? ( + + ) : ( + + )} + + {imageCaption} + + ); +}; diff --git a/x-pack/plugins/uptime/public/components/monitor/ping_list/columns/ping_timestamp.test.tsx b/x-pack/plugins/uptime/public/components/monitor/ping_list/columns/ping_timestamp/ping_timestamp.test.tsx similarity index 70% rename from x-pack/plugins/uptime/public/components/monitor/ping_list/columns/ping_timestamp.test.tsx rename to x-pack/plugins/uptime/public/components/monitor/ping_list/columns/ping_timestamp/ping_timestamp.test.tsx index 1baeb8a69d34c..a934f6fa39b22 100644 --- a/x-pack/plugins/uptime/public/components/monitor/ping_list/columns/ping_timestamp.test.tsx +++ b/x-pack/plugins/uptime/public/components/monitor/ping_list/columns/ping_timestamp/ping_timestamp.test.tsx @@ -5,16 +5,17 @@ */ import React from 'react'; +import { fireEvent, waitFor } from '@testing-library/react'; import { PingTimestamp } from './ping_timestamp'; -import { mockReduxHooks } from '../../../../lib/helper/test_helpers'; -import { render } from '../../../../lib/helper/rtl_helpers'; -import { Ping } from '../../../../../common/runtime_types/ping'; -import * as observabilityPublic from '../../../../../../observability/public'; +import { mockReduxHooks } from '../../../../../lib/helper/test_helpers'; +import { render } from '../../../../../lib/helper/rtl_helpers'; +import { Ping } from '../../../../../../common/runtime_types/ping'; +import * as observabilityPublic from '../../../../../../../observability/public'; mockReduxHooks(); -jest.mock('../../../../../../observability/public', () => { - const originalModule = jest.requireActual('../../../../../../observability/public'); +jest.mock('../../../../../../../observability/public', () => { + const originalModule = jest.requireActual('../../../../../../../observability/public'); return { ...originalModule, @@ -92,4 +93,26 @@ describe('Ping Timestamp component', () => { const { container } = render(); expect(container.querySelector('img')?.src).toBe(src); }); + + it('displays popover image when mouse enters img caption, and hides onLeave', async () => { + const src = 'http://sample.com/sampleImageSrc.png'; + jest.spyOn(observabilityPublic, 'useFetcher').mockReturnValue({ + status: FETCH_STATUS.SUCCESS, + data: { src }, + refetch: () => null, + }); + const { getByAltText, getByText, queryByAltText } = render( + + ); + const caption = getByText('Nov 26, 2020 10:28:56 AM'); + fireEvent.mouseEnter(caption); + + const altText = `A larger version of the screenshot for this journey step's thumbnail.`; + + await waitFor(() => getByAltText(altText)); + + fireEvent.mouseLeave(caption); + + await waitFor(() => expect(queryByAltText(altText)).toBeNull()); + }); }); diff --git a/x-pack/plugins/uptime/public/components/monitor/ping_list/columns/ping_timestamp/ping_timestamp.tsx b/x-pack/plugins/uptime/public/components/monitor/ping_list/columns/ping_timestamp/ping_timestamp.tsx new file mode 100644 index 0000000000000..6d605f25f6f68 --- /dev/null +++ b/x-pack/plugins/uptime/public/components/monitor/ping_list/columns/ping_timestamp/ping_timestamp.tsx @@ -0,0 +1,120 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License; + * you may not use this file except in compliance with the Elastic License. + */ + +import React, { useContext, useEffect, useState } from 'react'; +import useIntersection from 'react-use/lib/useIntersection'; +import styled from 'styled-components'; +import { Ping } from '../../../../../../common/runtime_types/ping'; +import { useFetcher, FETCH_STATUS } from '../../../../../../../observability/public'; +import { getJourneyScreenshot } from '../../../../../state/api/journey'; +import { UptimeSettingsContext } from '../../../../../contexts'; +import { NavButtons } from './nav_buttons'; +import { NoImageDisplay } from './no_image_display'; +import { StepImageCaption } from './step_image_caption'; +import { StepImagePopover } from './step_image_popover'; +import { formatCaptionContent } from './translations'; + +const StepDiv = styled.div` + figure.euiImage { + div.stepArrowsFullScreen { + display: none; + } + } + + figure.euiImage-isFullScreen { + div.stepArrowsFullScreen { + display: flex; + } + } + position: relative; + div.stepArrows { + display: none; + } + :hover { + div.stepArrows { + display: flex; + } + } +`; + +interface Props { + timestamp: string; + ping: Ping; +} + +export const PingTimestamp = ({ timestamp, ping }: Props) => { + const [stepNumber, setStepNumber] = useState(1); + const [isImagePopoverOpen, setIsImagePopoverOpen] = useState(false); + + const [stepImages, setStepImages] = useState([]); + + const intersectionRef = React.useRef(null); + + const { basePath } = useContext(UptimeSettingsContext); + + const imgPath = `${basePath}/api/uptime/journey/screenshot/${ping.monitor.check_group}/${stepNumber}`; + + const intersection = useIntersection(intersectionRef, { + root: null, + rootMargin: '0px', + threshold: 1, + }); + + const { data, status } = useFetcher(() => { + if (intersection && intersection.intersectionRatio === 1 && !stepImages[stepNumber - 1]) + return getJourneyScreenshot(imgPath); + }, [intersection?.intersectionRatio, stepNumber]); + + useEffect(() => { + if (data) { + setStepImages((prevState) => [...prevState, data?.src]); + } + }, [data]); + + const imgSrc = stepImages[stepNumber] || data?.src; + + const captionContent = formatCaptionContent(stepNumber, data?.maxSteps); + + const ImageCaption = ( + + ); + + return ( + setIsImagePopoverOpen(true)} + onMouseLeave={() => setIsImagePopoverOpen(false)} + ref={intersectionRef} + > + {imgSrc ? ( + + ) : ( + + )} + + + ); +}; diff --git a/x-pack/plugins/uptime/public/components/monitor/ping_list/columns/ping_timestamp/step_image_caption.test.tsx b/x-pack/plugins/uptime/public/components/monitor/ping_list/columns/ping_timestamp/step_image_caption.test.tsx new file mode 100644 index 0000000000000..ef1d0cb388a18 --- /dev/null +++ b/x-pack/plugins/uptime/public/components/monitor/ping_list/columns/ping_timestamp/step_image_caption.test.tsx @@ -0,0 +1,89 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License; + * you may not use this file except in compliance with the Elastic License. + */ + +import { fireEvent, waitFor } from '@testing-library/react'; +import React from 'react'; +import { render } from '../../../../../lib/helper/rtl_helpers'; +import { StepImageCaption, StepImageCaptionProps } from './step_image_caption'; + +describe('StepImageCaption', () => { + let defaultProps: StepImageCaptionProps; + + beforeEach(() => { + defaultProps = { + captionContent: 'test caption content', + imgSrc: 'http://sample.com/sampleImageSrc.png', + maxSteps: 3, + setStepNumber: jest.fn(), + stepNumber: 2, + timestamp: '2020-11-26T15:28:56.896Z', + }; + }); + + it('labels prev and next buttons', () => { + const { getByLabelText } = render(); + + expect(getByLabelText('Previous step')); + expect(getByLabelText('Next step')); + }); + + it('increments step number on next click', async () => { + const { getByLabelText } = render(); + + const nextButton = getByLabelText('Next step'); + + fireEvent.click(nextButton); + + await waitFor(() => { + expect(defaultProps.setStepNumber).toHaveBeenCalledTimes(1); + expect(defaultProps.setStepNumber).toHaveBeenCalledWith(3); + }); + }); + + it('decrements step number on prev click', async () => { + const { getByLabelText } = render(); + + const nextButton = getByLabelText('Previous step'); + + fireEvent.click(nextButton); + + await waitFor(() => { + expect(defaultProps.setStepNumber).toHaveBeenCalledTimes(1); + expect(defaultProps.setStepNumber).toHaveBeenCalledWith(1); + }); + }); + + it('disables `next` button on final step', () => { + defaultProps.stepNumber = 3; + + const { getByLabelText } = render(); + + // getByLabelText('Next step'); + expect(getByLabelText('Next step')).toHaveAttribute('disabled'); + expect(getByLabelText('Previous step')).not.toHaveAttribute('disabled'); + }); + + it('disables `prev` button on final step', () => { + defaultProps.stepNumber = 1; + + const { getByLabelText } = render(); + + expect(getByLabelText('Next step')).not.toHaveAttribute('disabled'); + expect(getByLabelText('Previous step')).toHaveAttribute('disabled'); + }); + + it('renders a timestamp', () => { + const { getByText } = render(); + + getByText('Nov 26, 2020 10:28:56 AM'); + }); + + it('renders caption content', () => { + const { getByText } = render(); + + getByText('test caption content'); + }); +}); diff --git a/x-pack/plugins/uptime/public/components/monitor/ping_list/columns/ping_timestamp/step_image_caption.tsx b/x-pack/plugins/uptime/public/components/monitor/ping_list/columns/ping_timestamp/step_image_caption.tsx new file mode 100644 index 0000000000000..c5da98bacc431 --- /dev/null +++ b/x-pack/plugins/uptime/public/components/monitor/ping_list/columns/ping_timestamp/step_image_caption.tsx @@ -0,0 +1,68 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License; + * you may not use this file except in compliance with the Elastic License. + */ + +import { EuiButtonIcon, EuiFlexGroup, EuiFlexItem, EuiSpacer, EuiText } from '@elastic/eui'; +import React from 'react'; +import moment from 'moment'; +import { nextAriaLabel, prevAriaLabel } from './translations'; +import { getShortTimeStamp } from '../../../../overview/monitor_list/columns/monitor_status_column'; + +export interface StepImageCaptionProps { + captionContent: string; + imgSrc?: string; + maxSteps?: number; + setStepNumber: React.Dispatch>; + stepNumber: number; + timestamp: string; +} + +export const StepImageCaption: React.FC = ({ + captionContent, + imgSrc, + maxSteps, + setStepNumber, + stepNumber, + timestamp, +}) => { + return ( + <> +
+ {imgSrc && ( + + + { + setStepNumber(stepNumber - 1); + }} + iconType="arrowLeft" + aria-label={prevAriaLabel} + /> + + + {captionContent} + + + { + setStepNumber(stepNumber + 1); + }} + iconType="arrowRight" + aria-label={nextAriaLabel} + /> + + + )} +
+ {/* TODO: Add link to details page once it's available */} + {getShortTimeStamp(moment(timestamp))} + + + ); +}; diff --git a/x-pack/plugins/uptime/public/components/monitor/ping_list/columns/ping_timestamp/step_image_popover.test.tsx b/x-pack/plugins/uptime/public/components/monitor/ping_list/columns/ping_timestamp/step_image_popover.test.tsx new file mode 100644 index 0000000000000..184794c1465aa --- /dev/null +++ b/x-pack/plugins/uptime/public/components/monitor/ping_list/columns/ping_timestamp/step_image_popover.test.tsx @@ -0,0 +1,60 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License; + * you may not use this file except in compliance with the Elastic License. + */ + +import { fireEvent, waitFor } from '@testing-library/react'; +import React from 'react'; +import { StepImagePopover, StepImagePopoverProps } from './step_image_popover'; +import { render } from '../../../../../lib/helper/rtl_helpers'; + +describe('StepImagePopover', () => { + let defaultProps: StepImagePopoverProps; + + beforeEach(() => { + defaultProps = { + captionContent: 'test caption', + imageCaption:
test caption element
, + imgSrc: 'http://sample.com/sampleImageSrc.png', + isImagePopoverOpen: false, + }; + }); + + it('opens displays full-size image on click, hides after close is closed', async () => { + const { getByAltText, getByLabelText, queryByLabelText } = render( + + ); + + const closeFullScreenButton = 'Close full screen test caption image'; + + expect(queryByLabelText(closeFullScreenButton)).toBeNull(); + + const caption = getByAltText('test caption'); + fireEvent.click(caption); + + await waitFor(() => { + const closeButton = getByLabelText(closeFullScreenButton); + fireEvent.click(closeButton); + }); + + await waitFor(() => { + expect(queryByLabelText(closeFullScreenButton)).toBeNull(); + }); + }); + + it('shows the popover when `isOpen` is true', () => { + defaultProps.isImagePopoverOpen = true; + + const { getByAltText } = render(); + + expect(getByAltText(`A larger version of the screenshot for this journey step's thumbnail.`)); + }); + + it('renders caption content', () => { + const { getByRole } = render(); + const image = getByRole('img'); + expect(image).toHaveAttribute('alt', 'test caption'); + expect(image).toHaveAttribute('src', 'http://sample.com/sampleImageSrc.png'); + }); +}); diff --git a/x-pack/plugins/uptime/public/components/monitor/ping_list/columns/ping_timestamp/step_image_popover.tsx b/x-pack/plugins/uptime/public/components/monitor/ping_list/columns/ping_timestamp/step_image_popover.tsx new file mode 100644 index 0000000000000..fd7b7e6a886bb --- /dev/null +++ b/x-pack/plugins/uptime/public/components/monitor/ping_list/columns/ping_timestamp/step_image_popover.tsx @@ -0,0 +1,61 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License; + * you may not use this file except in compliance with the Elastic License. + */ + +import { EuiImage, EuiPopover } from '@elastic/eui'; +import React from 'react'; +import styled from 'styled-components'; +import { fullSizeImageAlt } from './translations'; + +const POPOVER_IMG_HEIGHT = 360; +const POPOVER_IMG_WIDTH = 640; + +const StepImage = styled(EuiImage)` + &&& { + display: flex; + figcaption { + white-space: nowrap; + align-self: center; + margin-left: 8px; + margin-top: 8px; + text-decoration: none !important; + } + } +`; +export interface StepImagePopoverProps { + captionContent: string; + imageCaption: JSX.Element; + imgSrc: string; + isImagePopoverOpen: boolean; +} + +export const StepImagePopover: React.FC = ({ + captionContent, + imageCaption, + imgSrc, + isImagePopoverOpen, +}) => ( + + } + isOpen={isImagePopoverOpen} + > + + +); diff --git a/x-pack/plugins/uptime/public/components/monitor/ping_list/columns/ping_timestamp/translations.ts b/x-pack/plugins/uptime/public/components/monitor/ping_list/columns/ping_timestamp/translations.ts new file mode 100644 index 0000000000000..ad49143a68057 --- /dev/null +++ b/x-pack/plugins/uptime/public/components/monitor/ping_list/columns/ping_timestamp/translations.ts @@ -0,0 +1,35 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License; + * you may not use this file except in compliance with the Elastic License. + */ + +import { i18n } from '@kbn/i18n'; + +export const prevAriaLabel = i18n.translate('xpack.uptime.synthetics.prevStepButton.airaLabel', { + defaultMessage: 'Previous step', +}); + +export const nextAriaLabel = i18n.translate('xpack.uptime.synthetics.nextStepButton.ariaLabel', { + defaultMessage: 'Next step', +}); + +export const imageLoadingSpinnerAriaLabel = i18n.translate( + 'xpack.uptime.synthetics.imageLoadingSpinner.ariaLabel', + { + defaultMessage: 'An animated spinner indicating the image is loading', + } +); + +export const fullSizeImageAlt = i18n.translate('xpack.uptime.synthetics.thumbnail.fullSize.alt', { + defaultMessage: `A larger version of the screenshot for this journey step's thumbnail.`, +}); + +export const formatCaptionContent = (stepNumber: number, stepName?: number) => + i18n.translate('xpack.uptime.synthetics.pingTimestamp.captionContent', { + defaultMessage: 'Step: {stepNumber} {stepName}', + values: { + stepNumber, + stepName, + }, + }); diff --git a/x-pack/plugins/uptime/public/components/monitor/synthetics/step_detail/step_detail_container.tsx b/x-pack/plugins/uptime/public/components/monitor/synthetics/step_detail/step_detail_container.tsx index bdc6dbf3f6de2..3d9b646931e7d 100644 --- a/x-pack/plugins/uptime/public/components/monitor/synthetics/step_detail/step_detail_container.tsx +++ b/x-pack/plugins/uptime/public/components/monitor/synthetics/step_detail/step_detail_container.tsx @@ -32,7 +32,7 @@ export const StepDetailContainer: React.FC = ({ checkGroup, stepIndex }) useEffect(() => { if (checkGroup) { - dispatch(getJourneySteps({ checkGroup })); + dispatch(getJourneySteps({ checkGroup, syntheticEventTypes: ['step/end'] })); } }, [dispatch, checkGroup]); diff --git a/x-pack/plugins/uptime/public/components/monitor/synthetics/step_screenshot_display.tsx b/x-pack/plugins/uptime/public/components/monitor/synthetics/step_screenshot_display.tsx index 3efcff196b55f..716e877c50943 100644 --- a/x-pack/plugins/uptime/public/components/monitor/synthetics/step_screenshot_display.tsx +++ b/x-pack/plugins/uptime/public/components/monitor/synthetics/step_screenshot_display.tsx @@ -98,7 +98,7 @@ export const StepScreenshotDisplay: FC = ({ closePopover={() => setIsImagePopoverOpen(false)} isOpen={isImagePopoverOpen} > - { = ({ } ) } - src={imgSrc} + url={imgSrc} style={{ width: POPOVER_IMG_WIDTH, height: POPOVER_IMG_HEIGHT, objectFit: 'contain' }} /> diff --git a/x-pack/plugins/uptime/public/components/settings/types.ts b/x-pack/plugins/uptime/public/components/settings/types.ts index faa1c7e72e47b..7a3af47524b20 100644 --- a/x-pack/plugins/uptime/public/components/settings/types.ts +++ b/x-pack/plugins/uptime/public/components/settings/types.ts @@ -9,7 +9,7 @@ import { JiraActionTypeId, PagerDutyActionTypeId, ServerLogActionTypeId, - ServiceNowActionTypeId, + ServiceNowITSMActionTypeId as ServiceNowActionTypeId, SlackActionTypeId, TeamsActionTypeId, WebhookActionTypeId, diff --git a/x-pack/plugins/uptime/public/state/actions/journey.ts b/x-pack/plugins/uptime/public/state/actions/journey.ts index 0d35559d97fc3..5931980c56947 100644 --- a/x-pack/plugins/uptime/public/state/actions/journey.ts +++ b/x-pack/plugins/uptime/public/state/actions/journey.ts @@ -9,6 +9,7 @@ import { SyntheticsJourneyApiResponse } from '../../../common/runtime_types'; export interface FetchJourneyStepsParams { checkGroup: string; + syntheticEventTypes?: string[]; } export interface GetJourneyFailPayload { diff --git a/x-pack/plugins/uptime/public/state/api/journey.ts b/x-pack/plugins/uptime/public/state/api/journey.ts index 1aeeb485e481f..684056b197f93 100644 --- a/x-pack/plugins/uptime/public/state/api/journey.ts +++ b/x-pack/plugins/uptime/public/state/api/journey.ts @@ -16,7 +16,7 @@ export async function fetchJourneySteps( ): Promise { return (await apiService.get( `/api/uptime/journey/${params.checkGroup}`, - undefined, + { syntheticEventTypes: params.syntheticEventTypes }, SyntheticsJourneyApiResponseType )) as SyntheticsJourneyApiResponse; } diff --git a/x-pack/plugins/uptime/server/lib/requests/get_journey_steps.test.ts b/x-pack/plugins/uptime/server/lib/requests/get_journey_steps.test.ts new file mode 100644 index 0000000000000..8c432ff6f1e0f --- /dev/null +++ b/x-pack/plugins/uptime/server/lib/requests/get_journey_steps.test.ts @@ -0,0 +1,196 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License; + * you may not use this file except in compliance with the Elastic License. + */ + +import { getJourneySteps, formatSyntheticEvents } from './get_journey_steps'; +import { getUptimeESMockClient } from './helper'; + +describe('getJourneySteps request module', () => { + describe('formatStepTypes', () => { + it('returns default steps if none are provided', () => { + expect(formatSyntheticEvents()).toMatchInlineSnapshot(` + Array [ + "step/end", + "stderr", + "cmd/status", + "step/screenshot", + ] + `); + }); + + it('returns provided step array if isArray', () => { + expect(formatSyntheticEvents(['step/end', 'stderr'])).toMatchInlineSnapshot(` + Array [ + "step/end", + "stderr", + ] + `); + }); + + it('returns provided step string in an array', () => { + expect(formatSyntheticEvents('step/end')).toMatchInlineSnapshot(` + Array [ + "step/end", + ] + `); + }); + }); + + describe('getJourneySteps', () => { + let data: any; + beforeEach(() => { + data = { + body: { + hits: { + hits: [ + { + _id: 'o6myXncBFt2V8m6r6z-r', + _source: { + '@timestamp': '2021-02-01T17:45:19.001Z', + synthetics: { + package_version: '0.0.1-alpha.8', + journey: { + name: 'inline', + id: 'inline', + }, + step: { + name: 'load homepage', + index: 1, + }, + type: 'step/end', + }, + monitor: { + name: 'My Monitor', + id: 'my-monitor', + check_group: '2bf952dc-64b5-11eb-8b3b-42010a84000d', + type: 'browser', + }, + }, + }, + { + _id: 'IjqzXncBn2sjqrYxYoCG', + _source: { + '@timestamp': '2021-02-01T17:45:49.944Z', + synthetics: { + package_version: '0.0.1-alpha.8', + journey: { + name: 'inline', + id: 'inline', + }, + step: { + name: 'hover over products menu', + index: 2, + }, + type: 'step/end', + }, + monitor: { + name: 'My Monitor', + timespan: { + lt: '2021-02-01T17:46:49.945Z', + gte: '2021-02-01T17:45:49.945Z', + }, + id: 'my-monitor', + check_group: '2bf952dc-64b5-11eb-8b3b-42010a84000d', + type: 'browser', + }, + }, + }, + ], + }, + }, + }; + }); + + it('formats ES result', async () => { + const { esClient: mockEsClient, uptimeEsClient } = getUptimeESMockClient(); + + mockEsClient.search.mockResolvedValueOnce(data as any); + const result: any = await getJourneySteps({ + uptimeEsClient, + checkGroup: '2bf952dc-64b5-11eb-8b3b-42010a84000d', + }); + expect(mockEsClient.search).toHaveBeenCalledTimes(1); + const call: any = mockEsClient.search.mock.calls[0][0]; + + // check that default `synthetics.type` value is supplied, + expect(call.body.query.bool.filter[0]).toMatchInlineSnapshot(` + Object { + "terms": Object { + "synthetics.type": Array [ + "step/end", + "stderr", + "cmd/status", + "step/screenshot", + ], + }, + } + `); + + // given check group is used for the terms filter + expect(call.body.query.bool.filter[1]).toMatchInlineSnapshot(` + Object { + "term": Object { + "monitor.check_group": "2bf952dc-64b5-11eb-8b3b-42010a84000d", + }, + } + `); + + // should sort by step index, then timestamp + expect(call.body.sort).toMatchInlineSnapshot(` + Array [ + Object { + "synthetics.step.index": Object { + "order": "asc", + }, + }, + Object { + "@timestamp": Object { + "order": "asc", + }, + }, + ] + `); + + expect(result).toHaveLength(2); + // `getJourneySteps` is responsible for formatting these fields, so we need to check them + result.forEach((step: any) => { + expect(['2021-02-01T17:45:19.001Z', '2021-02-01T17:45:49.944Z']).toContain(step.timestamp); + expect(['o6myXncBFt2V8m6r6z-r', 'IjqzXncBn2sjqrYxYoCG']).toContain(step.docId); + expect(step.synthetics.screenshotExists).toBeDefined(); + }); + }); + + it('notes screenshot exists when a document of type step/screenshot is included', async () => { + const { esClient: mockEsClient, uptimeEsClient } = getUptimeESMockClient(); + + data.body.hits.hits[0]._source.synthetics.type = 'step/screenshot'; + data.body.hits.hits[0]._source.synthetics.step.index = 2; + mockEsClient.search.mockResolvedValueOnce(data as any); + + const result: any = await getJourneySteps({ + uptimeEsClient, + checkGroup: '2bf952dc-64b5-11eb-8b3b-42010a84000d', + syntheticEventTypes: ['stderr', 'step/end'], + }); + + const call: any = mockEsClient.search.mock.calls[0][0]; + + // assert that filters for only the provided step types are used + expect(call.body.query.bool.filter[0]).toMatchInlineSnapshot(` + Object { + "terms": Object { + "synthetics.type": Array [ + "stderr", + "step/end", + ], + }, + } + `); + + expect(result).toHaveLength(1); + expect(result[0].synthetics.screenshotExists).toBe(true); + }); + }); +}); diff --git a/x-pack/plugins/uptime/server/lib/requests/get_journey_steps.ts b/x-pack/plugins/uptime/server/lib/requests/get_journey_steps.ts index c330e1b66fe93..60d2a97c99f7d 100644 --- a/x-pack/plugins/uptime/server/lib/requests/get_journey_steps.ts +++ b/x-pack/plugins/uptime/server/lib/requests/get_journey_steps.ts @@ -9,11 +9,23 @@ import { Ping } from '../../../common/runtime_types'; interface GetJourneyStepsParams { checkGroup: string; + syntheticEventTypes?: string | string[]; } +const defaultEventTypes = ['step/end', 'stderr', 'cmd/status', 'step/screenshot']; + +export const formatSyntheticEvents = (eventTypes?: string | string[]) => { + if (!eventTypes) { + return defaultEventTypes; + } else { + return Array.isArray(eventTypes) ? eventTypes : [eventTypes]; + } +}; + export const getJourneySteps: UMElasticsearchQueryFn = async ({ uptimeEsClient, checkGroup, + syntheticEventTypes, }) => { const params = { query: { @@ -21,7 +33,7 @@ export const getJourneySteps: UMElasticsearchQueryFn checkGroup: schema.string(), _debug: schema.maybe(schema.boolean()), }), + query: schema.object({ + // provides a filter for the types of synthetic events to include + // when fetching a journey's data + syntheticEventTypes: schema.maybe( + schema.oneOf([schema.arrayOf(schema.string()), schema.string()]) + ), + }), }, handler: async ({ uptimeEsClient, request }): Promise => { const { checkGroup } = request.params; + const { syntheticEventTypes } = request.query; const result = await libs.requests.getJourneySteps({ uptimeEsClient, checkGroup, + syntheticEventTypes, }); const details = await libs.requests.getJourneyDetails({ diff --git a/x-pack/test/accessibility/apps/home.ts b/x-pack/test/accessibility/apps/home.ts index 280769bc09bc9..6857383b7db5b 100644 --- a/x-pack/test/accessibility/apps/home.ts +++ b/x-pack/test/accessibility/apps/home.ts @@ -9,12 +9,10 @@ import { FtrProviderContext } from '../ftr_provider_context'; export default function ({ getService, getPageObjects }: FtrProviderContext) { const PageObjects = getPageObjects(['common', 'home']); const a11y = getService('a11y'); - const retry = getService('retry'); - const globalNav = getService('globalNav'); const testSubjects = getService('testSubjects'); + const find = getService('find'); - // FLAKY: https://github.com/elastic/kibana/issues/80929 - describe.skip('Kibana Home', () => { + describe('Kibana Home', () => { before(async () => { await PageObjects.common.navigateToApp('home'); }); @@ -23,64 +21,74 @@ export default function ({ getService, getPageObjects }: FtrProviderContext) { await a11y.testAppSnapshot(); }); - it('all plugins view page meets a11y requirements', async () => { - await PageObjects.home.clickAllKibanaPlugins(); + it('Kibana overview page meets a11y requirements ', async () => { + await testSubjects.click('homSolutionPanel homSolutionPanel_kibana'); await a11y.testAppSnapshot(); }); - it('visualize & explore details tab meets a11y requirements', async () => { - await PageObjects.home.clickVisualizeExplorePlugins(); + it('toggle side nav meets a11y requirements', async () => { + await testSubjects.click('toggleNavButton'); await a11y.testAppSnapshot(); }); - it('administrative detail tab meets a11y requirements', async () => { - await PageObjects.home.clickAdminPlugin(); + it('Enterprise search overview page meets a11y requirements ', async () => { + await testSubjects.click('homeLink'); + await testSubjects.click('homSolutionPanel homSolutionPanel_enterpriseSearch'); await a11y.testAppSnapshot(); }); - it('navigating to console app from administration tab meets a11y requirements', async () => { - await PageObjects.home.clickOnConsole(); - // wait till dev tools app is loaded (lazy loading the bundle) - await retry.waitFor( - 'switched to dev tools', - async () => (await globalNav.getLastBreadcrumb()) === 'Dev Tools' - ); + it('Observability overview page meets a11y requirements ', async () => { + await testSubjects.click('toggleNavButton'); + await testSubjects.click('homeLink'); + await testSubjects.click('homSolutionPanel homSolutionPanel_observability'); await a11y.testAppSnapshot(); }); - it('navigating back to home page from console meets a11y requirements', async () => { - await PageObjects.home.clickOnLogo(); + it('Security overview page meets a11y requirements ', async () => { + await testSubjects.click('toggleNavButton'); + await testSubjects.click('homeLink'); + await testSubjects.click('homSolutionPanel homSolutionPanel_securitySolution'); await a11y.testAppSnapshot(); }); - it('click on Add logs panel to open all log examples page meets a11y requirements ', async () => { - await PageObjects.home.clickOnAddData(); + it('Add data page meets a11y requirements ', async () => { + await testSubjects.click('toggleNavButton'); + await testSubjects.click('homeLink'); + await testSubjects.click('homeAddData'); await a11y.testAppSnapshot(); }); - it('click on ActiveMQ logs panel to open tutorial meets a11y requirements', async () => { - await PageObjects.home.clickOnLogsTutorial(); + it('Sample data page meets a11y requirements ', async () => { + await testSubjects.click('homeTab-sampleData'); await a11y.testAppSnapshot(); }); - it('click on cloud tutorial meets a11y requirements', async () => { - await PageObjects.home.clickOnCloudTutorial(); + it('click on Add logs panel to open all log examples page meets a11y requirements ', async () => { + await testSubjects.click('sampleDataSetCardlogs'); + await a11y.testAppSnapshot(); + }); + + it('click on ActiveMQ logs panel to open tutorial meets a11y requirements', async () => { + await testSubjects.click('homeTab-all'); + await testSubjects.click('homeSynopsisLinkactivemqlogs'); await a11y.testAppSnapshot(); }); - it('click on side nav to see all the side nav menu', async () => { - await PageObjects.home.clickOnLogo(); - await PageObjects.home.clickOnToggleNavButton(); + it('click on cloud tutorial meets a11y requirements', async () => { + await testSubjects.click('onCloudTutorial'); await a11y.testAppSnapshot(); }); it('Dock the side nav', async () => { + await testSubjects.click('toggleNavButton'); await PageObjects.home.dockTheSideNav(); await a11y.testAppSnapshot(); }); it('click on collapse on observability in side nav to test a11y of collapse button', async () => { - await PageObjects.home.collapseObservabibilitySideNav(); + await find.clickByCssSelector( + '[data-test-subj="collapsibleNavGroup-observability"] .euiCollapsibleNavGroup__title' + ); await a11y.testAppSnapshot(); }); @@ -91,8 +99,7 @@ export default function ({ getService, getPageObjects }: FtrProviderContext) { }); it('passes with searchbox open', async () => { - await PageObjects.common.navigateToApp('home'); - await testSubjects.click('header-search'); + await testSubjects.click('nav-search-popover'); await a11y.testAppSnapshot(); }); }); diff --git a/x-pack/test/alerting_api_integration/common/fixtures/plugins/actions_simulators/server/plugin.ts b/x-pack/test/alerting_api_integration/common/fixtures/plugins/actions_simulators/server/plugin.ts index 18f3c83b00141..dfdacb230763f 100644 --- a/x-pack/test/alerting_api_integration/common/fixtures/plugins/actions_simulators/server/plugin.ts +++ b/x-pack/test/alerting_api_integration/common/fixtures/plugins/actions_simulators/server/plugin.ts @@ -38,6 +38,7 @@ export function getAllExternalServiceSimulatorPaths(): string[] { getExternalServiceSimulatorPath(service) ); allPaths.push(`/api/_${NAME}/${ExternalServiceSimulator.SERVICENOW}/api/now/v2/table/incident`); + allPaths.push(`/api/_${NAME}/${ExternalServiceSimulator.SERVICENOW}/api/now/v2/table/sys_choice`); allPaths.push( `/api/_${NAME}/${ExternalServiceSimulator.SERVICENOW}/api/now/v2/table/sys_dictionary` ); diff --git a/x-pack/test/alerting_api_integration/common/fixtures/plugins/actions_simulators/server/servicenow_simulation.ts b/x-pack/test/alerting_api_integration/common/fixtures/plugins/actions_simulators/server/servicenow_simulation.ts index 2c3138a36f071..b94bb89fc6f4a 100644 --- a/x-pack/test/alerting_api_integration/common/fixtures/plugins/actions_simulators/server/servicenow_simulation.ts +++ b/x-pack/test/alerting_api_integration/common/fixtures/plugins/actions_simulators/server/servicenow_simulation.ts @@ -127,6 +127,51 @@ export function initPlugin(router: IRouter, path: string) { }); } ); + + router.get( + { + path: `${path}/api/now/v2/table/sys_choice`, + options: { + authRequired: false, + }, + validate: {}, + }, + async function ( + context: RequestHandlerContext, + req: KibanaRequest, + res: KibanaResponseFactory + ): Promise> { + return jsonResponse(res, 200, { + result: [ + { + dependent_value: '', + label: '1 - Critical', + value: '1', + }, + { + dependent_value: '', + label: '2 - High', + value: '2', + }, + { + dependent_value: '', + label: '3 - Moderate', + value: '3', + }, + { + dependent_value: '', + label: '4 - Low', + value: '4', + }, + { + dependent_value: '', + label: '5 - Planning', + value: '5', + }, + ], + }); + } + ); } function jsonResponse(res: KibanaResponseFactory, code: number, object?: Record) { diff --git a/x-pack/test/alerting_api_integration/security_and_spaces/tests/actions/builtin_action_types/servicenow.ts b/x-pack/test/alerting_api_integration/security_and_spaces/tests/actions/builtin_action_types/servicenow.ts index e448ad1f9c2ad..5f7146b43bfdb 100644 --- a/x-pack/test/alerting_api_integration/security_and_spaces/tests/actions/builtin_action_types/servicenow.ts +++ b/x-pack/test/alerting_api_integration/security_and_spaces/tests/actions/builtin_action_types/servicenow.ts @@ -216,7 +216,7 @@ export default function servicenowTest({ getService }: FtrProviderContext) { // Cannot destructure property 'value' of 'undefined' as it is undefined. // // The error seems to come from the exact same place in the code based on the - // exact same circomstances: + // exact same circumstances: // // https://github.com/elastic/kibana/blob/b0a223ebcbac7e404e8ae6da23b2cc6a4b509ff1/packages/kbn-config-schema/src/types/literal_type.ts#L28 // @@ -247,7 +247,7 @@ export default function servicenowTest({ getService }: FtrProviderContext) { status: 'error', retry: false, message: - 'error validating action params: types that failed validation:\n- [0.subAction]: expected value to equal [getFields]\n- [1.subAction]: expected value to equal [getIncident]\n- [2.subAction]: expected value to equal [handshake]\n- [3.subAction]: expected value to equal [pushToService]', + 'error validating action params: types that failed validation:\n- [0.subAction]: expected value to equal [getFields]\n- [1.subAction]: expected value to equal [getIncident]\n- [2.subAction]: expected value to equal [handshake]\n- [3.subAction]: expected value to equal [pushToService]\n- [4.subAction]: expected value to equal [getChoices]', }); }); }); @@ -265,7 +265,7 @@ export default function servicenowTest({ getService }: FtrProviderContext) { status: 'error', retry: false, message: - 'error validating action params: types that failed validation:\n- [0.subAction]: expected value to equal [getFields]\n- [1.subAction]: expected value to equal [getIncident]\n- [2.subAction]: expected value to equal [handshake]\n- [3.subActionParams.incident.short_description]: expected value of type [string] but got [undefined]', + 'error validating action params: types that failed validation:\n- [0.subAction]: expected value to equal [getFields]\n- [1.subAction]: expected value to equal [getIncident]\n- [2.subAction]: expected value to equal [handshake]\n- [3.subActionParams.incident.short_description]: expected value of type [string] but got [undefined]\n- [4.subAction]: expected value to equal [getChoices]', }); }); }); @@ -288,7 +288,7 @@ export default function servicenowTest({ getService }: FtrProviderContext) { status: 'error', retry: false, message: - 'error validating action params: types that failed validation:\n- [0.subAction]: expected value to equal [getFields]\n- [1.subAction]: expected value to equal [getIncident]\n- [2.subAction]: expected value to equal [handshake]\n- [3.subActionParams.incident.short_description]: expected value of type [string] but got [undefined]', + 'error validating action params: types that failed validation:\n- [0.subAction]: expected value to equal [getFields]\n- [1.subAction]: expected value to equal [getIncident]\n- [2.subAction]: expected value to equal [handshake]\n- [3.subActionParams.incident.short_description]: expected value of type [string] but got [undefined]\n- [4.subAction]: expected value to equal [getChoices]', }); }); }); @@ -315,7 +315,7 @@ export default function servicenowTest({ getService }: FtrProviderContext) { status: 'error', retry: false, message: - 'error validating action params: types that failed validation:\n- [0.subAction]: expected value to equal [getFields]\n- [1.subAction]: expected value to equal [getIncident]\n- [2.subAction]: expected value to equal [handshake]\n- [3.subActionParams.comments]: types that failed validation:\n - [subActionParams.comments.0.0.commentId]: expected value of type [string] but got [undefined]\n - [subActionParams.comments.1]: expected value to equal [null]', + 'error validating action params: types that failed validation:\n- [0.subAction]: expected value to equal [getFields]\n- [1.subAction]: expected value to equal [getIncident]\n- [2.subAction]: expected value to equal [handshake]\n- [3.subActionParams.comments]: types that failed validation:\n - [subActionParams.comments.0.0.commentId]: expected value of type [string] but got [undefined]\n - [subActionParams.comments.1]: expected value to equal [null]\n- [4.subAction]: expected value to equal [getChoices]', }); }); }); @@ -342,10 +342,33 @@ export default function servicenowTest({ getService }: FtrProviderContext) { status: 'error', retry: false, message: - 'error validating action params: types that failed validation:\n- [0.subAction]: expected value to equal [getFields]\n- [1.subAction]: expected value to equal [getIncident]\n- [2.subAction]: expected value to equal [handshake]\n- [3.subActionParams.comments]: types that failed validation:\n - [subActionParams.comments.0.0.comment]: expected value of type [string] but got [undefined]\n - [subActionParams.comments.1]: expected value to equal [null]', + 'error validating action params: types that failed validation:\n- [0.subAction]: expected value to equal [getFields]\n- [1.subAction]: expected value to equal [getIncident]\n- [2.subAction]: expected value to equal [handshake]\n- [3.subActionParams.comments]: types that failed validation:\n - [subActionParams.comments.0.0.comment]: expected value of type [string] but got [undefined]\n - [subActionParams.comments.1]: expected value to equal [null]\n- [4.subAction]: expected value to equal [getChoices]', }); }); }); + + describe('getChoices', () => { + it('should fail when field is not provided', async () => { + await supertest + .post(`/api/actions/action/${simulatedActionId}/_execute`) + .set('kbn-xsrf', 'foo') + .send({ + params: { + subAction: 'getChoices', + subActionParams: {}, + }, + }) + .then((resp: any) => { + expect(resp.body).to.eql({ + actionId: simulatedActionId, + status: 'error', + retry: false, + message: + 'error validating action params: types that failed validation:\n- [0.subAction]: expected value to equal [getFields]\n- [1.subAction]: expected value to equal [getIncident]\n- [2.subAction]: expected value to equal [handshake]\n- [3.subAction]: expected value to equal [pushToService]\n- [4.subActionParams.fields]: expected value of type [array] but got [undefined]', + }); + }); + }); + }); }); describe('Execution', () => { @@ -376,6 +399,54 @@ export default function servicenowTest({ getService }: FtrProviderContext) { }, }); }); + + describe('getChoices', () => { + it('should get choices', async () => { + const { body: result } = await supertest + .post(`/api/actions/action/${simulatedActionId}/_execute`) + .set('kbn-xsrf', 'foo') + .send({ + params: { + subAction: 'getChoices', + subActionParams: { fields: ['priority'] }, + }, + }) + .expect(200); + + expect(proxyHaveBeenCalled).to.equal(true); + expect(result).to.eql({ + status: 'ok', + actionId: simulatedActionId, + data: [ + { + dependent_value: '', + label: '1 - Critical', + value: '1', + }, + { + dependent_value: '', + label: '2 - High', + value: '2', + }, + { + dependent_value: '', + label: '3 - Moderate', + value: '3', + }, + { + dependent_value: '', + label: '4 - Low', + value: '4', + }, + { + dependent_value: '', + label: '5 - Planning', + value: '5', + }, + ], + }); + }); + }); }); after(() => { diff --git a/x-pack/test/alerting_api_integration/security_and_spaces/tests/alerting/index.ts b/x-pack/test/alerting_api_integration/security_and_spaces/tests/alerting/index.ts index 8ed979a171169..e1502b496f77e 100644 --- a/x-pack/test/alerting_api_integration/security_and_spaces/tests/alerting/index.ts +++ b/x-pack/test/alerting_api_integration/security_and_spaces/tests/alerting/index.ts @@ -10,7 +10,8 @@ import { setupSpacesAndUsers, tearDown } from '..'; // eslint-disable-next-line import/no-default-export export default function alertingTests({ loadTestFile, getService }: FtrProviderContext) { describe('Alerts', () => { - describe('legacy alerts', () => { + // FLAKY: https://github.com/elastic/kibana/issues/86952 + describe.skip('legacy alerts', () => { before(async () => { await setupSpacesAndUsers(getService); }); diff --git a/x-pack/test/alerting_api_integration/spaces_only/tests/alerting/create.ts b/x-pack/test/alerting_api_integration/spaces_only/tests/alerting/create.ts index 8bf0a2a0f034f..7e25707c10c78 100644 --- a/x-pack/test/alerting_api_integration/spaces_only/tests/alerting/create.ts +++ b/x-pack/test/alerting_api_integration/spaces_only/tests/alerting/create.ts @@ -111,6 +111,75 @@ export default function createAlertTests({ getService }: FtrProviderContext) { }); }); + it('should allow providing custom saved object ids (uuid v1)', async () => { + const customId = '09570bb0-6299-11eb-8fde-9fe5ce6ea450'; + const response = await supertest + .post(`${getUrlPrefix(Spaces.space1.id)}/api/alerts/alert/${customId}`) + .set('kbn-xsrf', 'foo') + .send(getTestAlertData()); + + expect(response.status).to.eql(200); + objectRemover.add(Spaces.space1.id, response.body.id, 'alert', 'alerts'); + expect(response.body.id).to.eql(customId); + // Ensure AAD isn't broken + await checkAAD({ + supertest, + spaceId: Spaces.space1.id, + type: 'alert', + id: customId, + }); + }); + + it('should allow providing custom saved object ids (uuid v4)', async () => { + const customId = 'b3bc6d83-3192-4ffd-9702-ad4fb88617ba'; + const response = await supertest + .post(`${getUrlPrefix(Spaces.space1.id)}/api/alerts/alert/${customId}`) + .set('kbn-xsrf', 'foo') + .send(getTestAlertData()); + + expect(response.status).to.eql(200); + objectRemover.add(Spaces.space1.id, response.body.id, 'alert', 'alerts'); + expect(response.body.id).to.eql(customId); + // Ensure AAD isn't broken + await checkAAD({ + supertest, + spaceId: Spaces.space1.id, + type: 'alert', + id: customId, + }); + }); + + it('should not allow providing simple custom ids (non uuid)', async () => { + const customId = '1'; + const response = await supertest + .post(`${getUrlPrefix(Spaces.space1.id)}/api/alerts/alert/${customId}`) + .set('kbn-xsrf', 'foo') + .send(getTestAlertData()); + + expect(response.status).to.eql(400); + expect(response.body).to.eql({ + statusCode: 400, + error: 'Bad Request', + message: + 'Predefined IDs are not allowed for saved objects with encrypted attributes unless the ID is a UUID.: Bad Request', + }); + }); + + it('should return 409 when document with id already exists', async () => { + const customId = '5031f8f0-629a-11eb-b500-d1931a8e5df7'; + const createdAlertResponse = await supertest + .post(`${getUrlPrefix(Spaces.space1.id)}/api/alerts/alert/${customId}`) + .set('kbn-xsrf', 'foo') + .send(getTestAlertData()) + .expect(200); + objectRemover.add(Spaces.space1.id, createdAlertResponse.body.id, 'alert', 'alerts'); + await supertest + .post(`${getUrlPrefix(Spaces.space1.id)}/api/alerts/alert/${customId}`) + .set('kbn-xsrf', 'foo') + .send(getTestAlertData()) + .expect(409); + }); + it('should handle create alert request appropriately when consumer is unknown', async () => { const response = await supertest .post(`${getUrlPrefix(Spaces.space1.id)}/api/alerts/alert`) diff --git a/x-pack/test/api_integration/apis/maps/migrations.js b/x-pack/test/api_integration/apis/maps/migrations.js index b634e7117e607..9f9082c959ca5 100644 --- a/x-pack/test/api_integration/apis/maps/migrations.js +++ b/x-pack/test/api_integration/apis/maps/migrations.js @@ -41,7 +41,7 @@ export default function ({ getService }) { type: 'index-pattern', }, ]); - expect(resp.body.migrationVersion).to.eql({ map: '7.10.0' }); + expect(resp.body.migrationVersion).to.eql({ map: '7.12.0' }); expect(resp.body.attributes.layerListJSON.includes('indexPatternRefName')).to.be(true); }); }); diff --git a/x-pack/test/api_integration/apis/search/session.ts b/x-pack/test/api_integration/apis/search/session.ts index ee2e4337adc95..5111ce06a4d7d 100644 --- a/x-pack/test/api_integration/apis/search/session.ts +++ b/x-pack/test/api_integration/apis/search/session.ts @@ -12,6 +12,20 @@ export default function ({ getService }: FtrProviderContext) { describe('search session', () => { describe('session management', () => { + it('should fail to create a session with no name', async () => { + const sessionId = `my-session-${Math.random()}`; + await supertest + .post(`/internal/session`) + .set('kbn-xsrf', 'foo') + .send({ + sessionId, + appId: 'discover', + expires: '123', + urlGeneratorId: 'discover', + }) + .expect(400); + }); + it('should create and get a session', async () => { const sessionId = `my-session-${Math.random()}`; await supertest @@ -52,7 +66,7 @@ export default function ({ getService }: FtrProviderContext) { await supertest.get(`/internal/session/${sessionId}`).set('kbn-xsrf', 'foo').expect(404); }); - it('should sync search ids into session', async () => { + it('should sync search ids into persisted session', async () => { const sessionId = `my-session-${Math.random()}`; // run search @@ -76,7 +90,7 @@ export default function ({ getService }: FtrProviderContext) { const { id: id1 } = searchRes1.body; - // create session + // persist session await supertest .post(`/internal/session`) .set('kbn-xsrf', 'foo') @@ -108,21 +122,135 @@ export default function ({ getService }: FtrProviderContext) { const { id: id2 } = searchRes2.body; - // wait 10 seconds for ids to be synced - // TODO: make the refresh interval dynamic, so we can speed it up! - await new Promise((resolve) => setTimeout(resolve, 10000)); - const resp = await supertest .get(`/internal/session/${sessionId}`) .set('kbn-xsrf', 'foo') .expect(200); - const { idMapping } = resp.body.attributes; + const { name, touched, created, persisted, idMapping } = resp.body.attributes; + expect(persisted).to.be(true); + expect(name).to.be('My Session'); + expect(touched).not.to.be(undefined); + expect(created).not.to.be(undefined); const idMappings = Object.values(idMapping).map((value: any) => value.id); expect(idMappings).to.contain(id1); expect(idMappings).to.contain(id2); }); }); + + it('should sync search ids into not persisted session', async () => { + const sessionId = `my-session-${Math.random()}`; + + // run search + const searchRes1 = await supertest + .post(`/internal/search/ese`) + .set('kbn-xsrf', 'foo') + .send({ + sessionId, + params: { + body: { + query: { + term: { + agent: '1', + }, + }, + }, + wait_for_completion_timeout: '1ms', + }, + }) + .expect(200); + + const { id: id1 } = searchRes1.body; + + // run search + const searchRes2 = await supertest + .post(`/internal/search/ese`) + .set('kbn-xsrf', 'foo') + .send({ + sessionId, + params: { + body: { + query: { + match_all: {}, + }, + }, + wait_for_completion_timeout: '1ms', + }, + }) + .expect(200); + + const { id: id2 } = searchRes2.body; + + const resp = await supertest + .get(`/internal/session/${sessionId}`) + .set('kbn-xsrf', 'foo') + .expect(200); + + const { appId, name, touched, created, persisted, idMapping } = resp.body.attributes; + expect(persisted).to.be(false); + expect(name).to.be(undefined); + expect(appId).to.be(undefined); + expect(touched).not.to.be(undefined); + expect(created).not.to.be(undefined); + + const idMappings = Object.values(idMapping).map((value: any) => value.id); + expect(idMappings).to.contain(id1); + expect(idMappings).to.contain(id2); + }); + + it('touched time updates when you poll on an search', async () => { + const sessionId = `my-session-${Math.random()}`; + + // run search + const searchRes1 = await supertest + .post(`/internal/search/ese`) + .set('kbn-xsrf', 'foo') + .send({ + sessionId, + params: { + body: { + query: { + term: { + agent: '1', + }, + }, + }, + wait_for_completion_timeout: '1ms', + }, + }) + .expect(200); + + const { id: id1 } = searchRes1.body; + + // it might take the session a moment to be created + await new Promise((resolve) => setTimeout(resolve, 1000)); + + const getSessionFirstTime = await supertest + .get(`/internal/session/${sessionId}`) + .set('kbn-xsrf', 'foo') + .expect(200); + + // poll on search + await supertest + .post(`/internal/search/ese/${id1}`) + .set('kbn-xsrf', 'foo') + .send({ + sessionId, + }) + .expect(200); + + const getSessionSecondTime = await supertest + .get(`/internal/session/${sessionId}`) + .set('kbn-xsrf', 'foo') + .expect(200); + + expect(getSessionFirstTime.body.attributes.sessionId).to.be.equal( + getSessionSecondTime.body.attributes.sessionId + ); + expect(getSessionFirstTime.body.attributes.touched).to.be.lessThan( + getSessionSecondTime.body.attributes.touched + ); + }); }); } diff --git a/x-pack/test/api_integration/config.ts b/x-pack/test/api_integration/config.ts index 546b23ab4f26c..8563d60ca68fc 100644 --- a/x-pack/test/api_integration/config.ts +++ b/x-pack/test/api_integration/config.ts @@ -31,6 +31,8 @@ export async function getApiIntegrationConfig({ readConfigFile }: FtrConfigProvi '--xpack.fleet.enabled=true', '--xpack.fleet.agents.pollingRequestTimeout=5000', // 5 seconds '--xpack.data_enhanced.search.sessions.enabled=true', // enable WIP send to background UI + '--xpack.data_enhanced.search.sessions.notTouchedTimeout=15s', // shorten notTouchedTimeout for quicker testing + '--xpack.data_enhanced.search.sessions.trackingInterval=5s', // shorten trackingInterval for quicker testing ], }, esTestCluster: { diff --git a/x-pack/test/fleet_api_integration/apis/epm/data_stream.ts b/x-pack/test/fleet_api_integration/apis/epm/data_stream.ts index 574ff6dd615ad..a43f51a1655e5 100644 --- a/x-pack/test/fleet_api_integration/apis/epm/data_stream.ts +++ b/x-pack/test/fleet_api_integration/apis/epm/data_stream.ts @@ -12,8 +12,6 @@ export default function (providerContext: FtrProviderContext) { const { getService } = providerContext; const supertest = getService('supertest'); const es = getService('es'); - const dockerServers = getService('dockerServers'); - const server = dockerServers.get('registry'); const pkgName = 'datastreams'; const pkgVersion = '0.1.0'; const pkgUpdateVersion = '0.2.0'; @@ -21,6 +19,7 @@ export default function (providerContext: FtrProviderContext) { const pkgUpdateKey = `${pkgName}-${pkgUpdateVersion}`; const logsTemplateName = `logs-${pkgName}.test_logs`; const metricsTemplateName = `metrics-${pkgName}.test_metrics`; + const namespaces = ['default', 'foo', 'bar']; const uninstallPackage = async (pkg: string) => { await supertest.delete(`/api/fleet/epm/packages/${pkg}`).set('kbn-xsrf', 'xxxx'); @@ -35,86 +34,105 @@ export default function (providerContext: FtrProviderContext) { describe('datastreams', async () => { skipIfNoDockerRegistry(providerContext); + beforeEach(async () => { await installPackage(pkgKey); - await es.transport.request({ - method: 'POST', - path: `/${logsTemplateName}-default/_doc`, - body: { - '@timestamp': '2015-01-01', - logs_test_name: 'test', - data_stream: { - dataset: `${pkgName}.test_logs`, - namespace: 'default', - type: 'logs', - }, - }, - }); - await es.transport.request({ - method: 'POST', - path: `/${metricsTemplateName}-default/_doc`, - body: { - '@timestamp': '2015-01-01', - logs_test_name: 'test', - data_stream: { - dataset: `${pkgName}.test_metrics`, - namespace: 'default', - type: 'metrics', - }, - }, - }); + await Promise.all( + namespaces.map(async (namespace) => { + const createLogsRequest = es.transport.request({ + method: 'POST', + path: `/${logsTemplateName}-${namespace}/_doc`, + body: { + '@timestamp': '2015-01-01', + logs_test_name: 'test', + data_stream: { + dataset: `${pkgName}.test_logs`, + namespace, + type: 'logs', + }, + }, + }); + const createMetricsRequest = es.transport.request({ + method: 'POST', + path: `/${metricsTemplateName}-${namespace}/_doc`, + body: { + '@timestamp': '2015-01-01', + logs_test_name: 'test', + data_stream: { + dataset: `${pkgName}.test_metrics`, + namespace, + type: 'metrics', + }, + }, + }); + return Promise.all([createLogsRequest, createMetricsRequest]); + }) + ); }); + afterEach(async () => { - if (!server.enabled) return; - await es.transport.request({ - method: 'DELETE', - path: `/_data_stream/${logsTemplateName}-default`, - }); - await es.transport.request({ - method: 'DELETE', - path: `/_data_stream/${metricsTemplateName}-default`, - }); + await Promise.all( + namespaces.map(async (namespace) => { + const deleteLogsRequest = es.transport.request({ + method: 'DELETE', + path: `/_data_stream/${logsTemplateName}-${namespace}`, + }); + const deleteMetricsRequest = es.transport.request({ + method: 'DELETE', + path: `/_data_stream/${metricsTemplateName}-${namespace}`, + }); + return Promise.all([deleteLogsRequest, deleteMetricsRequest]); + }) + ); await uninstallPackage(pkgKey); await uninstallPackage(pkgUpdateKey); }); + it('should list the logs and metrics datastream', async function () { - const resLogsDatastream = await es.transport.request({ - method: 'GET', - path: `/_data_stream/${logsTemplateName}-default`, - }); - const resMetricsDatastream = await es.transport.request({ - method: 'GET', - path: `/_data_stream/${metricsTemplateName}-default`, + namespaces.forEach(async (namespace) => { + const resLogsDatastream = await es.transport.request({ + method: 'GET', + path: `/_data_stream/${logsTemplateName}-${namespace}`, + }); + const resMetricsDatastream = await es.transport.request({ + method: 'GET', + path: `/_data_stream/${metricsTemplateName}-${namespace}`, + }); + expect(resLogsDatastream.body.data_streams.length).equal(1); + expect(resLogsDatastream.body.data_streams[0].indices.length).equal(1); + expect(resMetricsDatastream.body.data_streams.length).equal(1); + expect(resMetricsDatastream.body.data_streams[0].indices.length).equal(1); }); - expect(resLogsDatastream.body.data_streams.length).equal(1); - expect(resLogsDatastream.body.data_streams[0].indices.length).equal(1); - expect(resMetricsDatastream.body.data_streams.length).equal(1); - expect(resMetricsDatastream.body.data_streams[0].indices.length).equal(1); }); it('after update, it should have rolled over logs datastream because mappings are not compatible and not metrics', async function () { await installPackage(pkgUpdateKey); - const resLogsDatastream = await es.transport.request({ - method: 'GET', - path: `/_data_stream/${logsTemplateName}-default`, - }); - const resMetricsDatastream = await es.transport.request({ - method: 'GET', - path: `/_data_stream/${metricsTemplateName}-default`, + namespaces.forEach(async (namespace) => { + const resLogsDatastream = await es.transport.request({ + method: 'GET', + path: `/_data_stream/${logsTemplateName}-${namespace}`, + }); + const resMetricsDatastream = await es.transport.request({ + method: 'GET', + path: `/_data_stream/${metricsTemplateName}-${namespace}`, + }); + expect(resLogsDatastream.body.data_streams[0].indices.length).equal(2); + expect(resMetricsDatastream.body.data_streams[0].indices.length).equal(1); }); - expect(resLogsDatastream.body.data_streams[0].indices.length).equal(2); - expect(resMetricsDatastream.body.data_streams[0].indices.length).equal(1); }); + it('should be able to upgrade a package after a rollover', async function () { - await es.transport.request({ - method: 'POST', - path: `/${logsTemplateName}-default/_rollover`, - }); - const resLogsDatastream = await es.transport.request({ - method: 'GET', - path: `/_data_stream/${logsTemplateName}-default`, + namespaces.forEach(async (namespace) => { + await es.transport.request({ + method: 'POST', + path: `/${logsTemplateName}-${namespace}/_rollover`, + }); + const resLogsDatastream = await es.transport.request({ + method: 'GET', + path: `/_data_stream/${logsTemplateName}-${namespace}`, + }); + expect(resLogsDatastream.body.data_streams[0].indices.length).equal(2); }); - expect(resLogsDatastream.body.data_streams[0].indices.length).equal(2); await installPackage(pkgUpdateKey); }); }); diff --git a/x-pack/test/functional/apps/canvas/index.js b/x-pack/test/functional/apps/canvas/index.js index b7031cf0e55da..d5f7540f48c83 100644 --- a/x-pack/test/functional/apps/canvas/index.js +++ b/x-pack/test/functional/apps/canvas/index.js @@ -26,6 +26,7 @@ export default function canvasApp({ loadTestFile, getService }) { loadTestFile(require.resolve('./custom_elements')); loadTestFile(require.resolve('./feature_controls/canvas_security')); loadTestFile(require.resolve('./feature_controls/canvas_spaces')); + loadTestFile(require.resolve('./lens')); loadTestFile(require.resolve('./reports')); }); } diff --git a/x-pack/test/functional/apps/canvas/lens.ts b/x-pack/test/functional/apps/canvas/lens.ts new file mode 100644 index 0000000000000..e74795de6c7ea --- /dev/null +++ b/x-pack/test/functional/apps/canvas/lens.ts @@ -0,0 +1,30 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License; + * you may not use this file except in compliance with the Elastic License. + */ + +import { FtrProviderContext } from '../../ftr_provider_context'; + +export default function canvasLensTest({ getService, getPageObjects }: FtrProviderContext) { + const PageObjects = getPageObjects(['canvas', 'common', 'header', 'lens']); + const esArchiver = getService('esArchiver'); + + describe('lens in canvas', function () { + before(async () => { + await esArchiver.load('canvas/lens'); + // open canvas home + await PageObjects.common.navigateToApp('canvas'); + // load test workpad + await PageObjects.common.navigateToApp('canvas', { + hash: '/workpad/workpad-1705f884-6224-47de-ba49-ca224fe6ec31/page/1', + }); + }); + + it('renders lens visualization', async () => { + await PageObjects.header.waitUntilLoadingHasFinished(); + + await PageObjects.lens.assertMetric('Maximum of bytes', '16,788'); + }); + }); +} diff --git a/x-pack/test/functional/apps/dashboard/dashboard_lens_by_value.ts b/x-pack/test/functional/apps/dashboard/dashboard_lens_by_value.ts new file mode 100644 index 0000000000000..0a153aecec323 --- /dev/null +++ b/x-pack/test/functional/apps/dashboard/dashboard_lens_by_value.ts @@ -0,0 +1,72 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License; + * you may not use this file except in compliance with the Elastic License. + */ + +import expect from '@kbn/expect'; +import { FtrProviderContext } from '../../ftr_provider_context'; + +export default function ({ getPageObjects, getService }: FtrProviderContext) { + const PageObjects = getPageObjects(['common', 'dashboard', 'visualize', 'lens']); + + const find = getService('find'); + const esArchiver = getService('esArchiver'); + const dashboardPanelActions = getService('dashboardPanelActions'); + const dashboardVisualizations = getService('dashboardVisualizations'); + + describe('dashboard lens by value', function () { + before(async () => { + await esArchiver.loadIfNeeded('logstash_functional'); + await esArchiver.loadIfNeeded('lens/basic'); + await PageObjects.common.navigateToApp('dashboard'); + await PageObjects.dashboard.preserveCrossAppState(); + await PageObjects.dashboard.clickNewDashboard(); + }); + + it('can add a lens panel by value', async () => { + await dashboardVisualizations.ensureNewVisualizationDialogIsShowing(); + await PageObjects.lens.createAndAddLensFromDashboard({}); + const newPanelCount = await PageObjects.dashboard.getPanelCount(); + expect(newPanelCount).to.eql(1); + }); + + it('edits to a by value lens panel are properly applied', async () => { + await PageObjects.dashboard.waitForRenderComplete(); + await dashboardPanelActions.openContextMenu(); + await dashboardPanelActions.clickEdit(); + await PageObjects.lens.switchToVisualization('donut'); + await PageObjects.lens.saveAndReturn(); + await PageObjects.dashboard.waitForRenderComplete(); + + const pieExists = await find.existsByCssSelector('.lnsPieExpression__container'); + expect(pieExists).to.be(true); + }); + + it('editing and saving a lens by value panel retains number of panels', async () => { + const originalPanelCount = await PageObjects.dashboard.getPanelCount(); + await PageObjects.dashboard.waitForRenderComplete(); + await dashboardPanelActions.openContextMenu(); + await dashboardPanelActions.clickEdit(); + await PageObjects.lens.switchToVisualization('treemap'); + await PageObjects.lens.saveAndReturn(); + await PageObjects.dashboard.waitForRenderComplete(); + const newPanelCount = await PageObjects.dashboard.getPanelCount(); + expect(newPanelCount).to.eql(originalPanelCount); + }); + + it('updates panel on dashboard when a by value panel is saved to library', async () => { + const newTitle = 'look out library, here I come!'; + const originalPanelCount = await PageObjects.dashboard.getPanelCount(); + await PageObjects.dashboard.waitForRenderComplete(); + await dashboardPanelActions.openContextMenu(); + await dashboardPanelActions.clickEdit(); + await PageObjects.lens.save(newTitle, false, true); + await PageObjects.dashboard.waitForRenderComplete(); + const newPanelCount = await PageObjects.dashboard.getPanelCount(); + expect(newPanelCount).to.eql(originalPanelCount); + const titles = await PageObjects.dashboard.getPanelTitles(); + expect(titles.indexOf(newTitle)).to.not.be(-1); + }); + }); +} diff --git a/x-pack/test/functional/apps/dashboard/index.ts b/x-pack/test/functional/apps/dashboard/index.ts index 1ba87f89762a1..d6c0c4394e24a 100644 --- a/x-pack/test/functional/apps/dashboard/index.ts +++ b/x-pack/test/functional/apps/dashboard/index.ts @@ -15,5 +15,6 @@ export default function ({ loadTestFile }: FtrProviderContext) { loadTestFile(require.resolve('./drilldowns')); loadTestFile(require.resolve('./sync_colors')); loadTestFile(require.resolve('./_async_dashboard')); + loadTestFile(require.resolve('./dashboard_lens_by_value')); }); } diff --git a/x-pack/test/functional/apps/dashboard_mode/dashboard_empty_screen.js b/x-pack/test/functional/apps/dashboard_mode/dashboard_empty_screen.js index bd35374643e9b..9408bee7dc868 100644 --- a/x-pack/test/functional/apps/dashboard_mode/dashboard_empty_screen.js +++ b/x-pack/test/functional/apps/dashboard_mode/dashboard_empty_screen.js @@ -7,7 +7,6 @@ import expect from '@kbn/expect'; export default function ({ getPageObjects, getService }) { - const log = getService('log'); const testSubjects = getService('testSubjects'); const esArchiver = getService('esArchiver'); const dashboardVisualizations = getService('dashboardVisualizations'); @@ -27,40 +26,12 @@ export default function ({ getPageObjects, getService }) { await PageObjects.dashboard.gotoDashboardLandingPage(); }); - async function createAndAddLens(title, saveAsNew = false, redirectToOrigin = true) { - log.debug(`createAndAddLens(${title})`); - const inViewMode = await PageObjects.dashboard.getIsInViewMode(); - if (inViewMode) { - await PageObjects.dashboard.switchToEditMode(); - } - await PageObjects.visualize.clickLensWidget(); - await PageObjects.lens.goToTimeRange(); - await PageObjects.lens.configureDimension({ - dimension: 'lnsXY_xDimensionPanel > lns-empty-dimension', - operation: 'date_histogram', - field: '@timestamp', - }); - - await PageObjects.lens.configureDimension({ - dimension: 'lnsXY_yDimensionPanel > lns-empty-dimension', - operation: 'avg', - field: 'bytes', - }); - - await PageObjects.lens.configureDimension({ - dimension: 'lnsXY_splitDimensionPanel > lns-empty-dimension', - operation: 'terms', - field: 'ip', - }); - await PageObjects.lens.save(title, saveAsNew, redirectToOrigin); - } - it('adds Lens visualization to empty dashboard', async () => { const title = 'Dashboard Test Lens'; await testSubjects.exists('addVisualizationButton'); await testSubjects.click('addVisualizationButton'); await dashboardVisualizations.ensureNewVisualizationDialogIsShowing(); - await createAndAddLens(title); + await PageObjects.lens.createAndAddLensFromDashboard({ title, redirectToOrigin: true }); await PageObjects.dashboard.waitForRenderComplete(); await testSubjects.exists(`embeddablePanelHeading-${title}`); }); @@ -118,7 +89,7 @@ export default function ({ getPageObjects, getService }) { await testSubjects.exists('dashboardAddNewPanelButton'); await testSubjects.click('dashboardAddNewPanelButton'); await dashboardVisualizations.ensureNewVisualizationDialogIsShowing(); - await createAndAddLens(title, false, false); + await PageObjects.lens.createAndAddLensFromDashboard({ title }); await PageObjects.lens.notLinkedToOriginatingApp(); await PageObjects.common.navigateToApp('dashboard'); }); diff --git a/x-pack/test/functional/apps/ml/data_frame_analytics/classification_creation.ts b/x-pack/test/functional/apps/ml/data_frame_analytics/classification_creation.ts index 6b42306c08c92..b0f1e316e626a 100644 --- a/x-pack/test/functional/apps/ml/data_frame_analytics/classification_creation.ts +++ b/x-pack/test/functional/apps/ml/data_frame_analytics/classification_creation.ts @@ -40,6 +40,17 @@ export default function ({ getService }: FtrProviderContext) { modelMemory: '60mb', createIndexPattern: true, expected: { + scatterplotMatrixColorStats: [ + // background + { key: '#000000', value: 94 }, + // tick/grid/axis + { key: '#DDDDDD', value: 1 }, + { key: '#D3DAE6', value: 1 }, + { key: '#F5F7FA', value: 1 }, + // scatterplot circles + { key: '#6A717D', value: 1 }, + { key: '#54B39A', value: 1 }, + ], row: { type: 'classification', status: 'stopped', @@ -89,6 +100,12 @@ export default function ({ getService }: FtrProviderContext) { await ml.testExecution.logTestStep('displays the include fields selection'); await ml.dataFrameAnalyticsCreation.assertIncludeFieldsSelectionExists(); + await ml.testExecution.logTestStep('displays the scatterplot matrix'); + await ml.dataFrameAnalyticsScatterplot.assertScatterplotMatrix( + 'mlAnalyticsCreateJobWizardScatterplotMatrixFormRow', + testData.expected.scatterplotMatrixColorStats + ); + await ml.testExecution.logTestStep('continues to the additional options step'); await ml.dataFrameAnalyticsCreation.continueToAdditionalOptionsStep(); @@ -207,6 +224,10 @@ export default function ({ getService }: FtrProviderContext) { await ml.dataFrameAnalyticsResults.assertResultsTableExists(); await ml.dataFrameAnalyticsResults.assertResultsTableTrainingFiltersExist(); await ml.dataFrameAnalyticsResults.assertResultsTableNotEmpty(); + await ml.dataFrameAnalyticsScatterplot.assertScatterplotMatrix( + 'mlDFExpandableSection-splom', + testData.expected.scatterplotMatrixColorStats + ); }); it('displays the analytics job in the map view', async () => { diff --git a/x-pack/test/functional/apps/ml/data_frame_analytics/cloning.ts b/x-pack/test/functional/apps/ml/data_frame_analytics/cloning.ts index 532de930bc1a1..91ca0e6f32fd8 100644 --- a/x-pack/test/functional/apps/ml/data_frame_analytics/cloning.ts +++ b/x-pack/test/functional/apps/ml/data_frame_analytics/cloning.ts @@ -13,7 +13,8 @@ export default function ({ getService }: FtrProviderContext) { const esArchiver = getService('esArchiver'); const ml = getService('ml'); - describe('jobs cloning supported by UI form', function () { + // Failing ES promotion, see https://github.com/elastic/kibana/issues/89980 + describe.skip('jobs cloning supported by UI form', function () { const testDataList: Array<{ suiteTitle: string; archive: string; diff --git a/x-pack/test/functional/apps/ml/data_frame_analytics/outlier_detection_creation.ts b/x-pack/test/functional/apps/ml/data_frame_analytics/outlier_detection_creation.ts index 53daa0cae2522..419239d1d15ca 100644 --- a/x-pack/test/functional/apps/ml/data_frame_analytics/outlier_detection_creation.ts +++ b/x-pack/test/functional/apps/ml/data_frame_analytics/outlier_detection_creation.ts @@ -49,6 +49,27 @@ export default function ({ getService }: FtrProviderContext) { { chartAvailable: true, id: 'Exterior2nd', legend: '3 categories' }, { chartAvailable: true, id: 'Fireplaces', legend: '0 - 3' }, ], + scatterplotMatrixColorStatsWizard: [ + // background + { key: '#000000', value: 91 }, + // tick/grid/axis + { key: '#6A717D', value: 2 }, + { key: '#F5F7FA', value: 2 }, + { key: '#D3DAE6', value: 1 }, + // scatterplot circles + { key: '#54B399', value: 1 }, + { key: '#54B39A', value: 1 }, + ], + scatterplotMatrixColorStatsResults: [ + // background + { key: '#000000', value: 91 }, + // tick/grid/axis, grey markers + // the red outlier color is not above the 1% threshold. + { key: '#6A717D', value: 2 }, + { key: '#98A2B3', value: 1 }, + { key: '#F5F7FA', value: 2 }, + { key: '#D3DAE6', value: 1 }, + ], row: { type: 'outlier_detection', status: 'stopped', @@ -105,6 +126,12 @@ export default function ({ getService }: FtrProviderContext) { await ml.testExecution.logTestStep('displays the include fields selection'); await ml.dataFrameAnalyticsCreation.assertIncludeFieldsSelectionExists(); + await ml.testExecution.logTestStep('displays the scatterplot matrix'); + await ml.dataFrameAnalyticsScatterplot.assertScatterplotMatrix( + 'mlAnalyticsCreateJobWizardScatterplotMatrixFormRow', + testData.expected.scatterplotMatrixColorStatsWizard + ); + await ml.testExecution.logTestStep('continues to the additional options step'); await ml.dataFrameAnalyticsCreation.continueToAdditionalOptionsStep(); @@ -221,6 +248,10 @@ export default function ({ getService }: FtrProviderContext) { await ml.dataFrameAnalyticsResults.assertOutlierTablePanelExists(); await ml.dataFrameAnalyticsResults.assertResultsTableExists(); await ml.dataFrameAnalyticsResults.assertResultsTableNotEmpty(); + await ml.dataFrameAnalyticsScatterplot.assertScatterplotMatrix( + 'mlDFExpandableSection-splom', + testData.expected.scatterplotMatrixColorStatsResults + ); }); it('displays the analytics job in the map view', async () => { diff --git a/x-pack/test/functional/apps/ml/data_frame_analytics/regression_creation.ts b/x-pack/test/functional/apps/ml/data_frame_analytics/regression_creation.ts index fef22fcebc3ed..f1d19a82caa9b 100644 --- a/x-pack/test/functional/apps/ml/data_frame_analytics/regression_creation.ts +++ b/x-pack/test/functional/apps/ml/data_frame_analytics/regression_creation.ts @@ -39,6 +39,16 @@ export default function ({ getService }: FtrProviderContext) { modelMemory: '20mb', createIndexPattern: true, expected: { + scatterplotMatrixColorStats: [ + // background + { key: '#000000', value: 80 }, + // tick/grid/axis + { key: '#6A717D', value: 1 }, + { key: '#F5F7FA', value: 2 }, + { key: '#D3DAE6', value: 1 }, + // because a continuous color scale is used for the scatterplot circles, + // none of the generated colors is above the 1% threshold. + ], row: { type: 'regression', status: 'stopped', @@ -89,6 +99,12 @@ export default function ({ getService }: FtrProviderContext) { await ml.testExecution.logTestStep('displays the include fields selection'); await ml.dataFrameAnalyticsCreation.assertIncludeFieldsSelectionExists(); + await ml.testExecution.logTestStep('displays the scatterplot matrix'); + await ml.dataFrameAnalyticsScatterplot.assertScatterplotMatrix( + 'mlAnalyticsCreateJobWizardScatterplotMatrixFormRow', + testData.expected.scatterplotMatrixColorStats + ); + await ml.testExecution.logTestStep('continues to the additional options step'); await ml.dataFrameAnalyticsCreation.continueToAdditionalOptionsStep(); @@ -207,6 +223,10 @@ export default function ({ getService }: FtrProviderContext) { await ml.dataFrameAnalyticsResults.assertResultsTableExists(); await ml.dataFrameAnalyticsResults.assertResultsTableTrainingFiltersExist(); await ml.dataFrameAnalyticsResults.assertResultsTableNotEmpty(); + await ml.dataFrameAnalyticsScatterplot.assertScatterplotMatrix( + 'mlDFExpandableSection-splom', + testData.expected.scatterplotMatrixColorStats + ); }); it('displays the analytics job in the map view', async () => { diff --git a/x-pack/test/functional/config.js b/x-pack/test/functional/config.js index 1815942a06a9a..fc508f8477ebe 100644 --- a/x-pack/test/functional/config.js +++ b/x-pack/test/functional/config.js @@ -232,6 +232,7 @@ export default async function ({ readConfigFile }) { { feature: { canvas: ['all'], + visualize: ['all'], }, spaces: ['*'], }, diff --git a/x-pack/test/functional/es_archives/canvas/lens/data.json b/x-pack/test/functional/es_archives/canvas/lens/data.json new file mode 100644 index 0000000000000..dca7d31d71082 --- /dev/null +++ b/x-pack/test/functional/es_archives/canvas/lens/data.json @@ -0,0 +1,190 @@ +{ + "type": "doc", + "value": { + "id": "space:default", + "index": ".kibana_1", + "source": { + "space": { + "_reserved": true, + "color": "#00bfb3", + "description": "This is your default space!", + "name": "Default" + }, + "type": "space", + "updated_at": "2018-11-06T18:20:26.703Z" + } + } +} + +{ + "type": "doc", + "value": { + "id": "canvas-workpad:workpad-1705f884-6224-47de-ba49-ca224fe6ec31", + "index": ".kibana_1", + "source": { + "canvas-workpad": { + "@created": "2018-11-19T19:17:12.646Z", + "@timestamp": "2018-11-19T19:36:28.499Z", + "assets": { + }, + "colors": [ + "#37988d", + "#c19628", + "#b83c6f", + "#3f9939", + "#1785b0", + "#ca5f35", + "#45bdb0", + "#f2bc33", + "#e74b8b", + "#4fbf48", + "#1ea6dc", + "#fd7643", + "#72cec3", + "#f5cc5d", + "#ec77a8", + "#7acf74", + "#4cbce4", + "#fd986f", + "#a1ded7", + "#f8dd91", + "#f2a4c5", + "#a6dfa2", + "#86d2ed", + "#fdba9f", + "#000000", + "#444444", + "#777777", + "#BBBBBB", + "#FFFFFF", + "rgba(255,255,255,0)" + ], + "height": 920, + "id": "workpad-1705f884-6224-47de-ba49-ca224fe6ec31", + "isWriteable": true, + "name": "Test Workpad", + "page": 0, + "pages": [ + { + "elements": [ + { + "expression": "savedLens id=\"my-lens-vis\" timerange={timerange from=\"2014-01-01\" to=\"2018-01-01\"}", + "id": "element-8f64a10a-01f3-4a71-a682-5b627cbe4d0e", + "position": { + "angle": 0, + "height": 238, + "left": 33.5, + "top": 20, + "width": 338 + } + } + ], + "id": "page-c38cd459-10fe-45f9-847b-2cbd7ec74319", + "style": { + "background": "#fff" + }, + "transition": { + } + } + ], + "width": 840 + }, + "type": "canvas-workpad", + "updated_at": "2018-11-19T19:36:28.511Z" + } + } +} + +{ + "type": "doc", + "value": { + "id": "lens:my-lens-vis", + "index": ".kibana_1", + "source": { + "lens": { + "expression": "", + "state": { + "datasourceMetaData": { + "filterableIndexPatterns": [ + { + "id": "logstash-lens", + "title": "logstash-lens" + } + ] + }, + "datasourceStates": { + "indexpattern": { + "currentIndexPatternId": "logstash-lens", + "layers": { + "c61a8afb-a185-4fae-a064-fb3846f6c451": { + "columnOrder": [ + "2cd09808-3915-49f4-b3b0-82767eba23f7" + ], + "columns": { + "2cd09808-3915-49f4-b3b0-82767eba23f7": { + "dataType": "number", + "isBucketed": false, + "label": "Maximum of bytes", + "operationType": "max", + "scale": "ratio", + "sourceField": "bytes" + } + }, + "indexPatternId": "logstash-lens" + } + } + } + }, + "filters": [], + "query": { + "language": "kuery", + "query": "" + }, + "visualization": { + "accessor": "2cd09808-3915-49f4-b3b0-82767eba23f7", + "layerId": "c61a8afb-a185-4fae-a064-fb3846f6c451" + } + }, + "title": "Artistpreviouslyknownaslens", + "visualizationType": "lnsMetric" + }, + "references": [], + "type": "lens", + "updated_at": "2019-10-16T00:28:08.979Z" + } + } +} + +{ + "type": "doc", + "value": { + "index": "logstash-lens", + "id": "1", + "source": { + "@timestamp": "2015-09-20T02:00:00.000Z", + "bytes": 16788 + } + } +} + +{ + "type": "doc", + "value": { + "id": "index-pattern:logstash-lens", + "index": ".kibana_1", + "source": { + "index-pattern" : { + "title" : "logstash-lens", + "timeFieldName" : "@timestamp", + "fields" : "[]" + }, + "type" : "index-pattern", + "references" : [ ], + "migrationVersion" : { + "index-pattern" : "7.6.0" + }, + "updated_at" : "2020-08-19T08:39:09.998Z" + }, + "type": "_doc" + } +} \ No newline at end of file diff --git a/x-pack/test/functional/es_archives/canvas/lens/mappings.json b/x-pack/test/functional/es_archives/canvas/lens/mappings.json new file mode 100644 index 0000000000000..811bfaaae0d2c --- /dev/null +++ b/x-pack/test/functional/es_archives/canvas/lens/mappings.json @@ -0,0 +1,409 @@ +{ + "type": "index", + "value": { + "aliases": { + ".kibana": { + } + }, + "index": ".kibana_1", + "mappings": { + "dynamic": "strict", + "properties": { + "canvas-workpad": { + "dynamic": "false", + "properties": { + "@created": { + "type": "date" + }, + "@timestamp": { + "type": "date" + }, + "id": { + "index": false, + "type": "text" + }, + "name": { + "fields": { + "keyword": { + "type": "keyword" + } + }, + "type": "text" + } + } + }, + "config": { + "dynamic": "true", + "properties": { + "buildNum": { + "type": "keyword" + } + } + }, + "references": { + "properties": { + "id": { + "type": "keyword" + }, + "name": { + "type": "keyword" + }, + "type": { + "type": "keyword" + } + }, + "type": "nested" + }, + "dashboard": { + "properties": { + "description": { + "type": "text" + }, + "hits": { + "type": "integer" + }, + "kibanaSavedObjectMeta": { + "properties": { + "searchSourceJSON": { + "type": "text" + } + } + }, + "optionsJSON": { + "type": "text" + }, + "panelsJSON": { + "type": "text" + }, + "refreshInterval": { + "properties": { + "display": { + "type": "keyword" + }, + "pause": { + "type": "boolean" + }, + "section": { + "type": "integer" + }, + "value": { + "type": "integer" + } + } + }, + "timeFrom": { + "type": "keyword" + }, + "timeRestore": { + "type": "boolean" + }, + "timeTo": { + "type": "keyword" + }, + "title": { + "type": "text" + }, + "uiStateJSON": { + "type": "text" + }, + "version": { + "type": "integer" + } + } + }, + "graph-workspace": { + "properties": { + "description": { + "type": "text" + }, + "kibanaSavedObjectMeta": { + "properties": { + "searchSourceJSON": { + "type": "text" + } + } + }, + "numLinks": { + "type": "integer" + }, + "numVertices": { + "type": "integer" + }, + "title": { + "type": "text" + }, + "version": { + "type": "integer" + }, + "wsState": { + "type": "text" + } + } + }, + "index-pattern": { + "properties": { + "fieldFormatMap": { + "type": "text" + }, + "fields": { + "type": "text" + }, + "intervalName": { + "type": "keyword" + }, + "notExpandable": { + "type": "boolean" + }, + "sourceFilters": { + "type": "text" + }, + "timeFieldName": { + "type": "keyword" + }, + "title": { + "type": "text" + }, + "type": { + "type": "keyword" + }, + "typeMeta": { + "type": "keyword" + } + } + }, + "lens": { + "properties": { + "expression": { + "index": false, + "type": "keyword" + }, + "state": { + "type": "flattened" + }, + "title": { + "type": "text" + }, + "visualizationType": { + "type": "keyword" + } + } + }, + "kql-telemetry": { + "properties": { + "optInCount": { + "type": "long" + }, + "optOutCount": { + "type": "long" + } + } + }, + "migrationVersion": { + "dynamic": "true", + "type": "object" + }, + "namespace": { + "type": "keyword" + }, + "search": { + "properties": { + "columns": { + "type": "keyword" + }, + "description": { + "type": "text" + }, + "hits": { + "type": "integer" + }, + "kibanaSavedObjectMeta": { + "properties": { + "searchSourceJSON": { + "type": "text" + } + } + }, + "sort": { + "type": "keyword" + }, + "title": { + "type": "text" + }, + "version": { + "type": "integer" + } + } + }, + "server": { + "properties": { + "uuid": { + "type": "keyword" + } + } + }, + "space": { + "properties": { + "_reserved": { + "type": "boolean" + }, + "color": { + "type": "keyword" + }, + "description": { + "type": "text" + }, + "initials": { + "type": "keyword" + }, + "disabledFeatures": { + "type": "keyword" + }, + "name": { + "fields": { + "keyword": { + "ignore_above": 2048, + "type": "keyword" + } + }, + "type": "text" + } + } + }, + "telemetry": { + "properties": { + "enabled": { + "type": "boolean" + } + } + }, + "timelion-sheet": { + "properties": { + "description": { + "type": "text" + }, + "hits": { + "type": "integer" + }, + "kibanaSavedObjectMeta": { + "properties": { + "searchSourceJSON": { + "type": "text" + } + } + }, + "timelion_chart_height": { + "type": "integer" + }, + "timelion_columns": { + "type": "integer" + }, + "timelion_interval": { + "type": "keyword" + }, + "timelion_other_interval": { + "type": "keyword" + }, + "timelion_rows": { + "type": "integer" + }, + "timelion_sheet": { + "type": "text" + }, + "title": { + "type": "text" + }, + "version": { + "type": "integer" + } + } + }, + "type": { + "type": "keyword" + }, + "updated_at": { + "type": "date" + }, + "url": { + "properties": { + "accessCount": { + "type": "long" + }, + "accessDate": { + "type": "date" + }, + "createDate": { + "type": "date" + }, + "url": { + "fields": { + "keyword": { + "ignore_above": 2048, + "type": "keyword" + } + }, + "type": "text" + } + } + }, + "visualization": { + "properties": { + "description": { + "type": "text" + }, + "kibanaSavedObjectMeta": { + "properties": { + "searchSourceJSON": { + "type": "text" + } + } + }, + "savedSearchId": { + "type": "keyword" + }, + "title": { + "type": "text" + }, + "uiStateJSON": { + "type": "text" + }, + "version": { + "type": "integer" + }, + "visState": { + "type": "text" + } + } + } + } + }, + "settings": { + "index": { + "number_of_replicas": "1", + "number_of_shards": "1" + } + } + } +} + +{ + "type": "index", + "value": { + "index": "logstash-lens", + "mappings": { + "properties": { + "@timestamp": { + "type": "date" + }, + "bytes": { + "type": "float" + } + } + }, + "settings": { + "index": { + "number_of_shards": "1", + "number_of_replicas": "0" + } + } + } +} \ No newline at end of file diff --git a/x-pack/test/functional/es_archives/data/search_sessions/data.json.gz b/x-pack/test/functional/es_archives/data/search_sessions/data.json.gz index 28260ee99e4dc..51e8c09f19247 100644 Binary files a/x-pack/test/functional/es_archives/data/search_sessions/data.json.gz and b/x-pack/test/functional/es_archives/data/search_sessions/data.json.gz differ diff --git a/x-pack/test/functional/es_archives/data/search_sessions/mappings.json b/x-pack/test/functional/es_archives/data/search_sessions/mappings.json index 24bbcbea23385..4492bcae7047d 100644 --- a/x-pack/test/functional/es_archives/data/search_sessions/mappings.json +++ b/x-pack/test/functional/es_archives/data/search_sessions/mappings.json @@ -310,6 +310,9 @@ "created": { "type": "date" }, + "touched": { + "type": "date" + }, "expires": { "type": "date" }, @@ -324,6 +327,9 @@ "name": { "type": "keyword" }, + "persisted": { + "type": "boolean" + }, "restoreState": { "enabled": false, "type": "object" diff --git a/x-pack/test/functional/page_objects/lens_page.ts b/x-pack/test/functional/page_objects/lens_page.ts index dabead6ffbdad..292e91866d690 100644 --- a/x-pack/test/functional/page_objects/lens_page.ts +++ b/x-pack/test/functional/page_objects/lens_page.ts @@ -16,7 +16,7 @@ export function LensPageProvider({ getService, getPageObjects }: FtrProviderCont const find = getService('find'); const comboBox = getService('comboBox'); const browser = getService('browser'); - const PageObjects = getPageObjects(['header', 'timePicker', 'common', 'visualize']); + const PageObjects = getPageObjects(['header', 'timePicker', 'common', 'visualize', 'dashboard']); return logWrapper('lensPage', log, { /** @@ -202,7 +202,7 @@ export function LensPageProvider({ getService, getPageObjects }: FtrProviderCont }) .lnsDragDrop`; const dropping = `[data-test-subj='${dimension}']:nth-of-type(${ endIndex + 1 - }) [data-test-subj='lnsDragDrop-reorderableDrop'`; + }) [data-test-subj='lnsDragDrop-reorderableDropLayer'`; await browser.html5DragAndDrop(dragging, dropping); await PageObjects.header.waitUntilLoadingHasFinished(); }, @@ -586,5 +586,49 @@ export function LensPageProvider({ getService, getPageObjects }: FtrProviderCont // TODO: target dimensionTrigger color element after merging https://github.com/elastic/kibana/pull/76871 await testSubjects.getAttribute('colorPickerAnchor', color); }, + + /** + * Creates and saves a lens visualization from a dashboard + * + * @param title - title for the new lens. If left undefined, the panel will be created by value + * @param redirectToOrigin - whether to redirect back to the dashboard after saving the panel + */ + async createAndAddLensFromDashboard({ + title, + redirectToOrigin, + }: { + title?: string; + redirectToOrigin?: boolean; + }) { + log.debug(`createAndAddLens${title}`); + const inViewMode = await PageObjects.dashboard.getIsInViewMode(); + if (inViewMode) { + await PageObjects.dashboard.switchToEditMode(); + } + await PageObjects.visualize.clickLensWidget(); + await this.goToTimeRange(); + await this.configureDimension({ + dimension: 'lnsXY_xDimensionPanel > lns-empty-dimension', + operation: 'date_histogram', + field: '@timestamp', + }); + + await this.configureDimension({ + dimension: 'lnsXY_yDimensionPanel > lns-empty-dimension', + operation: 'avg', + field: 'bytes', + }); + + await this.configureDimension({ + dimension: 'lnsXY_splitDimensionPanel > lns-empty-dimension', + operation: 'terms', + field: 'ip', + }); + if (title) { + await this.save(title, false, redirectToOrigin); + } else { + await this.saveAndReturn(); + } + }, }); } diff --git a/x-pack/test/functional/services/canvas_element.ts b/x-pack/test/functional/services/canvas_element.ts index e2a42c5dc43c3..08ac38d970225 100644 --- a/x-pack/test/functional/services/canvas_element.ts +++ b/x-pack/test/functional/services/canvas_element.ts @@ -43,9 +43,13 @@ export async function CanvasElementProvider({ getService }: FtrProviderContext) public async getImageData(selector: string): Promise { return await driver.executeScript( ` - const el = document.querySelector('${selector}'); - const ctx = el.getContext('2d'); - return ctx.getImageData(0, 0, el.width, el.height).data; + try { + const el = document.querySelector('${selector}'); + const ctx = el.getContext('2d'); + return ctx.getImageData(0, 0, el.width, el.height).data; + } catch(e) { + return []; + } ` ); } diff --git a/x-pack/test/functional/services/ml/data_frame_analytics_scatterplot.ts b/x-pack/test/functional/services/ml/data_frame_analytics_scatterplot.ts new file mode 100644 index 0000000000000..3472e5079c79a --- /dev/null +++ b/x-pack/test/functional/services/ml/data_frame_analytics_scatterplot.ts @@ -0,0 +1,40 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License; + * you may not use this file except in compliance with the Elastic License. + */ +import expect from '@kbn/expect'; + +import { FtrProviderContext } from '../../ftr_provider_context'; + +export function MachineLearningDataFrameAnalyticsScatterplotProvider({ + getService, +}: FtrProviderContext) { + const canvasElement = getService('canvasElement'); + const testSubjects = getService('testSubjects'); + + return new (class AnalyticsScatterplot { + public async assertScatterplotMatrix( + dataTestSubj: string, + expectedColorStats: Array<{ + key: string; + value: number; + }> + ) { + await testSubjects.existOrFail(dataTestSubj); + await testSubjects.existOrFail('mlScatterplotMatrix'); + + const actualColorStats = await canvasElement.getColorStats( + `[data-test-subj="mlScatterplotMatrix"] canvas`, + expectedColorStats, + 1 + ); + expect(actualColorStats.every((d) => d.withinTolerance)).to.eql( + true, + `Color stats for scatterplot matrix should be within tolerance. Expected: '${JSON.stringify( + expectedColorStats + )}' (got '${JSON.stringify(actualColorStats)}')` + ); + } + })(); +} diff --git a/x-pack/test/functional/services/ml/index.ts b/x-pack/test/functional/services/ml/index.ts index c1a9ac304dd69..aa87bc5dc4772 100644 --- a/x-pack/test/functional/services/ml/index.ts +++ b/x-pack/test/functional/services/ml/index.ts @@ -17,6 +17,7 @@ import { MachineLearningDataFrameAnalyticsProvider } from './data_frame_analytic import { MachineLearningDataFrameAnalyticsCreationProvider } from './data_frame_analytics_creation'; import { MachineLearningDataFrameAnalyticsEditProvider } from './data_frame_analytics_edit'; import { MachineLearningDataFrameAnalyticsResultsProvider } from './data_frame_analytics_results'; +import { MachineLearningDataFrameAnalyticsScatterplotProvider } from './data_frame_analytics_scatterplot'; import { MachineLearningDataFrameAnalyticsMapProvider } from './data_frame_analytics_map'; import { MachineLearningDataFrameAnalyticsTableProvider } from './data_frame_analytics_table'; import { MachineLearningDataVisualizerProvider } from './data_visualizer'; @@ -63,6 +64,9 @@ export function MachineLearningProvider(context: FtrProviderContext) { const dataFrameAnalyticsResults = MachineLearningDataFrameAnalyticsResultsProvider(context); const dataFrameAnalyticsMap = MachineLearningDataFrameAnalyticsMapProvider(context); const dataFrameAnalyticsTable = MachineLearningDataFrameAnalyticsTableProvider(context); + const dataFrameAnalyticsScatterplot = MachineLearningDataFrameAnalyticsScatterplotProvider( + context + ); const dataVisualizer = MachineLearningDataVisualizerProvider(context); const dataVisualizerTable = MachineLearningDataVisualizerTableProvider(context, commonUI); @@ -105,6 +109,7 @@ export function MachineLearningProvider(context: FtrProviderContext) { dataFrameAnalyticsResults, dataFrameAnalyticsMap, dataFrameAnalyticsTable, + dataFrameAnalyticsScatterplot, dataVisualizer, dataVisualizerFileBased, dataVisualizerIndexBased, diff --git a/x-pack/test/load/runner.ts b/x-pack/test/load/runner.ts index e9a1cadfddc55..52448a6b32a99 100644 --- a/x-pack/test/load/runner.ts +++ b/x-pack/test/load/runner.ts @@ -7,23 +7,57 @@ import { withProcRunner } from '@kbn/dev-utils'; import { resolve } from 'path'; import { REPO_ROOT } from '@kbn/utils'; +import Fs from 'fs'; +import { createFlagError } from '@kbn/dev-utils'; import { FtrProviderContext } from './../functional/ftr_provider_context'; +const baseSimulationPath = 'src/test/scala/org/kibanaLoadTest/simulation'; +const simulationPackage = 'org.kibanaLoadTest.simulation'; +const simulationFIleExtension = '.scala'; +const gatlingProjectRootPath: string = + process.env.GATLING_PROJECT_PATH || resolve(REPO_ROOT, '../kibana-load-testing'); +const simulationEntry: string = process.env.GATLING_SIMULATIONS || 'DemoJourney'; + +if (!Fs.existsSync(gatlingProjectRootPath)) { + throw createFlagError( + `Incorrect path to load testing project: '${gatlingProjectRootPath}'\n + Clone 'elastic/kibana-load-testing' and set path using 'GATLING_PROJECT_PATH' env var` + ); +} + +const dropEmptyLines = (s: string) => s.split(',').filter((i) => i.length > 0); +const simulationClasses = dropEmptyLines(simulationEntry); +const simulationsRootPath = resolve(gatlingProjectRootPath, baseSimulationPath); + +simulationClasses.map((className) => { + const simulationClassPath = resolve( + simulationsRootPath, + className.replace('.', '/') + simulationFIleExtension + ); + if (!Fs.existsSync(simulationClassPath)) { + throw createFlagError(`Simulation class is not found: '${simulationClassPath}'`); + } +}); + +/** + * + * GatlingTestRunner is used to run load simulation against local Kibana instance + * + * Use GATLING_SIMULATIONS to pass comma-separated class names + * Use GATLING_PROJECT_PATH to override path to 'kibana-load-testing' project + */ export async function GatlingTestRunner({ getService }: FtrProviderContext) { const log = getService('log'); - const gatlingProjectRootPath = resolve(REPO_ROOT, '../kibana-load-testing'); await withProcRunner(log, async (procs) => { - await procs.run('gatling', { + await procs.run('mvn: clean compile', { cmd: 'mvn', args: [ - 'clean', - '-q', + '-Dmaven.wagon.http.retryHandler.count=3', '-Dmaven.test.failure.ignore=true', - 'compile', - 'gatling:test', '-q', - '-Dgatling.simulationClass=org.kibanaLoadTest.simulation.DemoJourney', + 'clean', + 'compile', ], cwd: gatlingProjectRootPath, env: { @@ -31,5 +65,20 @@ export async function GatlingTestRunner({ getService }: FtrProviderContext) { }, wait: true, }); + for (const simulationClass of simulationClasses) { + await procs.run('gatling: test', { + cmd: 'mvn', + args: [ + 'gatling:test', + '-q', + `-Dgatling.simulationClass=${simulationPackage}.${simulationClass}`, + ], + cwd: gatlingProjectRootPath, + env: { + ...process.env, + }, + wait: true, + }); + } }); } diff --git a/x-pack/test/send_search_to_background_integration/tests/apps/dashboard/async_search/send_to_background.ts b/x-pack/test/send_search_to_background_integration/tests/apps/dashboard/async_search/send_to_background.ts index 7e878e763bfc1..35ee15472f346 100644 --- a/x-pack/test/send_search_to_background_integration/tests/apps/dashboard/async_search/send_to_background.ts +++ b/x-pack/test/send_search_to_background_integration/tests/apps/dashboard/async_search/send_to_background.ts @@ -80,6 +80,7 @@ export default function ({ getService, getPageObjects }: FtrProviderContext) { ); // load URL to restore a saved session + // TODO: replace with clicking on "Re-run link" const url = await browser.getCurrentUrl(); const savedSessionURL = `${url}&searchSessionId=${savedSessionId}`; await browser.get(savedSessionURL); @@ -96,6 +97,10 @@ export default function ({ getService, getPageObjects }: FtrProviderContext) { // should leave session state untouched await PageObjects.dashboard.switchToEditMode(); await searchSessions.expectState('restored'); + + // navigating to a listing page clears the session + await PageObjects.dashboard.gotoDashboardLandingPage(); + await searchSessions.missingOrFail(); }); }); } diff --git a/x-pack/test/send_search_to_background_integration/tests/apps/dashboard/async_search/send_to_background_relative_time.ts b/x-pack/test/send_search_to_background_integration/tests/apps/dashboard/async_search/send_to_background_relative_time.ts index 25291fd74b322..5d5cdb29523bd 100644 --- a/x-pack/test/send_search_to_background_integration/tests/apps/dashboard/async_search/send_to_background_relative_time.ts +++ b/x-pack/test/send_search_to_background_integration/tests/apps/dashboard/async_search/send_to_background_relative_time.ts @@ -19,13 +19,13 @@ export default function ({ getService, getPageObjects }: FtrProviderContext) { 'home', 'timePicker', 'maps', + 'searchSessionsManagement', ]); const dashboardPanelActions = getService('dashboardPanelActions'); const inspector = getService('inspector'); const pieChart = getService('pieChart'); const find = getService('find'); const dashboardExpect = getService('dashboardExpect'); - const browser = getService('browser'); const searchSessions = getService('searchSessions'); describe('send to background with relative time', () => { @@ -59,23 +59,20 @@ export default function ({ getService, getPageObjects }: FtrProviderContext) { await PageObjects.timePicker.pauseAutoRefresh(); // sample data has auto-refresh on await PageObjects.header.waitUntilLoadingHasFinished(); await PageObjects.dashboard.waitForRenderComplete(); - await checkSampleDashboardLoaded(); await searchSessions.expectState('completed'); await searchSessions.save(); await searchSessions.expectState('backgroundCompleted'); - const savedSessionId = await dashboardPanelActions.getSearchSessionIdByTitle( - '[Flights] Airline Carrier' - ); - const resolvedTimeRange = await getResolvedTimeRangeFromPanel('[Flights] Airline Carrier'); + + await checkSampleDashboardLoaded(); // load URL to restore a saved session - const url = await browser.getCurrentUrl(); - const savedSessionURL = `${url}&searchSessionId=${savedSessionId}` - .replace('now-24h', `'${resolvedTimeRange.gte}'`) - .replace('now', `'${resolvedTimeRange.lte}'`); - log.debug('Trying to restore session by URL:', savedSessionId); - await browser.get(savedSessionURL); + await PageObjects.searchSessionsManagement.goTo(); + const searchSessionList = await PageObjects.searchSessionsManagement.getList(); + + // navigate to dashboard + await searchSessionList[0].view(); + await PageObjects.header.waitUntilLoadingHasFinished(); await PageObjects.dashboard.waitForRenderComplete(); await checkSampleDashboardLoaded(); @@ -87,16 +84,6 @@ export default function ({ getService, getPageObjects }: FtrProviderContext) { // HELPERS - async function getResolvedTimeRangeFromPanel( - panelTitle: string - ): Promise<{ gte: string; lte: string }> { - await dashboardPanelActions.openInspectorByTitle(panelTitle); - await inspector.openInspectorRequestsView(); - await (await inspector.getOpenRequestDetailRequestButton()).click(); - const request = JSON.parse(await inspector.getCodeEditorValue()); - return request.query.bool.filter.find((f: any) => f.range).range.timestamp; - } - async function checkSampleDashboardLoaded() { log.debug('Checking no error labels'); await testSubjects.missingOrFail('embeddableErrorLabel'); diff --git a/x-pack/test/tsconfig.json b/x-pack/test/tsconfig.json index 5232af0dd304b..a9d76eea80d8f 100644 --- a/x-pack/test/tsconfig.json +++ b/x-pack/test/tsconfig.json @@ -24,6 +24,7 @@ { "path": "../../src/plugins/kibana_react/tsconfig.json" }, { "path": "../../src/plugins/kibana_usage_collection/tsconfig.json" }, { "path": "../../src/plugins/kibana_utils/tsconfig.json" }, + { "path": "../../src/plugins/legacy_export/tsconfig.json" }, { "path": "../../src/plugins/navigation/tsconfig.json" }, { "path": "../../src/plugins/newsfeed/tsconfig.json" }, { "path": "../../src/plugins/saved_objects/tsconfig.json" }, @@ -35,9 +36,10 @@ { "path": "../../src/plugins/usage_collection/tsconfig.json" }, { "path": "../../src/plugins/ui_actions/tsconfig.json" }, { "path": "../../src/plugins/url_forwarding/tsconfig.json" }, + { "path": "../../src/plugins/index_pattern_management/tsconfig.json" }, - { "path": "../plugins/actions/tsconfig.json"}, - { "path": "../plugins/alerts/tsconfig.json"}, + { "path": "../plugins/actions/tsconfig.json" }, + { "path": "../plugins/alerts/tsconfig.json" }, { "path": "../plugins/console_extensions/tsconfig.json" }, { "path": "../plugins/data_enhanced/tsconfig.json" }, { "path": "../plugins/enterprise_search/tsconfig.json" }, @@ -60,7 +62,10 @@ { "path": "../plugins/cloud/tsconfig.json" }, { "path": "../plugins/saved_objects_tagging/tsconfig.json" }, { "path": "../plugins/global_search_bar/tsconfig.json" }, + { "path": "../plugins/ingest_pipelines/tsconfig.json" }, { "path": "../plugins/license_management/tsconfig.json" }, + { "path": "../plugins/snapshot_restore/tsconfig.json" }, + { "path": "../plugins/grokdebugger/tsconfig.json" }, { "path": "../plugins/painless_lab/tsconfig.json" }, { "path": "../plugins/watcher/tsconfig.json" } ] diff --git a/x-pack/test/upgrade_assistant_integration/upgrade_assistant/reindexing.js b/x-pack/test/upgrade_assistant_integration/upgrade_assistant/reindexing.js index f57d2184d4b8d..c70b90b384d13 100644 --- a/x-pack/test/upgrade_assistant_integration/upgrade_assistant/reindexing.js +++ b/x-pack/test/upgrade_assistant_integration/upgrade_assistant/reindexing.js @@ -71,6 +71,8 @@ export default function ({ getService }) { expect(indexSummary[newIndexName]).to.be.an('object'); // The original index name is aliased to the new one expect(indexSummary[newIndexName].aliases.dummydata).to.be.an('object'); + // Verify mappings exist on new index + expect(indexSummary[newIndexName].mappings.properties).to.be.an('object'); // The number of documents in the new index matches what we expect expect((await es.count({ index: lastState.newIndexName })).body.count).to.be(3); diff --git a/x-pack/tsconfig.json b/x-pack/tsconfig.json index 1be6b5cf84cda..7f64a552a5169 100644 --- a/x-pack/tsconfig.json +++ b/x-pack/tsconfig.json @@ -40,9 +40,12 @@ "plugins/cloud/**/*", "plugins/saved_objects_tagging/**/*", "plugins/global_search_bar/**/*", + "plugins/ingest_pipelines/**/*", "plugins/license_management/**/*", + "plugins/snapshot_restore/**/*", "plugins/painless_lab/**/*", "plugins/watcher/**/*", + "plugins/grokdebugger/**/*", "test/**/*" ], "compilerOptions": { @@ -67,6 +70,7 @@ { "path": "../src/plugins/kibana_react/tsconfig.json" }, { "path": "../src/plugins/kibana_usage_collection/tsconfig.json" }, { "path": "../src/plugins/kibana_utils/tsconfig.json" }, + { "path": "../src/plugins/legacy_export/tsconfig.json" }, { "path": "../src/plugins/management/tsconfig.json" }, { "path": "../src/plugins/navigation/tsconfig.json" }, { "path": "../src/plugins/newsfeed/tsconfig.json" }, @@ -82,6 +86,7 @@ { "path": "../src/plugins/ui_actions/tsconfig.json" }, { "path": "../src/plugins/url_forwarding/tsconfig.json" }, { "path": "../src/plugins/usage_collection/tsconfig.json" }, + { "path": "../src/plugins/index_pattern_management/tsconfig.json" }, { "path": "./plugins/actions/tsconfig.json"}, { "path": "./plugins/alerts/tsconfig.json"}, { "path": "./plugins/beats_management/tsconfig.json" }, @@ -110,12 +115,19 @@ { "path": "./plugins/searchprofiler/tsconfig.json" }, { "path": "./plugins/security/tsconfig.json" }, { "path": "./plugins/spaces/tsconfig.json" }, - { "path": "./plugins/stack_alerts/tsconfig.json"}, + { "path": "./plugins/stack_alerts/tsconfig.json" }, { "path": "./plugins/task_manager/tsconfig.json" }, { "path": "./plugins/telemetry_collection_xpack/tsconfig.json" }, { "path": "./plugins/translations/tsconfig.json" }, { "path": "./plugins/triggers_actions_ui/tsconfig.json"}, + { "path": "./plugins/stack_alerts/tsconfig.json"}, + { "path": "./plugins/snapshot_restore/tsconfig.json" }, + { "path": "./plugins/grokdebugger/tsconfig.json" }, + { "path": "./plugins/ingest_pipelines/tsconfig.json"}, + { "path": "./plugins/license_management/tsconfig.json" }, + { "path": "./plugins/painless_lab/tsconfig.json" }, + { "path": "./plugins/triggers_actions_ui/tsconfig.json" }, { "path": "./plugins/ui_actions_enhanced/tsconfig.json" }, - { "path": "./plugins/watcher/tsconfig.json" }, + { "path": "./plugins/watcher/tsconfig.json" } ] } diff --git a/x-pack/tsconfig.refs.json b/x-pack/tsconfig.refs.json index ed209cd241586..43a488e8727cc 100644 --- a/x-pack/tsconfig.refs.json +++ b/x-pack/tsconfig.refs.json @@ -36,6 +36,19 @@ { "path": "./plugins/telemetry_collection_xpack/tsconfig.json" }, { "path": "./plugins/translations/tsconfig.json" }, { "path": "./plugins/triggers_actions_ui/tsconfig.json"}, + { "path": "./plugins/spaces/tsconfig.json" }, + { "path": "./plugins/security/tsconfig.json" }, + { "path": "./plugins/stack_alerts/tsconfig.json"}, + { "path": "./plugins/encrypted_saved_objects/tsconfig.json" }, + { "path": "./plugins/beats_management/tsconfig.json" }, + { "path": "./plugins/cloud/tsconfig.json" }, + { "path": "./plugins/saved_objects_tagging/tsconfig.json" }, + { "path": "./plugins/global_search_bar/tsconfig.json" }, + { "path": "./plugins/snapshot_restore/tsconfig.json" }, + { "path": "./plugins/grokdebugger/tsconfig.json" }, + { "path": "./plugins/ingest_pipelines/tsconfig.json" }, + { "path": "./plugins/license_management/tsconfig.json" }, + { "path": "./plugins/painless_lab/tsconfig.json" }, { "path": "./plugins/ui_actions_enhanced/tsconfig.json" }, { "path": "./plugins/watcher/tsconfig.json" } ]