Merge remote-tracking branch 'origin/master' into security/audit_trail

.github/CODEOWNERS

@@ -119,18 +119,18 @@
 #CC# /x-pack/plugins/beats_management/ @elastic/beats
 
 # Canvas
-/src/plugins/dashboard/ @elastic/kibana-app
-/src/plugins/input_control_vis/ @elastic/kibana-app
-/src/plugins/vis_type_markdown/ @elastic/kibana-app
+/src/plugins/dashboard/ @elastic/kibana-canvas
+/src/plugins/input_control_vis/ @elastic/kibana-canvas
+/src/plugins/vis_type_markdown/ @elastic/kibana-canvas
 /x-pack/plugins/canvas/  @elastic/kibana-canvas
-/x-pack/plugins/dashboard_enhanced/ @elastic/kibana-app
+/x-pack/plugins/dashboard_enhanced/ @elastic/kibana-canvas
 /x-pack/test/functional/apps/canvas/  @elastic/kibana-canvas
-#CC# /src/legacy/core_plugins/kibana/public/dashboard/ @elastic/kibana-app
-#CC# /src/legacy/core_plugins/input_control_vis @elastic/kibana-app
+#CC# /src/legacy/core_plugins/kibana/public/dashboard/ @elastic/kibana-canvas
+#CC# /src/legacy/core_plugins/input_control_vis @elastic/kibana-canvas
 #CC# /src/plugins/kibana_react/public/code_editor/ @elastic/kibana-canvas
 #CC# /x-pack/legacy/plugins/canvas/ @elastic/kibana-canvas
-#CC# /x-pack/plugins/dashboard_mode @elastic/kibana-app
-#CC# /x-pack/legacy/plugins/dashboard_mode/ @elastic/kibana-app
+#CC# /x-pack/plugins/dashboard_mode @elastic/kibana-canvas
+#CC# /x-pack/legacy/plugins/dashboard_mode/ @elastic/kibana-canvas
 
 # Core UI
 # Exclude tutorials folder for now because they are not owned by Kibana app and most will move out soon

.github/ISSUE_TEMPLATE/security_solution_bug_report.md

@@ -1,7 +1,8 @@
 ---
-name: Security Solution Bug Report
-about: Things break. Help us identify those things so we can fix them!
+name: Bug report for Security Solution
+about: Help us identify bugs in Elastic Security, SIEM, and Endpoint so we can fix them!
 title: '[Security Solution]'
+labels: Team: SecuritySolution
 ---
 
 **Describe the bug:**

docs/api/saved-objects.asciidoc

@@ -28,6 +28,8 @@ The following saved objects APIs are available:
 
 * <<saved-objects-api-resolve-import-errors, Resolve import errors API>> to resolve errors from the import API
 
+* <<saved-objects-api-rotate-encryption-key, Rotate encryption key API>> to rotate the encryption key for encrypted saved objects
+
 include::saved-objects/get.asciidoc[]
 include::saved-objects/bulk_get.asciidoc[]
 include::saved-objects/find.asciidoc[]
@@ -38,3 +40,4 @@ include::saved-objects/delete.asciidoc[]
 include::saved-objects/export.asciidoc[]
 include::saved-objects/import.asciidoc[]
 include::saved-objects/resolve_import_errors.asciidoc[]
+include::saved-objects/rotate_encryption_key.asciidoc[]

docs/api/saved-objects/rotate_encryption_key.asciidoc

@@ -0,0 +1,110 @@
+[role="xpack"]
+[[saved-objects-api-rotate-encryption-key]]
+=== Rotate encryption key API
+++++
+<titleabbrev>Rotate encryption key</titleabbrev>
+++++
+
+experimental[] Rotate the encryption key for encrypted saved objects.
+
+If a saved object cannot be decrypted using the primary encryption key, then {kib} will attempt to decrypt it using the specified <<xpack-encryptedSavedObjects-keyRotation-decryptionOnlyKeys, decryption-only keys>>. In most of the cases this overhead is negligible, but if you're dealing with a large number of saved objects and experiencing performance issues, you may want to rotate the encryption key.
+
+[IMPORTANT]
+============================================================================
+Bulk key rotation can consume a considerable amount of resources and hence only user with a `superuser` role can trigger it.
+============================================================================
+
+[[saved-objects-api-rotate-encryption-key-request]]
+==== Request
+
+`POST <kibana host>:<port>/api/encrypted_saved_objects/_rotate_key`
+
+[[saved-objects-api-rotate-encryption-key-request-query-params]]
+==== Query parameters
+
+`type`::
+(Optional, string) Limits encryption key rotation only to the saved objects with the specified type. By default, {kib} tries to rotate the encryption key for all saved object types that may contain encrypted attributes.
+
+`batchSize`::
+(Optional, number) Specifies a maximum number of saved objects that {kib} can process in a single batch. Bulk key rotation is an iterative process since {kib} may not be able to fetch and process all required saved objects in one go and splits processing into consequent batches. By default, the batch size is 10000, which is also a maximum allowed value.
+
+[[saved-objects-api-rotate-encryption-key-response-body]]
+==== Response body
+
+`total`::
+(number) Indicates the total number of _all_ encrypted saved objects (optionally filtered by the requested `type`), regardless of the key {kib} used for encryption.
+
+`successful`::
+(number) Indicates the total number of _all_ encrypted saved objects (optionally filtered by the requested `type`), regardless of the key {kib} used for encryption.
++
+NOTE: In most cases, `total` will be greater than `successful` even if `failed` is zero. The reason is that {kib} may not need or may not be able to rotate encryption keys for all encrypted saved objects.
+
+`failed`::
+(number) Indicates the number of the saved objects that were still encrypted with one of the old encryption keys that {kib} failed to re-encrypt with the primary key.
+
+[[saved-objects-api-rotate-encryption-key-response-codes]]
+==== Response code
+
+`200`::
+Indicates a successful call.
+
+`400`::
+Indicates that either query parameters are wrong or <<xpack-encryptedSavedObjects-keyRotation-decryptionOnlyKeys, decryption-only keys>> aren't configured.
+
+`429`::
+Indicates that key rotation is already in progress.
+
+[[saved-objects-api-rotate-encryption-key-example]]
+==== Examples
+
+[[saved-objects-api-rotate-encryption-key-example-1]]
+===== Encryption key rotation with default parameters
+
+[source,sh]
+--------------------------------------------------
+$ curl -X POST /api/encrypted_saved_objects/_rotate_key
+--------------------------------------------------
+// KIBANA
+
+The API returns the following:
+
+[source,sh]
+--------------------------------------------------
+{
+  "total": 1000,
+  "successful": 300,
+  "failed": 0
+}
+--------------------------------------------------
+
+The result indicates that the encryption key was successfully rotated for 300 out of 1000 saved objects with encrypted attributes, and 700 of the saved objects either didn't require key rotation, or were encrypted with an unknown encryption key.
+
+[[saved-objects-api-rotate-encryption-key-example-2]]
+===== Encryption key rotation for the specific type with reduce batch size
+
+[IMPORTANT]
+============================================================================
+Default parameters are optimized for speed. Change the parameters only when necessary. However, if you're experiencing any issues with this API, you may want to decrease a batch size or rotate the encryption keys for the specific types only. In this case, you may need to run key rotation multiple times in a row.
+============================================================================
+
+In this example, key rotation is performed for all saved objects with the `alert` type in batches of 5000.
+
+[source,sh]
+--------------------------------------------------
+$ curl -X POST /api/encrypted_saved_objects/_rotate_key?type=alert&batchSize=5000
+--------------------------------------------------
+// KIBANA
+
+The API returns the following:
+
+[source,sh]
+--------------------------------------------------
+{
+  "total": 100,
+  "successful": 100,
+  "failed": 0
+}
+--------------------------------------------------
+
+The result indicates that the encryption key was successfully rotated for all 100 saved objects with the `alert` type.
+

docs/developer/best-practices/typescript.asciidoc

@@ -28,7 +28,7 @@ This architecture imposes several limitations to which we must comply:
 [discrete]
 ==== Prerequisites
 Since project refs rely on generated `d.ts` files, the migration order does matter. You can migrate your plugin only when all the plugin dependencies already have migrated. It creates a situation where commonly used plugins (such as `data` or `kibana_react`) have to migrate first.
-https://github.com/elastic/kibana/issues/79343 is going to provide a tool for identifying a plugin dependency tree.
+Run `node scripts/find_plugins_without_ts_refs.js --id your_plugin_id` to get a list of plugins that should be switched to TS project refs to unblock your plugin migration.
 
 [discrete]
 ==== Implementation

docs/developer/plugin-list.asciidoc

@@ -496,8 +496,8 @@ routes, etc.
 |Gathers all usage collection, retrieving them from both: OSS and X-Pack plugins.
 
 
-|{kib-repo}blob/{branch}/x-pack/plugins/transform[transform]
-|WARNING: Missing README.
+|{kib-repo}blob/{branch}/x-pack/plugins/transform/readme.md[transform]
+|This plugin provides access to the transforms features provided by Elastic.
 
 
 |{kib-repo}blob/{branch}/x-pack/plugins/translations[translations]

docs/development/plugins/data/public/kibana-plugin-plugins-data-public.isearchoptions.md

@@ -15,5 +15,6 @@ export interface ISearchOptions
 |  Property | Type | Description |
 |  --- | --- | --- |
 |  [abortSignal](./kibana-plugin-plugins-data-public.isearchoptions.abortsignal.md) | <code>AbortSignal</code> | An <code>AbortSignal</code> that allows the caller of <code>search</code> to abort a search request. |
+|  [sessionId](./kibana-plugin-plugins-data-public.isearchoptions.sessionid.md) | <code>string</code> | A session ID, grouping multiple search requests into a single session. |
 |  [strategy](./kibana-plugin-plugins-data-public.isearchoptions.strategy.md) | <code>string</code> | Use this option to force using a specific server side search strategy. Leave empty to use the default strategy. |
 

docs/development/plugins/data/public/kibana-plugin-plugins-data-public.isearchoptions.sessionid.md

@@ -0,0 +1,13 @@
+<!-- Do not edit this file. It is automatically generated by API Documenter. -->
+
+[Home](./index.md) &gt; [kibana-plugin-plugins-data-public](./kibana-plugin-plugins-data-public.md) &gt; [ISearchOptions](./kibana-plugin-plugins-data-public.isearchoptions.md) &gt; [sessionId](./kibana-plugin-plugins-data-public.isearchoptions.sessionid.md)
+
+## ISearchOptions.sessionId property
+
+A session ID, grouping multiple search requests into a single session.
+
+<b>Signature:</b>
+
+```typescript
+sessionId?: string;
+```

docs/development/plugins/data/public/kibana-plugin-plugins-data-public.isearchsetup.md

@@ -17,5 +17,6 @@ export interface ISearchSetup
 |  Property | Type | Description |
 |  --- | --- | --- |
 |  [aggs](./kibana-plugin-plugins-data-public.isearchsetup.aggs.md) | <code>AggsSetup</code> |  |
+|  [session](./kibana-plugin-plugins-data-public.isearchsetup.session.md) | <code>ISessionService</code> | session management  |
 |  [usageCollector](./kibana-plugin-plugins-data-public.isearchsetup.usagecollector.md) | <code>SearchUsageCollector</code> |  |
 

docs/development/plugins/data/public/kibana-plugin-plugins-data-public.isearchsetup.session.md

@@ -0,0 +1,13 @@
+<!-- Do not edit this file. It is automatically generated by API Documenter. -->
+
+[Home](./index.md) &gt; [kibana-plugin-plugins-data-public](./kibana-plugin-plugins-data-public.md) &gt; [ISearchSetup](./kibana-plugin-plugins-data-public.isearchsetup.md) &gt; [session](./kibana-plugin-plugins-data-public.isearchsetup.session.md)
+
+## ISearchSetup.session property
+
+session management 
+
+<b>Signature:</b>
+
+```typescript
+session: ISessionService;
+```

docs/development/plugins/data/public/kibana-plugin-plugins-data-public.isearchstart.md

@@ -19,5 +19,6 @@ export interface ISearchStart
 |  [aggs](./kibana-plugin-plugins-data-public.isearchstart.aggs.md) | <code>AggsStart</code> | agg config sub service [AggsStart](./kibana-plugin-plugins-data-public.aggsstart.md) |
 |  [search](./kibana-plugin-plugins-data-public.isearchstart.search.md) | <code>ISearchGeneric</code> | low level search [ISearchGeneric](./kibana-plugin-plugins-data-public.isearchgeneric.md) |
 |  [searchSource](./kibana-plugin-plugins-data-public.isearchstart.searchsource.md) | <code>ISearchStartSearchSource</code> | high level search [ISearchStartSearchSource](./kibana-plugin-plugins-data-public.isearchstartsearchsource.md) |
+|  [session](./kibana-plugin-plugins-data-public.isearchstart.session.md) | <code>ISessionService</code> | session management  |
 |  [showError](./kibana-plugin-plugins-data-public.isearchstart.showerror.md) | <code>(e: Error) =&gt; void</code> |  |
 

docs/development/plugins/data/public/kibana-plugin-plugins-data-public.isearchstart.session.md

@@ -0,0 +1,13 @@
+<!-- Do not edit this file. It is automatically generated by API Documenter. -->
+
+[Home](./index.md) &gt; [kibana-plugin-plugins-data-public](./kibana-plugin-plugins-data-public.md) &gt; [ISearchStart](./kibana-plugin-plugins-data-public.isearchstart.md) &gt; [session](./kibana-plugin-plugins-data-public.isearchstart.session.md)
+
+## ISearchStart.session property
+
+session management 
+
+<b>Signature:</b>
+
+```typescript
+session: ISessionService;
+```

docs/development/plugins/data/public/kibana-plugin-plugins-data-public.searchinterceptor.handlesearcherror.md

@@ -7,7 +7,7 @@
 <b>Signature:</b>
 
 ```typescript
-protected handleSearchError(e: any, request: IKibanaSearchRequest, timeoutSignal: AbortSignal, appAbortSignal?: AbortSignal): Error;
+protected handleSearchError(e: any, request: IKibanaSearchRequest, timeoutSignal: AbortSignal, options?: ISearchOptions): Error;
 ```
 
 ## Parameters
@@ -17,7 +17,7 @@ protected handleSearchError(e: any, request: IKibanaSearchRequest, timeoutSignal
 |  e | <code>any</code> |  |
 |  request | <code>IKibanaSearchRequest</code> |  |
 |  timeoutSignal | <code>AbortSignal</code> |  |
-|  appAbortSignal | <code>AbortSignal</code> |  |
+|  options | <code>ISearchOptions</code> |  |
 
 <b>Returns:</b>
 

docs/development/plugins/data/public/kibana-plugin-plugins-data-public.searchinterceptor.md

@@ -27,7 +27,7 @@ export declare class SearchInterceptor
 |  Method | Modifiers | Description |
 |  --- | --- | --- |
 |  [getTimeoutMode()](./kibana-plugin-plugins-data-public.searchinterceptor.gettimeoutmode.md) |  |  |
-|  [handleSearchError(e, request, timeoutSignal, appAbortSignal)](./kibana-plugin-plugins-data-public.searchinterceptor.handlesearcherror.md) |  |  |
+|  [handleSearchError(e, request, timeoutSignal, options)](./kibana-plugin-plugins-data-public.searchinterceptor.handlesearcherror.md) |  |  |
 |  [search(request, options)](./kibana-plugin-plugins-data-public.searchinterceptor.search.md) |  | Searches using the given <code>search</code> method. Overrides the <code>AbortSignal</code> with one that will abort either when <code>cancelPending</code> is called, when the request times out, or when the original <code>AbortSignal</code> is aborted. Updates <code>pendingCount$</code> when the request is started/finalized. |
 |  [showError(e)](./kibana-plugin-plugins-data-public.searchinterceptor.showerror.md) |  |  |
 

docs/development/plugins/data/public/kibana-plugin-plugins-data-public.searchinterceptordeps.md

@@ -15,6 +15,7 @@ export interface SearchInterceptorDeps
 |  Property | Type | Description |
 |  --- | --- | --- |
 |  [http](./kibana-plugin-plugins-data-public.searchinterceptordeps.http.md) | <code>CoreSetup['http']</code> |  |
+|  [session](./kibana-plugin-plugins-data-public.searchinterceptordeps.session.md) | <code>ISessionService</code> |  |
 |  [startServices](./kibana-plugin-plugins-data-public.searchinterceptordeps.startservices.md) | <code>Promise&lt;[CoreStart, any, unknown]&gt;</code> |  |
 |  [toasts](./kibana-plugin-plugins-data-public.searchinterceptordeps.toasts.md) | <code>ToastsSetup</code> |  |
 |  [uiSettings](./kibana-plugin-plugins-data-public.searchinterceptordeps.uisettings.md) | <code>CoreSetup['uiSettings']</code> |  |

docs/development/plugins/data/public/kibana-plugin-plugins-data-public.searchinterceptordeps.session.md

@@ -0,0 +1,11 @@
+<!-- Do not edit this file. It is automatically generated by API Documenter. -->
+
+[Home](./index.md) &gt; [kibana-plugin-plugins-data-public](./kibana-plugin-plugins-data-public.md) &gt; [SearchInterceptorDeps](./kibana-plugin-plugins-data-public.searchinterceptordeps.md) &gt; [session](./kibana-plugin-plugins-data-public.searchinterceptordeps.session.md)
+
+## SearchInterceptorDeps.session property
+
+<b>Signature:</b>
+
+```typescript
+session: ISessionService;
+```

docs/development/plugins/data/server/kibana-plugin-plugins-data-server.isearchoptions.md

@@ -15,5 +15,6 @@ export interface ISearchOptions
 |  Property | Type | Description |
 |  --- | --- | --- |
 |  [abortSignal](./kibana-plugin-plugins-data-server.isearchoptions.abortsignal.md) | <code>AbortSignal</code> | An <code>AbortSignal</code> that allows the caller of <code>search</code> to abort a search request. |
+|  [sessionId](./kibana-plugin-plugins-data-server.isearchoptions.sessionid.md) | <code>string</code> | A session ID, grouping multiple search requests into a single session. |
 |  [strategy](./kibana-plugin-plugins-data-server.isearchoptions.strategy.md) | <code>string</code> | Use this option to force using a specific server side search strategy. Leave empty to use the default strategy. |
 

docs/development/plugins/data/server/kibana-plugin-plugins-data-server.isearchoptions.sessionid.md

@@ -0,0 +1,13 @@
+<!-- Do not edit this file. It is automatically generated by API Documenter. -->
+
+[Home](./index.md) &gt; [kibana-plugin-plugins-data-server](./kibana-plugin-plugins-data-server.md) &gt; [ISearchOptions](./kibana-plugin-plugins-data-server.isearchoptions.md) &gt; [sessionId](./kibana-plugin-plugins-data-server.isearchoptions.sessionid.md)
+
+## ISearchOptions.sessionId property
+
+A session ID, grouping multiple search requests into a single session.
+
+<b>Signature:</b>
+
+```typescript
+sessionId?: string;
+```

docs/development/plugins/embeddable/public/kibana-plugin-plugins-embeddable-public.rangeselectcontext.data.md

@@ -8,7 +8,7 @@
 
 ```typescript
 data: {
-        table: KibanaDatatable;
+        table: Datatable;
         column: number;
         range: number[];
         timeFieldName?: string;

docs/development/plugins/embeddable/public/kibana-plugin-plugins-embeddable-public.rangeselectcontext.md

@@ -14,6 +14,6 @@ export interface RangeSelectContext<T extends IEmbeddable = IEmbeddable>
 
 |  Property | Type | Description |
 |  --- | --- | --- |
-|  [data](./kibana-plugin-plugins-embeddable-public.rangeselectcontext.data.md) | <code>{</code><br/><code>        table: KibanaDatatable;</code><br/><code>        column: number;</code><br/><code>        range: number[];</code><br/><code>        timeFieldName?: string;</code><br/><code>    }</code> |  |
+|  [data](./kibana-plugin-plugins-embeddable-public.rangeselectcontext.data.md) | <code>{</code><br/><code>        table: Datatable;</code><br/><code>        column: number;</code><br/><code>        range: number[];</code><br/><code>        timeFieldName?: string;</code><br/><code>    }</code> |  |
 |  [embeddable](./kibana-plugin-plugins-embeddable-public.rangeselectcontext.embeddable.md) | <code>T</code> |  |
 

docs/development/plugins/embeddable/public/kibana-plugin-plugins-embeddable-public.valueclickcontext.data.md

@@ -9,7 +9,7 @@
 ```typescript
 data: {
         data: Array<{
-            table: Pick<KibanaDatatable, 'rows' | 'columns'>;
+            table: Pick<Datatable, 'rows' | 'columns'>;
             column: number;
             row: number;
             value: any;

docs/development/plugins/embeddable/public/kibana-plugin-plugins-embeddable-public.valueclickcontext.md

@@ -14,6 +14,6 @@ export interface ValueClickContext<T extends IEmbeddable = IEmbeddable>
 
 |  Property | Type | Description |
 |  --- | --- | --- |
-|  [data](./kibana-plugin-plugins-embeddable-public.valueclickcontext.data.md) | <code>{</code><br/><code>        data: Array&lt;{</code><br/><code>            table: Pick&lt;KibanaDatatable, 'rows' &#124; 'columns'&gt;;</code><br/><code>            column: number;</code><br/><code>            row: number;</code><br/><code>            value: any;</code><br/><code>        }&gt;;</code><br/><code>        timeFieldName?: string;</code><br/><code>        negate?: boolean;</code><br/><code>    }</code> |  |
+|  [data](./kibana-plugin-plugins-embeddable-public.valueclickcontext.data.md) | <code>{</code><br/><code>        data: Array&lt;{</code><br/><code>            table: Pick&lt;Datatable, 'rows' &#124; 'columns'&gt;;</code><br/><code>            column: number;</code><br/><code>            row: number;</code><br/><code>            value: any;</code><br/><code>        }&gt;;</code><br/><code>        timeFieldName?: string;</code><br/><code>        negate?: boolean;</code><br/><code>    }</code> |  |
 |  [embeddable](./kibana-plugin-plugins-embeddable-public.valueclickcontext.embeddable.md) | <code>T</code> |  |
 

docs/development/plugins/expressions/public/kibana-plugin-plugins-expressions-public.datatablecolumntype.md

@@ -4,10 +4,10 @@
 
 ## DatatableColumnType type
 
-This type represents the `type` of any `DatatableColumn` in a `Datatable`<!-- -->.
+This type represents the `type` of any `DatatableColumn` in a `Datatable`<!-- -->. its duplicated from KBN\_FIELD\_TYPES
 
 <b>Signature:</b>
 
 ```typescript
-export declare type DatatableColumnType = 'string' | 'number' | 'boolean' | 'date' | 'null';
+export declare type DatatableColumnType = '_source' | 'attachment' | 'boolean' | 'date' | 'geo_point' | 'geo_shape' | 'ip' | 'murmur3' | 'number' | 'string' | 'unknown' | 'conflict' | 'object' | 'nested' | 'histogram' | 'null';
 ```