Skip to content

Commit

Permalink
Merge branch 'es-kuery/remove-convert' into es-query/field-base
Browse files Browse the repository at this point in the history
  • Loading branch information
Liza K committed Jun 27, 2021
2 parents f8e7e15 + 5b40ebc commit 435c547
Show file tree
Hide file tree
Showing 1,091 changed files with 25,306 additions and 18,171 deletions.
22 changes: 13 additions & 9 deletions NOTICE.txt
View file
Original file line number Diff line number Diff line change
Expand Up @@ -149,17 +149,17 @@ SOFTWARE.

---
Detection Rules
Copyright 2020 Elasticsearch B.V.
Copyright 2021 Elasticsearch B.V.

---
This product bundles rules based on https://github.com/BlueTeamLabs/sentinel-attack
which is available under a "MIT" license. The files based on this license are:
which is available under a "MIT" license. The rules based on this license are:

- defense_evasion_via_filter_manager
- discovery_process_discovery_via_tasklist_command
- persistence_priv_escalation_via_accessibility_features
- persistence_via_application_shimming
- defense_evasion_execution_via_trusted_developer_utilities
- "Potential Evasion via Filter Manager" (06dceabf-adca-48af-ac79-ffdf4c3b1e9a)
- "Process Discovery via Tasklist" (cc16f774-59f9-462d-8b98-d27ccd4519ec)
- "Potential Modification of Accessibility Binaries" (7405ddf1-6c8e-41ce-818f-48bea6bcaed8)
- "Potential Application Shimming via Sdbinst" (fd4a992d-6130-4802-9ff8-829b89ae801f)
- "Trusted Developer Application Usage" (9d110cb3-5f4b-4c9a-b9f5-53f0a1707ae1)

MIT License

Expand All @@ -185,9 +185,9 @@ SOFTWARE.

---
This product bundles rules based on https://github.com/FSecureLABS/leonidas
which is available under a "MIT" license. The files based on this license are:
which is available under a "MIT" license. The rules based on this license are:

- credential_access_secretsmanager_getsecretvalue.toml
- "AWS Access Secret in Secrets Manager" (a00681e3-9ed6-447c-ab2c-be648821c622)

MIT License

Expand Down Expand Up @@ -235,6 +235,10 @@ LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

---
Portions of this code are licensed under the following license:
For license information please see https://edge.fullstory.com/s/fs.js.LICENSE.txt

---
This product bundles [email protected] which is available under a
"MIT" license.
Expand Down
2 changes: 2 additions & 0 deletions docs/developer/getting-started/monorepo-packages.asciidoc
View file
Original file line number Diff line number Diff line change
Expand Up @@ -70,6 +70,7 @@ yarn kbn watch-bazel
- @kbn/apm-utils
- @kbn/babel-code-parser
- @kbn/babel-preset
- @kbn/cli-dev-mode
- @kbn/config
- @kbn/config-schema
- @kbn/crypto
Expand All @@ -87,6 +88,7 @@ yarn kbn watch-bazel
- @kbn/mapbox-gl
- @kbn/monaco
- @kbn/optimizer
- @kbn/plugin-helpers
- @kbn/rule-data-utils
- @kbn/securitysolution-es-utils
- @kbn/securitysolution-hook-utils
Expand Down
11 changes: 11 additions & 0 deletions docs/development/core/server/kibana-plugin-core-server.cspconfig.__private_.md
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
<!-- Do not edit this file. It is automatically generated by API Documenter. -->

[Home](./index.md) &gt; [kibana-plugin-core-server](./kibana-plugin-core-server.md) &gt; [CspConfig](./kibana-plugin-core-server.cspconfig.md) &gt; ["\#private"](./kibana-plugin-core-server.cspconfig.__private_.md)

## CspConfig."\#private" property

<b>Signature:</b>

```typescript
#private;
```
1 change: 1 addition & 0 deletions docs/development/core/server/kibana-plugin-core-server.cspconfig.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ The constructor for this class is marked as internal. Third-party code should no
| Property | Modifiers | Type | Description |
| --- | --- | --- | --- |
| ["\#private"](./kibana-plugin-core-server.cspconfig.__private_.md) | | <code></code> | |
| [DEFAULT](./kibana-plugin-core-server.cspconfig.default.md) | <code>static</code> | <code>CspConfig</code> | |
| [disableEmbedding](./kibana-plugin-core-server.cspconfig.disableembedding.md) | | <code>boolean</code> | |
| [header](./kibana-plugin-core-server.cspconfig.header.md) | | <code>string</code> | |
Expand Down
1 change: 1 addition & 0 deletions ...evelopment/plugins/embeddable/public/kibana-plugin-plugins-embeddable-public.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,7 @@
| [isSavedObjectEmbeddableInput(input)](./kibana-plugin-plugins-embeddable-public.issavedobjectembeddableinput.md) | |
| [openAddPanelFlyout(options)](./kibana-plugin-plugins-embeddable-public.openaddpanelflyout.md) | |
| [plugin(initializerContext)](./kibana-plugin-plugins-embeddable-public.plugin.md) | |
| [useEmbeddableFactory({ input, factory, onInputUpdated, })](./kibana-plugin-plugins-embeddable-public.useembeddablefactory.md) | |

## Interfaces

Expand Down
22 changes: 22 additions & 0 deletions ...beddable/public/kibana-plugin-plugins-embeddable-public.useembeddablefactory.md
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
<!-- Do not edit this file. It is automatically generated by API Documenter. -->

[Home](./index.md) &gt; [kibana-plugin-plugins-embeddable-public](./kibana-plugin-plugins-embeddable-public.md) &gt; [useEmbeddableFactory](./kibana-plugin-plugins-embeddable-public.useembeddablefactory.md)

## useEmbeddableFactory() function

<b>Signature:</b>

```typescript
export declare function useEmbeddableFactory<I extends EmbeddableInput>({ input, factory, onInputUpdated, }: EmbeddableRendererWithFactory<I>): readonly [ErrorEmbeddable | IEmbeddable<I, import("./i_embeddable").EmbeddableOutput> | undefined, boolean, string | undefined];
```

## Parameters

| Parameter | Type | Description |
| --- | --- | --- |
| { input, factory, onInputUpdated, } | <code>EmbeddableRendererWithFactory&lt;I&gt;</code> | |

<b>Returns:</b>

`readonly [ErrorEmbeddable | IEmbeddable<I, import("./i_embeddable").EmbeddableOutput> | undefined, boolean, string | undefined]`

3 changes: 3 additions & 0 deletions docs/management/advanced-options.asciidoc
View file
Original file line number Diff line number Diff line change
Expand Up @@ -482,6 +482,9 @@ of buckets to try to represent.
[[visualization-visualize-chartslibrary]]`visualization:visualize:legacyChartsLibrary`::
Enables the legacy charts library for aggregation-based area, line, and bar charts in *Visualize*.

[[visualization-visualize-pieChartslibrary]]`visualization:visualize:legacyPieChartsLibrary`::
Enables the legacy charts library for aggregation-based pie charts in *Visualize*.

[[visualization-colormapping]]`visualization:colorMapping`::
**This setting is deprecated and will not be supported as of 8.0.**
Maps values to specific colors in charts using the *Compatibility* palette.
Expand Down
8 changes: 7 additions & 1 deletion docs/settings/task-manager-settings.asciidoc
View file
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,13 @@ Task Manager runs background tasks by polling for work on an interval. You can
| The maximum number of tasks that this Kibana instance will run simultaneously. Defaults to 10.
Starting in 8.0, it will not be possible to set the value greater than 100.

| `xpack.task_manager.monitored_stats_warn_delayed_task_start_in_seconds`
| `xpack.task_manager.`
`monitored_stats_health_verbose_log.enabled`
| This flag will enable automatic warn and error logging if task manager self detects a performance issue, such as the time between when a task is scheduled to execute and when it actually executes. Defaults to false.

| `xpack.task_manager.`
`monitored_stats_health_verbose_log.`
`warn_delayed_task_start_in_seconds`
| The amount of seconds we allow a task to delay before printing a warning server log. Defaults to 60.
|===

Expand Down
51 changes: 48 additions & 3 deletions docs/setup/settings.asciidoc
View file
Original file line number Diff line number Diff line change
Expand Up @@ -36,11 +36,57 @@ Set to `false` to disable Console. *Default: `true`*
<<ops-cGroupOverrides-cpuAcctPath, `ops.cGroupOverrides.cpuAcctPath`>>.

| `csp.rules:`
| A https://w3c.github.io/webappsec-csp/[content-security-policy] template
| deprecated:[7.14.0,"In 8.0 and later, this setting will no longer be supported."]
A https://w3c.github.io/webappsec-csp/[Content Security Policy] template
that disables certain unnecessary and potentially insecure capabilities in
the browser. It is strongly recommended that you keep the default CSP rules
that ship with {kib}.

| `csp.script_src:`
| Add sources for the https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Content-Security-Policy/script-src[Content Security Policy `script-src` directive].

| `csp.worker_src:`
| Add sources for the https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Content-Security-Policy/worker-src[Content Security Policy `worker-src` directive].

| `csp.style_src:`
| Add sources for the https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Content-Security-Policy/style-src[Content Security Policy `style-src` directive].

| `csp.connect_src:`
| Add sources for the https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Content-Security-Policy/connect-src[Content Security Policy `connect-src` directive].

| `csp.default_src:`
| Add sources for the https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Content-Security-Policy/default-src[Content Security Policy `default-src` directive].

| `csp.font_src:`
| Add sources for the https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Content-Security-Policy/font-src[Content Security Policy `font-src` directive].

| `csp.frame_src:`
| Add sources for the https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Content-Security-Policy/frame-src[Content Security Policy `frame-src` directive].

| `csp.img_src:`
| Add sources for the https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Content-Security-Policy/img-src[Content Security Policy `img-src` directive].

| `csp.frame_ancestors:`
| Add sources for the https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Content-Security-Policy/frame-ancestors[Content Security Policy `frame-ancestors` directive].

|===

[NOTE]
============
The `frame-ancestors` directive can also be configured by using
<<server-securityResponseHeaders-disableEmbedding, `server.securityResponseHeaders.disableEmbedding`>>. In that case, that takes precedence and any values in `csp.frame_ancestors`
are ignored.
============

[cols="2*<"]
|===

| `csp.report_uri:`
| Add sources for the https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Content-Security-Policy/report-uri[Content Security Policy `report-uri` directive].

| `csp.report_to:`
| Add sources for the https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Content-Security-Policy/report-to[Content Security Policy `report-to` directive].

|[[csp-strict]] `csp.strict:`
| Blocks {kib} access to any browser that
does not enforce even rudimentary CSP rules. In practice, this disables
Expand Down Expand Up @@ -538,8 +584,7 @@ a|`server.securityResponseHeaders:`
is used in all responses to the client from the {kib} server, and specifies what value is used. Allowed values are any text value or `null`.
To disable, set to `null`. *Default:* `null`

[[server-securityResponseHeaders-disableEmbedding]]
a|`server.securityResponseHeaders:`
|[[server-securityResponseHeaders-disableEmbedding]]`server.securityResponseHeaders:`
`disableEmbedding:`
| Controls whether the https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Content-Security-Policy[`Content-Security-Policy`] and
https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/X-Frame-Options[`X-Frame-Options`] headers are configured to disable embedding
Expand Down
49 changes: 41 additions & 8 deletions docs/setup/upgrade/upgrade-migrations.asciidoc
View file
Original file line number Diff line number Diff line change
Expand Up @@ -55,22 +55,55 @@ This section highlights common causes of {kib} upgrade failures and how to preve
There is a known issue in v7.12.0 for users who tried the fleet beta. Upgrade migrations fail because of a large number of documents in the `.kibana` index.

This can cause Kibana to log errors like:
> Error: Unable to complete saved object migrations for the [.kibana] index. Please check the health of your Elasticsearch cluster and try again. Error: [receive_timeout_transport_exception]: [instance-0000000002][10.32.1.112:19541][cluster:monitor/task/get] request_id [2648] timed out after [59940ms]
> Error: Unable to complete saved object migrations for the [.kibana] index. Please check the health of your Elasticsearch cluster and try again. Error: [timeout_exception]: Timed out waiting for completion of [org.elasticsearch.index.reindex.BulkByScrollTask@6a74c54]

[source,sh]
--------------------------------------------
Error: Unable to complete saved object migrations for the [.kibana] index. Please check the health of your Elasticsearch cluster and try again. Error: [receive_timeout_transport_exception]: [instance-0000000002][10.32.1.112:19541][cluster:monitor/task/get] request_id [2648] timed out after [59940ms]
Error: Unable to complete saved object migrations for the [.kibana] index. Please check the health of your Elasticsearch cluster and try again. Error: [timeout_exception]: Timed out waiting for completion of [org.elasticsearch.index.reindex.BulkByScrollTask@6a74c54]
--------------------------------------------

See https://github.com/elastic/kibana/issues/95321 for instructions to work around this issue.

[float]
===== Corrupt saved objects
We highly recommend testing your {kib} upgrade in a development cluster to discover and remedy problems caused by corrupt documents, especially when there are custom integrations creating saved objects in your environment. Saved objects that were corrupted through manual editing or integrations will cause migration failures with a log message like `Failed to transform document. Transform: index-pattern:7.0.0\n Doc: {...}` or `Unable to migrate the corrupt Saved Object document ...`. Corrupt documents will have to be fixed or deleted before an upgrade migration can succeed.
We highly recommend testing your {kib} upgrade in a development cluster to discover and remedy problems caused by corrupt documents, especially when there are custom integrations creating saved objects in your environment.

Saved objects that were corrupted through manual editing or integrations will cause migration failures with a log message like `Failed to transform document. Transform: index-pattern:7.0.0\n Doc: {...}` or `Unable to migrate the corrupt Saved Object document ...`. Corrupt documents will have to be fixed or deleted before an upgrade migration can succeed.

For example, given the following error message:
> Unable to migrate the corrupt saved object document with _id: 'marketing_space:dashboard:e3c5fc71-ac71-4805-bcab-2bcc9cc93275'. To allow migrations to proceed, please delete this document from the [.kibana_7.12.0_001] index.

The following steps must be followed to allow the upgrade migration to succeed.
Please be aware the Dashboard having ID `e3c5fc71-ac71-4805-bcab-2bcc9cc93275` belonging to the space `marketing_space` will no more be available:
1. Delete the corrupt document with `DELETE .kibana_7.12.0_001/_doc/marketing_space:dashboard:e3c5fc71-ac71-4805-bcab-2bcc9cc93275`
2. Restart {kib}
[source,sh]
--------------------------------------------
Unable to migrate the corrupt saved object document with _id: 'marketing_space:dashboard:e3c5fc71-ac71-4805-bcab-2bcc9cc93275'. To allow migrations to proceed, please delete this document from the [.kibana_7.12.0_001] index.
--------------------------------------------

The following steps must be followed to delete the document that is causing the migration to fail:

. Remove the write block which the migration system has placed on the previous index:
+
[source,sh]
--------------------------------------------
PUT .kibana_7.12.1_001/_settings
{
"index": {
"blocks.write": false
}
}
--------------------------------------------

. Delete the corrupt document:
+
[source,sh]
--------------------------------------------
DELETE .kibana_7.12.0_001/_doc/marketing_space:dashboard:e3c5fc71-ac71-4805-bcab-2bcc9cc93275
--------------------------------------------

. Restart {kib}.

In this example, the Dashboard with ID `e3c5fc71-ac71-4805-bcab-2bcc9cc93275` that belongs to the space `marketing_space` **will no longer be available**.

Be sure you have a snapshot before you delete the corrupt document. If restoring from a snapshot is not an option, it is recommended to also delete the `temp` and `target` indices the migration created before restarting {kib} and retrying.

[float]
===== User defined index templates that causes new `.kibana*` indices to have incompatible settings or mappings
Expand Down
Loading

0 comments on commit 435c547

Please sign in to comment.