Merge remote-tracking branch 'upstream/master' into react-beats-views

elastic · Sep 27, 2021 · 9380c4c · 9380c4c
2 parents d6e94d0 + d27c723
commit 9380c4c
Show file tree

Hide file tree

Showing 698 changed files with 21,262 additions and 5,903 deletions.
diff --git a/.buildkite/scripts/steps/storybooks/build_and_upload.js b/.buildkite/scripts/steps/storybooks/build_and_upload.js
@@ -6,13 +6,20 @@ const path = require('path');
 const STORYBOOKS = [
   'apm',
   'canvas',
+  'codeeditor',
   'ci_composite',
   'url_template_editor',
-  'codeeditor',
   'dashboard',
   'dashboard_enhanced',
   'data_enhanced',
   'embeddable',
+  'expression_error',
+  'expression_image',
+  'expression_metric',
+  'expression_repeat_image',
+  'expression_reveal_image',
+  'expression_shape',
+  'expression_tagcloud',
   'fleet',
   'infra',
   'security_solution',

diff --git a/dev_docs/key_concepts/persistable_state.mdx b/dev_docs/key_concepts/persistable_state.mdx
@@ -0,0 +1,83 @@
+---
+id: kibDevDocsPersistableStateIntro
+slug: /kibana-dev-docs/persistable-state-intro
+title: Persistable State
+summary: Persitable state is a key concept to understand when building a Kibana plugin.
+date: 2021-02-02
+tags: ['kibana','dev', 'contributor', 'api docs']
+---
+
+ “Persistable state” is developer-defined state that supports being persisted by a plugin other than the one defining it. Persistable State needs to be serializable and the owner can/should provide utilities to migrate it, extract and inject any <DocLink id="kibDevDocsSavedObjectsIntro" section="references" text="references to Saved Objects"/> it may contain, as well as telemetry collection utilities.
+
+## Exposing state that can be persisted
+
+Any plugin that exposes state that another plugin might persist should implement <DocLink id="kibKibanaUtilsPluginApi" section="def-common.PersistableStateService" text="`PersistableStateService`"/> interface on their `setup` contract. This will allow plugins persisting the state to easily access migrations and other utilities.
+
+Example: Data plugin allows you to generate filters. Those filters can be persisted by applications in their saved 
+objects or in the URL. In order to allow apps to migrate the filters in case the structure changes in the future, the Data plugin implements `PersistableStateService` on <DocLink id="kibDataQueryPluginApi" section="def-public.FilterManager" text="`data.query.filterManager`"/>.
+
+note: There is currently no obvious way for a plugin to know which state is safe to persist. The developer must manually look for a matching `PersistableStateService`, or ad-hoc provided migration utilities (as is the case with Rule Type Parameters).
+In the future, we hope to make it easier for producers of state to understand when they need to write a migration with changes, and also make it easier for consumers of such state, to understand whether it is safe to persist.
+
+## Exposing state that can be persisted but is not owned by plugin exposing it (registry)
+
+Any plugin that owns collection of items (registry) whose state/configuration can be persisted should implement `PersistableStateService`
+interface on their `setup` contract and each item in the collection should implement <DocLink id="kibKibanaUtilsPluginApi" section="def-common.PersistableStateDefinition" text="`PersistableStateDefinition`"/> interface.
+
+Example: Embeddable plugin owns the registry of embeddable factories to which other plugins can register new embeddable factories. Dashboard plugin
+stores a bunch of embeddable panels input in its saved object and URL. Embeddable plugin setup contract implements `PersistableStateService`
+interface and each `EmbeddableFactory`  needs to implement `PersistableStateDefinition` interface.
+
+Embeddable plugin exposes this interfaces:
+```
+// EmbeddableInput implements Serializable
+
+export interface EmbeddableRegistryDefinition extends PersistableStateDefinition<EmbeddableInput> {
+  id: string;
+  ...
+}
+
+export interface EmbeddableSetup extends PersistableStateService<EmbeddableInput>;
+```
+
+Note: if your plugin doesn't expose the state (it is the only one storing state), the plugin doesn't need to implement the `PersistableStateService` interface.
+If the state your plugin is storing can be provided by other plugins (your plugin owns a registry) items in that registry still need to implement `PersistableStateDefinition` interface.
+
+## Storing persistable state as part of saved object
+
+Any plugin that stores any persistable state as part of their saved object should make sure that its saved object migration 
+and reference extraction and injection methods correctly use the matching `PersistableStateService` implementation for the state they are storing.
+
+Take a look at [example saved object](https://github.com/elastic/kibana/blob/master/examples/embeddable_examples/server/searchable_list_saved_object.ts#L32) which stores an embeddable state. Note how the `migrations`, `extractReferences` and `injectReferences` are defined.
+
+## Storing persistable state as part of URL
+
+When storing persistable state as part of URL you must make sure your URL is versioned. When loading the state `migrateToLatest` method
+of `PersistableStateService` should be called, which will migrate the state from its original version to latest.
+
+note: Currently there is no recommended way on how to store version in url and its up to every application to decide on how to implement that.
+
+## Available state operations
+
+### Extraction/Injection of References
+
+In order to support import and export, and space-sharing capabilities, Saved Objects need to explicitly list any references they contain to other Saved Objects.
+To support persisting your state in saved objects owned by another plugin, the <DocLink id="kibKibanaUtilsPluginApi" section="def-common.PersistableState.extract" text="`extract`"/> and <DocLink id="kibKibanaUtilsPluginApi" section="def-common.PersistableState.inject" text="`inject`"/> methods of Persistable State interface should be implemented.
+
+<DocLink id="kibDevTutorialSavedObject" section="references" text="Learn how to define Saved Object references"/>
+
+[See example embeddable providing extract/inject functions](https://github.com/elastic/kibana/blob/master/examples/embeddable_examples/public/migrations/migrations_embeddable_factory.ts)
+
+### Migrations and Backward compatibility
+
+As your plugin evolves, you may need to change your state in a breaking way. If that happens, you should write a migration to upgrade the state that existed prior to the change.
+
+<DocLink id="kibDevTutorialSavedObject" section="migrations" text="How to write a migration"/>.
+
+[See an example saved object storing embeddable state implementing saved object migration function](https://github.com/elastic/kibana/blob/master/examples/embeddable_examples/server/searchable_list_saved_object.ts)
+
+[See example embeddable providing migration functions](https://github.com/elastic/kibana/blob/master/examples/embeddable_examples/public/migrations/migrations_embeddable_factory.ts)
+
+## Telemetry
+
+You might want to collect statistics about how your state is used. If that is the case you should implement the telemetry method of Persistable State interface. 
diff --git a/docs/apm/dependencies.asciidoc b/docs/apm/dependencies.asciidoc
@@ -0,0 +1,32 @@
+[role="xpack"]
+[[dependencies]]
+=== Dependencies
+
+APM agents collect details about external calls made from instrumented services.
+Sometimes, these external calls resolve into a downstream service that's instrumented -- in these cases,
+you can utilize <<distributed-tracing,distributed tracing>> to drill down into problematic downstream services.
+Other times, though, it's not possible to instrument a downstream dependency --
+like with a database or third-party service.
+**Dependencies** gives you a window into these uninstrumented, downstream dependencies.
+
+[role="screenshot"]
+image::apm/images/dependencies.png[Dependencies view in the APM app in Kibana]
+
+Many application issues are caused by slow or unresponsive downstream dependencies.
+And because a single, slow dependency can significantly impact the end-user experience,
+it's important to be able to quickly identify these problems and determine the root cause.
+
+Select a dependency to see detailed latency, throughput, and failed transaction rate metrics.
+
+[role="screenshot"]
+image::apm/images/dependencies-drilldown.png[Dependencies drilldown view in the APM app in Kibana]
+
+When viewing a dependency, consider your pattern of usage with that dependency.
+If your usage pattern _hasn't_ increased or decreased,
+but the experience has been negatively effected -- either with an increase in latency or errors,
+there's likely a problem with the dependency that needs to be addressed.
+
+If your usage pattern _has_ changed, the dependency view can quickly show you whether
+that pattern change exists in all upstream services, or just a subset of your services.
+You might then start digging into traces coming from
+impacted services to determine why that pattern change has occurred.
diff --git a/docs/apm/errors.asciidoc b/docs/apm/errors.asciidoc
@@ -4,19 +4,21 @@
 
 TIP: {apm-overview-ref-v}/errors.html[Errors] are groups of exceptions with a similar exception or log message.
 
-The *Errors* overview provides a high-level view of the error message and culprit,
-the number of occurrences, and the most recent occurrence.
-Just like the transaction overview, you'll notice we group together like errors.
-This makes it very easy to quickly see which errors are affecting your services,
+The *Errors* overview provides a high-level view of the exceptions that APM agents catch,
+or that users manually report with APM agent APIs.
+Like errors are grouped together to make it easy to quickly see which errors are affecting your services,
 and to take actions to rectify them.
 
+A service returning a 5xx code from a request handler, controller, etc., will not create
+an exception that an APM agent can catch, and will therefore not show up in this view.
+
 [role="screenshot"]
-image::apm/images/apm-errors-overview.png[Example view of the errors overview in the APM app in Kibana]
+image::apm/images/apm-errors-overview.png[APM Errors overview]
 
 Selecting an error group ID or error message brings you to the *Error group*.
 
 [role="screenshot"]
-image::apm/images/apm-error-group.png[Example view of the error group page in the APM app in Kibana]
+image::apm/images/apm-error-group.png[APM Error group]
 
 Here, you'll see the error message, culprit, and the number of occurrences over time.
 

diff --git a/docs/apm/getting-started.asciidoc b/docs/apm/getting-started.asciidoc
@@ -29,6 +29,7 @@ start with:
 
 * <<services>>
 * <<traces>>
+* <<dependencies>>
 * <<service-maps>>
 
 Notice something awry? Select a service or trace and dive deeper with:
@@ -46,6 +47,8 @@ include::services.asciidoc[]
 
 include::traces.asciidoc[]
 
+include::dependencies.asciidoc[]
+
 include::service-maps.asciidoc[]
 
 include::service-overview.asciidoc[]

diff --git a/docs/apm/images/all-instances.png b/docs/apm/images/all-instances.png
diff --git a/docs/apm/images/apm-distributed-tracing.png b/docs/apm/images/apm-distributed-tracing.png
diff --git a/docs/apm/images/apm-geo-ui.png b/docs/apm/images/apm-geo-ui.png
diff --git a/docs/apm/images/apm-logs-tab.png b/docs/apm/images/apm-logs-tab.png
diff --git a/docs/apm/images/apm-services-overview.png b/docs/apm/images/apm-services-overview.png
diff --git a/docs/apm/images/apm-span-detail.png b/docs/apm/images/apm-span-detail.png
diff --git a/docs/apm/images/apm-transaction-duration-dist.png b/docs/apm/images/apm-transaction-duration-dist.png
diff --git a/docs/apm/images/apm-transaction-sample.png b/docs/apm/images/apm-transaction-sample.png
diff --git a/docs/apm/images/apm-transactions-overview.png b/docs/apm/images/apm-transactions-overview.png
diff --git a/docs/apm/images/apm-transactions-table.png b/docs/apm/images/apm-transactions-table.png
diff --git a/docs/apm/images/dependencies-drilldown.png b/docs/apm/images/dependencies-drilldown.png
diff --git a/docs/apm/images/dependencies.png b/docs/apm/images/dependencies.png
diff --git a/docs/apm/images/error-rate.png b/docs/apm/images/error-rate.png
diff --git a/docs/apm/images/spans-dependencies.png b/docs/apm/images/spans-dependencies.png
diff --git a/docs/apm/index.asciidoc b/docs/apm/index.asciidoc
@@ -4,8 +4,7 @@
 
 [partintro]
 --
-The APM app in {kib} is provided with the basic license.
-It allows you to monitor your software services and applications in real-time;
+The APM app in {kib} allows you to monitor your software services and applications in real-time;
 visualize detailed performance information on your services,
 identify and analyze errors,
 and monitor host-level and agent-specific metrics like JVM and Go runtime metrics.

diff --git a/docs/apm/service-overview.asciidoc b/docs/apm/service-overview.asciidoc
@@ -69,34 +69,43 @@ image::apm/images/traffic-transactions.png[Traffic and transactions]
 
 [discrete]
 [[service-error-rates]]
-=== Error rate and errors
+=== Failed transaction rate and errors
 
-The *Error rate* chart displays the average error rates relating to the service, within a specific time range.
-An HTTP response code greater than 400 does not necessarily indicate a failed transaction.
-<<transaction-error-rate,Learn more>>.
+The failed transaction rate represents the percentage of failed transactions from the perspective of the selected service.
+It's useful for visualizing unexpected increases, decreases, or irregular patterns in a service's transactions.
++
+[TIP]
+====
+HTTP **transactions** from the HTTP server perspective do not consider a `4xx` status code (client error) as a failure
+because the failure was caused by the caller, not the HTTP server. Thus, `event.outcome=success` and there will be no increase in failed transaction rate.
+
+HTTP **spans** from the client perspective however, are considered failures if the HTTP status code is ≥ 400.
+These spans will set `event.outcome=failure` and increase the failed transaction rate.
+
+If there is no HTTP status, both transactions and spans are considered successful unless an error is reported.
+====
 
 The *Errors* table provides a high-level view of each error message when it first and last occurred,
 along with the total number of occurrences. This makes it very easy to quickly see which errors affect
 your services and take actions to rectify them. To do so, click *View errors*.
 
 [role="screenshot"]
-image::apm/images/error-rate.png[Error rate and errors]
+image::apm/images/error-rate.png[failed transaction rate and errors]
 
 [discrete]
 [[service-span-duration]]
 === Span types average duration and dependencies
 
-The *Average duration by span type* chart visualizes each span type's average duration and helps you determine
+The *Time spent by span type* chart visualizes each span type's average duration and helps you determine
 which spans could be slowing down transactions. The "app" label displayed under the
 chart indicates that something was happening within the application. This could signal that the
 agent does not have auto-instrumentation for whatever was happening during that time or that the time was spent in the
 application code and not in database or external requests.
 
 The *Dependencies* table displays a list of downstream services or external connections relevant
-to the service at the selected time range. The table displays latency, traffic, error rate, and the impact of
+to the service at the selected time range. The table displays latency, throughput, failed transaction rate, and the impact of
 each dependency. By default, dependencies are sorted by _Impact_ to show the most used and the slowest dependency.
-If there is a particular dependency you are interested in, click *View service map* to view the related
-<<service-maps, service map>>.
+If there is a particular dependency you are interested in, click *<<dependencies,View dependencies>>* to learn more about it.
 
 NOTE: Displaying dependencies for services instrumented with the Real User Monitoring (RUM) agent
 requires an agent version ≥ v5.6.3.
@@ -106,11 +115,11 @@ image::apm/images/spans-dependencies.png[Span type duration and dependencies]
 
 [discrete]
 [[service-instances]]
-=== All instances
+=== Instances
 
-The *All instances* table displays a list of all the available service instances within the selected time range.
-Depending on how the service runs, the instance could be a host or a container. The table displays latency, traffic,
-errors, CPU usage, and memory usage for each instance. By default, instances are sorted by _Throughput_.
+The *Instances* table displays a list of all the available service instances within the selected time range.
+Depending on how the service runs, the instance could be a host or a container. The table displays latency, throughput,
+failed transaction, CPU usage, and memory usage for each instance. By default, instances are sorted by _Throughput_.
 
 [role="screenshot"]
 image::apm/images/all-instances.png[All instances]

diff --git a/docs/apm/set-up.asciidoc b/docs/apm/set-up.asciidoc
@@ -8,7 +8,7 @@
 
 APM is available via the navigation sidebar in {Kib}.
 If you have not already installed and configured Elastic APM,
-the *Setup Instructions* in Kibana will get you started.
+the *Add data* page will get you started.
 
 [role="screenshot"]
 image::apm/images/apm-setup.png[Installation instructions on the APM page in Kibana]
@@ -17,10 +17,9 @@ image::apm/images/apm-setup.png[Installation instructions on the APM page in Kib
 [[apm-configure-index-pattern]]
 === Load the index pattern
 
-Index patterns tell Kibana which Elasticsearch indices you want to explore.
+Index patterns tell {kib} which {es} indices you want to explore.
 An APM index pattern is necessary for certain features in the APM app, like the query bar.
-To set up the correct index pattern,
-simply click *Load Kibana objects* at the bottom of the Setup Instructions.
+To set up the correct index pattern, on the *Add data* page, click *Load Kibana objects*.
 
 [role="screenshot"]
 image::apm/images/apm-index-pattern.png[Setup index pattern for APM in Kibana]

diff --git a/docs/apm/transactions.asciidoc b/docs/apm/transactions.asciidoc
@@ -8,7 +8,7 @@ APM agents automatically collect performance metrics on HTTP requests, database
 [role="screenshot"]
 image::apm/images/apm-transactions-overview.png[Example view of transactions table in the APM app in Kibana]
 
-The *Latency*, *transactions per minute*, *Error rate*, and *Average duration by span type*
+The *Latency*, *transactions per minute*, *Failed transaction rate*, and *Average duration by span type*
 charts display information on all transactions associated with the selected service:
 
 *Latency*::
@@ -23,17 +23,17 @@ Useful for determining if more responses than usual are being served with a part
 Like in the latency graph, you can zoom in on anomalies to further investigate them.
 
 [[transaction-error-rate]]
-*Error rate*::
-The error rate represents the percentage of failed transactions from the perspective of the selected service.
+*Failed transaction rate*::
+The failed transaction rate represents the percentage of failed transactions from the perspective of the selected service.
 It's useful for visualizing unexpected increases, decreases, or irregular patterns in a service's transactions.
 +
 [TIP]
 ====
 HTTP **transactions** from the HTTP server perspective do not consider a `4xx` status code (client error) as a failure
-because the failure was caused by the caller, not the HTTP server. Thus, there will be no increase in error rate.
+because the failure was caused by the caller, not the HTTP server. Thus, `event.outcome=success` and there will be no increase in failed transaction rate.
 
 HTTP **spans** from the client perspective however, are considered failures if the HTTP status code is ≥ 400.
-These spans will increase the error rate.
+These spans will set `event.outcome=failure` and increase the failed transaction rate.
 
 If there is no HTTP status, both transactions and spans are considered successful unless an error is reported.
 ====
@@ -97,7 +97,7 @@ This page is visually similar to the transaction overview, but it shows data fro
 the selected transaction group.
 
 [role="screenshot"]
-image::apm/images/apm-transaction-response-dist.png[Example view of response time distribution]
+image::apm/images/apm-transactions-overview.png[Example view of response time distribution]
 
 [[transaction-duration-distribution]]
 ==== Latency distribution
@@ -110,10 +110,10 @@ It's the requests on the right, the ones taking longer than average, that we pro
 [role="screenshot"]
 image::apm/images/apm-transaction-duration-dist.png[Example view of latency distribution graph]
 
-Select a latency duration _bucket_ to display up to ten trace samples.
+Click and drag to select a latency duration _bucket_ to display up to 500 trace samples.
 
 [[transaction-trace-sample]]
-==== Trace sample
+==== Trace samples
 
 Trace samples are based on the _bucket_ selection in the *Latency distribution* chart;
 update the samples by selecting a new _bucket_.
@@ -167,4 +167,11 @@ and solve problems.
 [role="screenshot"]
 image::apm/images/apm-logs-tab.png[APM logs tab]
 
-// To do: link to log correlation
+[[transaction-latency-correlations]]
+==== Correlations
+
+Correlations surface attributes of your data that are potentially correlated with high-latency or erroneous transactions.
+To learn more, see <<correlations>>.
+
+[role="screenshot"]
+image::apm/images/correlations-hover.png[APM lattency correlations]
diff --git a/docs/dev-tools/console/console.asciidoc b/docs/dev-tools/console/console.asciidoc
@@ -134,6 +134,7 @@ shortcuts, click *Help*.
 [[console-settings]]
 === Disable Console
 
+deprecated:[7.16.0,"In 8.0 and later, this setting will no longer be supported."]
 If you don’t want to use *Console*, you can disable it by setting `console.enabled`
 to `false` in your `kibana.yml` configuration file. Changing this setting
 causes the server to regenerate assets on the next startup,

diff --git a/docs/management/advanced-options.asciidoc b/docs/management/advanced-options.asciidoc
@@ -142,6 +142,9 @@ The default placeholder value to use in
 Fields that exist outside of `_source`. Kibana merges these fields into the
 document when displaying it.
 
+[[metrics:allowStringIndices]]`metrics:allowStringIndices`::
+Enables you to use {es} indices in *TSVB* visualizations.
+
 [[metrics-maxbuckets]]`metrics:max_buckets`::
 Affects the *TSVB* histogram density. Must be set higher than `histogram:maxBars`.