From 497f840fc4e3d81603313b386883732365fcab45 Mon Sep 17 00:00:00 2001 From: Brandon Morelli Date: Wed, 15 Jan 2020 10:32:41 -0800 Subject: [PATCH 1/4] docs: add indexed troubleshooting --- docs/apm/troubleshooting.asciidoc | 28 ++++++++++++++++++++++++++++ 1 file changed, 28 insertions(+) diff --git a/docs/apm/troubleshooting.asciidoc b/docs/apm/troubleshooting.asciidoc index 22279b69b70fe..a46e472e0cb30 100644 --- a/docs/apm/troubleshooting.asciidoc +++ b/docs/apm/troubleshooting.asciidoc @@ -69,3 +69,31 @@ or because something is happening to the request that the Agent doesn't understa To resolve this, you'll need to head over to the relevant {apm-agents-ref}[Agent documentation]. Specifically, view the Agent's supported technologies page. You can also use the Agent's public API to manually set a name for the transaction. + +==== Fields are not indexed or searchable + +In Elasticsearch, index templates are used to define settings and mappings that determine how fields should be analyzed. +The recommended index template file for APM Server is installed by the APM Server packages. +This template, by default, enables and disables indexing on certain fields. + +As an example, some agents store cookie values in `http.request.cookies`. +Since `http.request` has disabled dynamic indexing, and `http.request.cookies` is not declared in a custom mapping, +the values in `http.request.cookies` are not indexed and thus not searchable. + +There are two things you can do to ensure indexed searchable: + +1. Index your additional data as {apm-overview-ref}/metadata.html[labels]. +These are dynamic by default, which means they will get indexed and become searchable and aggregatable. + +2. Use the experimental {apm-server-ref}/configuration-template.html[`append_fields`] feature. As an example, +adding the following to `apm-server.yml` will enable dynamic indexing for `http.request.cookies`: + +[source,yml] +---- +setup.template.enabled: true +setup.template.overwrite: true +setup.template.append_fields: + - name: http.request.cookies + type: object + dynamic: true +----- From 28abdedeb4a99a597c28c1435e70af409b1440c3 Mon Sep 17 00:00:00 2001 From: Brandon Morelli Date: Wed, 15 Jan 2020 10:41:05 -0800 Subject: [PATCH 2/4] docs: fix broken code block --- docs/apm/troubleshooting.asciidoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/apm/troubleshooting.asciidoc b/docs/apm/troubleshooting.asciidoc index a46e472e0cb30..79cfb41c05479 100644 --- a/docs/apm/troubleshooting.asciidoc +++ b/docs/apm/troubleshooting.asciidoc @@ -96,4 +96,4 @@ setup.template.append_fields: - name: http.request.cookies type: object dynamic: true ------ +---- From 5b539fb0ce44517fee1a73d9a0648f2df33b02a3 Mon Sep 17 00:00:00 2001 From: Brandon Morelli Date: Fri, 24 Jan 2020 11:08:56 -0800 Subject: [PATCH 3/4] docs: touch up --- docs/apm/troubleshooting.asciidoc | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/apm/troubleshooting.asciidoc b/docs/apm/troubleshooting.asciidoc index 79cfb41c05479..728b3e7463f43 100644 --- a/docs/apm/troubleshooting.asciidoc +++ b/docs/apm/troubleshooting.asciidoc @@ -80,12 +80,12 @@ As an example, some agents store cookie values in `http.request.cookies`. Since `http.request` has disabled dynamic indexing, and `http.request.cookies` is not declared in a custom mapping, the values in `http.request.cookies` are not indexed and thus not searchable. -There are two things you can do to ensure indexed searchable: +There are two things you can do to if you'd like to ensure a field is indexed and searchable: 1. Index your additional data as {apm-overview-ref}/metadata.html[labels]. -These are dynamic by default, which means they will get indexed and become searchable and aggregatable. +These are dynamic by default, which means they will be indexed and become searchable and aggregatable. -2. Use the experimental {apm-server-ref}/configuration-template.html[`append_fields`] feature. As an example, +2. Use the {apm-server-ref}/configuration-template.html[`append_fields`] feature. As an example, adding the following to `apm-server.yml` will enable dynamic indexing for `http.request.cookies`: [source,yml] From 5765ef95cb4c367de98464fa9b8b458dff71416a Mon Sep 17 00:00:00 2001 From: Brandon Morelli Date: Tue, 4 Feb 2020 12:24:03 -0800 Subject: [PATCH 4/4] docs: feedback from @ogupte --- docs/apm/troubleshooting.asciidoc | 23 +++++++++++++++++------ 1 file changed, 17 insertions(+), 6 deletions(-) diff --git a/docs/apm/troubleshooting.asciidoc b/docs/apm/troubleshooting.asciidoc index 728b3e7463f43..c4611f3b41e55 100644 --- a/docs/apm/troubleshooting.asciidoc +++ b/docs/apm/troubleshooting.asciidoc @@ -6,6 +6,7 @@ your proposed changes at https://github.com/elastic/kibana. Also check out the https://discuss.elastic.co/c/apm[APM discussion forum]. +[[no-apm-data-found]] ==== No APM data found This section can help with any of the following: @@ -70,19 +71,29 @@ To resolve this, you'll need to head over to the relevant {apm-agents-ref}[Agent Specifically, view the Agent's supported technologies page. You can also use the Agent's public API to manually set a name for the transaction. -==== Fields are not indexed or searchable +==== Fields are not searchable -In Elasticsearch, index templates are used to define settings and mappings that determine how fields should be analyzed. -The recommended index template file for APM Server is installed by the APM Server packages. -This template, by default, enables and disables indexing on certain fields. +In Elasticsearch, index patterns are used to define settings and mappings that determine how fields should be analyzed. +The recommended index template file for APM Server is installed when Kibana starts. +This template defines which fields are available in Kibana for features like the Kuery bar, +or for linking to other plugins like Logs, Uptime, and Discover. As an example, some agents store cookie values in `http.request.cookies`. Since `http.request` has disabled dynamic indexing, and `http.request.cookies` is not declared in a custom mapping, the values in `http.request.cookies` are not indexed and thus not searchable. -There are two things you can do to if you'd like to ensure a field is indexed and searchable: +*Ensure an index pattern exists* +As a first step, you should ensure the correct index pattern exists. +In Kibana, navigate to *Management > Kibana > Index Patterns*. +In the pattern list, you should see an apm index pattern; The default is `apm-*`. +If you don't, the index pattern doesn't exist. See <> for information on how to fix this problem. -1. Index your additional data as {apm-overview-ref}/metadata.html[labels]. +Selecting the `apm-*` index pattern shows a listing of every field defined in the pattern. + +*Ensure a field is searchable* +There are two things you can do to if you'd like to ensure a field is searchable: + +1. Index your additional data as {apm-overview-ref}/metadata.html[labels] instead. These are dynamic by default, which means they will be indexed and become searchable and aggregatable. 2. Use the {apm-server-ref}/configuration-template.html[`append_fields`] feature. As an example,