[DOCS] Add fields parameter to EQL search API (#69634)

elastic · Mar 1, 2021 · 783769d · 783769d
1 parent 8b8a20b
commit 783769d
Show file tree

Hide file tree

Showing 3 changed files with 155 additions and 5 deletions.
diff --git a/docs/reference/eql/eql-search-api.asciidoc b/docs/reference/eql/eql-search-api.asciidoc
@@ -185,6 +185,34 @@ returned.
 +
 A greater `fetch_size` value often increases search speed but uses more memory.
 
+`fields`::
+(Optional, array of strings and objects)
+Array of wildcard (`*`) patterns. The response returns values for field names
+matching these patterns in the `fields` property of each hit.
++
+You can specify items in the array as a string or object.
++
+.Properties of `fields` objects
+[%collapsible%open]
+====
+`field`::
+(Required, string)
+Wildcard pattern. The request returns values for field names matching this
+pattern.
+
+`format`::
+(Optional, string)
+Format in which the values are returned.
++
+<<date,`date`>> and <<date_nanos, `date_nanos`>> fields accept a
+<<mapping-date-format,date format>>. <<spatial_datatypes, Spatial fields>>
+accept either `geojson` for http://www.geojson.org[GeoJSON] (the default) or
+`wkt` for {wikipedia}/Well-known_text_representation_of_geometry[Well Known
+Text].
++
+For other field data types, this parameter is not supported.
+====
+
 `filter`::
 (Optional, <<query-dsl,Query DSL object>>)
 Query, written in Query DSL, used to filter the events on which the EQL query

diff --git a/docs/reference/eql/eql.asciidoc b/docs/reference/eql/eql.asciidoc
@@ -136,9 +136,16 @@ GET /my-index-000001/_eql/search
 ----
 // TEST[setup:sec_logs]
 
-Use the <<common-options-response-filtering,`filter_path`>> query parameter to
-filter the API response. For example, the following search returns only the
-timestamp and PID for each matching event.
+[discrete]
+[[retrieve-selected-fields]]
+== Retrieve selected fields
+
+By default, each hit in the search response includes the document `_source`,
+which is the entire JSON object that was provided when indexing the document.
+
+You can use the <<common-options-response-filtering,`filter_path`>> query
+parameter to filter the API response. For example, the following search returns
+only the timestamp and PID from the `_source` of each matching event.
 
 [source,console]
 ----
@@ -179,6 +186,117 @@ The API returns the following response.
 }
 ----
 
+You can also use the `fields` parameter to retrieve and format specific fields
+in the response.
+
+include::{es-repo-dir}/search/search-your-data/retrieve-selected-fields.asciidoc[tag=fields-param-desc]
+
+The following EQL search uses the `fields` parameter to retrieve values for the
+`event.type` field, all fields starting with `process.`, and the `@timestamp`
+field. The request also uses the `filter_path` query parameter to exclude the
+`_source` of each hit.
+
+[source,console]
+----
+GET /my-index-000001/_eql/search?filter_path=-hits.events._source
+{
+  "query": """
+    process where process.name == "regsvr32.exe"
+  """,
+  "fields": [
+    "event.type",
+    "process.*",                <1>
+    {
+      "field": "@timestamp",    <2>
+      "format": "epoch_millis"  
+    }
+  ]
+}
+----
+// TEST[setup:sec_logs]
+
+include::{es-repo-dir}/search/search-your-data/retrieve-selected-fields.asciidoc[tag=fields-param-callouts]
+
+The values are returned as a flat list in the `fields` section of each hit:
+
+[source,console-result]
+----
+{
+  "is_partial": false,
+  "is_running": false,
+  "took": 60,
+  "timed_out": false,
+  "hits": {
+    "total": {
+      "value": 2,
+      "relation": "eq"
+    },
+    "events": [
+      {
+        "_index": "my-index-000001",
+        "_id": "OQmfCaduce8zoHT93o4H",
+        "fields": {
+          "process.name": [
+            "regsvr32.exe"
+          ],
+          "process.name.keyword": [
+            "regsvr32.exe"
+          ],
+          "@timestamp": [
+            "4100324829000"
+          ],
+          "process.command_line": [
+            "regsvr32.exe  /s /u /i:https://...RegSvr32.sct scrobj.dll"
+          ],
+          "process.command_line.keyword": [
+            "regsvr32.exe  /s /u /i:https://...RegSvr32.sct scrobj.dll"
+          ],
+          "process.executable.keyword": [
+            "C:\\Windows\\System32\\regsvr32.exe"
+          ],
+          "process.pid": [
+            2012
+          ],
+          "process.executable": [
+            "C:\\Windows\\System32\\regsvr32.exe"
+          ]
+        }
+      },
+      {
+        "_index": "my-index-000001",
+        "_id": "xLkCaj4EujzdNSxfYLbO",
+        "fields": {
+          "process.name": [
+            "regsvr32.exe"
+          ],
+          "process.name.keyword": [
+            "regsvr32.exe"
+          ],
+          "@timestamp": [
+            "4100324830000"
+          ],
+          "event.type": [
+            "termination"
+          ],
+          "process.executable.keyword": [
+            "C:\\Windows\\System32\\regsvr32.exe"
+          ],
+          "process.pid": [
+            2012
+          ],
+          "process.executable": [
+            "C:\\Windows\\System32\\regsvr32.exe"
+          ]
+        }
+      }
+    ]
+  }
+}
+----
+// TESTRESPONSE[s/"took": 60/"took": $body.took/]
+// TESTRESPONSE[s/"_id": "OQmfCaduce8zoHT93o4H"/"_id": $body.hits.events.0._id/]
+// TESTRESPONSE[s/"_id": "xLkCaj4EujzdNSxfYLbO"/"_id": $body.hits.events.1._id/]
+
 [discrete]
 [[eql-search-sequence]]
 === Search for a sequence of events

diff --git a/docs/reference/search/search-your-data/retrieve-selected-fields.asciidoc b/docs/reference/search/search-your-data/retrieve-selected-fields.asciidoc
@@ -60,11 +60,13 @@ following sections:
 === Fields
 beta::[]
 
+// tag::fields-param-desc[]
 The `fields` parameter allows for retrieving a list of document fields in
 the search response. It consults both the document `_source` and the index
 mappings to return each value in a standardized way that matches its mapping
 type. By default, date fields are formatted according to the
 <<mapping-date-format,date format>> parameter in their mappings.
+// end::fields-param-desc[]
 
 The following search request uses the `fields` parameter to retrieve values
 for the `user.id` field, all fields starting with `http.response.`, and the
@@ -92,15 +94,17 @@ POST my-index-000001/_search
 ----
 // TEST[setup:my_index]
 
+// tag::fields-param-callouts[]
 <1> Both full field names and wildcard patterns are accepted.
 <2> Using object notation, you can pass a `format` parameter to apply a custom
-    format for the field's values. The date fields
-    <<date,`date`>> and <<date_nanos, `date_nanos`>> accept a
+    format for the field's values.
+    <<date,`date`>> and <<date_nanos, `date_nanos`>> fields accept a
     <<mapping-date-format,date format>>. <<spatial_datatypes, Spatial fields>>
     accept either `geojson` for http://www.geojson.org[GeoJSON] (the default)
     or `wkt` for
     {wikipedia}/Well-known_text_representation_of_geometry[Well Known Text].
     Other field types do not support the `format` parameter.
+// end::fields-param-callouts[]
 
 The values are returned as a flat list in the `fields` section in each hit: