Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[DOCS] Add fields parameter to EQL search API #69634

Merged
merged 1 commit into from
Mar 1, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 28 additions & 0 deletions docs/reference/eql/eql-search-api.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -185,6 +185,34 @@ returned.
+
A greater `fetch_size` value often increases search speed but uses more memory.

`fields`::
(Optional, array of strings and objects)
Array of wildcard (`*`) patterns. The response returns values for field names
matching these patterns in the `fields` property of each hit.
+
You can specify items in the array as a string or object.
+
.Properties of `fields` objects
[%collapsible%open]
====
`field`::
(Required, string)
Wildcard pattern. The request returns values for field names matching this
pattern.

`format`::
(Optional, string)
Format in which the values are returned.
+
<<date,`date`>> and <<date_nanos, `date_nanos`>> fields accept a
<<mapping-date-format,date format>>. <<spatial_datatypes, Spatial fields>>
accept either `geojson` for http://www.geojson.org[GeoJSON] (the default) or
`wkt` for {wikipedia}/Well-known_text_representation_of_geometry[Well Known
Text].
+
For other field data types, this parameter is not supported.
====

`filter`::
(Optional, <<query-dsl,Query DSL object>>)
Query, written in Query DSL, used to filter the events on which the EQL query
Expand Down
124 changes: 121 additions & 3 deletions docs/reference/eql/eql.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -136,9 +136,16 @@ GET /my-index-000001/_eql/search
----
// TEST[setup:sec_logs]

Use the <<common-options-response-filtering,`filter_path`>> query parameter to
filter the API response. For example, the following search returns only the
timestamp and PID for each matching event.
[discrete]
[[retrieve-selected-fields]]
== Retrieve selected fields

By default, each hit in the search response includes the document `_source`,
which is the entire JSON object that was provided when indexing the document.

You can use the <<common-options-response-filtering,`filter_path`>> query
parameter to filter the API response. For example, the following search returns
only the timestamp and PID from the `_source` of each matching event.

[source,console]
----
Expand Down Expand Up @@ -179,6 +186,117 @@ The API returns the following response.
}
----

You can also use the `fields` parameter to retrieve and format specific fields
in the response.

include::{es-repo-dir}/search/search-your-data/retrieve-selected-fields.asciidoc[tag=fields-param-desc]

The following EQL search uses the `fields` parameter to retrieve values for the
`event.type` field, all fields starting with `process.`, and the `@timestamp`
field. The request also uses the `filter_path` query parameter to exclude the
`_source` of each hit.

[source,console]
----
GET /my-index-000001/_eql/search?filter_path=-hits.events._source
{
"query": """
process where process.name == "regsvr32.exe"
""",
"fields": [
"event.type",
"process.*", <1>
{
"field": "@timestamp", <2>
"format": "epoch_millis"
}
]
}
----
// TEST[setup:sec_logs]

include::{es-repo-dir}/search/search-your-data/retrieve-selected-fields.asciidoc[tag=fields-param-callouts]

The values are returned as a flat list in the `fields` section of each hit:

[source,console-result]
----
{
"is_partial": false,
"is_running": false,
"took": 60,
"timed_out": false,
"hits": {
"total": {
"value": 2,
"relation": "eq"
},
"events": [
{
"_index": "my-index-000001",
"_id": "OQmfCaduce8zoHT93o4H",
"fields": {
"process.name": [
"regsvr32.exe"
],
"process.name.keyword": [
"regsvr32.exe"
],
"@timestamp": [
"4100324829000"
],
"process.command_line": [
"regsvr32.exe /s /u /i:https://...RegSvr32.sct scrobj.dll"
],
"process.command_line.keyword": [
"regsvr32.exe /s /u /i:https://...RegSvr32.sct scrobj.dll"
],
"process.executable.keyword": [
"C:\\Windows\\System32\\regsvr32.exe"
],
"process.pid": [
2012
],
"process.executable": [
"C:\\Windows\\System32\\regsvr32.exe"
]
}
},
{
"_index": "my-index-000001",
"_id": "xLkCaj4EujzdNSxfYLbO",
"fields": {
"process.name": [
"regsvr32.exe"
],
"process.name.keyword": [
"regsvr32.exe"
],
"@timestamp": [
"4100324830000"
],
"event.type": [
"termination"
],
"process.executable.keyword": [
"C:\\Windows\\System32\\regsvr32.exe"
],
"process.pid": [
2012
],
"process.executable": [
"C:\\Windows\\System32\\regsvr32.exe"
]
}
}
]
}
}
----
// TESTRESPONSE[s/"took": 60/"took": $body.took/]
// TESTRESPONSE[s/"_id": "OQmfCaduce8zoHT93o4H"/"_id": $body.hits.events.0._id/]
// TESTRESPONSE[s/"_id": "xLkCaj4EujzdNSxfYLbO"/"_id": $body.hits.events.1._id/]

[discrete]
[[eql-search-sequence]]
=== Search for a sequence of events
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -60,11 +60,13 @@ following sections:
=== Fields
beta::[]

// tag::fields-param-desc[]
The `fields` parameter allows for retrieving a list of document fields in
the search response. It consults both the document `_source` and the index
mappings to return each value in a standardized way that matches its mapping
type. By default, date fields are formatted according to the
<<mapping-date-format,date format>> parameter in their mappings.
// end::fields-param-desc[]

The following search request uses the `fields` parameter to retrieve values
for the `user.id` field, all fields starting with `http.response.`, and the
Expand Down Expand Up @@ -92,15 +94,17 @@ POST my-index-000001/_search
----
// TEST[setup:my_index]

// tag::fields-param-callouts[]
<1> Both full field names and wildcard patterns are accepted.
<2> Using object notation, you can pass a `format` parameter to apply a custom
format for the field's values. The date fields
<<date,`date`>> and <<date_nanos, `date_nanos`>> accept a
format for the field's values.
<<date,`date`>> and <<date_nanos, `date_nanos`>> fields accept a
<<mapping-date-format,date format>>. <<spatial_datatypes, Spatial fields>>
accept either `geojson` for http://www.geojson.org[GeoJSON] (the default)
or `wkt` for
{wikipedia}/Well-known_text_representation_of_geometry[Well Known Text].
Other field types do not support the `format` parameter.
// end::fields-param-callouts[]

The values are returned as a flat list in the `fields` section in each hit:

Expand Down