diff --git a/CHANGELOG.md b/CHANGELOG.md index 0a218f9515..920442ef3d 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -11,6 +11,9 @@ All notable changes to this project will be documented in this file based on the * Rename log.message to log.original. #106 * Rename `event.raw` to `event.original`. * Rename `user_agent.raw` to `user_agent.original` and make it a keyword. +* Rename `file.path.raw` to `file.path.keyword`, `file.target_path.raw` to `file.target_path.keyword`, + `url.href.raw` to `url.href.keyword`, `url.path.raw` to `url.path.keyword`, + `url.query.raw` to `url.query.keyword`, and `network.name.raw` to `network.name.keyword`. ### Bugfixes diff --git a/README.md b/README.md index 0dfdef4d1f..06a5c715f9 100644 --- a/README.md +++ b/README.md @@ -198,9 +198,9 @@ File fields provide details about each file. | Field | Description | Type | Multi Field | Example | |---|---|---|---|---| | file.path | Path to the file. | text | | | -| file.path.raw | Path to the file. This is a non-analyzed field that is useful for aggregations. | keyword | 1 | | +| file.path.keyword | Path to the file. This is a non-analyzed field that is useful for aggregations. | keyword | 1 | | | file.target_path | Target path for symlinks. | text | | | -| file.target_path.raw | Path to the file. This is a non-analyzed field that is useful for aggregations. | keyword | 1 | | +| file.target_path.keyword | Path to the file. This is a non-analyzed field that is useful for aggregations. | keyword | 1 | | | file.extension | File extension.
This should allow easy filtering by file extensions. | keyword | | `png` | | file.type | File type (file, dir, or symlink). | keyword | | | | file.device | Device that is the source of the file. | keyword | | | @@ -299,7 +299,7 @@ Fields related to network data. | Field | Description | Type | Multi Field | Example | |---|---|---|---|---| | network.name | Name given by operators to sections of their network. | text | | `Guest Wifi` | -| network.name.raw | Name given by operators to sections of their network. | keyword | 1 | | +| network.name.keyword | Name given by operators to sections of their network. | keyword | 1 | | | network.protocol | Network protocol name. | keyword | | `http` | | network.direction | Direction of the network traffic.
Recommended values are:
* inbound
* outbound
* unknown | keyword | | `inbound` | | network.forwarded_ip | Host IP address when the source IP address is the proxy. | ip | | `192.1.1.2` | @@ -404,14 +404,14 @@ URL fields provide a complete URL, with scheme, host, and path. The URL object c | Field | Description | Type | Multi Field | Example | |---|---|---|---|---| | url.href | Full url. The field is stored as keyword.
`url.href` is a [multi field](https://www.elastic.co/guide/en/ elasticsearch/reference/6.2/ multi-fields.html#_multi_fields_with_multiple_analyzers). The data is stored as keyword `url.href` and test `url.href.analyzed`. These fields enable you to run a query against part of the url still works splitting up the URL at ingest time.
`href` is an analyzed field so the parsed information can be accessed through `href.analyzed` in queries. | text | | `https://elastic.co:443/search?q=elasticsearch#top` | -| url.href.raw | The full URL. This is a non-analyzed field that is useful for aggregations. | keyword | 1 | | +| url.href.keyword | The full URL. This is a non-analyzed field that is useful for aggregations. | keyword | 1 | | | url.scheme | Scheme of the request, such as "https".
Note: The `:` is not part of the scheme. | keyword | | `https` | | url.host.name | Hostname of the request, such as "example.com".
For correlation the this field can be copied into the `host.name` field. | keyword | | `elastic.co` | | url.port | Port of the request, such as 443. | integer | | `443` | | url.path | Path of the request, such as "/search". | text | | | -| url.path.raw | URL path. A non-analyzed field that is useful for aggregations. | keyword | 1 | | +| url.path.keyword | URL path. A non-analyzed field that is useful for aggregations. | keyword | 1 | | | url.query | The query field describes the query string of the request, such as "q=elasticsearch".
The `?` is excluded from the query string. If a URL contains no `?`, there is no query field. If there is a `?` but no query, the query field exists with an empty string. The `exists` query can be used to differentiate between the two cases. | text | | | -| url.query.raw | URL query part. A non-analyzed field that is useful for aggregations. | keyword | 1 | | +| url.query.keyword | URL query part. A non-analyzed field that is useful for aggregations. | keyword | 1 | | | url.fragment | Portion of the url after the `#`, such as "top".
The `#` is not part of the fragment. | keyword | | | | url.username | Username of the request. | keyword | | | | url.password | Password of the request. | keyword | | | diff --git a/fields.yml b/fields.yml index 3706556bd3..07296d8f28 100644 --- a/fields.yml +++ b/fields.yml @@ -454,7 +454,7 @@ type: text description: Path to the file. multi_fields: - - name: raw + - name: keyword type: keyword description: > Path to the file. This is a non-analyzed field that is useful @@ -464,7 +464,7 @@ type: text description: Target path for symlinks. multi_fields: - - name: raw + - name: keyword type: keyword description: > Path to the file. This is a non-analyzed field that is useful @@ -764,7 +764,7 @@ Name given by operators to sections of their network. example: Guest Wifi multi_fields: - - name: raw + - name: keyword type: keyword description: > Name given by operators to sections of their network. @@ -1083,7 +1083,7 @@ through `href.analyzed` in queries. multi_fields: - - name: raw + - name: keyword type: keyword description: > The full URL. This is a non-analyzed field that is useful @@ -1114,7 +1114,7 @@ description: > Path of the request, such as "/search". multi_fields: - - name: raw + - name: keyword type: keyword description: > URL path. A non-analyzed field that is useful @@ -1130,7 +1130,7 @@ the query field exists with an empty string. The `exists` query can be used to differentiate between the two cases. multi_fields: - - name: raw + - name: keyword type: keyword description: > URL query part. A non-analyzed field that is useful diff --git a/schemas/file.yml b/schemas/file.yml index d2dbe3c094..4c87e91bd4 100644 --- a/schemas/file.yml +++ b/schemas/file.yml @@ -10,7 +10,7 @@ type: text description: Path to the file. multi_fields: - - name: raw + - name: keyword type: keyword description: > Path to the file. This is a non-analyzed field that is useful @@ -20,7 +20,7 @@ type: text description: Target path for symlinks. multi_fields: - - name: raw + - name: keyword type: keyword description: > Path to the file. This is a non-analyzed field that is useful diff --git a/schemas/network.yml b/schemas/network.yml index 1eef90caef..e4c5201d9a 100644 --- a/schemas/network.yml +++ b/schemas/network.yml @@ -12,7 +12,7 @@ Name given by operators to sections of their network. example: Guest Wifi multi_fields: - - name: raw + - name: keyword type: keyword description: > Name given by operators to sections of their network. diff --git a/schemas/url.yml b/schemas/url.yml index 25c738fb91..11847af15d 100644 --- a/schemas/url.yml +++ b/schemas/url.yml @@ -24,7 +24,7 @@ through `href.analyzed` in queries. multi_fields: - - name: raw + - name: keyword type: keyword description: > The full URL. This is a non-analyzed field that is useful @@ -55,7 +55,7 @@ description: > Path of the request, such as "/search". multi_fields: - - name: raw + - name: keyword type: keyword description: > URL path. A non-analyzed field that is useful @@ -71,7 +71,7 @@ the query field exists with an empty string. The `exists` query can be used to differentiate between the two cases. multi_fields: - - name: raw + - name: keyword type: keyword description: > URL query part. A non-analyzed field that is useful diff --git a/template.json b/template.json index 5c66d5a963..978473c848 100644 --- a/template.json +++ b/template.json @@ -297,7 +297,7 @@ }, "path": { "fields": { - "raw": { + "keyword": { "ignore_above": 1024, "type": "keyword" } @@ -310,7 +310,7 @@ }, "target_path": { "fields": { - "raw": { + "keyword": { "ignore_above": 1024, "type": "keyword" } @@ -514,7 +514,7 @@ }, "name": { "fields": { - "raw": { + "keyword": { "ignore_above": 1024, "type": "keyword" } @@ -700,7 +700,7 @@ }, "href": { "fields": { - "raw": { + "keyword": { "ignore_above": 1024, "type": "keyword" } @@ -714,7 +714,7 @@ }, "path": { "fields": { - "raw": { + "keyword": { "ignore_above": 1024, "type": "keyword" } @@ -727,7 +727,7 @@ }, "query": { "fields": { - "raw": { + "keyword": { "ignore_above": 1024, "type": "keyword" }