diff --git a/CHANGELOG.md b/CHANGELOG.md index bb32a3b18c..5eca3870bd 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -14,7 +14,8 @@ All notable changes to this project will be documented in this file based on the * Rename `file.path.raw` to `file.path.keyword`, `file.target_path.raw` to `file.target_path.keyword`, `url.href.raw` to `url.href.keyword`, `url.path.raw` to `url.path.keyword`, `url.query.raw` to `url.query.keyword`, and `network.name.raw` to `network.name.keyword`. -* Remove `log.offset` and `log.line` as to specific for ECS. +* Remove `log.offset` and `log.line` as too specific for ECS. +* Remove top level objects `kubernetes` and `tls`. ### Bugfixes diff --git a/README.md b/README.md index 4aded39861..4434cacee6 100644 --- a/README.md +++ b/README.md @@ -47,7 +47,6 @@ ECS defines these fields. * [Geo fields](#geo) * [Host fields](#host) * [HTTP fields](#http) - * [Kubernetes fields](#kubernetes) * [Log fields](#log) * [Network fields](#network) * [Organization fields](#organization) @@ -55,7 +54,6 @@ ECS defines these fields. * [Process fields](#process) * [Service fields](#service) * [Source fields](#source) - * [TLS fields](#tls) * [URL fields](#url) * [User fields](#user) * [User agent fields](#user_agent) @@ -264,20 +262,6 @@ Fields related to HTTP requests and responses. | http.version | Http version. | keyword | | `1.1` | -## Kubernetes fields - -Kubernetes fields are used for Kubernetes meta information. This information helps correlate data from Kubernetes setups. - - -| Field | Description | Type | Multi Field | Example | -|---|---|---|---|---| -| kubernetes.pod.name | Kubernetes pod name | keyword | | | -| kubernetes.namespace | Kubernetes namespace | keyword | | | -| kubernetes.labels | Kubernetes labels map | object | | | -| kubernetes.annotations | Kubernetes annotations map | object | | | -| kubernetes.container.name | Kubernetes container name. This name is unique within the pod only. It is different from the underlying `container.name` field. | keyword | | | - - ## Log fields Fields which are specific to log events. @@ -378,22 +362,6 @@ Source fields describe details about the source of the event. | source.subdomain | Source subdomain. | keyword | | | -## TLS fields - -The tls fields contain the TLS related data about a specific connection. - - -| Field | Description | Type | Multi Field | Example | -|---|---|---|---|---| -| tls.version | TLS version. | keyword | | `TLSv1.2` | -| tls.certificates | An array of certificates. | keyword | | | -| tls.servername | Server name requested by the client. | keyword | | `localhost` | -| tls.ciphersuite | Name of the cipher used for the communication. | keyword | | `ECDHE-ECDSA-AES-128-CBC-SHA` | - - -As an example in the case of Filebeat and the TCP input, the `version` field would be the version of the TLS protocol in use, the `certificates` would be the chain of certificates provided by the client and the `ciphersuite` is the encryption algorithm used for the communication. - - ## URL fields URL fields provide a complete URL, with scheme, host, and path. The URL object can be reused in other prefixes, such as `host.url.*` for example. Keep the structure consistent whenever you use URL fields. @@ -463,8 +431,10 @@ Contributions of additional uses cases on top of ECS are welcome. * [Auditbeat](https://github.com/elastic/ecs/blob/master/use-cases/auditbeat.md) * [Beats](https://github.com/elastic/ecs/blob/master/use-cases/beats.md) * [Filebeat Apache](https://github.com/elastic/ecs/blob/master/use-cases/filebeat-apache-access.md) + * [Kubernetes](https://github.com/elastic/ecs/blob/master/use-cases/kubernetes.md) * [Logging](https://github.com/elastic/ecs/blob/master/use-cases/logging.md) * [Metricbeat](https://github.com/elastic/ecs/blob/master/use-cases/metricbeat.md) + * [TLS](https://github.com/elastic/ecs/blob/master/use-cases/tls.md) diff --git a/fields.yml b/fields.yml index 01ef6016e0..dc2a21ca8c 100644 --- a/fields.yml +++ b/fields.yml @@ -674,41 +674,6 @@ Http version. example: 1.1 - - name: kubernetes - title: Kubernetes - group: 2 - description: > - Kubernetes fields are used for Kubernetes meta information. This - information helps correlate data from Kubernetes setups. - - type: group - fields: - - name: pod.name - type: keyword - description: > - Kubernetes pod name - - - name: namespace - type: keyword - description: > - Kubernetes namespace - - - name: labels - type: object - description: > - Kubernetes labels map - - - name: annotations - type: object - description: > - Kubernetes annotations map - - - name: container.name - type: keyword - description: > - Kubernetes container name. This name is unique within the pod only. - It is different from the underlying `container.name` field. - - name: log title: Log description: > @@ -1010,44 +975,6 @@ description: > Source subdomain. - - name: tls - title: TLS - group: 2 - description: > - The tls fields contain the TLS related data about a specific connection. - - footnote: > - As an example in the case of Filebeat and the TCP input, the `version` field would be the - version of the TLS protocol in use, the `certificates` would be the chain of certificates - provided by the client and the `ciphersuite` is the encryption algorithm used for the - communication. - type: group - fields: - - name: version - type: keyword - description: > - TLS version. - - example: TLSv1.2 - - name: certificates - type: keyword - description: > - An array of certificates. - ignore_above: -1 - doc_values: false - - name: servername - type: keyword - description: > - Server name requested by the client. - - example: localhost - - name: ciphersuite - type: keyword - description: > - Name of the cipher used for the communication. - - example: ECDHE-ECDSA-AES-128-CBC-SHA - - name: url title: URL description: > diff --git a/schema.csv b/schema.csv index 67ce6459db..dcefdb1012 100644 --- a/schema.csv +++ b/schema.csv @@ -85,11 +85,6 @@ http.request.method,keyword,0,"GET, POST, PUT" http.response.body,text,0,Hello world http.response.status_code,long,0,404 http.version,keyword,0,1.1 -kubernetes.annotations,object,0, -kubernetes.container.name,keyword,0, -kubernetes.labels,object,0, -kubernetes.namespace,keyword,0, -kubernetes.pod.name,keyword,0, log.level,keyword,0,ERR log.original,keyword,1,Sep 19 08:26:10 localhost My log network.direction,keyword,0,inbound @@ -126,10 +121,6 @@ source.ip,ip,0, source.mac,keyword,1, source.port,long,1, source.subdomain,keyword,1, -tls.certificates,keyword,0, -tls.ciphersuite,keyword,0,ECDHE-ECDSA-AES-128-CBC-SHA -tls.servername,keyword,0,localhost -tls.version,keyword,0,TLSv1.2 url.fragment,keyword,0, url.host.name,keyword,0,elastic.co url.href,text,0,https://elastic.co:443/search?q=elasticsearch#top diff --git a/schemas/kubernetes.yml b/schemas/kubernetes.yml deleted file mode 100644 index a667ade2aa..0000000000 --- a/schemas/kubernetes.yml +++ /dev/null @@ -1,35 +0,0 @@ ---- -- name: kubernetes - title: Kubernetes - group: 2 - description: > - Kubernetes fields are used for Kubernetes meta information. This - information helps correlate data from Kubernetes setups. - - type: group - fields: - - name: pod.name - type: keyword - description: > - Kubernetes pod name - - - name: namespace - type: keyword - description: > - Kubernetes namespace - - - name: labels - type: object - description: > - Kubernetes labels map - - - name: annotations - type: object - description: > - Kubernetes annotations map - - - name: container.name - type: keyword - description: > - Kubernetes container name. This name is unique within the pod only. - It is different from the underlying `container.name` field. diff --git a/template.json b/template.json index 3d53da71e5..504cb2290e 100644 --- a/template.json +++ b/template.json @@ -436,36 +436,6 @@ } } }, - "kubernetes": { - "properties": { - "annotations": { - "type": "object" - }, - "container": { - "properties": { - "name": { - "ignore_above": 1024, - "type": "keyword" - } - } - }, - "labels": { - "type": "object" - }, - "namespace": { - "ignore_above": 1024, - "type": "keyword" - }, - "pod": { - "properties": { - "name": { - "ignore_above": 1024, - "type": "keyword" - } - } - } - } - }, "labels": { "type": "object" }, @@ -658,26 +628,6 @@ "ignore_above": 1024, "type": "keyword" }, - "tls": { - "properties": { - "certificates": { - "doc_values": false, - "type": "keyword" - }, - "ciphersuite": { - "ignore_above": 1024, - "type": "keyword" - }, - "servername": { - "ignore_above": 1024, - "type": "keyword" - }, - "version": { - "ignore_above": 1024, - "type": "keyword" - } - } - }, "url": { "properties": { "fragment": { diff --git a/use-cases/kubernetes.md b/use-cases/kubernetes.md new file mode 100644 index 0000000000..cce9d10f8b --- /dev/null +++ b/use-cases/kubernetes.md @@ -0,0 +1,21 @@ +## Kubernetes use case + +You can monitor containers running in a Kubernetes cluster by adding Kubernetes-specific information under `kubernetes.` + + +### Kubernetes fields + + +| Field | Description | Type | Multi Field | Example | +|---|---|---|---|---| +| [container.id](https://github.com/elastic/ecs#container.id) | Unique container id. | keyword | | `fdbef803fa2b` | +| [container.name](https://github.com/elastic/ecs#container.name) | Container name. | keyword | | | +| [host.name](https://github.com/elastic/ecs#host.name) | host.name is the hostname of the host.
It can contain what `hostname` returns on Unix systems, the fully qualified domain name, or a name specified by the user. The sender decides which value to use. | keyword | | `kube-high-cpu-42` | +| *kubernetes.pod.name* | *Kubernetes pod name* | keyword | | `foo-webserver` | +| *kubernetes.namespace* | *Kubernetes namespace* | keyword | | `foo-team` | +| *kubernetes.labels* | *Kubernetes labels map* | object | | | +| *kubernetes.annotations* | *Kubernetes annotations map* | object | | | +| *kubernetes.container.name* | *Kubernetes container name. This name is unique within the pod only. It is different from the `container.name` field.* | keyword | | | + + + diff --git a/use-cases/kubernetes.yml b/use-cases/kubernetes.yml new file mode 100644 index 0000000000..be5c93d677 --- /dev/null +++ b/use-cases/kubernetes.yml @@ -0,0 +1,51 @@ +name: kubernetes +title: Kubernetes +description: > + You can monitor containers running in a Kubernetes cluster by adding + Kubernetes-specific information under `kubernetes.` + +fields: +- name: container + fields: + + - name: id + example: fdbef803fa2b + + - name: name + +- name: host + fields: + + - name: name + example: kube-high-cpu-42 + +- name: kubernetes + fields: + + - name: pod.name + type: keyword + description: > + Kubernetes pod name + example: foo-webserver + + - name: namespace + type: keyword + description: > + Kubernetes namespace + example: foo-team + + - name: labels + type: object + description: > + Kubernetes labels map + + - name: annotations + type: object + description: > + Kubernetes annotations map + + - name: container.name + type: keyword + description: > + Kubernetes container name. This name is unique within the pod only. + It is different from the `container.name` field. diff --git a/use-cases/tls.md b/use-cases/tls.md new file mode 100644 index 0000000000..e60c32a570 --- /dev/null +++ b/use-cases/tls.md @@ -0,0 +1,20 @@ +## TLS use case + +You can store TLS-related metadata under `tls.`, when appropriate. + + +### TLS fields + + +| Field | Description | Type | Multi Field | Example | +|---|---|---|---|---| +| [source.ip](https://github.com/elastic/ecs#source.ip) | IP address of the source.
Can be one or multiple IPv4 or IPv6 addresses. | ip | | `10.1.1.10` | +| [destination.ip](https://github.com/elastic/ecs#destination.ip) | IP address of the destination.
Can be one or multiple IPv4 or IPv6 addresses. | ip | | `5.5.5.5` | +| [destination.port](https://github.com/elastic/ecs#destination.port) | Port of the destination. | long | | `443` | +| *tls.version* | *TLS version.* | keyword | | `TLSv1.2` | +| *tls.certificates* | *An array of certificates.* | keyword | | | +| *tls.servername* | *Server name requested by the client.* | keyword | | `localhost` | +| *tls.ciphersuite* | *Name of the cipher used for the communication.* | keyword | | `ECDHE-ECDSA-AES-128-CBC-SHA` | + + + diff --git a/schemas/tls.yml b/use-cases/tls.yml similarity index 56% rename from schemas/tls.yml rename to use-cases/tls.yml index 581eaeb8ac..81621ec839 100644 --- a/schemas/tls.yml +++ b/use-cases/tls.yml @@ -1,38 +1,45 @@ ---- -- name: tls - title: TLS - group: 2 - description: > - The tls fields contain the TLS related data about a specific connection. +name: tls +title: TLS +description: > + You can store TLS-related metadata under `tls.`, when appropriate. + +fields: +- name: source + fields: + - name: ip + example: 10.1.1.10 - footnote: > - As an example in the case of Filebeat and the TCP input, the `version` field would be the - version of the TLS protocol in use, the `certificates` would be the chain of certificates - provided by the client and the `ciphersuite` is the encryption algorithm used for the - communication. - type: group +- name: destination + fields: + - name: ip + example: 5.5.5.5 + + - name: port + example: 443 + +- name: tls fields: - name: version type: keyword description: > TLS version. - example: TLSv1.2 + - name: certificates type: keyword description: > An array of certificates. ignore_above: -1 doc_values: false + - name: servername type: keyword description: > Server name requested by the client. - example: localhost + - name: ciphersuite type: keyword description: > Name of the cipher used for the communication. - example: ECDHE-ECDSA-AES-128-CBC-SHA