Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding elastic common schema (ECS) to appendix A #4

Closed
wants to merge 2 commits into from
Closed

Adding elastic common schema (ECS) to appendix A #4

Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
c272a25
Adding elastic common schema (ECS) to appendix A
May 13, 2020
9cb86f4
fixed @tigrannajaryan's comments and reformatted quotes
May 20, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
321 changes: 321 additions & 0 deletions text/0097-log-data-model.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,7 @@ Introduce Data Model for Log Records as it is understood by OpenTelemetry.
* [Apache HTTP Server access log](#apache-http-server-access-log)
* [CloudTrail Log Event](#cloudtrail-log-event)
* [Google Cloud Logging](#google-cloud-logging)
* [Elastic Common Schema](#elastic-common-schema)
* [Appendix B: `SeverityNumber` example mappings](#appendix-b-severitynumber-example-mappings)
* [References](#references)

Expand Down Expand Up @@ -1002,6 +1003,326 @@ span_id | string | The span ID within the trace associated
labels | map<string,string> | A set of user-defined (key, value) data that provides additional information about the log entry. | Attributes
All other fields | | | Attributes["google.*"]


## Elastic Common Schema

<table>
<tr>
<td>Field</td>
<td>Type</td>
<td>Description</td>
<td>Maps to Unified Model Field</td>
</tr>
<tr>
<td>@timestamp</td>
<td>datetime</td>
<td>Time the event was recorded</td>
<td>timestamp</td>
</tr>
<tr>
<td>message</td>
<td>string</td>
<td>Any type of message</td>
<td>body</td>
</tr>
<tr>
<td>labels</td>
<td>key/value</td>
<td>Arbitrary labels related to the event</td>
<td>attributes[*]</td>
</tr>
<tr>
<td>tags</td>
<td>array of string</td>
<td>List of values related to the event</td>
<td>?</td>
</tr>
<tr>
<td>trace.id</td>
<td>string</td>
<td>Trace ID</td>
<td>trace_id</td>
</tr>
<tr>
<td>span.id*</td>
<td>string</td>
<td>Span ID</td>
<td>span_id</td>
</tr>
<tr>
<td>agent.ephemeral_id</td>
<td>string</td>
<td>Ephemeral ID created by agent</td>
<td>**resource</td>
</tr>
<tr>
<td>agent.id</td>
<td>string</td>
<td>Unique identifier of this agent</td>
<td>**resource</td>
</tr>
<tr>
<td>agent.name</td>
<td>string</td>
<td>Name given to the agent</td>
<td>resource["telemetry.sdk.name"]</td>
</tr>
<tr>
<td>agent.type</td>
<td>string</td>
<td>Type of agent</td>
<td>resource[“telemetry.sdk.language”]</td>
</tr>
<tr>
<td>agent.version</td>
<td>string</td>
<td>Version of agent</td>
<td>resource[“telemetry.sdk.version”]</td>
</tr>
<tr>
<td>source.ip, client.ip</td>
<td>string</td>
<td>The IP address that the request was made from.</td>
<td>resource[“net.peer.ip”] or resource[“net.host.ip”]</td>
roncohen marked this conversation as resolved.
Show resolved Hide resolved
</tr>
<tr>
<td>cloud.account.id</td>
<td>string</td>
<td>ID of the account in the given cloud</td>
<td>resource[“cloud.account.id”]</td>
</tr>
<tr>
<td>cloud.availability_zone</td>
<td>string</td>
<td>Availability zone in which this host is running.</td>
<td>resource[“cloud.zone”]</td>
</tr>
<tr>
<td>cloud.instance.id</td>
<td>string</td>
<td>Instance ID of the host machine.</td>
<td>**resource</td>
</tr>
<tr>
<td>cloud.instance.name</td>
<td>string</td>
<td>Instance name of the host machine.</td>
<td>**resource</td>
</tr>
<tr>
<td>cloud.machine.type</td>
<td>string</td>
<td>Machine type of the host machine.</td>
<td>**resource</td>
</tr>
<tr>
<td>cloud.provider</td>
<td>string</td>
<td>Name of the cloud provider. Example values are aws, azure, gcp, or digitalocean.</td>
<td>resource[“cloud.provider”]</td>
</tr>
<tr>
<td>cloud.region</td>
<td>string</td>
<td>Region in which this host is running.</td>
<td>resource[“cloud.region”]</td>
</tr>
<tr>
<td>cloud.image.id*</td>
<td>string</td>
<td></td>
<td>resource[“host.image.name”]</td>
</tr>
<tr>
<td>container.id</td>
<td>string</td>
<td>Unique container id</td>
<td>resource[“container.id”]</td>
</tr>
<tr>
<td>container.image.name</td>
<td>string</td>
<td>Name of the image the container was built on.</td>
<td>resource[“container.image.name”]</td>
</tr>
<tr>
<td>container.image.tag</td>
<td>Array of string</td>
<td>Container image tags.</td>
<td>**resource</td>
</tr>
<tr>
<td>container.labels</td>
<td>key/value</td>
<td>Image labels.</td>
<td>attributes[*]</td>
</tr>
<tr>
<td>container.name</td>
<td>string</td>
<td>Container name.</td>
<td>resource[“container.name”]</td>
</tr>
<tr>
<td>container.runtime</td>
<td>string</td>
<td>Runtime managing this container. Example: “docker”</td>
<td>**resource</td>
</tr>
<tr>
<td>destination.address</td>
<td>string</td>
<td>Destination address for the event</td>
<td>attributes[“destination.address”]</td>
</tr>
<tr>
<td>error.code</td>
<td>string</td>
<td>Error code describing the error.</td>
<td>attributes[“error.code”]</td>
</tr>
<tr>
<td>error.id</td>
<td>string</td>
<td>Unique identifier for the error.</td>
<td>attributes[“error.id”]</td>
</tr>
<tr>
<td>error.message</td>
<td>string</td>
<td>Error message.</td>
<td>attributes[“error.stack_trace”]</td>
roncohen marked this conversation as resolved.
Show resolved Hide resolved
</tr>
<tr>
<td>error.stack_trace</td>
<td>string</td>
<td>The stack trace of this error in plain text.</td>
<td>**resource</td>
roncohen marked this conversation as resolved.
Show resolved Hide resolved
</tr>
<tr>
<td>host.architecture</td>
<td>string</td>
<td>Operating system architecture</td>
<td>**resource</td>
</tr>
<tr>
<td>host.domain</td>
<td>string</td>
<td>Name of the domain of which the host is a member.

For example, on Windows this could be the host’s Active Directory domain or NetBIOS domain name. For Linux this could be the domain of the host’s LDAP provider.</td>

<td>**resource</td>
</tr>
<tr>
<td>host.hostname</td>
<td>string</td>
<td>Hostname of the host.

It normally contains what the hostname command returns on the host machine.</td>

<td>resource[“host.hostname”]</td>

</tr>
<tr>
<td>host.id</td>
<td>string</td>
<td>Unique host id.</td>
<td>resource[“host.id”]</td>
</tr>
<tr>
<td>host.ip</td>
<td>Array of string</td>
<td>Host IP</td>
<td>resource[“host.ip”]</td>
</tr>
<tr>
<td>host.mac</td>
<td>array of string</td>
<td>MAC addresses of the host</td>
<td>resource[“host.mac”]</td>
</tr>
<tr>
<td>host.name</td>
<td>string</td>
<td>Name of the host.

It may contain what hostname returns on Unix systems, the fully qualified, or a name specified by the user. </td>

<td>resource[“host.name”]</td>

</tr>
<tr>
<td>host.type</td>
<td>string</td>
<td>Type of host.</td>
<td>resource[“host.type”]</td>
</tr>
<tr>
<td>host.uptime</td>
<td>string</td>
<td>Seconds the host has been up.</td>
<td>?</td>
</tr>
<tr>
<td>service.ephemeral_id

</td>
<td>string</td>
<td>Ephemeral identifier of this service</td>
<td>**resource</td>
</tr>
<tr>
<td>service.id</td>
<td>string</td>
<td>Unique identifier of the running service. If the service is comprised of many nodes, the service.id should be the same for all nodes.</td>
<td>**resource</td>
</tr>
<tr>
<td>service.name</td>
<td>string</td>
<td>Name of the service data is collected from.</td>
<td>resource["service.name"]</td>
</tr>
<tr>
<td>service.node.name</td>
<td>string</td>
<td>Specific node serving that service</td>
<td>resource[“service.instance.id”]</td>
</tr>
<tr>
<td>service.state</td>
<td>string</td>
<td>Current state of the service.</td>
<td>**resource</td>
roncohen marked this conversation as resolved.
Show resolved Hide resolved
</tr>
<tr>
<td>service.type</td>
<td>string</td>
<td>The type of the service data is collected from.</td>
<td>**resource</td>
</tr>
<tr>
<td>service.version</td>
<td>string</td>
<td>Version of the service the data was collected from.</td>
<td>resource[“service.version”]</td>
</tr>
<tr>
<td></td>
<td></td>
<td></td>
<td></td>
</tr>
</table>

\* Not yet formalized into ECS

\*\* A resource that doesn’t exist in the [OpenTelemetry resource semantic convention](https://github.com/open-telemetry/opentelemetry-specification/tree/master/specification/resource/semantic_conventions)

This is a selection of the most relevant fields. See [for the full reference](https://www.elastic.co/guide/en/ecs/current/ecs-field-reference.html) for an exhaustive list.


## Appendix B: `SeverityNumber` example mappings

|Syslog |WinEvtLog |Log4j |Zap |java.util.logging|SeverityNumber|
Expand Down