data-model-appendix.md

Data Model Appendix

Note: this document is NOT a spec, it is provided to support the Logs Data Model specification. These examples provided purely for demonstrative purposes and are not exhaustive or canonical; please refer to the respective exporter documentation if exact details are required.

• Appendix A. Example Mappings
  • RFC5424 Syslog
  • Windows Event Log
  • SignalFx Events
  • Splunk HEC
  • Log4j
  • Zap
  • Apache HTTP Server access log
  • CloudTrail Log Event
  • Google Cloud Logging
  • Elastic Common Schema
• Appendix B: SeverityNumber example mappings
• References

Appendix A. Example Mappings

This section contains examples of mapping of other events and logs formats to this data model.

RFC5424 Syslog

Property	Type	Description	Maps to Unified Model Field
TIMESTAMP	Timestamp	Time when an event occurred measured by the origin clock.	Timestamp
SEVERITY	enum	Defines the importance of the event. Example: Debug	Severity
FACILITY	enum	Describes where the event originated. A predefined list of Unix processes. Part of event source identity. Example: mail system	Attributes["syslog.facility"]
VERSION	number	Meta: protocol version, orthogonal to the event.	Attributes["syslog.version"]
HOSTNAME	string	Describes the location where the event originated. Possible values are FQDN, IP address, etc.	Resource["host.name"]
APP-NAME	string	User-defined app name. Part of event source identity.	Resource["service.name"]
PROCID	string	Not well defined. May be used as a meta field for protocol operation purposes or may be part of event source identity.	Attributes["syslog.procid"]
MSGID	string	Defines the type of the event. Part of event source identity. Example: "TCPIN"	Attributes["syslog.msgid"]
STRUCTURED-DATA	array of maps of string to string	A variety of use cases depending on the SDID: Can describe event source identity. Can include data that describes particular occurrence of the event. Can be meta-information, e.g. quality of timestamp value.	SDID origin.swVersion map to Resource["service.version"]. SDID origin.ip map to Attributes["client.address"]. Rest of SDIDs -> Attributes["syslog.*"]
MSG	string	Free-form text message about the event. Typically human readable.	Body

Windows Event Log

Property	Type	Description	Maps to Unified Model Field
TimeCreated	Timestamp	The time stamp that identifies when the event was logged.	Timestamp
Level	enum	Contains the severity level of the event.	Severity
Computer	string	The name of the computer on which the event occurred.	Resource["host.name"]
EventID	uint	The identifier that the provider used to identify the event.	Attributes["winlog.event_id"]
Message	string	The message string.	Body
Rest of the fields.	any	All other fields in the event.	Attributes["winlog.*"]

SignalFx Events

Field	Type	Description	Maps to Unified Model Field
Timestamp	Timestamp	Time when the event occurred measured by the origin clock.	Timestamp
EventType	string	Short machine understandable string describing the event type. SignalFx specific concept. Non-namespaced. Example: k8s Event Reason field.	Attributes["com.splunk.signalfx.event_type"]
Category	enum	Describes where the event originated and why. SignalFx specific concept. Example: AGENT.	Attributes["com.splunk.signalfx.event_category"]
Dimensions	map<string, string>	Helps to define the identity of the event source together with EventType and Category. Multiple occurrences of events coming from the same event source can happen across time and they all have the value of Dimensions.	Resource
Properties	map<string, any>	Additional information about the specific event occurrence. Unlike Dimensions which are fixed for a particular event source, Properties can have different values for each occurrence of the event coming from the same event source.	Attributes

Splunk HEC

We apply this mapping from HEC to the unified model:

Field	Type	Description	Maps to Unified Model Field
time	numeric, string	The event time in epoch time format, in seconds.	Timestamp
host	string	The host value to assign to the event data. This is typically the host name of the client that you are sending data from.	Resource["host.name"]
source	string	The source value to assign to the event data. For example, if you are sending data from an app you are developing, you could set this key to the name of the app.	Resource["com.splunk.source"]
sourcetype	string	The sourcetype value to assign to the event data.	Resource["com.splunk.sourcetype"]
event	any	The JSON representation of the raw body of the event. It can be a string, number, string array, number array, JSON object, or a JSON array.	Body
fields	map<string, any>	Specifies a JSON object that contains explicit custom fields.	Attributes
index	string	The name of the index by which the event data is to be indexed. The index you specify here must be within the list of allowed indexes if the token has the indexes parameter set.	Attributes["com.splunk.index"]

When mapping from the unified model to HEC, we apply this additional mapping:

Unified model element	Type	Description	Maps to HEC
SeverityText	string	The severity of the event as a human-readable string.	fields['otel.log.severity.text']
SeverityNumber	string	The severity of the event as a number.	fields['otel.log.severity.number']
Name	string	Short event identifier that does not contain varying parts.	fields['otel.log.name']
TraceId	string	Request trace id.	fields['trace_id']
SpanId	string	Request span id.	fields['span_id']
TraceFlags	string	W3C trace flags.	fields['trace_flags']

Log4j

Field	Type	Description	Maps to Unified Model Field
Instant	Timestamp	Time when an event occurred measured by the origin clock.	Timestamp
Level	enum	Log level.	Severity
Message	string	Human readable message.	Body
All other fields	any	Structured data.	Attributes

Zap

Field	Type	Description	Maps to Unified Model Field
ts	Timestamp	Time when an event occurred measured by the origin clock.	Timestamp
level	enum	Logging level.	Severity
caller	string	Calling function's filename and line number.	Attributes, key=TBD
msg	string	Human readable message.	Body
All other fields	any	Structured data.	Attributes

Apache HTTP Server access log

Field	Type	Description	Maps to Unified Model Field
%t	Timestamp	Time when an event occurred measured by the origin clock.	Timestamp
%a	string	Client address	Attributes["network.peer.address"]
%A	string	Server address	Attributes["network.local.address"]
%h	string	Client hostname.	Attributes["client.address"]
%m	string	The request method.	Attributes["http.request.method"]
%v,%p,%U,%q	string	Multiple fields that can be composed into URL.	Attributes["url.full"]
%>s	string	Response status.	Attributes["http.response.status_code"]
All other fields	any	Structured data.	Attributes, key=TBD

CloudTrail Log Event

Field	Type	Description	Maps to Unified Model Field
eventTime	string	The date and time the request was made, in coordinated universal time (UTC).	Timestamp
eventSource	string	The service that the request was made to. This name is typically a short form of the service name without spaces plus .amazonaws.com.	Resource["service.name"]?
awsRegion	string	The AWS region that the request was made to, such as us-east-2.	Resource["cloud.region"]
sourceIPAddress	string	The IP address that the request was made from.	Attributes["client.address"]
errorCode	string	The AWS service error if the request returns an error.	Attributes["cloudtrail.error_code"]
errorMessage	string	If the request returns an error, the description of the error.	Body
All other fields	*		Attributes["cloudtrail.*"]

Google Cloud Logging

Field	Type	Description	Maps to Unified Model Field
timestamp	string	The time the event described by the log entry occurred.	Timestamp
resource	MonitoredResource	The monitored resource that produced this log entry.	Resource
log_name	string	The URL-encoded LOG_ID suffix of the log_name field identifies which log stream this entry belongs to.	Attributes["gcp.log_name"]
json_payload	google.protobuf.Struct	The log entry payload, represented as a structure that is expressed as a JSON object.	Body
proto_payload	google.protobuf.Any	The log entry payload, represented as a protocol buffer.	Body
text_payload	string	The log entry payload, represented as a Unicode string (UTF-8).	Body
severity	LogSeverity	The severity of the log entry.	Severity
trace	string	The trace associated with the log entry, if any.	TraceId
span_id	string	The span ID within the trace associated with the log entry.	SpanId
labels	map<string,string>	A set of user-defined (key, value) data that provides additional information about the log entry.	Attributes
http_request	HttpRequest	The HTTP request associated with the log entry, if any.	Attributes["gcp.http_request"]
trace_sampled	boolean	The sampling decision of the trace associated with the log entry.	TraceFlags.SAMPLED
All other fields			Attributes["gcp.*"]

Elastic Common Schema

Field	Type	Description	Maps to Unified Model Field
@timestamp	datetime	Time the event was recorded	Timestamp
message	string	Any type of message	Body
labels	key/value	Arbitrary labels related to the event	Attributes[*]
tags	array of string	List of values related to the event	?
trace.id	string	Trace ID	TraceId
span.id*	string	Span ID	SpanId
agent.ephemeral_id	string	Ephemeral ID created by agent	**Resource
agent.id	string	Unique identifier of this agent	**Resource
agent.name	string	Name given to the agent	Resource["telemetry.sdk.name"]
agent.type	string	Type of agent	Resource["telemetry.sdk.language"]
agent.version	string	Version of agent	Resource["telemetry.sdk.version"]
source.ip, client.ip	string	The IP address that the request was made from.	Attributes["client.address"]
cloud.account.id	string	ID of the account in the given cloud	Resource["cloud.account.id"]
cloud.availability_zone	string	Availability zone in which this host is running.	Resource["cloud.zone"]
cloud.instance.id	string	Instance ID of the host machine.	**Resource
cloud.instance.name	string	Instance name of the host machine.	**Resource
cloud.machine.type	string	Machine type of the host machine.	**Resource
cloud.provider	string	Name of the cloud provider. Example values are aws, azure, gcp, or digitalocean.	Resource["cloud.provider"]
cloud.region	string	Region in which this host is running.	Resource["cloud.region"]
cloud.image.id*	string		Resource["host.image.name"]
container.id	string	Unique container id	Resource["container.id"]
container.image.name	string	Name of the image the container was built on.	Resource["container.image.name"]
container.image.tag	Array of string	Container image tags.	**Resource
container.labels	key/value	Image labels.	Attributes[*]
container.name	string	Container name.	Resource["container.name"]
container.runtime	string	Runtime managing this container. Example: "docker"	**Resource
destination.address	string	Destination address for the event	Attributes["destination.address"]
error.code	string	Error code describing the error.	Attributes["error.code"]
error.id	string	Unique identifier for the error.	Attributes["error.id"]
error.message	string	Error message.	Attributes["error.message"]
error.stack_trace	string	The stack trace of this error in plain text.	Attributes["error.stack_trace]
host.architecture	string	Operating system architecture	**Resource
host.domain	string	Name of the domain of which the host is a member. For example, on Windows this could be the host’s Active Directory domain or NetBIOS domain name. For Linux this could be the domain of the host’s LDAP provider.	**Resource
host.name	string	Hostname of the host. It normally contains what the hostname command returns on the host machine.	Resource["host.name"]
host.id	string	Unique host id.	Resource["host.id"]
host.ip	Array of string	Host IP	Resource["host.ip"]
host.mac	array of string	MAC addresses of the host	Resource["host.mac"]
host.name	string	Name of the host. It may contain what hostname returns on Unix systems, the fully qualified, or a name specified by the user.	Resource["host.name"]
host.type	string	Type of host.	Resource["host.type"]
host.uptime	string	Seconds the host has been up.	?
service.ephemeral_id	string	Ephemeral identifier of this service	**Resource
service.id	string	Unique identifier of the running service. If the service is comprised of many nodes, the service.id should be the same for all nodes.	**Resource
service.name	string	Name of the service data is collected from.	Resource["service.name"]
service.node.name	string	Specific node serving that service	Resource["service.instance.id"]
service.state	string	Current state of the service.	Attributes["service.state"]
service.type	string	The type of the service data is collected from.	**Resource
service.version	string	Version of the service the data was collected from.	Resource["service.version"]

* Not yet formalized into ECS.

** A resource that doesn’t exist in the OpenTelemetry resource semantic convention.

This is a selection of the most relevant fields. See for the full reference for an exhaustive list.

Appendix B: SeverityNumber example mappings

Syslog	WinEvtLog	Log4j	Zap	java.util.logging	.NET (Microsoft.Extensions.Logging)	SeverityNumber
		TRACE		FINEST	LogLevel.Trace	TRACE
Debug	Verbose	DEBUG	Debug	FINER	LogLevel.Debug	DEBUG
				FINE		DEBUG2
				CONFIG		DEBUG3
Informational	Information	INFO	Info	INFO	LogLevel.Information	INFO
Notice						INFO2
Warning	Warning	WARN	Warn	WARNING	LogLevel.Warning	WARN
Error	Error	ERROR	Error	SEVERE	LogLevel.Error	ERROR
Critical	Critical		Dpanic			ERROR2
Alert			Panic			ERROR3
Emergency		FATAL	Fatal		LogLevel.Critical	FATAL

References

• OTEP0097 Log Data Model, Appendix A. Example Mappings