Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Align log sdk naming with api #2768

Merged
merged 6 commits into from
Sep 13, 2022
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 17 additions & 17 deletions spec-compliance-matrix.md
Original file line number Diff line number Diff line change
Expand Up @@ -210,23 +210,23 @@ formats is required. Implementing more than one format is optional.

Disclaimer: this list of features is still a work in progress, please refer to the specification if in any doubt.

| Feature | Optional | Go | Java | JS | Python | Ruby | Erlang | PHP | Rust | C++ | .NET | Swift |
|--------------------------------------------------------------|----------|-----|------|-----|--------|------|--------|-----|------|-----|------|-------|
| **[Logging SDK](specification/logs/logging-library-sdk.md)** | Optional | Go | Java | JS | Python | Ruby | Erlang | PHP | Rust | C++ | .NET | Swift |
| LogEmitterProvider.Get LogEmitter | | | + | | + | | | | | | - | |
| LogEmitterProvider.Get LogEmitter accepts attributes | | | | | | | | | | | | |
| LogEmitterProvider.Shutdown | | | + | | + | | | | | | - | |
| LogEmitterProvider.ForceFlush | | | + | | + | | | | | | - | |
| LogEmitter.Emit(LogRecord) | | | + | | + | | | | | | - | |
| SimpleLogProcessor | | | + | | + | | | | | | + | |
| BatchLogProcessor | | | + | | + | | | | | | + | |
| Can plug custom log processor | | | + | | + | | | | | | + | |
| OTLP/gRPC exporter | | | + | | + | | | | | | + | |
| OTLP/HTTP exporter | | | + | | + | | | | | | + | |
| OTLP File exporter | | | - | | - | | | | | | - | |
| Can plug custom log exporter | | | + | | + | | | | | | + | |
| Implicit Context Injection | | | - | | + | | | | | | + | |
| Explicit Context | | | + | | - | | | | | | - | |
jack-berg marked this conversation as resolved.
Show resolved Hide resolved
| Feature | Optional | Go | Java | JS | Python | Ruby | Erlang | PHP | Rust | C++ | .NET | Swift |
|----------------------------------------------|----------|-----|------|-----|--------|------|--------|-----|------|-----|------|-------|
| **[Logging SDK](specification/logs/sdk.md)** | Optional | Go | Java | JS | Python | Ruby | Erlang | PHP | Rust | C++ | .NET | Swift |
| LoggerProvider.Get Logger | | | + | | + | | | | | | - | |
| LoggerProvider.Get Logger accepts attributes | | | | | | | | | | | | |
| LoggerProvider.Shutdown | | | + | | + | | | | | | - | |
| LoggerProvider.ForceFlush | | | + | | + | | | | | | - | |
| Logger.Emit(LogRecord) | | | + | | + | | | | | | - | |
| SimpleLogRecordProcessor | | | + | | + | | | | | | + | |
| BatchLogRecordProcessor | | | + | | + | | | | | | + | |
| Can plug custom LogRecordProcessor | | | + | | + | | | | | | + | |
| OTLP/gRPC exporter | | | + | | + | | | | | | + | |
| OTLP/HTTP exporter | | | + | | + | | | | | | + | |
| OTLP File exporter | | | - | | - | | | | | | - | |
| Can plug custom LogRecordExporter | | | + | | + | | | | | | + | |
| Implicit Context Injection | | | - | | + | | | | | | + | |
| Explicit Context | | | + | | - | | | | | | - | |

## Resource

Expand Down
2 changes: 1 addition & 1 deletion specification/common/attribute-type-mapping.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ The mapping is needed when OpenTelemetry needs to convert a value produced outsi
OpenTelemetry into a value that can be exported using an OTLP exporter, or otherwise be
converted to be used inside OpenTelemetry boundaries. Example use cases are the following:

- In [Logging Library SDK](../logs/logging-library-sdk.md)s, to convert values received
- In [Logging Library SDK](../logs/sdk.md)s, to convert values received
jack-berg marked this conversation as resolved.
Show resolved Hide resolved
from logging libraries into OpenTelemetry representation.
- In the Collector, to convert values received from various data sources into
[pdata](https://github.com/open-telemetry/opentelemetry-collector/blob/4998703dadd19fa91a88eabf7ccc68d728bee3fd/model/pdata/common.go#L84)
Expand Down
165 changes: 138 additions & 27 deletions specification/logs/api.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,12 +9,18 @@

<!-- toc -->

- [Data Model](#data-model)
- [LoggerProvider](#loggerprovider)
* [LoggerProvider operations](#loggerprovider-operations)
+ [Get a Logger](#get-a-logger)
- [Logger](#logger)
* [Logger operations](#logger-operations)
+ [Emit Event](#emit-event)
+ [Emit LogRecord](#emit-logrecord)
- [LogRecord](#logrecord)
- [Usage](#usage)
* [How to Create Log4J Style Appender](#how-to-create-log4j-style-appender)
* [Implicit Context Injection](#implicit-context-injection)
* [Explicit Context](#explicit-context)

<!-- tocstop -->

Expand All @@ -23,7 +29,7 @@
The Events and Logs API consist of these main classes:

* LoggerProvider is the entry point of the API. It provides access to Loggers.
* Logger is the class responsible for creating events and logs using Log records.
* Logger is the class responsible for creating Events and Logs using LogRecords.
jack-berg marked this conversation as resolved.
Show resolved Hide resolved

LoggerProvider/Logger are analogous to TracerProvider/Tracer.

Expand All @@ -34,10 +40,6 @@ graph TD
B --> D(Log)
```

## Data Model

The API creates Events and Logs using the `LogRecord` data model. See `LogRecord` [data model](data-model.md) for the list of fields.

## LoggerProvider

`Logger`s can be accessed with a `LoggerProvider`.
Expand All @@ -51,10 +53,10 @@ LoggerProvider.

Notwithstanding any global LoggerProvider, some applications may want to or have
to use multiple LoggerProvider instances, e.g. to have different configuration
(like LogProcessors) for each (and consequently for the Loggers obtained
from them), or because it's easier with dependency injection frameworks. Thus,
implementations of LoggerProvider SHOULD allow creating an arbitrary number of
instances.
(like [LogRecordProcessors](sdk.md#logrecordprocessor)) for each (and
consequently for the Loggers obtained from them), or because it's easier with
dependency injection frameworks. Thus, implementations of LoggerProvider SHOULD
allow creating an arbitrary number of instances.

### LoggerProvider operations

Expand Down Expand Up @@ -87,10 +89,10 @@ produced by this library.
the scope has a version (e.g. a library version). Example value: 1.0.0.
- `schema_url` (optional): Specifies the Schema URL that should be recorded in
the emitted telemetry.
- `event_domain` (optional): Specifies the domain for the events created, which
- `event_domain` (optional): Specifies the domain for the Events emitted, which
should be added as `event.domain` attribute of the instrumentation scope.
- `include_trace_context` (optional): Specifies whether the Trace Context should
automatically be passed on to the events and logs created by the Logger. This
automatically be passed on to the Events and Logs emitted by the Logger. This
SHOULD be true by default.
- `attributes` (optional): Specifies the instrumentation scope attributes to
associate with emitted telemetry.
Expand All @@ -101,7 +103,7 @@ instance is a valid implementation. The only exception to this rule is the no-op
`Logger`: implementations MAY return the same instance regardless of parameter
values.

Implementations MUST NOT require users to repeatedly obtain an Logger again with
Implementations MUST NOT require users to repeatedly obtain a Logger again with
the same name+version+schema_url+event_domain+include_trace_context+attributes
to pick up configuration changes. This can be achieved either by allowing to
work with an outdated configuration or by ensuring that new configuration
Expand All @@ -121,7 +123,7 @@ the emitted data format is capable of representing such association.

## Logger

The `Logger` is responsible for creating Events and Logs.
The `Logger` is responsible for emitting Events and Logs.

Note that `Logger`s should not be responsible for configuration. This should be
the responsibility of the `LoggerProvider` instead.
Expand All @@ -130,16 +132,125 @@ the responsibility of the `LoggerProvider` instead.

The Logger MUST provide functions to:

- Create a `LogRecord`, representing an `Event` and emit it to the processing
pipeline.
- The API MUST accept an event name as a parameter. The event name provided
should be recorded as an attribute with key `event.name`. Care MUST be taken
by the implementation to not override or delete this attribute while the
`Event` is created to preserve its identity.
- Events require the `event.domain` attribute. The API MUST not allow creating
an event if the Logger instance doesn't have `event.domain` scope attribute.
- This function MAY be named `logEvent`.
- Create a `LogRecord` and emit it to the processing pipeline.
- This function MAY be named `logRecord`.
- This API is intended for use by Log Appenders, and SHOULD not be used by end
users or other instrumentation.
#### Emit Event
jack-berg marked this conversation as resolved.
Show resolved Hide resolved
jack-berg marked this conversation as resolved.
Show resolved Hide resolved

Emit a `LogRecord` representing an Event to the processing pipeline.

This function MAY be named `logEvent`.

**Parameters:**

* `name` - the Event name. This argument MUST be recorded as an attribute with
jack-berg marked this conversation as resolved.
Show resolved Hide resolved
the key `event.name`. Care MUST be taken by the implementation to not override
or delete this attribute while the Event is emitted to preserve its identity.
* `logRecord` - the [LogRecord](#logrecord) representing the Event.

Events require the `event.domain` attribute. The API MUST not allow creating an
Event if the Logger instance doesn't have `event.domain` scope attribute.
Comment on lines +151 to +152
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jack-berg @scheler

Events require the event.domain attribute. The API MUST not allow creating an
Event if the Logger instance doesn't have event.domain scope attribute.

Can you clarify for me what to do in the case of missing event.domain to the EmitEvent call? Is it a no-op or a throw? Do I generate a log to alert the user?

Also, is it valid if name is passed as null or empty?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@CodeBlanch both domain and name are required for an Event and also cannot be empty or null. So EmitEvent call should throw an error if event.domain attribute is missing or is null/empty. Same with name - if it is null or empty, EmitEvent should return failure or throw an error.


#### Emit LogRecord

Emit a `LogRecord` to the processing pipeline.

This function MAY be named `logRecord`.

This API is intended for use
by [Log Appenders](#how-to-create-log4j-style-appender), and SHOULD not be used
by end users or other instrumentation.

**Parameters:**

* `logRecord` - the [LogRecord](#logrecord).

## LogRecord

The API emits [Events](#emit-event) and [LogRecords](#emit-logrecord) using
the `LogRecord` [data model](data-model.md).

A function receiving this as an argument MUST be able to set the following
fields:

- [Timestamp](./data-model.md#field-timestamp)
- [Observed Timestamp](./data-model.md#field-observedtimestamp)
- [Trace Context](./data-model.md#trace-context-fields)
- [Severity Number](./data-model.md#field-severitynumber)
- [Severity Text](./data-model.md#field-severitytext)
- [Body](./data-model.md#field-body)
- [Attributes](./data-model.md#field-attributes)

## Usage

### How to Create Log4J Style Appender

An Appender implementation can be used to allow emitting `LogRecords` via
OpenTelemetry [LogRecordExporters](sdk.md#logrecordexporter). This approach is
typically used for applications which are fine with changing the log transport
and is [one of the supported](README.md#direct-to-collector) log collection
approaches.

The Appender implementation will typically acquire a [Logger](#logger) from the
global [LoggerProvider](#loggerprovider) at startup time, then
call [Emit LogRecord](#emit-logrecord) for `LogRecords` received from the
jack-berg marked this conversation as resolved.
Show resolved Hide resolved
application.

For languages with implicit Context, the Appender may call Context API to get
the currently [active Span](../trace/api.md#context-interaction) and populate
`TraceContext` fields of the `LogRecord` before emitting it. The log
library may also have an alternate way to inject the context into `LogRecords`
(e.g. MDC in Log4j).

TODO: update image to reflect current naming conventions
jack-berg marked this conversation as resolved.
Show resolved Hide resolved

![Appender](img/appender.png)

This same approach can be also used for example for:

- Python logging library by creating a Handler.
- Go zap logging library by implementing the Core interface. Note that since
there is no implicit Context in Go it is not possible to get and use the
active Span.

Appenders can be created in OpenTelemetry language libraries by OpenTelemetry
maintainers, or by 3rd parties for any logging library that supports a similar
extension mechanism. This specification recommends each OpenTelemetry language
library to include out-of-the-box Appender implementation for at least one
popular logging library.

### Implicit Context Injection

When Context is implicitly available (e.g. in Java) it may be fetched by the log
library extension synchronously for every `LogRecord` by calling the
OpenTelemetry Context API and injecting the span context fields into the
`LogRecord` before emitting it.
jack-berg marked this conversation as resolved.
Show resolved Hide resolved

Some log libraries have mechanisms specifically tailored for injecting
contextual information into log records. An example of such a mechanism is Log4j
MDC. When available such mechanisms may be the preferable place to fetch the
span context and inject it into the log records, since it usually allows
jack-berg marked this conversation as resolved.
Show resolved Hide resolved
fetching of the context to work correctly even when log records are emitted
asynchronously (which otherwise can result in the incorrect implicit context
being fetched.

TODO: clarify how works or doesn't work when the log statement call site and the
log appender are executed on different threads.

### Explicit Context

In languages where the Context must be provided explicitly (e.g. Go) the end
user must capture the context and explicitly pass it to the logging subsystem in
order for trace context to be recorded in Log records.

Support for OpenTelemetry for logging libraries in these languages typically can
be implemented in the form of logger wrappers that can capture the context once,
when the span is created and then use the wrapped logger to execute log
statements in a normal way. The wrapper will be responsible for injecting the
captured context in the `LogRecords`.

This specification does not define how exactly it is achieved since the actual
mechanism depends on the language and the particular logging library used. In
any case the wrappers are expected to make use of the Trace Context API to get
the current active span.

See
[an example](https://docs.google.com/document/d/15vR7D1x2tKd7u3zaTF0yH1WaHkUr2T4hhr7OyiZgmBg/edit#heading=h.4xuru5ljcups)
of how it can be done for zap logging library for Go.
Binary file not shown.
Binary file not shown.
Loading