Skip to content

Commit

Permalink
Align log sdk naming with api (open-telemetry#2768)
Browse files Browse the repository at this point in the history
Resolves open-telemetry#2752.

This aligns log SDK and API concepts which have diverged after the merged of open-telemetry#2676. This PR brings alignment to the log API and SDK, and in brings the log signal into alignment with tracing and metrics where there is conceptual overlap. There shouldn't be any new concepts introduced here.

- Rename `../logs/logging-library-sdk.md` to `../logs/sdk.md`
- Remove wording from SDK that implies that an API doesn't exist, like [this](https://github.com/open-telemetry/opentelemetry-specification/blame/main/specification/logs/logging-library-sdk.md#L60-L62).
- Move [How to Create Log4j Style Appender](https://github.com/open-telemetry/opentelemetry-specification/blame/main/specification/logs/logging-library-sdk.md#L219) to `api.md` since it describes an API use case.
- Move [Implicit / Explicit Context Injection](https://github.com/open-telemetry/opentelemetry-specification/blame/main/specification/logs/logging-library-sdk.md#L270-L288) sections to `api.md` since they describe API level considerations.
- Rename Logger [create](https://github.com/open-telemetry/opentelemetry-specification/blame/main/specification/logs/api.md#L133) method to be emit, to align with SDK concept of `LogRecordProcessor#onEmit(..)`.
- Rename `LogProcessor`, `LogExporter` to `LogRecordProcessor`, `LogRecordExporter`. 
- Fill in various SDK level TODOs related to shutdown and flushing. The language from these was taken directly from the metrics / tracing SDK - no new concepts were introduced.
  • Loading branch information
jack-berg authored Sep 13, 2022
1 parent 0292d0f commit 5df7710
Show file tree
Hide file tree
Showing 12 changed files with 533 additions and 373 deletions.
4 changes: 3 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,9 @@ release.
- Update log SDK to allow log processors to mutate log records
([#2681](https://github.com/open-telemetry/opentelemetry-specification/pull/2681)).
- Add experimental Events and Logs API specification
([#2676](https://github.com/open-telemetry/opentelemetry-specification/pull/2676))
([#2676](https://github.com/open-telemetry/opentelemetry-specification/pull/2676)).
- Align log SDK and API component naming
([#2768](https://github.com/open-telemetry/opentelemetry-specification/pull/2768)).

### Resource

Expand Down
34 changes: 17 additions & 17 deletions spec-compliance-matrix.md
Original file line number Diff line number Diff line change
Expand Up @@ -210,23 +210,23 @@ formats is required. Implementing more than one format is optional.

Disclaimer: this list of features is still a work in progress, please refer to the specification if in any doubt.

| Feature | Optional | Go | Java | JS | Python | Ruby | Erlang | PHP | Rust | C++ | .NET | Swift |
|--------------------------------------------------------------|----------|-----|------|-----|--------|------|--------|-----|------|-----|------|-------|
| **[Logging SDK](specification/logs/logging-library-sdk.md)** | Optional | Go | Java | JS | Python | Ruby | Erlang | PHP | Rust | C++ | .NET | Swift |
| LogEmitterProvider.Get LogEmitter | | | + | | + | | | | | | - | |
| LogEmitterProvider.Get LogEmitter accepts attributes | | | | | | | | | | | | |
| LogEmitterProvider.Shutdown | | | + | | + | | | | | | - | |
| LogEmitterProvider.ForceFlush | | | + | | + | | | | | | - | |
| LogEmitter.Emit(LogRecord) | | | + | | + | | | | | | - | |
| SimpleLogProcessor | | | + | | + | | | | | | + | |
| BatchLogProcessor | | | + | | + | | | | | | + | |
| Can plug custom log processor | | | + | | + | | | | | | + | |
| OTLP/gRPC exporter | | | + | | + | | | | | | + | |
| OTLP/HTTP exporter | | | + | | + | | | | | | + | |
| OTLP File exporter | | | - | | - | | | | | | - | |
| Can plug custom log exporter | | | + | | + | | | | | | + | |
| Implicit Context Injection | | | - | | + | | | | | | + | |
| Explicit Context | | | + | | - | | | | | | - | |
| Feature | Optional | Go | Java | JS | Python | Ruby | Erlang | PHP | Rust | C++ | .NET | Swift |
|----------------------------------------------|----------|-----|------|-----|--------|------|--------|-----|------|-----|------|-------|
| **[Logging SDK](specification/logs/sdk.md)** | Optional | Go | Java | JS | Python | Ruby | Erlang | PHP | Rust | C++ | .NET | Swift |
| LoggerProvider.Get Logger | | | | | | | | | | | - | |
| LoggerProvider.Get Logger accepts attributes | | | | | | | | | | | | |
| LoggerProvider.Shutdown | | | | | | | | | | | - | |
| LoggerProvider.ForceFlush | | | | | | | | | | | - | |
| Logger.Emit(LogRecord) | | | | | | | | | | | - | |
| SimpleLogRecordProcessor | | | | | | | | | | | | |
| BatchLogRecordProcessor | | | | | | | | | | | | |
| Can plug custom LogRecordProcessor | | | | | | | | | | | | |
| OTLP/gRPC exporter | | | + | | + | | | | | | + | |
| OTLP/HTTP exporter | | | + | | + | | | | | | + | |
| OTLP File exporter | | | - | | - | | | | | | - | |
| Can plug custom LogRecordExporter | | | | | | | | | | | | |
| Implicit Context Injection | | | - | | + | | | | | | + | |
| Explicit Context | | | + | | - | | | | | | - | |

## Resource

Expand Down
2 changes: 1 addition & 1 deletion specification/common/attribute-type-mapping.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ The mapping is needed when OpenTelemetry needs to convert a value produced outsi
OpenTelemetry into a value that can be exported using an OTLP exporter, or otherwise be
converted to be used inside OpenTelemetry boundaries. Example use cases are the following:

- In [Logging Library SDK](../logs/logging-library-sdk.md)s, to convert values received
- In the [Logging SDK](../logs/sdk.md)s, to convert values received
from logging libraries into OpenTelemetry representation.
- In the Collector, to convert values received from various data sources into
[pdata](https://github.com/open-telemetry/opentelemetry-collector/blob/4998703dadd19fa91a88eabf7ccc68d728bee3fd/model/pdata/common.go#L84)
Expand Down
162 changes: 135 additions & 27 deletions specification/logs/api.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,12 +9,18 @@

<!-- toc -->

- [Data Model](#data-model)
- [LoggerProvider](#loggerprovider)
* [LoggerProvider operations](#loggerprovider-operations)
+ [Get a Logger](#get-a-logger)
- [Logger](#logger)
* [Logger operations](#logger-operations)
+ [Emit Event](#emit-event)
+ [Emit LogRecord](#emit-logrecord)
- [LogRecord](#logrecord)
- [Usage](#usage)
* [How to Create Log4J Style Appender](#how-to-create-log4j-style-appender)
* [Implicit Context Injection](#implicit-context-injection)
* [Explicit Context Injection](#explicit-context-injection)

<!-- tocstop -->

Expand All @@ -23,7 +29,9 @@
The Events and Logs API consist of these main classes:

* LoggerProvider is the entry point of the API. It provides access to Loggers.
* Logger is the class responsible for creating events and logs using Log records.
* Logger is the class responsible for
creating [Events](./semantic_conventions/events.md)
and [Logs](./data-model.md#log-and-event-record-definition) as LogRecords.

LoggerProvider/Logger are analogous to TracerProvider/Tracer.

Expand All @@ -34,10 +42,6 @@ graph TD
B --> D(Log)
```

## Data Model

The API creates Events and Logs using the `LogRecord` data model. See `LogRecord` [data model](data-model.md) for the list of fields.

## LoggerProvider

`Logger`s can be accessed with a `LoggerProvider`.
Expand All @@ -51,10 +55,10 @@ LoggerProvider.

Notwithstanding any global LoggerProvider, some applications may want to or have
to use multiple LoggerProvider instances, e.g. to have different configuration
(like LogProcessors) for each (and consequently for the Loggers obtained
from them), or because it's easier with dependency injection frameworks. Thus,
implementations of LoggerProvider SHOULD allow creating an arbitrary number of
instances.
(like [LogRecordProcessors](sdk.md#logrecordprocessor)) for each (and
consequently for the Loggers obtained from them), or because it's easier with
dependency injection frameworks. Thus, implementations of LoggerProvider SHOULD
allow creating an arbitrary number of instances.

### LoggerProvider operations

Expand Down Expand Up @@ -87,10 +91,10 @@ produced by this library.
the scope has a version (e.g. a library version). Example value: 1.0.0.
- `schema_url` (optional): Specifies the Schema URL that should be recorded in
the emitted telemetry.
- `event_domain` (optional): Specifies the domain for the events created, which
- `event_domain` (optional): Specifies the domain for the Events emitted, which
should be added as `event.domain` attribute of the instrumentation scope.
- `include_trace_context` (optional): Specifies whether the Trace Context should
automatically be passed on to the events and logs created by the Logger. This
automatically be passed on to the Events and Logs emitted by the Logger. This
SHOULD be true by default.
- `attributes` (optional): Specifies the instrumentation scope attributes to
associate with emitted telemetry.
Expand All @@ -101,7 +105,7 @@ instance is a valid implementation. The only exception to this rule is the no-op
`Logger`: implementations MAY return the same instance regardless of parameter
values.

Implementations MUST NOT require users to repeatedly obtain an Logger again with
Implementations MUST NOT require users to repeatedly obtain a Logger again with
the same name+version+schema_url+event_domain+include_trace_context+attributes
to pick up configuration changes. This can be achieved either by allowing to
work with an outdated configuration or by ensuring that new configuration
Expand All @@ -121,7 +125,7 @@ the emitted data format is capable of representing such association.

## Logger

The `Logger` is responsible for creating Events and Logs.
The `Logger` is responsible for emitting Events and Logs.

Note that `Logger`s should not be responsible for configuration. This should be
the responsibility of the `LoggerProvider` instead.
Expand All @@ -130,16 +134,120 @@ the responsibility of the `LoggerProvider` instead.

The Logger MUST provide functions to:

- Create a `LogRecord`, representing an `Event` and emit it to the processing
pipeline.
- The API MUST accept an event name as a parameter. The event name provided
should be recorded as an attribute with key `event.name`. Care MUST be taken
by the implementation to not override or delete this attribute while the
`Event` is created to preserve its identity.
- Events require the `event.domain` attribute. The API MUST not allow creating
an event if the Logger instance doesn't have `event.domain` scope attribute.
- This function MAY be named `logEvent`.
- Create a `LogRecord` and emit it to the processing pipeline.
- This function MAY be named `logRecord`.
- This API is intended for use by Log Appenders, and SHOULD not be used by end
users or other instrumentation.
#### Emit Event

Emit a `LogRecord` representing an Event to the processing pipeline.

This function MAY be named `logEvent`.

**Parameters:**

* `name` - the Event name. This argument MUST be recorded as a `LogRecord`
attribute with the key `event.name`. Care MUST be taken by the implementation
to not override or delete this attribute while the Event is emitted to
preserve its identity.
* `logRecord` - the [LogRecord](#logrecord) representing the Event.

Events require the `event.domain` attribute. The API MUST not allow creating an
Event if the Logger instance doesn't have `event.domain` scope attribute.

#### Emit LogRecord

Emit a `LogRecord` to the processing pipeline.

This function MAY be named `logRecord`.

This API is intended for use
by [Log Appenders](#how-to-create-log4j-style-appender), and SHOULD not be used
by end users or other instrumentation.

**Parameters:**

* `logRecord` - the [LogRecord](#logrecord).

## LogRecord

The API emits [Events](#emit-event) and [LogRecords](#emit-logrecord) using
the `LogRecord` [data model](data-model.md).

A function receiving this as an argument MUST be able to set the following
fields:

- [Timestamp](./data-model.md#field-timestamp)
- [Observed Timestamp](./data-model.md#field-observedtimestamp)
- [Trace Context](./data-model.md#trace-context-fields)
- [Severity Number](./data-model.md#field-severitynumber)
- [Severity Text](./data-model.md#field-severitytext)
- [Body](./data-model.md#field-body)
- [Attributes](./data-model.md#field-attributes)

## Usage

### How to Create Log4J Style Appender

An Appender implementation can be used to allow emitting logs via
OpenTelemetry [LogRecordExporters](sdk.md#logrecordexporter). This approach is
typically used for applications which are fine with changing the log transport
and is [one of the supported](README.md#direct-to-collector) log collection
approaches.

The Appender implementation will typically acquire a [Logger](#logger) from the
global [LoggerProvider](#loggerprovider) at startup time, then
call [Emit LogRecord](#emit-logrecord) for `LogRecords` received from the
application.

[Implicit Context Injection](#implicit-context-injection)
and [Explicit Context Injection](#explicit-context-injection) describe how an
Appender injects `TraceContext` into `LogRecords`.

![Appender](img/appender.png)

This same approach can be also used for example for:

- Python logging library by creating a Handler.
- Go zap logging library by implementing the Core interface. Note that since
there is no implicit Context in Go it is not possible to get and use the
active Span.

Appenders can be created in OpenTelemetry language libraries by OpenTelemetry
maintainers, or by 3rd parties for any logging library that supports a similar
extension mechanism. This specification recommends each OpenTelemetry language
library to include out-of-the-box Appender implementation for at least one
popular logging library.

### Implicit Context Injection

When Context is implicitly available (e.g. in Java) the log library extension
can rely on automatic context propagation
by [obtaining a Logger](#get-a-logger) with `include_trace_context=true`.

Some log libraries have mechanism specifically tailored for injecting contextual
information into logs, such as MDC in Log4j. When available such mechanisms may
be the preferable place to fetch the `TraceContext` and inject it into
the `LogRecord`, since it usually allows fetching of the context to work
correctly even when log records are emitted asynchronously which otherwise can
result in the incorrect implicit context being fetched.

TODO: clarify how works or doesn't work when the log statement call site and the
log appender are executed on different threads.

### Explicit Context Injection

In languages where the Context must be provided explicitly (e.g. Go) the end
user must capture the context and explicitly pass it to the logging subsystem in
order for `TraceContext` to be recorded in `LogRecords`

Support for OpenTelemetry for logging libraries in these languages typically can
be implemented in the form of logger wrappers that can capture the context once,
when the span is created and then use the wrapped logger to execute log
statements in a normal way. The wrapper will be responsible for injecting the
captured context in the `LogRecords`.

This specification does not define how exactly it is achieved since the actual
mechanism depends on the language and the particular logging library used. In
any case the wrappers are expected to make use of the Trace Context API to get
the current active span.

See
[an example](https://docs.google.com/document/d/15vR7D1x2tKd7u3zaTF0yH1WaHkUr2T4hhr7OyiZgmBg/edit#heading=h.4xuru5ljcups)
of how it can be done for zap logging library for Go.
Binary file added specification/logs/img/appender.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file added specification/logs/img/otlp-file.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading

0 comments on commit 5df7710

Please sign in to comment.