From b1f26d26d5148144144f21cf97a56009a0f136bb Mon Sep 17 00:00:00 2001 From: jack-berg <34418638+jack-berg@users.noreply.github.com> Date: Sat, 22 Apr 2023 01:06:06 -0500 Subject: [PATCH] Add java documentation for log signal (#2583) Co-authored-by: Phillip Carter --- .../en/docs/instrumentation/java/manual.md | 167 +++++++++++++++++- 1 file changed, 166 insertions(+), 1 deletion(-) diff --git a/content/en/docs/instrumentation/java/manual.md b/content/en/docs/instrumentation/java/manual.md index 80d990d4f0c0..9a598af50931 100644 --- a/content/en/docs/instrumentation/java/manual.md +++ b/content/en/docs/instrumentation/java/manual.md @@ -71,6 +71,8 @@ telemetry source-identifying info. ``` +See [releases][releases] for a full list of artifact coordinates. + ### Gradle ```kotlin @@ -82,6 +84,8 @@ dependencies { } ``` +See [releases][releases] for a full list of artifact coordinates. + ### Imports ```java @@ -116,6 +120,8 @@ SdkMeterProvider sdkMeterProvider = SdkMeterProvider.builder() .setResource(resource) .build(); +// TODO: add log configuration when stable + OpenTelemetry openTelemetry = OpenTelemetrySdk.builder() .setTracerProvider(sdkTracerProvider) .setMeterProvider(sdkMeterProvider) @@ -616,6 +622,90 @@ meter }); ``` +## Logs + +Logs are distinct from Metrics and Tracing in that there is no user-facing logs +API. Instead, there is tooling to bridge logs from existing popular log +frameworks (e.g. SLF4j, JUL, Logback, Log4j) into the OpenTelemetry ecosystem. + +The two typical workflows discussed below each cater to different application +requirements. + +### Direct to collector + +In the direct to collector workflow, logs are emitted directly from an +application to a collector using a network protocol (e.g. OTLP). This workflow +is simple to set up as it doesn't require any additional log forwarding +components, and allows an application to easily emit structured logs that +conform to the [log data model][log data model]. However, the overhead required +for applications to queue and export logs to a network location may not be +suitable for all applications. + +To use this workflow: + +- Install appropriate [Log Appender](#log-appenders). +- Configure the OpenTelemetry [Log SDK](#logs-sdk) to export log records to + desired target destination (the [collector][opentelemetry collector] or + other). + +#### Log appenders + +A log appender bridges logs from a log framework into the OpenTelemetry +[Log SDK](#logs-sdk) using the [Logs Bridge API][logs bridge API]. Log appenders +are available for various popular java log frameworks: + +- [Log4j2 Appender][log4j2 appender] +- [Logback Appender][logback appender] + +The links above contain full usage and installation documentation, but +installation is generally as follows: + +- Add required dependency via gradle or maven. +- Extend the application's log configuration (i.e. `logback.xml`, `log4j.xml`, + etc) to include a reference to the OpenTelemetry log appender. + - Optionally configure the log framework to determine which logs (i.e. filter + by severity or logger name) are passed to the appender. + - Optionally configure the appender to indicate how logs are mapped to + OpenTelemetry Log Records (i.e. capture thread information, context data, + markers, etc). + +Log appenders automatically include the trace context in log records, enabling +log correlation with traces. + +The [Log Appender example][log appender example] demonstrates setup for a +variety of scenarios. + +### Via file or stdout + +In the file or stdout workflow, logs are written to files or standout output. +Another component (e.g. FluentBit) is responsible for reading / tailing the +logs, parsing them to more structured format, and forwarding them a target, such +as the collector. This workflow may be preferable in situations where +application requirements do not permit additional overhead from +[direct to collector](#direct-to-collector). However, it requires that all log +fields required down stream are encoded into the logs, and that the component +reading the logs parse the data into the [log data model][log data model]. The +installation and configuration of log forwarding components is outside the scope +of this document. + +Log correlation with traces is available by installing +[log context instrumentation](#log-context-instrumentation). + +#### Log context instrumentation + +OpenTelemetry provides components which enrich log context with trace context +for various popular java log frameworks: + +- [Log4j context data instrumentation][log4j context instrumentation] +- [Logback MDC instrumentation][logback context instrumentation] + +This links above contain full usage and installation documentation, but +installation is generally as follows: + +- Add required dependency via gradle or maven. +- Extend the application's log configuration (i.e. `logback.xml` or `log4j.xml`, + etc) to reference the trace context fields in the log pattern. + ## SDK Configuration The configuration examples reported in this document only apply to the SDK @@ -798,6 +888,68 @@ view is replaced by the registered view. Additional registered views that match the instrument are additive, and result in multiple exported metrics for the instrument. +### Logs SDK + +The logs SDK dictates how logs are processed when using the +[direct to collector](#direct-to-collector) workflow. No log SDK is needed when +using the [log forwarding](#via-file-or-stdout) workflow. + +The typical log SDK configuration installs a log record processor and exporter. +For example, the following installs the +[BatchLogRecordProcessor](#logrecord-processor), which periodically exports to a +network location via the [OtlpGrpcLogRecordExporter](#logrecord-exporter): + +```java +SdkLoggerProvider loggerProvider = SdkLoggerProvider.builder() + .addLogRecordProcessor( + BatchLogRecordProcessor.builder( + OtlpGrpcLogRecordExporter.builder() + .setEndpoint("http://localhost:4317") + .build()) + .build()) + .build(); +``` + +See [releases][releases] for log specific artifact coordinates. + +#### LogRecord Processor + +LogRecord processors process LogRecords emitted by +[log appenders](#log-appenders). + +OpenTelemetry provides the following LogRecord processors out of the box: + +- `BatchLogRecordProcessor`: periodically sends batches of LogRecords to a + [LogRecordExporter](#logrecord-exporter). +- `SimpleLogRecordProcessor`: immediately sends each LogRecord to a + [LogRecordExporter](#logrecord-exporter). + +Custom LogRecord processors are supported by implementing the +`LogRecordProcessor` interface. Common use cases include enriching the +LogRecords with contextual data like baggage, or filtering / obfuscating +sensitive data. + +#### LogRecord Exporter + +`BatchLogRecordProcessor` and `SimpleLogRecordProcessor` are paired with +`LogRecordExporter`, which is responsible for sending telemetry data to a +particular backend. OpenTelemetry provides the following exporters out of the +box: + +- OpenTelemetry Protocol Exporter: sends the data in OTLP to the [OpenTelemetry + Collector] or other OTLP receivers. Varieties include + `OtlpGrpcLogRecordExporter` and `OtlpHttpLogRecordExporter`. +- `InMemoryLogRecordExporter`: keeps the data in memory, useful for testing and + debugging. +- Logging Exporter: saves the telemetry data into log streams. Varieties include + `SystemOutLogRecordExporter` and `OtlpJsonLoggingLogRecordExporter`. Note: + `OtlpJsonLoggingLogRecordExporter` logs to JUL, and may cause infinite loops + (i.e. JUL -> SLF4J -> Logback -> OpenTelemetry Appender -> OpenTelemetry Log + SDK -> JUL) if not carefully configured. + +Custom exporters are supported by implementing the `LogRecordExporter` +interface. + ### Auto Configuration Instead of manually creating the `OpenTelemetry` instance by using the SDK @@ -863,7 +1015,7 @@ AutoConfiguredOpenTelemetrySdk.builder() .build(); ``` -## Logging and Error Handling +## SDK Logging and Error Handling OpenTelemetry uses [java.util.logging](https://docs.oracle.com/javase/7/docs/api/java/util/logging/package-summary.html) @@ -930,12 +1082,25 @@ io.opentelemetry.sdk.trace.export.BatchSpanProcessor = io.opentelemetry.extensio [instrumented library]: /docs/reference/specification/glossary/#instrumented-library [library guidelines]: /docs/reference/specification/library-guidelines +[logs bridge API]: /docs/reference/specification/logs/bridge-api +[log data model]: /docs/reference/specification/logs/data-model +[log4j2 appender]: + https://github.com/open-telemetry/opentelemetry-java-instrumentation/tree/main/instrumentation/log4j/log4j-appender-2.17/library +[logback appender]: + https://github.com/open-telemetry/opentelemetry-java-instrumentation/tree/main/instrumentation/logback/logback-appender-1.0/library +[log appender example]: + https://github.com/open-telemetry/opentelemetry-java-docs/tree/main/log-appender +[log4j context instrumentation]: + https://github.com/open-telemetry/opentelemetry-java-instrumentation/tree/main/instrumentation/log4j/log4j-context-data/log4j-context-data-2.17/library-autoconfigure +[logback context instrumentation]: + https://github.com/open-telemetry/opentelemetry-java-instrumentation/tree/main/instrumentation/logback/logback-mdc-1.0/library [obtaining a tracer]: /docs/reference/specification/trace/api/#get-a-tracer [opentelemetry collector]: https://github.com/open-telemetry/opentelemetry-collector [opentelemetry registry]: /ecosystem/registry/?component=exporter&language=java [parentbased]: https://github.com/open-telemetry/opentelemetry-java/blob/main/sdk/trace/src/main/java/io/opentelemetry/sdk/trace/samplers/ParentBasedSampler.java +[releases]: https://github.com/open-telemetry/opentelemetry-java#releases [semantic conventions]: /docs/reference/specification/trace/semantic_conventions [traceidratiobased]: https://github.com/open-telemetry/opentelemetry-java/blob/main/sdk/trace/src/main/java/io/opentelemetry/sdk/trace/samplers/TraceIdRatioBasedSampler.java