Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add stdout log record exporter #6675

Merged
merged 47 commits into from
Sep 18, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
47 commits
Select commit Hold shift + click to select a range
aa57fff
add stdout log record exporter
zeitlinger Aug 27, 2024
3fb3c92
add stdout log record exporter
zeitlinger Aug 27, 2024
aeed8fc
simplify export
zeitlinger Aug 29, 2024
48634ea
rename OtlpJsonLoggingLogRecordExporterBuilder
zeitlinger Aug 29, 2024
75adbfa
no logger in InternalBuilder
zeitlinger Aug 29, 2024
61bff70
add EXPORTER_ARTIFACT_ID_BY_NAME
zeitlinger Aug 29, 2024
70087b8
remove InternalBuilder, OtlpStdoutLogRecordExporter delegates
zeitlinger Aug 29, 2024
daee6e6
we need two builders, because they produce different types
zeitlinger Aug 29, 2024
28628ec
return interface
zeitlinger Aug 29, 2024
77e6456
simplify
zeitlinger Aug 29, 2024
284e879
split test class
zeitlinger Aug 29, 2024
165e1fe
demo
jack-berg Aug 29, 2024
f56f794
allow to set logger
zeitlinger Aug 30, 2024
61ea461
remove create method
zeitlinger Aug 30, 2024
cf194c6
fix test
zeitlinger Aug 30, 2024
7dc1960
inline abstract class
zeitlinger Aug 30, 2024
95fa97f
inline abstract class
zeitlinger Aug 30, 2024
2ee23b9
get back old test
zeitlinger Aug 30, 2024
f2280c1
simplify tests
zeitlinger Aug 30, 2024
f2db40c
simplify tests
zeitlinger Aug 30, 2024
d52e8a9
simplify tests
zeitlinger Aug 30, 2024
335b3fc
Update buildSrc/src/main/kotlin/otel.java-conventions.gradle.kts
zeitlinger Sep 2, 2024
70eb011
add test for using file exporter
zeitlinger Sep 2, 2024
01fa923
add test for using file exporter
zeitlinger Sep 2, 2024
9eb0c45
test coverage
zeitlinger Sep 3, 2024
0726ce6
call close unless it's System.out or System.err
zeitlinger Sep 6, 2024
ee3ceb6
call close unless it's System.out or System.err
zeitlinger Sep 6, 2024
66b804d
rename setLogger to setOutput, don't set the logger for reporting errors
zeitlinger Sep 6, 2024
04c1d61
move provider to sdk-extensions/incubator and rename to otlp/stdout a…
zeitlinger Sep 6, 2024
0933b6e
move provider to sdk-extensions/incubator and rename to otlp/stdout a…
zeitlinger Sep 6, 2024
c3c6e86
move provider to sdk-extensions/incubator and rename to otlp/stdout a…
zeitlinger Sep 6, 2024
8e049b5
add tests
zeitlinger Sep 9, 2024
5e7af08
add tests
zeitlinger Sep 9, 2024
d1cfadf
Update buildSrc/src/main/kotlin/otel.java-conventions.gradle.kts
zeitlinger Sep 11, 2024
69b0014
Revert "move provider to sdk-extensions/incubator and rename to otlp/…
zeitlinger Sep 10, 2024
e97b117
Revert "move provider to sdk-extensions/incubator and rename to otlp/…
zeitlinger Sep 10, 2024
d99a97f
Revert "move provider to sdk-extensions/incubator and rename to otlp/…
zeitlinger Sep 10, 2024
ca75f71
change name to "experimental-otlp/stdout"
zeitlinger Sep 10, 2024
c09c634
close input stream on shutdown
zeitlinger Sep 11, 2024
77aec31
inline superclass
zeitlinger Sep 11, 2024
febd77c
add tests
zeitlinger Sep 11, 2024
0f88cb2
add tests
zeitlinger Sep 11, 2024
1baa575
add tests
zeitlinger Sep 11, 2024
382d311
Update exporters/logging-otlp/src/main/java/io/opentelemetry/exporter…
zeitlinger Sep 11, 2024
e65af38
Update exporters/logging-otlp/src/main/java/io/opentelemetry/exporter…
zeitlinger Sep 11, 2024
069db45
pr review
zeitlinger Sep 11, 2024
44de5ea
pr review
zeitlinger Sep 11, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -87,7 +87,6 @@ tasks {
"-Xlint:-processing",
// We suppress the "options" warning because it prevents compilation on modern JDKs
"-Xlint:-options",

// Fail build on any warning
"-Werror",
),
Expand Down
1 change: 1 addition & 0 deletions exporters/logging-otlp/build.gradle.kts
Original file line number Diff line number Diff line change
Expand Up @@ -20,5 +20,6 @@ dependencies {

testImplementation(project(":sdk:testing"))

testImplementation("com.google.guava:guava")
testImplementation("org.skyscreamer:jsonassert")
}
Original file line number Diff line number Diff line change
Expand Up @@ -5,18 +5,12 @@

package io.opentelemetry.exporter.logging.otlp;

import static io.opentelemetry.exporter.logging.otlp.JsonUtil.JSON_FACTORY;

import com.fasterxml.jackson.core.JsonGenerator;
import com.fasterxml.jackson.core.io.SegmentedStringWriter;
import io.opentelemetry.exporter.internal.otlp.logs.ResourceLogsMarshaler;
import io.opentelemetry.exporter.logging.otlp.internal.logs.OtlpStdoutLogRecordExporter;
import io.opentelemetry.exporter.logging.otlp.internal.logs.OtlpStdoutLogRecordExporterBuilder;
import io.opentelemetry.sdk.common.CompletableResultCode;
import io.opentelemetry.sdk.logs.data.LogRecordData;
import io.opentelemetry.sdk.logs.export.LogRecordExporter;
import java.io.IOException;
import java.util.Collection;
import java.util.concurrent.atomic.AtomicBoolean;
import java.util.logging.Level;
import java.util.logging.Logger;

/**
Expand All @@ -30,49 +24,31 @@
private static final Logger logger =
Logger.getLogger(OtlpJsonLoggingLogRecordExporter.class.getName());

private final AtomicBoolean isShutdown = new AtomicBoolean();
private final OtlpStdoutLogRecordExporter delegate;

/** Returns a new {@link OtlpJsonLoggingLogRecordExporter}. */
public static LogRecordExporter create() {
return new OtlpJsonLoggingLogRecordExporter();
OtlpStdoutLogRecordExporter delegate =
new OtlpStdoutLogRecordExporterBuilder(logger).setWrapperJsonObject(false).build();
return new OtlpJsonLoggingLogRecordExporter(delegate);
}

private OtlpJsonLoggingLogRecordExporter() {}
OtlpJsonLoggingLogRecordExporter(OtlpStdoutLogRecordExporter delegate) {
this.delegate = delegate;
}

@Override
public CompletableResultCode export(Collection<LogRecordData> logs) {
if (isShutdown.get()) {
return CompletableResultCode.ofFailure();
}

ResourceLogsMarshaler[] allResourceLogs = ResourceLogsMarshaler.create(logs);
for (ResourceLogsMarshaler resourceLogs : allResourceLogs) {
SegmentedStringWriter sw = new SegmentedStringWriter(JSON_FACTORY._getBufferRecycler());
try (JsonGenerator gen = JsonUtil.create(sw)) {
resourceLogs.writeJsonTo(gen);
} catch (IOException e) {
// Shouldn't happen in practice, just skip it.
continue;
}
try {
logger.log(Level.INFO, sw.getAndClear());
} catch (IOException e) {
logger.log(Level.WARNING, "Unable to read OTLP JSON log records", e);
}
}
return CompletableResultCode.ofSuccess();
return delegate.export(logs);
}
zeitlinger marked this conversation as resolved.
Show resolved Hide resolved

@Override
public CompletableResultCode flush() {
return CompletableResultCode.ofSuccess();
return delegate.flush();

Check warning on line 47 in exporters/logging-otlp/src/main/java/io/opentelemetry/exporter/logging/otlp/OtlpJsonLoggingLogRecordExporter.java

View check run for this annotation

Codecov / codecov/patch

exporters/logging-otlp/src/main/java/io/opentelemetry/exporter/logging/otlp/OtlpJsonLoggingLogRecordExporter.java#L47

Added line #L47 was not covered by tests
}

@Override
public CompletableResultCode shutdown() {
if (!isShutdown.compareAndSet(false, true)) {
logger.log(Level.INFO, "Calling shutdown() multiple times.");
}
return CompletableResultCode.ofSuccess();
return delegate.shutdown();
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -5,11 +5,12 @@

package io.opentelemetry.exporter.logging.otlp;

import static io.opentelemetry.exporter.logging.otlp.JsonUtil.JSON_FACTORY;
import static io.opentelemetry.exporter.logging.otlp.internal.writer.JsonUtil.JSON_FACTORY;

import com.fasterxml.jackson.core.JsonGenerator;
import com.fasterxml.jackson.core.io.SegmentedStringWriter;
import io.opentelemetry.exporter.internal.otlp.metrics.ResourceMetricsMarshaler;
import io.opentelemetry.exporter.logging.otlp.internal.writer.JsonUtil;
import io.opentelemetry.sdk.common.CompletableResultCode;
import io.opentelemetry.sdk.metrics.InstrumentType;
import io.opentelemetry.sdk.metrics.data.AggregationTemporality;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,9 +5,12 @@

package io.opentelemetry.exporter.logging.otlp;

import static io.opentelemetry.exporter.logging.otlp.internal.writer.JsonUtil.JSON_FACTORY;

import com.fasterxml.jackson.core.JsonGenerator;
import com.fasterxml.jackson.core.io.SegmentedStringWriter;
import io.opentelemetry.exporter.internal.otlp.traces.ResourceSpansMarshaler;
import io.opentelemetry.exporter.logging.otlp.internal.writer.JsonUtil;
import io.opentelemetry.sdk.common.CompletableResultCode;
import io.opentelemetry.sdk.trace.data.SpanData;
import io.opentelemetry.sdk.trace.export.SpanExporter;
Expand Down Expand Up @@ -43,8 +46,7 @@ public CompletableResultCode export(Collection<SpanData> spans) {

ResourceSpansMarshaler[] allResourceSpans = ResourceSpansMarshaler.create(spans);
for (ResourceSpansMarshaler resourceSpans : allResourceSpans) {
SegmentedStringWriter sw =
new SegmentedStringWriter(JsonUtil.JSON_FACTORY._getBufferRecycler());
SegmentedStringWriter sw = new SegmentedStringWriter(JSON_FACTORY._getBufferRecycler());
try (JsonGenerator gen = JsonUtil.create(sw)) {
resourceSpans.writeJsonTo(gen);
} catch (IOException e) {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
* SPDX-License-Identifier: Apache-2.0
*/

package io.opentelemetry.exporter.logging.otlp.internal;
package io.opentelemetry.exporter.logging.otlp.internal.logs;

import io.opentelemetry.exporter.logging.otlp.OtlpJsonLoggingLogRecordExporter;
import io.opentelemetry.sdk.autoconfigure.spi.ConfigProperties;
Expand All @@ -17,6 +17,7 @@
* at any time.
*/
public class LoggingLogRecordExporterProvider implements ConfigurableLogRecordExporterProvider {

@Override
public LogRecordExporter createExporter(ConfigProperties config) {
return OtlpJsonLoggingLogRecordExporter.create();
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,92 @@
/*
* Copyright The OpenTelemetry Authors
* SPDX-License-Identifier: Apache-2.0
*/

package io.opentelemetry.exporter.logging.otlp.internal.logs;

import io.opentelemetry.exporter.internal.otlp.logs.LogsRequestMarshaler;
import io.opentelemetry.exporter.internal.otlp.logs.ResourceLogsMarshaler;
import io.opentelemetry.exporter.logging.otlp.internal.writer.JsonWriter;
import io.opentelemetry.sdk.common.CompletableResultCode;
import io.opentelemetry.sdk.logs.data.LogRecordData;
import io.opentelemetry.sdk.logs.export.LogRecordExporter;
import java.util.Collection;
import java.util.StringJoiner;
import java.util.concurrent.atomic.AtomicBoolean;
import java.util.logging.Level;
import java.util.logging.Logger;

/**
* Exporter for sending OTLP log records to stdout.
*
* <p>This class is internal and is hence not for public use. Its APIs are unstable and can change
* at any time.
*/
public class OtlpStdoutLogRecordExporter implements LogRecordExporter {

private static final Logger LOGGER =
Logger.getLogger(OtlpStdoutLogRecordExporter.class.getName());

private final AtomicBoolean isShutdown = new AtomicBoolean();

private final Logger logger;
private final JsonWriter jsonWriter;
private final boolean wrapperJsonObject;

OtlpStdoutLogRecordExporter(Logger logger, JsonWriter jsonWriter, boolean wrapperJsonObject) {
this.logger = logger;
this.jsonWriter = jsonWriter;
this.wrapperJsonObject = wrapperJsonObject;
}

/** Returns a new {@link OtlpStdoutLogRecordExporterBuilder}. */
@SuppressWarnings("SystemOut")
public static OtlpStdoutLogRecordExporterBuilder builder() {
return new OtlpStdoutLogRecordExporterBuilder(LOGGER).setOutput(System.out);
}
zeitlinger marked this conversation as resolved.
Show resolved Hide resolved

@Override
public CompletableResultCode export(Collection<LogRecordData> logs) {
if (isShutdown.get()) {
return CompletableResultCode.ofFailure();
}

if (wrapperJsonObject) {
LogsRequestMarshaler request = LogsRequestMarshaler.create(logs);
return jsonWriter.write(request);
} else {
for (ResourceLogsMarshaler resourceLogs : ResourceLogsMarshaler.create(logs)) {
CompletableResultCode resultCode = jsonWriter.write(resourceLogs);
if (!resultCode.isSuccess()) {
// already logged
return resultCode;

Check warning on line 63 in exporters/logging-otlp/src/main/java/io/opentelemetry/exporter/logging/otlp/internal/logs/OtlpStdoutLogRecordExporter.java

View check run for this annotation

Codecov / codecov/patch

exporters/logging-otlp/src/main/java/io/opentelemetry/exporter/logging/otlp/internal/logs/OtlpStdoutLogRecordExporter.java#L63

Added line #L63 was not covered by tests
}
}
return CompletableResultCode.ofSuccess();
}
}

@Override
public CompletableResultCode flush() {
return jsonWriter.flush();
}

@Override
public CompletableResultCode shutdown() {
if (!isShutdown.compareAndSet(false, true)) {
logger.log(Level.INFO, "Calling shutdown() multiple times.");
} else {
jsonWriter.close();
}
return CompletableResultCode.ofSuccess();
}

@Override
public String toString() {
StringJoiner joiner = new StringJoiner(", ", "OtlpStdoutLogRecordExporter{", "}");
joiner.add("jsonWriter=" + jsonWriter);
joiner.add("wrapperJsonObject=" + wrapperJsonObject);
return joiner.toString();
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,76 @@
/*
* Copyright The OpenTelemetry Authors
* SPDX-License-Identifier: Apache-2.0
*/

package io.opentelemetry.exporter.logging.otlp.internal.logs;

import static java.util.Objects.requireNonNull;

import io.opentelemetry.exporter.logging.otlp.OtlpJsonLoggingLogRecordExporter;
import io.opentelemetry.exporter.logging.otlp.internal.writer.JsonWriter;
import io.opentelemetry.exporter.logging.otlp.internal.writer.LoggerJsonWriter;
import io.opentelemetry.exporter.logging.otlp.internal.writer.StreamJsonWriter;
import java.io.OutputStream;
import java.util.logging.Logger;

/**
* Builder for {@link OtlpJsonLoggingLogRecordExporter}.
*
* <p>This class is internal and is hence not for public use. Its APIs are unstable and can change
* at any time.
*/
public final class OtlpStdoutLogRecordExporterBuilder {

private static final String TYPE = "log records";

private final Logger logger;
private JsonWriter jsonWriter;
private boolean wrapperJsonObject = true;

public OtlpStdoutLogRecordExporterBuilder(Logger logger) {
this.logger = logger;
this.jsonWriter = new LoggerJsonWriter(logger, TYPE);
}

/**
* Sets the exporter to use the specified JSON object wrapper.
*
* @param wrapperJsonObject whether to wrap the JSON object in an outer JSON "resourceLogs"
* object.
*/
public OtlpStdoutLogRecordExporterBuilder setWrapperJsonObject(boolean wrapperJsonObject) {
this.wrapperJsonObject = wrapperJsonObject;
return this;
}

/**
* Sets the exporter to use the specified output stream.
*
* <p>The output stream will be closed when {@link OtlpStdoutLogRecordExporter#shutdown()} is
* called unless it's {@link System#out} or {@link System#err}.
*
* @param outputStream the output stream to use.
*/
public OtlpStdoutLogRecordExporterBuilder setOutput(OutputStream outputStream) {
requireNonNull(outputStream, "outputStream");
this.jsonWriter = new StreamJsonWriter(outputStream, TYPE);
return this;
}

/** Sets the exporter to use the specified logger. */
public OtlpStdoutLogRecordExporterBuilder setOutput(Logger logger) {
requireNonNull(logger, "logger");
this.jsonWriter = new LoggerJsonWriter(logger, TYPE);
jack-berg marked this conversation as resolved.
Show resolved Hide resolved
return this;
}

/**
* Constructs a new instance of the exporter based on the builder's values.
*
* @return a new exporter's instance
*/
public OtlpStdoutLogRecordExporter build() {
return new OtlpStdoutLogRecordExporter(logger, jsonWriter, wrapperJsonObject);
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
/*
* Copyright The OpenTelemetry Authors
* SPDX-License-Identifier: Apache-2.0
*/

package io.opentelemetry.exporter.logging.otlp.internal.logs;

import io.opentelemetry.sdk.autoconfigure.spi.internal.ComponentProvider;
import io.opentelemetry.sdk.autoconfigure.spi.internal.StructuredConfigProperties;
import io.opentelemetry.sdk.logs.export.LogRecordExporter;

/**
* File configuration SPI implementation for {@link OtlpStdoutLogRecordExporter}.
*
* <p>This class is internal and is hence not for public use. Its APIs are unstable and can change
* at any time.
*/
public class OtlpStdoutLogRecordExporterComponentProvider
implements ComponentProvider<LogRecordExporter> {

@Override
public Class<LogRecordExporter> getType() {
return LogRecordExporter.class;
}

@Override
public String getName() {
return "experimental-otlp/stdout";
}

@Override
public LogRecordExporter create(StructuredConfigProperties config) {
OtlpStdoutLogRecordExporterBuilder builder = OtlpStdoutLogRecordExporter.builder();
return builder.build();
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
/*
* Copyright The OpenTelemetry Authors
* SPDX-License-Identifier: Apache-2.0
*/

package io.opentelemetry.exporter.logging.otlp.internal.logs;

import io.opentelemetry.sdk.autoconfigure.spi.ConfigProperties;
import io.opentelemetry.sdk.autoconfigure.spi.logs.ConfigurableLogRecordExporterProvider;
import io.opentelemetry.sdk.logs.export.LogRecordExporter;

/**
* {@link LogRecordExporter} SPI implementation for {@link OtlpStdoutLogRecordExporter}.
*
* <p>This class is internal and is hence not for public use. Its APIs are unstable and can change
* at any time.
*/
public class OtlpStdoutLogRecordExporterProvider implements ConfigurableLogRecordExporterProvider {
@Override
public LogRecordExporter createExporter(ConfigProperties config) {
OtlpStdoutLogRecordExporterBuilder builder = OtlpStdoutLogRecordExporter.builder();
return builder.build();
}

@Override
public String getName() {
return "experimental-otlp/stdout";
}
}
Loading
Loading