Consider adding a logs API or allowing languages to present a logs API within the OpenTelemetry API

[anuraaga]

In Java, we have a technical requirement to have a separate logs interface than the SDK. In summary, it is because the Javaagent manages the OpenTelemetry SDK and actively prevents the user from using it - this is to minimize chances of dependency conflicts between user code and the SDK and reduce foot-shooting. Please consider this a hard requirement.

We currently have implemented a logs interface in the instrumentation repo, but are finding it awkward.

open-telemetry/opentelemetry-java-instrumentation#4935

If logging functionality is 100% just an implementation detail of instrumentation of logging libraries, this is not too bad, but our understanding is users may use logging functionality, notably under the umbrella term "events"

#2074 (comment)

A primary point of awkwardness is the entry point - because the javaagent manages the OpenTelemetry SDK behind the scenes, it provides a GlobalOpenTelemetry entry point into the OpenTelemetry API. For example, they will need to know a separate entry point, GlobalLogEmitterProvider to get into logs - this is not very user friendly.

It is still possible and sort of OK to have this split in entry points, but I am wondering if there is a good reason. My understanding is one reason there hasn't been a logs API defined is to avoid stepping on the toes of established / well-used logging libraries, in Java SLF4J. But this doesn't seem to be consistent given the same consideration wasn't given to Micrometer for metrics, for example.

I do see in the OTEP that a logging API may be added in the future

https://github.com/open-telemetry/oteps/blob/main/text/logs/0150-logging-library-sdk.md#specification

Because Java has a technical requirement for one now, I am wondering how to proceed - one approach is for us to make sure all logging-related APIs are internal (we use the .internal. package naming) until a logs API can be defined in the spec and ensure it is only used by official instrumentation we maintain. But perhaps others also feel a similar need for a logs API in the short term, not necessarily for Java's technical reasons. For example, it does seem to be the way to make sure the spec and user experience of logs is completely consistent with the other signals. Interested in hearing how people feel after implementing some logs functionality.

[jack-berg]

My understanding is that the intention is for users to use the log API of whatever framework is popular / idiomatic in their language as a stand in for a OpenTelemetry Log API. And that log adapter will be written to bridge between the log framework APIs and the OpenTelemetry Log SDK.

This works for logs, but the APIs of popular log frameworks cater to log use cases much more than events, which are more structured in nature. If you want to instrument an application to specifically capture opentelemetry events (i.e. logs), its awkward and inconsistent to do so using existing log frameworks. I wanted to do this for some projects and ended up calling the prototype opentelemetry java log SDK directly instead of going through a log framework.

[tigrannajaryan]

I don't know much about Javaagent to have an opinion on how to solve this problem in Java, but I would prefer to postpone for as long as possible the introduction of OpenTelemetry Logging API that is accessible to end users.

The design of a logging API requires a lot of thought and consideration and I do not want it to become a focal point for logging SIG right now, otherwise it will distract from a critical topic that blocks many other things: declaring the log data model stable (on which the OTLP stability depends and on which pdata stability in the Collector depends and all Collector components that work with logs depend).

Because Java has a technical requirement for one now, I am wondering how to proceed - one approach is for us to make sure all logging-related APIs are internal (we use the .internal. package naming) until a logs API can be defined in the spec and ensure it is only used by official instrumentation we maintain.

If it is very important for Java then I think it is a good approach to introduce an internal API that is not exposed to the end users. This may be a good playground for understanding what we expect from a logging API and can become the foundation of the logging API OTEP in the future.

[yurishkuro]

I agree with @tigrannajaryan and I also don't quite understand the motivation of this issue. Any existing Java application, and especially the existing frameworks and libraries such application might include, are already using one or more of the widely popular logging APIs (such as slf4j). How does introducing an OpenTelemetry logging API help with that case? Perhaps we're talking not about an API exposed to the end user, but some SPI for binding an OTEL-specific exporter to the existing logging frameworks?

[anuraaga]

@yurishkuro From what I understand it is intended for the user to directly use the logs SDK to send OTel data model events rather than logs. This won't be possible for javaagent agent users where the SDK is managed by the agent and hidden from the user.

@tigrannajaryan If we make the bridge API internal for use only from our instrumentation, this will still be the case, users won't have a way to send logs other than standard log libraries. If this is ok then the internal approach works fine.

[trask]

From what I understand it is intended for the user to directly use the logs SDK to send OTel data model events rather than logs.

^^ I think this is a key question.

@tigrannajaryan can you clarify whether it is intended for users to use the logs SDK directly if they need to emit structured events? (though this would be surprising to me given the SDK/API split for the other signals)

Btw, here's the lovely experience @jack-berg is talking about above when trying to use standard Java logging libraries to emit structured events:

Slf4j:

Map<String, String> attributes = new HashMap<>();
attributes.put("myattr1", "val1");
attributes.put("myattr2", "val2");
attributes.put("myattr3", "val3");

// stash the current thread local state so that it doesn't leak into the event
Map<String, String> stash = MDC.getCopyOfContextMap();
try {
  MDC.setContextMap(attributes);
  logger.info("my event");
} finally {
  // restore the thread local state
  MDC.setContextMap(stash);
}

Log4j:

Map<String, String> attributes = new HashMap<>();
attributes.put("myattr1", "val1");
attributes.put("myattr2", "val2");
attributes.put("myattr3", "val3");

// stash the current thread local state so that it doesn't leak into the event
Map<String, String> stash = ThreadContext.getImmutableContext();
try {
  ThreadContext.clearMap();
  ThreadContext.putAll(attributes);
  logger.info("my event");
} finally {
  // restore the thread local state
  ThreadContext.putAll(stash);
}

java.util.logging:

[j/k, java.util.logging doesn't support structured events]

[yurishkuro]

On the other hand, in Jaeger, which is in Go and uses Zap logger:

s.logger.Info(
        "Query server started",
        zap.Int("port", tcpPort),
        zap.String("addr", s.queryOptions.HTTPHostPort))

There are similar structured logging frameworks in other languages. Even in Java and SLF4J, there is way to log structured events with much cleaner syntax than what @trask showed, e.g. (from https://www.innoq.com/en/blog/structured-logging/)

import static net.logstash.logback.argument.StructuredArguments.keyValue;

String orderId = "123";
log.info("Order saved {}", keyValue("orderId", orderId));

Point is, structured logging APIs already exist and can be integrated with OTEL logging transport. Trying to develop the "15th standard" in OTEL is not only a large scope creep, but a huge uphill battle for adoption.

[trask]

this makes sense to me 👍

so is this a fair summary:

• users should not use the logging SDK to emit logs
• it's ok for our structured logging support to require an extension on top of one of the standard Java logging frameworks (e.g. logstash in @yurishkuro's example above)

[tigrannajaryan]

@tigrannajaryan can you clarify whether it is intended for users to use the logs SDK directly if they need to emit structured events? (though this would be surprising to me given the SDK/API split for the other signals)

This was not my intent when writing the Logging Library SDK OTEP. It is an SDK to be used to make logging libraries emit Otel logs. The goal is to allow end users to continue to use their logging library.

From the OTEP:

the functions and methods callable from this SDK package are intended to be used by Logging Libraries only and are NOT intended to be used by the end user and will NOT be exposed in the OpenTelemetry API package.

[anuraaga]

@tigrannajaryan Thanks for the clarification in that case all is good. I think I heard something about users directly using the SDK for events, especially in the context of RUM. If that's not the case then things make sense.

[tigrannajaryan]

I am going to close this since it appears we are all on the same page. If any further clarifications are needed please reopen.

[jkwatson]

@tigrannajaryan Thanks for the clarification in that case all is good. I think I heard something about users directly using the SDK for events, especially in the context of RUM. If that's not the case then things make sense.

I think in the RUM case, we will want to create a new API specifically for RUM that will be implemented via the SDK, just like our existing API is implemented by the SDK.