Define whether instrumentations can emit non-event log records

[pellared]

Imagine I'm instrumenting kafka:

• there could be some events I'd emit - e.g. cluster leader is unreachable - which are documented in the semconv
• there could also be things I just want to let my users know of via logs. E.g. certain configuration was picked, connection has dropped, etc. I could use 3rd party logging library, but why if I can use OTel logging API without adding a new dependency?

The library logs are emitted with is not relevant to users - they'd get the same logs if lib used 3rd party logger or OTel API. So I don't see why we should limit instrumentations from using OTel API to emit undocumented regular (but structured) logs.

Originally posted by @lmolkova in #4225 (comment)

[lmolkova]

To add a bit of context: client libraries or infra components usually emit some logs using some non-OTel logging API.
If they are instrumented natively with OTel they need to have 2 dependencies - OTel and 3rd party logging lib.

While it's unlikely that existing libraries will replace their existing logging with OTel one, by providing user-facing logging API we allow future native instrumentations that use OTel for metrics and tracing to also use OTel logging API instead of adding a new dependency.

Before #4225 it was not possible.

[austinlparker]

This actually seems to be two separate questions/issues -

1. I am authoring an instrumentation library for Library Foo, that will be consumed as an external dependency of Library Foo (e.g., a library instrumentation that can be injected via zero-code instrumentation or as middleware that can be added at compile-time). How should I emit LogRecords from my instrumentation library that may not have semantic events defined?
2. I am updating Library Bar to include native OpenTelemetry support. I currently use the standard logging library for my language. I need to emit semantic events as LogRecords, can I do so with my existing logging API, or do I need to use the OTel Logging API to emit those events and not do so for other messages?

In the first case, I think it is non-controversial to suggest that instrumentation libraries must only rely on OpenTelemetry APIs. I will eschew commentary on the second case for now.

[pellared]

Logs SIG meeting notes:
We decided to postpone this decision. We first need instrumentation libraries to emit events. We can later revisit if we need to emit log records as well (which are not events).

[lmolkova]

To add to @austinlparker comment #4234 (comment) - from native instrumentation perspective I'm mostly interested in the case 3:

3. I'm creating a new library that does not do any logging yet. It will use OTel tracing, metrics and events. It seems non-controversial that if I have any logging needs (not covered by events) I will also use OTel logging API. Otherwise, I'll need to pick a 3rd party logging facade which may require extra dependency.

[austinlparker]

@lmolkova I considered adding that use case to the comment but left it off deliberately in order to keep the discussion focused. I do agree with you, though, it is an important use case to consider.

[pellared]

One of OTel Go prototypes for adding support for events makes emitting non-event log records "natural": open-telemetry/opentelemetry-go#6018