Emitting log template + parameters?

[trask]

Area(s)

area:log

Is your change request related to a problem? Please describe.

Java log messages are often emitted as a template (with placeholders) + parameters, e.g.

log.info("Message {}, {} and {}", param1, param2, param3)

Using template + parameters is not really an indication from the user that this is a "structured log", as it is typically done as a performance optimization to avoid concatenating the string when the logging level is suppressed already (and also there are no parameter names).

Whereas there are other Java logging APIs which are intended for structured logging and using them is an indication from the user that this is a "structured log", e.g.

log.info(MapMessage)

So in the OpenTelemetry Java logging appenders we think it makes sense in the first case to emit the "formatted" (interpolated) message in the log body (string), and in the second case to emit a "structured log body".

The first purpose of this issue is to see if others agree with our thinking.

The second purpose of this issue is to see how we could optionally capture the log template and parameters in the first case (e.g. to support open-telemetry/opentelemetry-java-instrumentation#11865).

Our initial proposal would be something like:

• log.record.template
• log.record.parameters

where log.record.parameters would be a string array, though another option could be log.record.parameters.<key> to more generally support named parameters (do other languages have these? and if there are named parameters is that a reason to capture as structured log?)

Thanks!

Describe the solution you'd like

not sure

Describe alternatives you've considered

No response

Additional context

No response

[lmolkova]

my 2c (only confirming that there are a lot of unknows on how to map logs to otel):

• the parameters are not necessarily strings/primitives and need to be flattened if they are recorded in the attributes. The body seems like a better place for them
• arguably people want to query by parameters, in some languages parameters are named (e.g. .NET allows to do logger.LogInfo("{foo}", "bar")) and they work best if "foo"="bar" is preserved
• AFAIK there are corner cases like slf4j structured logging where I can do logger.atInfo().addKeyValue("foo", "bar").log("{}", "baz") and it's not clear where the context ("foo"="bar") should go (attributes? body?)

[Update] one solution could be to keep bridged log body string-based, put parameters into attributes and kindly ask people to use Event API for structured payloads.

[igor-suhorukov]

Using template + parameters is not really an indication from the user that this is a "structured log", as it is typically done as a performance optimization to avoid concatenating the string when the logging level is suppressed already

This seems like a valid use case, but in case of legacy applications and libraries, access to logger parameters is the only possible way to analyze structured application information. In this case, each logger call with an array can be considered as a structured message with unnamed parameters.

Also in the case of the new slf4f2 API this will allow to preserve the logger data structure as much as possible for log search functionality without using full-text search:
logger.atInfo().setMessage("In fact, the very fact that this entry appears in the log indicates that there is an implicit structure and it would be good to provide access for searching to the parameters {} and {}, {} long value {}, and not just to the concatenated string with the result").addArgument(arg1).addArgument(arg2).addArgument(arg3bool).addArgument(arg3long).log();

Unfortunately, legacy applications cannot be rewritten to use new shiny top APIs, and my PR offers a working solution to simplify the analysis of the workflow of such applications, when you can perform queries to the storage of almost structured logs without changing the source codes.

[cijothomas]

log.info("Message {}, {} and {}", param1, param2, param3)
This looks similar to https://messagetemplates.org/, if the placeholders are named like the .NET Example
logger.LogInformation("Hello World! Logging is {Description}.", "fun");

[pellared]

The log templating is usually logging library specific.

I think that a log appender/bridge can have its behavior configurable whether Body should be interpolated or not or a combination of both (then probably the template should be passed emitted as an attribute).