Provide a hook for configuring Logback programatically

[celcius112]

At the moment we are considering migrating few applications to spring-native. Everything works fine excepts our Logback configuration. We use a logback.xml, and as stated in spring-attic/spring-native#625 it will not work as intended, nor does it seem to be an easy problem to tackle.

We were already considering moving to the programmatic Logback configuration, using a Configurator, which would probably be easier to use with spring-native and would provide better application startup for regular applications. However, we cannot migrate all of our logback.xml configuration since we cannot import default spring configuration nor use springProperty/springProfile elements.

[sdeleuze]

Status update:

• Currently Spring Boot 3 only supports Logback configuration via application.properties or Yaml equivalent, logback.xml is unsupported.
• After a deeper look, I confirm that build time generation of programmatic Logback configuration from logback.xml is the only identified way to support it on native, there are already discussions and experiments in the JVM/GraalVM communities about that.
• Providing such hook to users would unlock those who need more advanced customization capabilities than what application.properties offers and could allow Spring community to experiment about build time logback.xml to programmatic version in an autonomous manner. To be evaluated against other priorities of course.
• Potential link with Logback 1.4 upgrade from Upgrade to SLF4J 2.0 and Logback 1.4 #12649

[philwebb]

It looks like a ServiceLoader could be used to plug-in a custom ch.qos.logback.classic.spi.Configurator, however, I don't think our logging initialization code would detect that happened and we we'd probably end up applying our configuration. The org.springframework.boot.logging.AbstractLoggingSystem class has a initializeWithConventions which attempts to detect logback.xml and if it's found not apply our defaults.

We do already have a package-private org.springframework.boot.logging.logback.DefaultLogbackConfiguration class which applies our defaults in a programmatic way. I don't really want to make that class public, especially if the GraalVM community is working on a logback.xml -> Configurator processor. Presumably, such a processor would need to deal with <include> tags, at which point our existing XML files could just be included and used directly.

Adding hook points for logging is quite a challenge since they're normally needed before the Spring ApplicationContext is loaded. I think we should see how the community experiments progress before we do too much on our side.

[jack-berg]

I'd like to see this issue solved to accommodate what I think is a similar use case:

OpenTelemetry java has logback and log4j appenders which bridge log data from those logging systems in the OpenTelemetry ecosystem. These appenders need to be configured with an OpenTelemetry instance, an object which contains APIs for bridging log records into OpenTelemetry, and which is programmatically configured by the user. The general workflow is for users to initialize an OpenTelemetry instance very early in the application lifecycle, and then use it when installing various bits of instrumentation. In spring environments, its typical to create an OpenTelemetry bean to participate in dependency injection.

The programmatic configuration of the OpenTelemetry instance is at odds with typical static log configuration. But both log4j and logback allow for programmatic configuration as well, where you can modify a log4j.xml or logback.xml configuration at runtime by adding / editing appenders.

It would nice to offer OpenTelemetry users a helper function public static void installOpenTelemetryAppender(OpenTelemetry), which modifies the log4j / logback configuration at runtime after the OpenTelemetry instance has been configured.

This workflow doesn't quite work in spring boot because programmatic modifications to the log4j / logback configuration are erased when spring boot is initialized (I believe this line is the culprit). Users could do the modification after spring initializes, but they'll miss out on valuable logs detailing spring initialization.

It would be great if spring boot could provide a hook for performing programmatic configuration of log4j / logback early enough that any appenders added are able to receive application startup logs.

[wilkinsona]

Thanks for the use case, @jack-berg. Unfortunately, unless I have misunderstood, I don't think what you have described is possible.

In spring environments, its typical to create an OpenTelemetry bean to participate in dependency injection

It would be great if spring boot could provide a hook for performing programmatic configuration of log4j / logback early enough that any appenders added are able to receive application startup logs.

These goals seem to be contradictory. If we're using an OpenTelemetry bean, it will not be created until some point during application context refresh. This is too late to receive (all) application startup logs.

It would nice to offer OpenTelemetry users a helper function public static void installOpenTelemetryAppender(OpenTelemetry), which modifies the log4j / logback configuration at runtime after the OpenTelemetry instance has been configured.

I don't think this belongs in Spring Boot. From what I understand, there's nothing Spring Boot-specific about wanting to configure Logback or Log4j with an Open Telemetry appender. For example, with Logback, it would be done through the standard ch.qos.logback.classic.LoggerContext API. As such, I think it should be part of OpenTelemetry so that it benefits all of its users not just those who are also using spring Boot.

[jack-berg]

Sorry I think I presented my case in a confusing way. I don't think spring boot should in any way be responsible for installing / configuring the opentelemetry appender. Rather, I was just trying to explain why programmatic configuration is important for the OpenTelemetry appenders, and express support for spring boot adding some sort of generic callback or hook that opentelemetry code can leverage to perform programmatic configuration soon after spring boot log configuration, such that the opentelemetry appenders can receive initialization logs.

[philwebb]

This might overlap with Logback's existing META-INF/services/ch.qos.logback.classic.spi.Configurator support.

[making]

I created samples to automatically configure Logback. I'm not sure if it's the right way to do it, but it works for me in the following two use cases:

• Automatically install Otel Logback Appender　https://github.com/making/otel-logs-autoconfigure/blob/develop/src/main/java/am/ik/spring/opentelemetry/logs/logback/LogbackAppenderInstallListener.java
• Automatically configure EcsEncoder https://github.com/making/logback-ecs-encoder-autoconfigure/blob/develop/src/main/java/am/ik/spring/ecs/logback/EcsEncoderConfigurationListener.java