JBoss.Logging in junit tests

[smil2k]

Describe the bug
In case I would like to use JUnit for the purpose it was conceived (testing methods within small confined test scenarios without starting quarkus) and would like to use SLF4J for logging, then JBoss Logging gets into the way. It is virtually impossible to configure jboss logging by dropping some file on classpath like logback-test.xml or whatnot.

Additionally the logging is delayed at startup by some sort of buffering. Therefore you will only see the result of the log later, so your observation about some parts of the test is slow is wrong.
Update: the delay is caused by surefire!

Expected behavior

Adding logback-classic into the classpath would work fine or alternative there should be some canonical way of configuring the logging for unit tests without paying the 4-5 sec penalty to start the quarkus instance.

Actual behavior

Need to exclude bindings from every extension:

io.quarkus
quarkus-hibernate-orm


slf4j-jboss-logging
org.jboss.slf4j

Workaround

By adding a dependency exclude to surefire, the logmanager can be removed from the classpath, then any provider can take its place:

        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-surefire-plugin</artifactId>
            <version>${surefire-plugin.version}</version>
            <configuration>
                <classpathDependencyExcludes>
                    <classpathDependencyExclude>org.jboss.slf4j:slf4j-jboss-logmanager</classpathDependencyExclude>
                </classpathDependencyExcludes>
            </configuration>
        </plugin>

<dependencies>
    <dependency>
      <groupId>ch.qos.logback</groupId>
      <artifactId>logback-classic</artifactId>
      <scope>test</scope>
    </dependency>
    <dependency>
      <groupId>org.slf4j</groupId>
      <artifactId>jul-to-slf4j</artifactId>
      <scope>test</scope>
    </dependency>
  </dependencies>

To Reproduce
Steps to reproduce the behavior:

1. create unit test
2. try to use SLF4J
3. nothing gets printed

Environment (please complete the following information):

• Output of uname -a or ver: Linux conegliano 4.15.0-76-generic Opentracing take2 (OKAY TO MERGE) #86-Ubuntu SMP Fri Jan 17 17:24:28 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

• Output of java -version:
  openjdk version "1.8.0_242"
  OpenJDK Runtime Environment (Zulu 8.44.0.11-CA-linux64) (build 1.8.0_242-b20)
  OpenJDK 64-Bit Server VM (Zulu 8.44.0.11-CA-linux64) (build 25.242-b20, mixed mode)

• GraalVM version (if different from Java):
  openjdk version "1.8.0_242"
  OpenJDK Runtime Environment (build 1.8.0_242-b06)
  OpenJDK 64-Bit GraalVM CE 19.3.1 (build 25.242-b06-jvmci-19.3-b07, mixed mode)

• Quarkus version or git rev: 1.2.0

• Build tool (ie. output of mvnw --version or gradlew --version):
  Apache Maven 3.6.0

Additional context
(Add any other context about the problem here.)

[jimbogithub]

I'm using a temporary work-around of placing slf4j-log4j12 as a test dependency placed before other dependencies.

[zerocode]

Agree, the way logging is currently implemented is a pain. It would be good to have some flexibility here.

[glandais]

A trick is to provide a different LoggerProvider using org.jboss.logging.provider system property for unit tests. In LoggerProviders, this is the first thing that is done for getting a LoggerProvider.
In our test lib, I've added log4j-core (2.14.0) and a log4j2.xml.
Then in failsafe/surefire Maven configuration, I've added <org.jboss.logging.provider>log4j2</org.jboss.logging.provider> in systemPropertyVariables

[dcaillia]

We're using slf4j api in all our code.

Quarkus puts an impl (from jboss) on the classpath.

When i run a normal (non quarkus) junit tests from my IDE, i have no clue how to configure logging. It is not explained anywhere.

What i get: "LogManager error of type WRITE_FAILURE: The delayed handler's queue was overrun and log record(s) were lost. Did you forget to configure logging?"

What to do?

• configure this logging backend from jboss when running junit tests from IDE: how?
• bypass this jboss backend somehow and enable me to use an slf4j impl of my choice (log4j-slf4j-impl) : how?

[francescopace]

Another trick is to completely remove slf4j-jboss-logmanager dependency during test:

        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-surefire-plugin</artifactId>
            <version>${surefire-plugin.version}</version>
            <configuration>
                <classpathDependencyExcludes>
                    <classpathDependencyExclude>org.jboss.slf4j:slf4j-jboss-logmanager</classpathDependencyExclude>
                </classpathDependencyExcludes>
            </configuration>
        </plugin>

Then you can add your logging library as test scope dependency (e.g. slf4j):

    <dependency>
        <groupId>org.slf4j</groupId>
        <artifactId>slf4j-log4j12</artifactId>
        <version>1.7.30</version>
        <scope>test</scope>
    </dependency>

And then, at the end, put a simple log4j.properties in test/resources directory.

It seems to work both on @QuarkusTest and normal junit test.

[jimbogithub]

Another trick is to completely remove slf4j-jboss-logmanager dependency during test:

Nice, although it looks like Eclipse doesn't honour that. Good on Maven cli though.

[dcaillia]

I can confirm that having this log4j2 slf4j binding as first dependency (test scoped) in the pom.xml

org.apache.logging.log4j log4j-slf4j-impl 2.14.1 test ... all other dependencies below

works for running plain old non-quarkus junit tests from eclipse: eclipse respects that order, slf4j complains about 2 impl's but then carries on with this log4j2 impl, who's config i control via JRE default VM argument -Dlog4j.configurationFile=file:///home/dcailliau/log4j2.properties

I think this is worth mentioning in the quarkus docs.

[famod]

I invite everyone to try out 2.7.0.CR1 which was released today! Update: 2.7.0.Final is now generally available.

It comes with #22920 that added a global JUnit callback that tries to set up "basic" logging before any JUnit test class, which includes non-Quarkus JUnit-based tests.
In general, you don't need to do anything special, just fire up a plain JUnit tests that tries to log something (via JBoss logging or SLF4J or probably also Log4j) and you should be able to see it.

If (and only if!) you have a custom junit-platform.properties you'll need to set this: https://github.com/quarkusio/quarkus/blob/2.7.0.CR1/test-framework/junit5-properties/src/main/resources/junit-platform.properties
(and I also suggest to set that QuarkusTestProfileAwareClassOrderer)
You should also exclude quarkus-junit5-properties like described here: https://github.com/quarkusio/quarkus/blob/2.7.0.CR1/docs/src/main/asciidoc/getting-started-testing.adoc#10-testing-different-profiles

Finally, a note on "basic": This setup will include most of the logging config but nothing contributed by extensions. If you are missing something very important, please let me know!

Feedback is very much appreciated!

[famod]

Small update: As of 2.7.1.Final, application.properties (and some other config sources) are not picked up for configuring basic logging in unit tests.
I'm going to fix this, hopefully in 2.7.2.Final.

[famod]

2.7.2 has further improved the situation, so I'll close this issue.

Please open a new issue in case you thinks something is not right or missing.

[calohmn]

@famod The above issue text mentions a scenario where logback is used, with a logback-test.xml configuration.
As it seems to me, this scenario is still not supported (?). At least I don't see the logback-test.xml being picked up.
So, is it that in such a case the logging configuration would have to be done in an application.yaml instead?
Do you see logback support as something that still can be added?

As for the BasicLoggingEnabler class added here, see also #24524.

[famod]

@calohmn yeah, logback-test.xml is out of scope here. You should have a look at https://github.com/quarkiverse/quarkus-logging-logback

[smil2k]

Just for the future reference, the initial delay is caused by surefire:
https://issues.apache.org/jira/browse/SUREFIRE-1827

[famod]

Yeah, I strongly recommend using surefire 3.0.0-M7.

[mklueh]

In case someone finds this in 2023+ and logging does not work with Slf4j in tests, these are the only changes that needed to be done

test {
    //remove
    //systemProperty "java.util.logging.manager", "org.jboss.logmanager.LogManager"

    //add
    useJUnitPlatform()
    testLogging.showStandardStreams = true
}