Error (RejectedExecutionException) when rolling more than 32 log files

[matthewkulich]

Description

• There is an issue in Logback when using RollingFileAppender to roll more than 32 files, especially when the file sizes are large. The default thread pool size of 32 in Logback leads to a RejectedExecutionException when the thread pool becomes exhausted, causing log rolling to fail and leaving .tmp files on disk.

Steps to Reproduce:

1. Configure Logback to use SizeAndTimeBasedRollingPolicy.
2. Set up a logging environment where more than 32 log files need to roll at the same time.
3. Ensure the file sizes are substantial enough that compression tasks take longer than usual.
4. At the time of rolling, the thread pool reaches its limit, and the RejectedExecutionException occurs.

Expected Behavior:

• Logback should successfully roll and compress all files without throwing a RejectedExecutionException. It should also not leave .tmp files on the disk.

Actual Behavior:

• Logback attempts to roll files, but when the thread pool (set to a fixed size of 32) is full, it throws a RejectedExecutionException.
• This prevents some files from being compressed, and leaves .tmp files in the log directory.

Error stacktrace:

09:18:57,097 |-ERROR in ch.qos.logback.core.rolling.RollingFileAppender[FILE-my_test_file_1] - Appender [FILE-my_test_file_1] failed to append. java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@51d4563b[Not completed, task = java.util.concurrent.Executors$RunnableAdapter@5ea7a8e5[Wrapped task = ch.qos.logback.core.rolling.helper.Compressor$CompressionRunnable@65f83938]] rejected from java.util.concurrent.ThreadPoolExecutor@262004ad[Running, pool size = 32, active threads = 32, queued tasks = 0, completed tasks = 0]
	at java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@51d4563b[Not completed, task = java.util.concurrent.Executors$RunnableAdapter@5ea7a8e5[Wrapped task = ch.qos.logback.core.rolling.helper.Compressor$CompressionRunnable@65f83938]] rejected from java.util.concurrent.ThreadPoolExecutor@262004ad[Running, pool size = 32, active threads = 32, queued tasks = 0, completed tasks = 0]
	at 	at java.base/java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2055)
	at 	at java.base/java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:825)
	at 	at java.base/java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1355)
	at 	at java.base/java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:118)
	at 	at ch.qos.logback.core.rolling.helper.Compressor.asyncCompress(Compressor.java:229)
	at 	at ch.qos.logback.core.rolling.TimeBasedRollingPolicy.renameRawAndAsyncCompress(TimeBasedRollingPolicy.java:196)
	at 	at ch.qos.logback.core.rolling.TimeBasedRollingPolicy.rollover(TimeBasedRollingPolicy.java:182)
	at 	at ch.qos.logback.core.rolling.RollingFileAppender.attemptRollover(RollingFileAppender.java:220)
	at 	at ch.qos.logback.core.rolling.RollingFileAppender.rollover(RollingFileAppender.java:198)
	at 	at ch.qos.logback.core.rolling.RollingFileAppender.subAppend(RollingFileAppender.java:246)
	at 	at ch.qos.logback.core.OutputStreamAppender.append(OutputStreamAppender.java:102)
	at 	at ch.qos.logback.core.UnsynchronizedAppenderBase.doAppend(UnsynchronizedAppenderBase.java:85)
	at 	at ch.qos.logback.core.sift.SiftingAppenderBase.append(SiftingAppenderBase.java:122)
	at 	at ch.qos.logback.core.AppenderBase.doAppend(AppenderBase.java:83)
	at 	at ch.qos.logback.core.spi.AppenderAttachableImpl.appendLoopOnAppenders(AppenderAttachableImpl.java:51)
	at 	at ch.qos.logback.classic.Logger.appendLoopOnAppenders(Logger.java:272)
	at 	at ch.qos.logback.classic.Logger.callAppenders(Logger.java:259)
	at 	at ch.qos.logback.classic.Logger.buildLoggingEventAndAppend(Logger.java:426)
	at 	at ch.qos.logback.classic.Logger.filterAndLog_0_Or3Plus(Logger.java:386)
	at 	at ch.qos.logback.classic.Logger.info(Logger.java:596)

My actual workaround:

• As a temporary workaround, I used Java reflections to adjust the ThreadPoolExecutor configuration in my application, and it resolved the issue.
• modified threadPoolExecutor in ContextBase.java to use LinkedBlockingQueue instead of SynchronousQueue

Information:

• logback version 1.4.14
• this bug was already reported here: https://jira.qos.ch/browse/LOGBACK-1740