ScanIterator.scan.stream().toList() causes "java.lang.IllegalStateException: Accept exceeded fixed size of 0" on Java 16+

[lwestby-chwy]

Bug Report

Current Behavior

Attempting to call toList() (a new stream operation as of Java 16) on a stream obtained from a ScanIterator throws an IllegalStateException originating from several layers deep within the JDK. Tested on both Java 17 and Java 21.

This appears to match the description for a similar issue encountered in Spring, if this helps: spring-projects/spring-data-commons#2519. In their case, it has to do with the size or characteristic passed along on creation of the stream leading to a FixedNodeBuilder expecting 0 elements.

The older .collect(Collectors.toList()) operation does not encounter this issue.

Stack trace

java.lang.IllegalStateException: Accept exceeded fixed size of 0
	at java.base/java.util.stream.Nodes$FixedNodeBuilder.accept(Nodes.java:1224)
	at java.base/java.util.Iterator.forEachRemaining(Iterator.java:133)
	at java.base/java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1845)
	at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
	at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
	at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:575)
	at java.base/java.util.stream.AbstractPipeline.evaluateToArrayNode(AbstractPipeline.java:260)
	at java.base/java.util.stream.ReferencePipeline.toArray(ReferencePipeline.java:616)
	at java.base/java.util.stream.ReferencePipeline.toArray(ReferencePipeline.java:622)
	at java.base/java.util.stream.ReferencePipeline.toList(ReferencePipeline.java:627)
	at io.lettuce.core.ScanIteratorIntegrationTests.testScanIteratorStream(ScanIteratorIntegrationTests.java:85)
	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)

Input Code

Input Code

Reproducible right within ScanIteratorIntegrationTests.java if running on Java 16 JRE or above:

@Test
void testScanIteratorStream() {
    redis.mset(KeysAndValues.MAP);

    List<String> result = ScanIterator.scan(redis, ScanArgs.Builder.limit(50).match("*"))
        .stream()
        .toList();
}

Expected behavior/code

The toList() operation returns a list successfully.

Environment

• Lettuce version(s): 6.3.2.RELEASE as well as latest main branch
• Redis version: latest Redis 7 on DockerHub

Possible Solution

spring-projects/spring-data-commons#2519 points to the commit where they fixed it, although I can't pretend I understand how much of the full commit is required to solve this here.

Updating ScanIterator.stream() to set the size to -1 on the spliterator does fix it locally, but not sure if there are knock-on effects that require more code:

public Stream<T> stream() {
    return StreamSupport.stream(Spliterators.spliterator(this, -1, 0), false);
}

Additional context

Java version info:

Java 21:

openjdk version "21.0.3" 2024-04-16 LTS
OpenJDK Runtime Environment Temurin-21.0.3+9 (build 21.0.3+9-LTS)
OpenJDK 64-Bit Server VM Temurin-21.0.3+9 (build 21.0.3+9-LTS, mixed mode)

Java 17:

openjdk version "17.0.11" 2024-04-16
OpenJDK Runtime Environment Temurin-17.0.11+9 (build 17.0.11+9)
OpenJDK 64-Bit Server VM Temurin-17.0.11+9 (build 17.0.11+9, mixed mode)

[tishun]

Hey @lwestby-chwy,
thanks for the detail analysis and report. IMHO the fix from the spring-data-commons repo should be applied here too. I will leave @mp911de comment if he thinks this is wrong, but as far as I can understand the problem is similar.

I will prepare a PR and try to get it in the 6.5.0 release.