java.lang.IndexOutOfBoundsException: No enough data in the stream java.io.ByteArrayInputStream@36f0f1be

[zhangchaoxiang]

Search before asking

• I had searched in the issues and found no similar issues.

Version

0.6.0

Component(s)

Java

Minimal reproduce step

    ThreadSafeFury fury = Fury.builder().requireClassRegistration(false).withCompatibleMode(CompatibleMode.COMPATIBLE).buildThreadSafeFury();
    ByteArrayOutputStream bas = new ByteArrayOutputStream();
    fury.serialize(bas, Lists.newArrayList(1, 2, 3));
    fury.serialize(bas,Lists.newArrayList(10,9,7));
    fury.serialize(bas,Lists.newArrayList(7,5,4));
    bas.flush();

    InputStream bis = new ByteArrayInputStream(bas.toByteArray());
    FuryInputStream stream = of(bis);
    Object lobj1 =  fury.deserialize(stream);
    Object lobj2 =  fury.deserialize(stream);
    Object lobj3 =  fury.deserialize(stream);

What did you expect to see?

no exception

What did you see instead?

java.lang.IndexOutOfBoundsException: No enough data in the stream java.io.ByteArrayInputStream@36f0f1be

at org.apache.fury.io.FuryInputStream.fillBuffer(FuryInputStream.java:66)
at org.apache.fury.memory.MemoryBuffer.readerIndex(MemoryBuffer.java:1274)
at org.apache.fury.resolver.ClassResolver.readClassDefs(ClassResolver.java:1469)
at org.apache.fury.Fury.deserialize(Fury.java:758)
at org.apache.fury.Fury.deserialize(Fury.java:780)
at org.apache.fury.Fury.deserialize(Fury.java:773)
at org.apache.fury.ThreadLocalFury.deserialize(ThreadLocalFury.java:160)

Anything Else?

In version 0.5.1 it ran successfully, but after upgrading to 0.6.0 an exception was thrown, which is related to enabling COMPATIBLE.

Are you willing to submit a PR?

• I'm willing to submit a PR!

[chaokunyang]

We enabled scoped meta share mode by default, but it doesn't support native streaming deserializaiton. Could you use org.apache.fury.io.BlockedStreamUtils instead?

[zhangchaoxiang]

I have checked the BlockedStreamUtils method. The deserialization process reads the entire byte stream before performing the actual deserialization. However, my file is large and requires streaming reads. It's not feasible to read the entire byte stream into memory.
Is there any other solution available?

[chaokunyang]

Could you disable scopedMetaShare option? I tested locally, It works