smallByteSize methods are very trappy in many classes -- should be changed or have warnings in javadocs

[hossman]

When integrating t-digest into Solr, the first patch contributed had code which looked like this...

ByteBuffer buf = ByteBuffer.allocate(tdigest.smallByteSize());
tdigest.asSmallBytes(buf);
res.add("percentiles", buf.array());

...the contributor who wrote that code did so based on the javadocs of ArrayDigest.smallByteSize, not realizing that under the covers hte internals of this method are:

• allocate a ByteBuffer based on the result of byteSize()
• do a complete serialization using asSmallBytes(ByteBuffer)
• return the final position of the ByteBuffer

...meaning that ultimately at least 2x the smallByteSize of RAM was being allocated everytime, and the ArrayDigest was being serialized twice.

Methods like ArrayDigest.smallByteSize (AVLTreeDigest.smallByteSize, etc...) that have such a high internal cost should either be changed to just call byteSize() (since the overallocation the user will have to do is still better for space/time performance then doing that same overallocation internally plus a behind the scenes serialiation that is thrown away) or warn users about this heavy internal cost in the javadocs.

[tdunning]

I am opting to warn users away from these methods. I see no way to find out the size without doing the compression. There is probably a better API based on streams that would avoid this need. Or maybe over-allocation idioms are the rule of the day.

Either way, I am taking the easy way out.