Make TDigest serializable

[gregakespret]

I was wondering why TDigest is not Serializable. I found myself writing some boilerplate code to serialize/deserialize it between tasks when using it in Spark and it would be cool if it was serializable by default. This is what I have to do now:

def serialize(tdigest: TDigest) = {
  val arr = new Array[Byte](tdigest.byteSize)
  tdigest.asBytes(ByteBuffer.wrap(arr))
  arr
}

def deserialize(arr: Array[Byte]) = {
  AVLTreeDigest.fromBytes(ByteBuffer.wrap(arr))
}

Do you have any thoughts on that? I could submit a PR for this if you think it'd be useful.

[tdunning]

Hmm...

I really dislike what Serializable does to code. It is definitely
important to have a way to save data, but it tends to make things really
brittle.

How bad is this for you?

On Sun, Apr 5, 2015 at 4:57 PM, Grega Kespret [email protected]
wrote:

I was wondering why TDigest is not Serializable. I found myself writing
some boilerplate code to serialize/deserialize it when using it in Spark
and it would be cool if it was serializable by default. This is what I have
to do now:

def serialize(tdigest: TDigest) = {
val arr = new ArrayByte
tdigest.asBytes(ByteBuffer.wrap(arr))
arr
}

def deserialize(arr: Array[Byte]) = {
AVLTreeDigest.fromBytes(ByteBuffer.wrap(arr))
}

Do you have any thoughts on that? I could submit a PR for this if you
think it'd be useful.

—
Reply to this email directly or view it on GitHub
#47.

[gregakespret]

It's not that bad, I have a working solution and can continue to use it (code above).