Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

know if a serialized piece of Arkworks structure has been compressed #818

Open
a-stevan opened this issue Apr 12, 2024 · 3 comments
Open

Comments

@a-stevan
Copy link

i couldn't find anything related in the issue tracker, so here i am 😇

let's say i have the following piece of code

fn ser(obj: impl CanonicalSerialize, compress: Compress) -> Vec<u8> {
    let mut serialized = vec![0; obj.serialized_size(compress)];
    obj.serialize_with_mode(&mut serialized[..], compress)?;
    serialized
}

is it possible, on the other end of the protocol, to deserialize the bytes without passing around the same compress: Compress, e.g. maybe the information about compression is inside the bytes or something?
i.e. i'm wondering if something like the following is possible

fn deser(bytes: &[u8], validate: Validate) -> Result<MyStruct> {
    let compress = is_it_compressed_or_not(bytes); // this is the part i couldn't find

    let my_struct = Block::deserialize_with_mode(&bytes[..], compress, validate)?
    Ok(my_struct)
}

thanks and cheers 🥳 👋

@mmagician
Copy link
Member

Compression is very data specific, so it's hard to answer this question in full generality without knowing what comprises your struct.
You can always try to deserialize with, and then without compression, and hopefully one of them fails - but note that it's not always the case:

  • some types might not even support compression, so both versions work. This is not much of an issue, just double the work.
  • some types might give you different results. I think the clearest example is Vec. Given e.g. a sequence of 96 bytes, you might successfully deserialize into either 2 EC points (assuming compressed) or 1 non compressed point on the curve. Goes without saying that they represent completely different underlying data.

@burdges
Copy link
Contributor

burdges commented Apr 13, 2024

Afaik, you should always compress for network wire formats, because compression's extra CPU time costs less than the extra network bytes, and blockchains magnify this. In particular, I'd expect the projective to affine conversions cost more, making batch_normalize more significant.

It's different when you're only storing cached data locally, well then maybe uncompressed and/or unvalidated makes sense. In principle, one could even serialize in projective locally, but in general projective serialization is extremely dangerous.

@a-stevan
Copy link
Author

@mmagician @burdges
thanks for the quick answers, this is valuable information i think 🙏

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants