-
Notifications
You must be signed in to change notification settings - Fork 105
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
snappy.UncompressError: stream missing snappy identifier #40
Comments
The implementation for file compression/decompression using streaming was done by @jtolds. He may have a better idea of what's going on. |
There's two forms of Snappy compression, the basic form and the streaming form. The basic form has the limitation that it all must fit in memory, so the streaming form exists to be able to compress larger amounts of data. The streaming format has a header and then subranges that are compressed. If the header is missing, it sounds like maybe you compressed using the basic form and are trying to uncompress with the streaming form. |
@jtolds so how to uncompressed with streaming form? any tutorial to help me figure it out? |
What did you do to uncompress the data? Can you paste the code you used? That way I can help with what needs to change. |
@jtolds The following were the snappy cli I used.
I dont know what's wrong with it. Please help me out with this issue. Thanks |
okay, the commandline tool only supports the streaming format. You will need to write a program that imports the snappy package and uses the plain uncompress method with the data from that file. import snappy, sys; sys.stdout.write(snappy.uncompress(file("./02_1473019352277439472_14838198.snappy").read())) |
Oh. I got it. It works like a charm. Thanks a lot. It confused me so long, And why not provide an option to process the file? |
Because unframed snappy compression has a maximum size limit, so if you're dealing with files you almost certainly want the streaming format |
OK. I got it. |
Hi, Thanks, |
@rajeevanair , can you please be more specific about your situation, how you installed, what versions you got and what you did to see the error. |
Hi Martin, However snappy library does not have an uncompress function. |
|
A thousand thanks, Martin! That worked. |
When I used snappy module to decompress a file it occurs the issue
The text was updated successfully, but these errors were encountered: