Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add optional support for zlib-ng #132

Open
jack-pappas opened this issue May 27, 2015 · 1 comment
Open

Add optional support for zlib-ng #132

jack-pappas opened this issue May 27, 2015 · 1 comment

Comments

@jack-pappas
Copy link
Contributor

zlib-ng is a fork of zlib which is working to "modernize" the zlib codebase. It includes some patches from Intel and Cloudflare which use the SSE 4.2 pclmulqdq instruction to optimize the CRC32 calculations for compressed blocks, leading to a significant speedup for both compression and decompression.

It seems like a promising project but it'd be important to perform in-depth testing before trusting any important data to it. Perhaps a compromise would be to import zlib-ng to this repository (or add it as a submodule so it's easier to stay up-to-date), then implement a CMake option to use it instead of the standard zlib when compiling blosc to allow for experimentation and verification.

@FrancescAlted
Copy link
Member

Yes, zlib-ng looks like a perfect match for Blosc. I agree with allowing a transition time where zlib-ng and classic zlib would coexist with a CMake option to allow to switch to zlib-ng. And provided that zlib-ng is still in-flux, I think it is a good idea to add it as a submodule.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants