You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
zlib-ng is a fork of zlib which is working to "modernize" the zlib codebase. It includes some patches from Intel and Cloudflare which use the SSE 4.2 pclmulqdq instruction to optimize the CRC32 calculations for compressed blocks, leading to a significant speedup for both compression and decompression.
It seems like a promising project but it'd be important to perform in-depth testing before trusting any important data to it. Perhaps a compromise would be to import zlib-ng to this repository (or add it as a submodule so it's easier to stay up-to-date), then implement a CMake option to use it instead of the standard zlib when compiling blosc to allow for experimentation and verification.
The text was updated successfully, but these errors were encountered:
Yes, zlib-ng looks like a perfect match for Blosc. I agree with allowing a transition time where zlib-ng and classic zlib would coexist with a CMake option to allow to switch to zlib-ng. And provided that zlib-ng is still in-flux, I think it is a good idea to add it as a submodule.
zlib-ng is a fork of zlib which is working to "modernize" the zlib codebase. It includes some patches from Intel and Cloudflare which use the SSE 4.2
pclmulqdq
instruction to optimize the CRC32 calculations for compressed blocks, leading to a significant speedup for both compression and decompression.It seems like a promising project but it'd be important to perform in-depth testing before trusting any important data to it. Perhaps a compromise would be to import zlib-ng to this repository (or add it as a submodule so it's easier to stay up-to-date), then implement a CMake option to use it instead of the standard zlib when compiling blosc to allow for experimentation and verification.
The text was updated successfully, but these errors were encountered: