Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Please support enabling zstd long-distance matching mode #133

Closed
joshtriplett opened this issue Nov 13, 2021 · 4 comments
Closed

Please support enabling zstd long-distance matching mode #133

joshtriplett opened this issue Nov 13, 2021 · 4 comments

Comments

@joshtriplett
Copy link
Contributor

async-compression only allows passing a numeric level. I'd love to enable the zstd "long-distance matching mode", which substantially improves compression on large files that may have similarity at long distances.

I'd be happy to supply a PR implementing this. Would you prefer to extend Level to contain internal flags for things like this, or would you prefer to add a method (or additional constructor) directly to the ZstdEncoder to set this parameter?

@joshtriplett
Copy link
Contributor Author

I submitted #134 implementing this. (Happy to take a different approach if you would prefer.)

@Nemo157
Copy link
Member

Nemo157 commented Nov 26, 2021

Unfortunately I want to keep the algorithmic dependencies of async-compression private to allow non-breaking updates.

I have long been meaning to split out another crate with the underlying generic implementation that exposes these and would allow full configurability for different libraries, with the downside of needing to either support multiple implementations/versions of the algorithms or doing breaking releases.

I think I had started an attempt at that a while ago, but found it less trivial than I was hoping. This is some motivation to look at it again, hopefully I can at least see how difficult it appears this weekend.

@joshtriplett
Copy link
Contributor Author

Would you consider a PR adding an abstraction over a handful of the options, to avoid exposing the underlying library? I'd be happy to prepare such a patch.

@joshtriplett
Copy link
Contributor Author

This is now fixed, with #134.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants