Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for compression levels, closes #1042 #1044

Merged
merged 2 commits into from
Feb 12, 2018

Conversation

bobrik
Copy link
Contributor

@bobrik bobrik commented Feb 12, 2018

This makes it possible to set compression level for gzip:

config.Producer.CompressionLevel = flate.BestCompression

There are no other compression codecs that support levels,
but the code is generic to easily add other codecs with
numeric levels.

Default compression level tracks what Go libraries
set as the default level.

This makes it possible to set compression level for gzip:

```
config.Producer.CompressionLevel = flate.BestCompression
```

There are no other compression codecs that support levels,
but the code is generic to easily add other codecs with
numeric levels.

Default compression level tracks what Go libraries
set as the default level.
@bobrik
Copy link
Contributor Author

bobrik commented Feb 12, 2018

I started with *int for compression level with nil being the default, but that turned out to be ugly, because you can't take address of a constant like flat.BestCompression.

@eapache
Copy link
Contributor

eapache commented Feb 12, 2018

I'm not sure if it's worth throwing any checks into Validate() or if we should just rely on the libraries we call during encode()?

@bobrik
Copy link
Contributor Author

bobrik commented Feb 12, 2018

@eapache I added another commit with validation check:

2018/02/12 19:51:34 Error creating producer: kafka: invalid configuration (gzip compression does not work with level 12: gzip: invalid compression level: 12)

@eapache
Copy link
Contributor

eapache commented Feb 12, 2018

CI is going to fail because you're not accounting for the default value the same way in Validate as in encode

@bobrik bobrik force-pushed the compression-levels branch from 1d6d3b3 to 624d438 Compare February 12, 2018 19:56
@bobrik
Copy link
Contributor Author

bobrik commented Feb 12, 2018

I just added the check.

@eapache
Copy link
Contributor

eapache commented Feb 12, 2018

Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants