Support for Gzip/Inflate

[Bonias]

Hi,

I implemented gzip/inflate decompression. It resolves #113.

Before:

HTTP.headers('Accept-Encoding' => 'gzip').get("https://github.com").to_s[0,12]
 => "\u001F\x8B\b\u0000\u0000\u0000\u0000\u0000\u0000\u0003\xED]"

After:

HTTP.headers('Accept-Encoding' => 'gzip').get("https://github.com").to_s[0,12]
=> "\n\n\n<!DOCTYPE"

Please, let me know if I need to change something here.

[ixti]

Awesome! Thanks! Couple of notes though...

This should not be auto-enabled. User should explicitly tell to use inflater on incoming data. Probably something like this:

HTTP.use(:auto_inflate).headers("Accept-Encoding" => "gzip").get("https://github.com")

Sometimes one may want not to inflate body and keep it as is. Also it would make sense to add deflate support as well. Second thing is we should NEVER modify response headers, regardless if body was auto-inflated or not.

I see inflater to be an IO wrapper. Or filter. So that it would be possible to re-use for example apart from HTTP gem.

[tarcieri]

This should not be auto-enabled.

👍 on that. Compression over TLS can lead to compression oracles (ala TIME, CRIME, BREACH)

[Bonias]

Second thing is we should NEVER modify response headers, regardless if body was auto-inflated or not

Are you sure about that? The body is no longer encoded, so Content-Encoding header can be misleading. Also it is what Net::HTTP is doing (https://github.com/ruby/ruby/blob/trunk/lib/net/http/response.rb#L257) now. In fact, this is the reason why I implemented it in such way. But I will change it if you certain of that.

[ixti]

@Bonias Hm. Good call, but I think Net::HTTP is doing that wrong.

[britishtea]

I think it's worth considering to enable it by default, even going as far as setting Accept-Encoding by default unless:

• Accept-Encoding is already set
• Cache-Control is set and contains no-transform
• the scheme is not "http"

To make it possible for clients to retrieve the compressed body, perhaps Response::Body#to_s can take an optional parameter to disable automatic inflating?

I had a stab at implementing this a while ago, but got stuck on writing proper tests and kind of forgot. Deflate seemed to come in two flavours.

[Bonias]

I updated PR. We can turn decoding on by setting use(:auto_inflate):

HTTP.use(:auto_inflate).headers("Accept-Encoding" => "gzip").get("https://github.com").to_s

We can turn encoding on (gzip by default):

HTTP.use(:auto_deflate).post("https://github.com", body: "Some body").to_s

Or both features at once:

HTTP.use(:auto_inflate, :auto_deflate).headers("Accept-Encoding" => "gzip").post("https://github.com", body: "Some body")

We can also use deflate instead of gzip:

HTTP.use(auto_deflate: { method: :deflate }).post("https://github.com", body: "Some body")

Unfortunately rubocop is reporting 3 offenses. I'm not sure how to fix them, but maybe I'll handle that later, when you will not have any other remarks to this PR.

What do you think?

[ixti]

So far looks good. Have some strange feelings, but can't formulate them yet -will take another deeper look later today or tomorrow.

[Overload119]

Ship it!

[tarcieri]

This looks good to me.

Regarding the RuboCop offenses, I think we should just disable ClassLength checks on the offending classes.

Re: MethodLength, I'm fine turning that up to 25 globally.

[ixti]

Sorry got lots of things to do this week. Will provide my input tomorrow.

[Bonias]

I've changed rubocop config for now.

[Bonias]

I missed JRuby failed tests before. They are fixed now.

[Bonias]

Hi guys. Is there anything else what is blocking this PR?

[tarcieri]

It looks ok to me. I'm down to merge it if @ixti is

[ixti]

Sorry, completely overlooked this one. Thanks again for all your hard work! Merged!