-
Notifications
You must be signed in to change notification settings - Fork 48
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add requests/urllib3 work-around for intercepting gzipped bytes. #36
Conversation
Go for it. |
@@ -18,6 +18,8 @@ | |||
import hashlib | |||
import logging | |||
|
|||
import urllib3.response |
This comment was marked as spam.
This comment was marked as spam.
Sorry, something went wrong.
I would suggest that rather than just Cc'ing the |
since the caller will no longer need to hash to decoded bytes. | ||
""" | ||
encoding = response_raw.headers.get(u'content-encoding', u'').lower() | ||
if encoding != u'gzip': |
This comment was marked as spam.
This comment was marked as spam.
Sorry, something went wrong.
This comment was marked as spam.
This comment was marked as spam.
Sorry, something went wrong.
This comment was marked as spam.
This comment was marked as spam.
Sorry, something went wrong.
This comment was marked as spam.
This comment was marked as spam.
Sorry, something went wrong.
This comment was marked as spam.
This comment was marked as spam.
Sorry, something went wrong.
This comment was marked as spam.
This comment was marked as spam.
Sorry, something went wrong.
@Lukasa @shazow We're curious what you think of this hack. For Google Cloud Storage (and likely other media APIs with checksumming) we "need" simultaneous access to the (gzip) compressed and uncompressed bytes so we can compute the checksum of the gzipped content (which is what is actually stored in "the cloud") while streaming the uncompressed bytes to a stream for the user. |
Opened #37 about the deflate encoding issue noted above. |
@dhermes Looks like you did what you could with the API we have. :) It really does feel like the decoder stuff should be a registry, doesn't it? |
Fixes #34.
/cc @tseaver @mfschwartz
@jonparrott Do you think we should cc the
requests
/urllib3
maintainers? I'm also working on a PR to fix the failing system test, but it is not going to be part of this PR.