-
Notifications
You must be signed in to change notification settings - Fork 1.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Swapping out google.streaming for google-resumable-media in GCS downloads. #3323
Swapping out google.streaming for google-resumable-media in GCS downloads. #3323
Conversation
storage/setup.py
Outdated
@@ -52,6 +52,9 @@ | |||
|
|||
REQUIREMENTS = [ | |||
'google-cloud-core >= 0.24.0, < 0.25dev', | |||
'google-auth', |
This comment was marked as spam.
This comment was marked as spam.
Sorry, something went wrong.
This comment was marked as spam.
This comment was marked as spam.
Sorry, something went wrong.
storage/setup.py
Outdated
@@ -52,6 +52,9 @@ | |||
|
|||
REQUIREMENTS = [ | |||
'google-cloud-core >= 0.24.0, < 0.25dev', | |||
'google-auth', | |||
'google-resumable-media == 0.0.2', | |||
'requests', |
This comment was marked as spam.
This comment was marked as spam.
Sorry, something went wrong.
This comment was marked as spam.
This comment was marked as spam.
Sorry, something went wrong.
This comment was marked as spam.
This comment was marked as spam.
Sorry, something went wrong.
storage/google/cloud/storage/blob.py
Outdated
@@ -30,6 +30,9 @@ | |||
import six | |||
from six.moves.urllib.parse import quote | |||
|
|||
import google.auth.transport.requests as tr_requests |
This comment was marked as spam.
This comment was marked as spam.
Sorry, something went wrong.
This comment was marked as spam.
This comment was marked as spam.
Sorry, something went wrong.
storage/google/cloud/storage/blob.py
Outdated
@@ -30,6 +30,9 @@ | |||
import six | |||
from six.moves.urllib.parse import quote | |||
|
|||
import google.auth.transport.requests as tr_requests |
This comment was marked as spam.
This comment was marked as spam.
Sorry, something went wrong.
This comment was marked as spam.
This comment was marked as spam.
Sorry, something went wrong.
This comment was marked as spam.
This comment was marked as spam.
Sorry, something went wrong.
if self.chunk_size is None: | ||
download = resumable_media.Download(download_url, headers=headers) | ||
response = download.consume(transport) | ||
file_obj.write(response.content) |
This comment was marked as spam.
This comment was marked as spam.
Sorry, something went wrong.
This comment was marked as spam.
This comment was marked as spam.
Sorry, something went wrong.
This comment was marked as spam.
This comment was marked as spam.
Sorry, something went wrong.
This comment was marked as spam.
This comment was marked as spam.
Sorry, something went wrong.
This comment was marked as spam.
This comment was marked as spam.
Sorry, something went wrong.
This comment was marked as spam.
This comment was marked as spam.
Sorry, something went wrong.
This comment was marked as spam.
This comment was marked as spam.
Sorry, something went wrong.
This comment was marked as spam.
This comment was marked as spam.
Sorry, something went wrong.
download.chunksize = self.chunk_size | ||
|
||
# Get the download URL. | ||
download_url = self._get_download_url() |
This comment was marked as spam.
This comment was marked as spam.
Sorry, something went wrong.
This comment was marked as spam.
This comment was marked as spam.
Sorry, something went wrong.
This comment was marked as spam.
This comment was marked as spam.
Sorry, something went wrong.
This comment was marked as spam.
This comment was marked as spam.
Sorry, something went wrong.
This comment was marked as spam.
This comment was marked as spam.
Sorry, something went wrong.
This comment was marked as spam.
This comment was marked as spam.
Sorry, something went wrong.
I filed googleapis/google-resumable-media-python#5 to carry on the discussion about streaming the contents in a more memory-efficient manner (when possible). For now, I am going to merge this based on an LGTM from @jonparrott (given in chat) |
…oads. (googleapis#3323) * Swapping out google.streaming for google-resumable-media in GCS downloads. * Dropping tr_requests import alias in storage blob module. * Putting lower bounds on new deps in storage package.
Check out https://googlecloudplatform.github.io/google-resumable-media-python/0.0.2/ for details on the new library.