Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Swapping out google.streaming for google-resumable-media in GCS downloads. #3323

Merged
merged 3 commits into from
Apr 24, 2017

Conversation

dhermes
Copy link
Contributor

@dhermes dhermes commented Apr 24, 2017

@dhermes dhermes added the api: storage Issues related to the Cloud Storage API. label Apr 24, 2017
@googlebot googlebot added the cla: yes This human has signed the Contributor License Agreement. label Apr 24, 2017
storage/setup.py Outdated
@@ -52,6 +52,9 @@

REQUIREMENTS = [
'google-cloud-core >= 0.24.0, < 0.25dev',
'google-auth',

This comment was marked as spam.

This comment was marked as spam.

storage/setup.py Outdated
@@ -52,6 +52,9 @@

REQUIREMENTS = [
'google-cloud-core >= 0.24.0, < 0.25dev',
'google-auth',
'google-resumable-media == 0.0.2',
'requests',

This comment was marked as spam.

This comment was marked as spam.

This comment was marked as spam.

@@ -30,6 +30,9 @@
import six
from six.moves.urllib.parse import quote

import google.auth.transport.requests as tr_requests

This comment was marked as spam.

This comment was marked as spam.

@@ -30,6 +30,9 @@
import six
from six.moves.urllib.parse import quote

import google.auth.transport.requests as tr_requests

This comment was marked as spam.

This comment was marked as spam.

This comment was marked as spam.

if self.chunk_size is None:
download = resumable_media.Download(download_url, headers=headers)
response = download.consume(transport)
file_obj.write(response.content)

This comment was marked as spam.

This comment was marked as spam.

This comment was marked as spam.

This comment was marked as spam.

This comment was marked as spam.

This comment was marked as spam.

This comment was marked as spam.

This comment was marked as spam.

download.chunksize = self.chunk_size

# Get the download URL.
download_url = self._get_download_url()

This comment was marked as spam.

This comment was marked as spam.

This comment was marked as spam.

This comment was marked as spam.

This comment was marked as spam.

This comment was marked as spam.

@dhermes
Copy link
Contributor Author

dhermes commented Apr 24, 2017

I filed googleapis/google-resumable-media-python#5 to carry on the discussion about streaming the contents in a more memory-efficient manner (when possible).

For now, I am going to merge this based on an LGTM from @jonparrott (given in chat)

@dhermes dhermes merged commit d0ef6a9 into googleapis:master Apr 24, 2017
@dhermes dhermes deleted the resumable-media-for-GCS-downloads branch April 24, 2017 22:36
richkadel pushed a commit to richkadel/google-cloud-python that referenced this pull request May 6, 2017
…oads. (googleapis#3323)

* Swapping out google.streaming for google-resumable-media in GCS downloads.

* Dropping tr_requests import alias in storage blob module.

* Putting lower bounds on new deps in storage package.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: storage Issues related to the Cloud Storage API. cla: yes This human has signed the Contributor License Agreement.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants