Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Support blob streaming for file-like objects #1914

Merged
merged 3 commits into from
Jul 14, 2016
Merged

Support blob streaming for file-like objects #1914

merged 3 commits into from
Jul 14, 2016

Conversation

linar-jether
Copy link
Contributor

When uploading file-like objects without a specified size, upload in chunks instead of exhausting the stream and keeping the data in-memory.

See #1909

@googlebot
Copy link

Thanks for your pull request. It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

📝 Please visit https://cla.developers.google.com/ to sign.

Once you've signed, please reply here (e.g. I signed it!) and we'll verify. Thanks.

  • If you've already signed a CLA, it's possible we don't have your GitHub username or you're using a different email address. Check your existing CLA data and verify that your email is set on your git commits.
  • If you signed the CLA as a corporation, please let us know the company's name.

@googlebot googlebot added the cla: no This human has *not* signed the Contributor License Agreement. label Jun 27, 2016
@linar-jether
Copy link
Contributor Author

Signed it

@googlebot
Copy link

CLAs look good, thanks!

@googlebot googlebot added cla: yes This human has signed the Contributor License Agreement. and removed cla: no This human has *not* signed the Contributor License Agreement. labels Jun 27, 2016
@tseaver tseaver added type: feature request ‘Nice-to-have’ improvement, new feature or different behavior or design. api: storage Issues related to the Cloud Storage API. labels Jun 27, 2016
tseaver
tseaver reviewed Jul 11, 2016
View reviewed changes
gcloud/streaming/buffered_stream.py
self._buffered_data = self._stream.read(size)
self._buffered_data = self._stream.read(size) \
if not hasattr(self._stream,
'closed') or not self._stream.closed else ''

This comment was marked as spam.

Sign in to view

@linar-jether
Copy link
Contributor Author

fixed according to comments

@tseaver
Copy link
Contributor

tseaver commented Jul 12, 2016

@linar-jether thanks for the changes. The pylint failures on Travis aren't due to this PR: they released a new version last Friday, which introduced new checks that we now fail. The master branch now has fixes for those failures: could you rebase your branch against it and re-push?

@linar-jether
Copy link
Contributor Author

All tests passed

daspecster
daspecster reviewed Jul 13, 2016
View reviewed changes
gcloud/storage/blob.py
@@ -17,7 +17,7 @@
import base64
import copy
import hashlib
from io import BytesIO
from io import BytesIO, UnsupportedOperation

This comment was marked as spam.

Sign in to view

@daspecster
Copy link
Contributor

@linar-jether, cool! Do you think you could squash these commits down?
Also, I noticed the imports issue.

Otherwise LGTM, but I defer to @tseaver/@dhermes for confirmation.

@linar-jether linar-jether force-pushed the master branch 2 times, most recently from 742a95e to 43464b7 Compare July 14, 2016 08:29
@linar-jether
Copy link
Contributor Author

Sure, squashed all previous commits and separated the imports

@tseaver tseaver merged commit bd6435a into googleapis:master Jul 14, 2016
@tseaver tseaver mentioned this pull request Jul 14, 2016
Closed
@tseaver
Copy link
Contributor

tseaver commented Jul 14, 2016

@linar-jether Thanks again for your work!

This was referenced Jul 18, 2016
Merged
Merged
This was referenced Aug 3, 2016
Closed
Merged
@linar-jether linar-jether mentioned this pull request Jun 27, 2017
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
api: storage Issues related to the Cloud Storage API. cla: yes This human has signed the Contributor License Agreement. type: feature request ‘Nice-to-have’ improvement, new feature or different behavior or design.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@linar-jether @googlebot @tseaver @daspecster