-
Notifications
You must be signed in to change notification settings - Fork 1.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Blob.upload_from_file doesn't work for file-like objects without size parameter #900
Comments
Can you give a stacktrace of the error message? The offending line is # Get the basic stats about the file.
total_bytes = size or os.fstat(file_obj.fileno()).st_size which as @jonparrott points out, assumes the file-like object is a |
|
Ha, ok. Doesn't provide anything we didn't know. |
Heh, let me know if there's anything else I can provide to help here. The file-like object is a flask file upload (werkzeug.FileStorage). It fails likewise when using |
In storage, file upload will now fail if no size is passed in and if the file object does not have a fileno which can be stat'ed. Fixes googleapis#900.
Source-Link: googleapis/googleapis@3820562 Source-Link: googleapis/googleapis-gen@a2cd2c3 Copy-Tag: eyJwIjoicGFja2FnZXMvZ29vZ2xlLWNsb3VkLWNvbXB1dGUvLk93bEJvdC55YW1sIiwiaCI6ImEyY2QyYzNiYWNiZjlhZWNmN2RkMWY4ODQ1MGE2YTE0NTdlMjQzOGEifQ==
…240430 (#900) (#12648) - [ ] Regenerate this pull request now. Source-Link: googleapis/googleapis@3820562 Source-Link: googleapis/googleapis-gen@a2cd2c3 Copy-Tag: eyJwIjoicGFja2FnZXMvZ29vZ2xlLWNsb3VkLWNvbXB1dGUvLk93bEJvdC55YW1sIiwiaCI6ImEyY2QyYzNiYWNiZjlhZWNmN2RkMWY4ODQ1MGE2YTE0NTdlMjQzOGEifQ== BEGIN_NESTED_COMMIT feat: [google-cloud-compute] Update Compute Engine API to revision 20240421 (#899) Source-Link: googleapis/googleapis@55499b5 Source-Link: googleapis/googleapis-gen@cf5049b Copy-Tag: eyJwIjoicGFja2FnZXMvZ29vZ2xlLWNsb3VkLWNvbXB1dGUvLk93bEJvdC55YW1sIiwiaCI6ImNmNTA0OWI3MDc5MjgyMDA2NWRiMzhlNzEyN2YzMmVhYjc3MDU5NDQifQ== END_NESTED_COMMIT BEGIN_NESTED_COMMIT feat: [google-cloud-compute] Update Compute Engine API to revision 20240407 (#890) Source-Link: googleapis/googleapis@018c0e3 Source-Link: googleapis/googleapis-gen@dad8476 Copy-Tag: eyJwIjoicGFja2FnZXMvZ29vZ2xlLWNsb3VkLWNvbXB1dGUvLk93bEJvdC55YW1sIiwiaCI6ImRhZDg0NzZjOTRlYjFlY2IyZmQ3MmYyOTY4MjVlOTYyMWVhOGI0NWQifQ== END_NESTED_COMMIT --------- Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com> Co-authored-by: Anthonios Partheniou <[email protected]>
🤖 I have created a release *beep* *boop* --- <details><summary>google-apps-chat: 0.1.6</summary> ## [0.1.6](google-apps-chat-v0.1.5...google-apps-chat-v0.1.6) (2024-05-16) ### Documentation * [google-apps-chat] update Chat API comments ([#12694](#12694)) ([f66440d](f66440d)) </details> <details><summary>google-cloud-batch: 0.17.20</summary> ## [0.17.20](google-cloud-batch-v0.17.19...google-cloud-batch-v0.17.20) (2024-05-16) ### Features * Add UpdateJob API to update the job spec, only task_count is supported now ([b855629](b855629)) * Update description on allowed_locations in LocationPolicy field ([b855629](b855629)) ### Documentation * [google-cloud-batch] Refine description for field `task_execution` ([f149fb8](f149fb8)) * [google-cloud-batch] Refine description for field `task_execution` ([#12693](#12693)) ([f149fb8](f149fb8)) * updated comments ([b855629](b855629)) </details> <details><summary>google-cloud-compute: 1.19.0</summary> ## [1.19.0](google-cloud-compute-v1.18.0...google-cloud-compute-v1.19.0) (2024-05-16) ### Features * [google-cloud-compute] Update Compute Engine API to revision ([f515bc6](f515bc6)) * [google-cloud-compute] Update Compute Engine API to revision ([f515bc6](f515bc6)) * [google-cloud-compute] Update Compute Engine API to revision 20240430 ([#900](#900)) ([#12648](#12648)) ([f515bc6](f515bc6)) </details> <details><summary>google-cloud-dataplex: 2.0.0</summary> ## [2.0.0](google-cloud-dataplex-v1.13.0...google-cloud-dataplex-v2.0.0) (2024-05-16) ### ⚠ BREAKING CHANGES * An existing field `entry` is removed from message `.google.cloud.dataplex.v1.SearchEntriesResult` * An existing field `display_name` is removed from message `.google.cloud.dataplex.v1.SearchEntriesResult` * An existing field `entry_type` is removed from message `.google.cloud.dataplex.v1.SearchEntriesResult` * An existing field `modify_time` is removed from message `.google.cloud.dataplex.v1.SearchEntriesResult` * An existing field `fully_qualified_name` is removed from message `.google.cloud.dataplex.v1.SearchEntriesResult` * An existing field `description` is removed from message `.google.cloud.dataplex.v1.SearchEntriesResult` * An existing field `relative_resource` is removed from message `.google.cloud.dataplex.v1.SearchEntriesResult` ### Features * updated client libraries for Dataplex Catalog: removed deprecated fields, updated comments ([fd6e39c](fd6e39c)) ### Bug Fixes * An existing field `description` is removed from message `.google.cloud.dataplex.v1.SearchEntriesResult` ([fd6e39c](fd6e39c)) * An existing field `display_name` is removed from message `.google.cloud.dataplex.v1.SearchEntriesResult` ([fd6e39c](fd6e39c)) * An existing field `entry_type` is removed from message `.google.cloud.dataplex.v1.SearchEntriesResult` ([fd6e39c](fd6e39c)) * An existing field `entry` is removed from message `.google.cloud.dataplex.v1.SearchEntriesResult` ([fd6e39c](fd6e39c)) * An existing field `fully_qualified_name` is removed from message `.google.cloud.dataplex.v1.SearchEntriesResult` ([fd6e39c](fd6e39c)) * An existing field `modify_time` is removed from message `.google.cloud.dataplex.v1.SearchEntriesResult` ([fd6e39c](fd6e39c)) * An existing field `relative_resource` is removed from message `.google.cloud.dataplex.v1.SearchEntriesResult` ([fd6e39c](fd6e39c)) ### Documentation * A comment for field `aspects` in message `.google.cloud.dataplex.v1.Entry` is changed ([fd6e39c](fd6e39c)) * A comment for field `filter` in message `.google.cloud.dataplex.v1.ListEntriesRequest` is changed ([fd6e39c](fd6e39c)) </details> <details><summary>google-cloud-deploy: 1.19.0</summary> ## [1.19.0](google-cloud-deploy-v1.18.1...google-cloud-deploy-v1.19.0) (2024-05-16) ### Features * [google-cloud-deploy] add Skaffold verbose support to Execution Environment properties ([#12701](#12701)) ([03c7b0c](03c7b0c)) </details> <details><summary>google-cloud-documentai: 2.27.1</summary> ## [2.27.1](google-cloud-documentai-v2.27.0...google-cloud-documentai-v2.27.1) (2024-05-16) ### Documentation * [google-cloud-documentai] clarify the unavailability of some features ([#12699](#12699)) ([3ec5a2c](3ec5a2c)) * updated comments ([9722673](9722673)) </details> <details><summary>google-cloud-gke-multicloud: 0.6.10</summary> ## [0.6.10](google-cloud-gke-multicloud-v0.6.9...google-cloud-gke-multicloud-v0.6.10) (2024-05-16) ### Features * [google-cloud-gke-multicloud] option to ignore_errors while deleting Azure clusters / nodepools ([#12686](#12686)) ([3d1901e](3d1901e)) </details> <details><summary>google-cloud-kms: 2.23.0</summary> ## [2.23.0](google-cloud-kms-v2.22.0...google-cloud-kms-v2.23.0) (2024-05-16) ### Features * add client library for KMS Autokey service, which enables automated KMS key provision and management ([b74c6c2](b74c6c2)) </details> <details><summary>google-cloud-parallelstore: 0.2.0</summary> ## [0.2.0](google-cloud-parallelstore-v0.1.2...google-cloud-parallelstore-v0.2.0) (2024-05-16) ### ⚠ BREAKING CHANGES * An existing field `end_time` is removed from message `.google.cloud.parallelstore.v1beta.TransferOperationMetadata` * An existing field `source` is removed from message `.google.cloud.parallelstore.v1beta.TransferOperationMetadata` * An existing field `destination` is removed from message `.google.cloud.parallelstore.v1beta.TransferOperationMetadata` * [google-cloud-parallelstore] An existing field `create_time` is removed from message `.google.cloud.parallelstore.v1beta.TransferOperationMetadata` * An existing field `destination_path` is renamed to `destination_parallelstore` in message `.google.cloud.parallelstore.v1beta.ImportDataRequest` * An existing field `source_path` is renamed to `source_parallelstore` in message `.google.cloud.parallelstore.v1beta.ExportDataRequest` * An existing field `destination_gcs_uri` is renamed to `destination_gcs_bucket` in message `.google.cloud.parallelstore.v1beta.ExportDataRequest` * An existing field `source_gcs_uri` is renamed to `source_gcs_bucket` in message `.google.cloud.parallelstore.v1beta.ImportDataRequest` ### Features * A new field `api_version` is added to message `.google.cloud.parallelstore.v1beta.ExportDataMetadata` ([2f57a44](2f57a44)) * A new field `api_version` is added to message `.google.cloud.parallelstore.v1beta.ImportDataMetadata` ([2f57a44](2f57a44)) * A new field `create_time` is added to message `.google.cloud.parallelstore.v1beta.ExportDataMetadata` ([2f57a44](2f57a44)) * A new field `create_time` is added to message `.google.cloud.parallelstore.v1beta.ImportDataMetadata` ([2f57a44](2f57a44)) * A new field `destination_gcs_bucket` is added to message `.google.cloud.parallelstore.v1beta.TransferOperationMetadata` ([2f57a44](2f57a44)) * A new field `destination_parallelstore` is added to message `.google.cloud.parallelstore.v1beta.TransferOperationMetadata` ([2f57a44](2f57a44)) * A new field `end_time` is added to message `.google.cloud.parallelstore.v1beta.ExportDataMetadata` ([2f57a44](2f57a44)) * A new field `end_time` is added to message `.google.cloud.parallelstore.v1beta.ImportDataMetadata` ([2f57a44](2f57a44)) * A new field `requested_cancellation` is added to message `.google.cloud.parallelstore.v1beta.ExportDataMetadata` ([2f57a44](2f57a44)) * A new field `requested_cancellation` is added to message `.google.cloud.parallelstore.v1beta.ImportDataMetadata` ([2f57a44](2f57a44)) * A new field `source_gcs_bucket` is added to message `.google.cloud.parallelstore.v1beta.TransferOperationMetadata` ([2f57a44](2f57a44)) * A new field `source_parallelstore` is added to message `.google.cloud.parallelstore.v1beta.TransferOperationMetadata` ([2f57a44](2f57a44)) * A new field `status_message` is added to message `.google.cloud.parallelstore.v1beta.ExportDataMetadata` ([2f57a44](2f57a44)) * A new field `status_message` is added to message `.google.cloud.parallelstore.v1beta.ImportDataMetadata` ([2f57a44](2f57a44)) * A new field `target` is added to message `.google.cloud.parallelstore.v1beta.ExportDataMetadata` ([2f57a44](2f57a44)) * A new field `target` is added to message `.google.cloud.parallelstore.v1beta.ImportDataMetadata` ([2f57a44](2f57a44)) * A new field `verb` is added to message `.google.cloud.parallelstore.v1beta.ExportDataMetadata` ([2f57a44](2f57a44)) * A new field `verb` is added to message `.google.cloud.parallelstore.v1beta.ImportDataMetadata` ([2f57a44](2f57a44)) * A new message `DestinationGcsBucket` is added ([e3c48d7](e3c48d7)) * A new message `DestinationParallelstore` is added ([e3c48d7](e3c48d7)) * A new message `SourceGcsBucket` is added ([e3c48d7](e3c48d7)) * A new message `SourceParallelstore` is added ([e3c48d7](e3c48d7)) ### Bug Fixes * [google-cloud-parallelstore] An existing field `create_time` is removed from message `.google.cloud.parallelstore.v1beta.TransferOperationMetadata` ([2f57a44](2f57a44)) * An existing field `destination_gcs_uri` is renamed to `destination_gcs_bucket` in message `.google.cloud.parallelstore.v1beta.ExportDataRequest` ([e3c48d7](e3c48d7)) * An existing field `destination_path` is renamed to `destination_parallelstore` in message `.google.cloud.parallelstore.v1beta.ImportDataRequest` ([e3c48d7](e3c48d7)) * An existing field `destination` is removed from message `.google.cloud.parallelstore.v1beta.TransferOperationMetadata` ([2f57a44](2f57a44)) * An existing field `end_time` is removed from message `.google.cloud.parallelstore.v1beta.TransferOperationMetadata` ([2f57a44](2f57a44)) * An existing field `source_gcs_uri` is renamed to `source_gcs_bucket` in message `.google.cloud.parallelstore.v1beta.ImportDataRequest` ([e3c48d7](e3c48d7)) * An existing field `source_path` is renamed to `source_parallelstore` in message `.google.cloud.parallelstore.v1beta.ExportDataRequest` ([e3c48d7](e3c48d7)) * An existing field `source` is removed from message `.google.cloud.parallelstore.v1beta.TransferOperationMetadata` ([2f57a44](2f57a44)) ### Documentation * A comment for field `counters` in message `.google.cloud.parallelstore.v1beta.TransferOperationMetadata` is changed ([2f57a44](2f57a44)) * A comment for field `transfer_type` in message `.google.cloud.parallelstore.v1beta.TransferOperationMetadata` is changed ([2f57a44](2f57a44)) </details> <details><summary>google-cloud-public-ca: 0.3.10</summary> ## [0.3.10](google-cloud-public-ca-v0.3.9...google-cloud-public-ca-v0.3.10) (2024-05-16) ### Features * added protos for publicca v1 ([f38aae4](f38aae4)) </details> <details><summary>google-cloud-service-directory: 1.11.4</summary> ## [1.11.4](google-cloud-service-directory-v1.11.3...google-cloud-service-directory-v1.11.4) (2024-05-16) ### Documentation * add maximum page_size to ListEndpoint API documentation ([f8ff81b](f8ff81b)) * add maximum page_size to ListNamespace API documentation ([f8ff81b](f8ff81b)) * add maximum page_size to ListService API documentation ([f8ff81b](f8ff81b)) </details> <details><summary>google-cloud-visionai: 0.1.0</summary> ## 0.1.0 (2024-05-16) ### Features * add initial files for google.cloud.visionai.v1 ([#12669](#12669)) ([33215ca](33215ca)) </details> <details><summary>google-maps-fleetengine: 0.2.0</summary> ## [0.2.0](google-maps-fleetengine-v0.1.9...google-maps-fleetengine-v0.2.0) (2024-05-16) ### ⚠ BREAKING CHANGES * An existing method `SearchFuzzedVehicles` is removed from service `VehicleService` * An existing message `UpdateVehicleLocationRequest` is removed * An existing method `UpdateVehicleLocation` is removed from service `VehicleService` ### Bug Fixes * An existing message `UpdateVehicleLocationRequest` is removed ([e6969d5](e6969d5)) * An existing method `SearchFuzzedVehicles` is removed from service `VehicleService` ([e6969d5](e6969d5)) * An existing method `UpdateVehicleLocation` is removed from service `VehicleService` ([e6969d5](e6969d5)) ### Documentation * [google-maps-fleetengine] mark TerminalPointId as deprecated ([#12698](#12698)) ([262ef80](262ef80)) </details> <details><summary>google-maps-fleetengine-delivery: 0.2.0</summary> ## [0.2.0](google-maps-fleetengine-delivery-v0.1.8...google-maps-fleetengine-delivery-v0.2.0) (2024-05-16) ### ⚠ BREAKING CHANGES * An existing method `SearchFuzzedVehicles` is removed from service `VehicleService` * An existing message `UpdateVehicleLocationRequest` is removed * An existing method `UpdateVehicleLocation` is removed from service `VehicleService` ### Bug Fixes * An existing message `UpdateVehicleLocationRequest` is removed ([6c696a2](6c696a2)) * An existing method `SearchFuzzedVehicles` is removed from service `VehicleService` ([6c696a2](6c696a2)) * An existing method `UpdateVehicleLocation` is removed from service `VehicleService` ([6c696a2](6c696a2)) ### Documentation * remove comment about deleted SearchTasks method ([11fb7ce](11fb7ce)) </details> <details><summary>google-maps-routeoptimization: 0.1.0</summary> ## 0.1.0 (2024-05-16) ### Features * add initial files for google.maps.routeoptimization.v1 ([#12670](#12670)) ([524cd1e](524cd1e)) </details> --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please). Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com>
When using
StringIO
,io.BytesIO
or other file-like objects (such as Flask/werkzeug's file upload object), trying to upload viaBlob.upload_from_file
doesn't work because it attempts to find the size of the object usingos.stat
throwing aAttributeError: StringIO instance has no attribute 'fileno'
. Passing in the size manually fixes this, but imo this inconsistency shouldn't burden the user as there are other ways than fileno to determine the size of a StringIO or io.BytesIO instance (seek
,tell
).At any rate, the error message could be more helpful.
The text was updated successfully, but these errors were encountered: