Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(storage): new option to disable decompressive transcoding #8834

Merged

Conversation

coryan
Copy link
Contributor

@coryan coryan commented Apr 30, 2022

For objects stored in gzip format (and with contentEncoding == "gzip")
GCS over HTTP automatically decompresses the object during download.
Some applications may want to read the object in compressed format.
Support this use-case with a new option (AcceptEncoding) for
Client::ReadObject(), and a new helper function
(AcceptEncodingGzip()) that returns this option with the correct value.

Fixes #8305


This change is Reviewable

For objects stored in gzip format (and with contentEncoding == "gzip")
GCS over HTTP automatically decompresses the object during download.
Some applications may want to read the object in compressed format.
Support this use-case with a new option (`AcceptEncoding`) for
`Client::ReadObject()`, and a new helper function (`AcceptEncodingGzip()
`) that returns this option with the correct value.
@product-auto-label product-auto-label bot added the api: storage Issues related to the Cloud Storage API. label Apr 30, 2022
@google-cloud-cpp-bot
Copy link
Collaborator

Google Cloud Build Logs
For commit: 94d49f02502934ab19935bdab0b6516be95d7eb7

ℹ️ NOTE: Kokoro logs are linked from "Details" below.

@codecov
Copy link

codecov bot commented Apr 30, 2022

Codecov Report

Merging #8834 (b935ed4) into main (c16f539) will decrease coverage by 0.03%.
The diff coverage is 27.94%.

@@            Coverage Diff             @@
##             main    #8834      +/-   ##
==========================================
- Coverage   94.50%   94.46%   -0.04%     
==========================================
  Files        1356     1356              
  Lines      120137   120205      +68     
==========================================
+ Hits       113533   113553      +20     
- Misses       6604     6652      +48     
Impacted Files Coverage Δ
google/cloud/storage/client.h 99.79% <ø> (ø)
google/cloud/storage/internal/object_requests.h 97.29% <ø> (ø)
...ests/decompressive_transcoding_integration_test.cc 11.50% <7.54%> (-3.50%) ⬇️
...e/cloud/storage/examples/storage_object_samples.cc 91.16% <100.00%> (+0.25%) ⬆️
google/cloud/storage/internal/curl_client.cc 98.82% <100.00%> (+<0.01%) ⬆️
google/cloud/storage/well_known_headers.h 93.54% <100.00%> (+0.44%) ⬆️
google/cloud/pubsub/samples/samples.cc 92.10% <0.00%> (+0.07%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c16f539...b935ed4. Read the comment docs.

@google-cloud-cpp-bot
Copy link
Collaborator

Google Cloud Build Logs
For commit: ed4e3775b1a1c2a01e7315b0d17d860778adc9dc

ℹ️ NOTE: Kokoro logs are linked from "Details" below.

@coryan coryan marked this pull request as ready for review April 30, 2022 21:51
@coryan coryan requested a review from a team as a code owner April 30, 2022 21:51
Copy link
Member

@dbolduc dbolduc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 5 of 6 files at r1, 1 of 1 files at r2, all commit messages.
Reviewable status: all files reviewed, 1 unresolved discussion (waiting on @coryan)


google/cloud/storage/well_known_headers.h line 295 at r2 (raw file):

 *     compressed with `gzip` and have their `content_encoding()` attribute
 *     set accordingly. At the time of this writing GCS does not decompress
 *     objects stored with other compression algorithms, nor does not it

s/nor does not it/nor does it/

@coryan coryan enabled auto-merge (squash) May 1, 2022 14:43
@google-cloud-cpp-bot
Copy link
Collaborator

Google Cloud Build Logs
For commit: b935ed4c765cf5baebde66005993e02e5231abed

ℹ️ NOTE: Kokoro logs are linked from "Details" below.

@coryan coryan merged commit 8658400 into googleapis:main May 1, 2022
@coryan coryan deleted the feat-storage-disable-decompressive-transcoding branch May 1, 2022 15:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: storage Issues related to the Cloud Storage API.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add option to inject Accept-Encoding: gzip.
3 participants