Skip to content
This repository has been archived by the owner on Nov 8, 2024. It is now read-only.

Eliminate GZIP compression on 32nd chunk during upload #56

Closed
nblair opened this issue Feb 20, 2020 · 0 comments · Fixed by #57
Closed

Eliminate GZIP compression on 32nd chunk during upload #56

nblair opened this issue Feb 20, 2020 · 0 comments · Fixed by #57

Comments

@nblair
Copy link
Contributor

nblair commented Feb 20, 2020

Version 1.104.0 of the google-cloud-storage library includes an improvement that eliminates a bit of technical debt in the code to upload the blob content to the storage bucket.

In the event we hit the 32nd "chunk" of the file, the previous version of the library would attempt to GZIP the remainder of the stream on the way to the bucket. The compression would result in a CPU spike for these larger files and lower overall throughput.

Version 1.104 includes the ability to skip GZIP compression. This is very relevant to the plugin as pretty much every single blob we store is already compressed in some way (jar, war, gzip, docker layers, etc). Updating to this version of the library will avoid the CPU spike and potentially increase overall throughput.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
1 participant