Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: use a larger chunk size when loading data #799

Merged
merged 3 commits into from
Jul 22, 2021

Conversation

jimfulton
Copy link
Contributor

@jimfulton jimfulton commented Jul 22, 2021

Thank you for opening a Pull Request! Before submitting your PR, there are a few things you can do to make sure it goes smoothly:

  • Make sure to open an issue as a bug/issue before writing your code! That way we can discuss the change, evaluate designs, and agree on the general idea
  • Ensure the tests and linter pass
  • Code coverage does not decrease (if any source code was changed)
  • Appropriate docs were updated (if necessary)

Fixes googleapis/google-resumable-media-python#238 🦕

@product-auto-label product-auto-label bot added the api: bigquery Issues related to the googleapis/python-bigquery API. label Jul 22, 2021
@google-cla google-cla bot added the cla: yes This human has signed the Contributor License Agreement. label Jul 22, 2021
@jimfulton jimfulton changed the title The chunk size used for data uploads was too small (1MB). Now it's 10… fix: The chunk size used for data uploads was too small Jul 22, 2021
@jimfulton jimfulton marked this pull request as ready for review July 22, 2021 18:13
@jimfulton jimfulton requested a review from a team July 22, 2021 18:13
@jimfulton jimfulton requested a review from a team as a code owner July 22, 2021 18:13
@jimfulton jimfulton requested review from stephaniewang526 and removed request for a team July 22, 2021 18:13
@tswast tswast mentioned this pull request Jul 22, 2021
@tswast tswast changed the title fix: The chunk size used for data uploads was too small fix: use a larger chunk size when loading data Jul 22, 2021
Copy link
Contributor

@tswast tswast left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@tswast tswast merged commit b804373 into master Jul 22, 2021
@tswast tswast deleted the riversnake-fix-chunk-size branch July 22, 2021 18:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: bigquery Issues related to the googleapis/python-bigquery API. cla: yes This human has signed the Contributor License Agreement.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Uploads Using Python Google Cloud Storage/Bigquery Client Libraries are Very Slow
3 participants