Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(bigquery): fix inserting missing repeated fields #10196

Merged
merged 4 commits into from
Jan 31, 2020

Conversation

plamut
Copy link
Contributor

@plamut plamut commented Jan 24, 2020

Fixes #9602.

This PR fixes the issue when inserting rows with missing REPEATED fields. Omitting a REPEATED field from the request does not result in an error anymore.

How to test

Best to run the ready-made script from one of the issue comments.

MIND:
Omitting None values only happens when the code needs to convert the data to JSON by itself, i.e. in insert_rows(). The following thus still fails when insert_rows_json() is used:

rows = [{"int_col": 9, "repeated_col": None}
errors = client.insert_rows_json(TABLE_PATH, rows)

If this inconsistency is not desired, we need to sync the logic in these two methods.

PR checklist

  • Make sure to open an issue as a bug/issue before writing your code! That way we can discuss the change, evaluate designs, and agree on the general idea
  • Ensure the tests and linter pass
  • Code coverage does not decrease (if any source code was changed)
  • Appropriate docs were updated (if necessary)

@plamut plamut added the api: bigquery Issues related to the BigQuery API. label Jan 24, 2020
@plamut plamut requested review from tswast and a team January 24, 2020 12:09
@googlebot googlebot added the cla: yes This human has signed the Contributor License Agreement. label Jan 24, 2020
@plamut
Copy link
Contributor Author

plamut commented Jan 24, 2020

The failing snippets test is unrelated, #10195 will fix it,

@plamut plamut added the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Jan 29, 2020
@yoshi-kokoro yoshi-kokoro removed the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Jan 29, 2020
@plamut
Copy link
Contributor Author

plamut commented Jan 29, 2020

The unit tests failure will be fixed in #10222.

@plamut plamut added the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Jan 30, 2020
@yoshi-kokoro yoshi-kokoro removed the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Jan 30, 2020
@plamut plamut added the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Jan 30, 2020
@yoshi-kokoro yoshi-kokoro removed kokoro:force-run Add this label to force Kokoro to re-run the tests. labels Jan 30, 2020
@plamut
Copy link
Contributor Author

plamut commented Jan 30, 2020

Weird, a couple of PermissionDenied errors in system tests. Re-running the jobs.

Update: I cannot reproduce this locally, neither on the PR branch, nor on on the latest master. Hmm...

Copy link
Contributor

@tswast tswast left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One nit, otherwise LGTM

bigquery/tests/unit/test_client.py Show resolved Hide resolved
This was referenced Feb 2, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: bigquery Issues related to the BigQuery API. cla: yes This human has signed the Contributor License Agreement.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

BigQuery: insert_rows() fails when a repeated field is missing
4 participants