Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: handle case when expirationMs is None #1553

Merged

Conversation

abdelmegahed
Copy link
Contributor

Right now, unsetting ['timePartitioning']['expirationMs'] on BigQuery table is not supported and raises google.api_core.exceptions.BadRequest: 400 PATCH https://bigquery.googleapis.com/bigquery/v2/projects/abdelrahmanm-bd-dev/datasets/partition_expiry_test/tables/sample_data_partitioned?prettyPrint=false: Invalid value at 'table.time_partitioning.expiration_ms.value' (TYPE_INT64), "None"

This happens since the backend doesn't treat "None" (string) as a null. This PR addresses this issue from the client side.

Sample code that fails with the above exception:

from google.cloud import bigquery
client = bigquery.Client(project='abdelrahmanm-bd-dev')
table_ref = client.dataset('partition_expiry_test').table('sample_data_partitioned')
table = client.get_table(table_ref)
table.partition_expiration = None # ISSUE HERE!! This get cast to string and the backend returns a 400 as a result.
client.update_table(table, ['partition_expiration'])

@abdelmegahed abdelmegahed requested review from a team as code owners April 17, 2023 20:16
@product-auto-label product-auto-label bot added size: xs Pull request size is extra small. api: bigquery Issues related to the googleapis/python-bigquery API. labels Apr 17, 2023
Copy link
Contributor

@tswast tswast left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @abdelmegahed

Could you write a unit test covering this case? I'd expect one similar to https://github.com/googleapis/python-bigquery/blob/main/tests/unit/test_table.py#L1178 test_to_api_repr_w_custom_field perhaps called test_to_api_repr_w_unsetting_expiration

@product-auto-label product-auto-label bot added size: s Pull request size is small. and removed size: xs Pull request size is extra small. labels Apr 18, 2023
@abdelmegahed
Copy link
Contributor Author

@tswast test added. Thanks for the pointers :)

@tswast tswast added the kokoro:run Add this label to force Kokoro to re-run the tests. label Apr 18, 2023
@yoshi-kokoro yoshi-kokoro removed the kokoro:run Add this label to force Kokoro to re-run the tests. label Apr 18, 2023
tests/unit/test_table.py Outdated Show resolved Hide resolved
@tswast tswast added the kokoro:run Add this label to force Kokoro to re-run the tests. label Apr 18, 2023
@yoshi-kokoro yoshi-kokoro removed the kokoro:run Add this label to force Kokoro to re-run the tests. label Apr 18, 2023
@tswast tswast changed the title hotfix: handle case when expirationMs is None fix: handle case when expirationMs is None May 16, 2023
@tswast tswast added kokoro:force-run Add this label to force Kokoro to re-run the tests. owlbot:run Add this label to trigger the Owlbot post processor. labels May 16, 2023
@gcf-owl-bot gcf-owl-bot bot removed the owlbot:run Add this label to trigger the Owlbot post processor. label May 16, 2023
@yoshi-kokoro yoshi-kokoro removed the kokoro:force-run Add this label to force Kokoro to re-run the tests. label May 16, 2023
@tswast tswast merged commit fa6e13d into googleapis:main May 17, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: bigquery Issues related to the googleapis/python-bigquery API. size: s Pull request size is small.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants