Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Support parameterized NUMERIC, BIGNUMERIC, STRING, and BYTES types #673

Merged
merged 16 commits into from
May 21, 2021

Conversation

jimfulton
Copy link
Contributor

@jimfulton jimfulton commented May 20, 2021

Thank you for opening a Pull Request! Before submitting your PR, there are a few things you can do to make sure it goes smoothly:

  • Make sure to open an issue as a bug/issue before writing your code! That way we can discuss the change, evaluate designs, and agree on the general idea
  • Ensure the tests and linter pass
  • Code coverage does not decrease (if any source code was changed)
  • Appropriate docs were updated (if necessary)

Fixes #668 🦕

Some notes:

  • I verified that the server:
    • enforces STRING maxLen and NUMERIC precision.
    • Enforces scale by truncating values.
  • select * loses type-parameter information. :( but client.list_rows doesn't.
  • I was thinking of trying to make sure that the Decimal objects created in Python matched the NUMERIC parameters, however
    • Because SELECT loses type information, it's mostly unavailable. IDK how often people use list_rows.
    • Because BQ sends data to the client as strings, this mostly just works, however:
      • The strings sent to the client lack trailing zeros. So, for example, given a scale of 2 and a value 123,
        BQ sends 123 rather than 123.00, so the Python Decimal doesn't have the right number of trailing digits.
        This is easy to fix, but incurs some performance overhead. IDK if we want to do this. If we do, we can do it
        in a follow-on PR.

@product-auto-label product-auto-label bot added the api: bigquery Issues related to the googleapis/python-bigquery API. label May 20, 2021
@jimfulton jimfulton changed the title feat: Support parameterized NUMERIC, STRING, and BYTES types. feat: Support parameterized NUMERIC, STRING, and BYTES types May 20, 2021
@google-cla google-cla bot added the cla: yes This human has signed the Contributor License Agreement. label May 20, 2021
@jimfulton jimfulton marked this pull request as ready for review May 20, 2021 17:55
@jimfulton jimfulton requested a review from a team May 20, 2021 17:55
@jimfulton jimfulton requested a review from a team as a code owner May 20, 2021 17:55
@jimfulton jimfulton requested review from prash-mi and removed request for a team May 20, 2021 17:55
@jimfulton jimfulton changed the title feat: Support parameterized NUMERIC, STRING, and BYTES types feat: Support parameterized NUMERIC, BIGNUMERIC, STRING, and BYTES types May 20, 2021
Copy link
Contributor

@plamut plamut left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, just a few minor remarks that were supposed (?) to be caught by the linters.

The only more significant one is about the (non) cleanup in system tests.

google/cloud/bigquery/schema.py Outdated Show resolved Hide resolved
google/cloud/bigquery/schema.py Outdated Show resolved Hide resolved
google/cloud/bigquery/schema.py Outdated Show resolved Hide resolved
tests/system/test_client.py Outdated Show resolved Hide resolved
Copy link
Contributor

@plamut plamut left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@plamut plamut merged commit 45421e7 into master May 21, 2021
@plamut plamut deleted the riversnake-669-parameterized-schemafield branch May 21, 2021 16:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: bigquery Issues related to the googleapis/python-bigquery API. cla: yes This human has signed the Contributor License Agreement.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Parameterized Types
2 participants