Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

support google-cloud-bigquery-storage v1 #629

Closed
tswast opened this issue Apr 26, 2021 · 2 comments · Fixed by #666
Closed

support google-cloud-bigquery-storage v1 #629

tswast opened this issue Apr 26, 2021 · 2 comments · Fixed by #666
Assignees
Labels
api: bigquery Issues related to the googleapis/python-bigquery API. type: feature request ‘Nice-to-have’ improvement, new feature or different behavior or design.

Comments

@tswast
Copy link
Contributor

tswast commented Apr 26, 2021

Is your feature request related to a problem? Please describe.

I'm filing this in response to internal feedback 186096865

Error: module 'google.cloud.bigquery_storage_v1.types' has no attribute 'DataFormat'

I believe this is because the customer is using a 1.x version of google-cloud-bigquery-storage.

Describe the solution you'd like

(A) Ideally we'd have some code to support either 1.x or 2.x versions of google-cloud-bigquery-storage

Describe alternatives you've considered

  • (B) Detect when 1.x is used and raise an appropriate exception asking to upgrade to 2.x
  • (C) Detect ... and fallback to tabledata.list / getQueryResults + warn that the package should be upgraded

Additional context

N/A

@product-auto-label product-auto-label bot added the api: bigquery Issues related to the googleapis/python-bigquery API. label Apr 26, 2021
@tswast
Copy link
Contributor Author

tswast commented Apr 26, 2021

If (A) is infeasible, I think (C) is our best fallback, so that to_dataframe can still be used, even if it's the slower path.

@tswast tswast added the type: feature request ‘Nice-to-have’ improvement, new feature or different behavior or design. label Apr 26, 2021
@plamut
Copy link
Contributor

plamut commented May 11, 2021

Started experimenting with option A, but a concern emerged that this could open a can of worms. In setup.py we already require BQ Storage 2.x since version 2.0.0, meaning that if BQ 2.x and BQ Storage 1.x are installed side-by-side should be treated as an environment error IMO.

Sadly, this is quite possible in practice, I managed to pip install incompatible versions into a local environment. But updating the library code just to get past the attribute error when using a dependency version already declared too old can reveal other incompatibility issues.

After discussing this with @tswast offline, we think it's less risky to encourage users to upgrade to BQ Storage 2.x by issuing a more user friendly error message when incompatibility is detected. It's a quicker fix, but it still keeps the door open for a "backward compatibility fix" later if too many users are stuck on 1.x versions.

Update: It turned out we will probably have a more graceful fallback after all.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: bigquery Issues related to the googleapis/python-bigquery API. type: feature request ‘Nice-to-have’ improvement, new feature or different behavior or design.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants