Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: transition the library to microgenerator #62

Merged
merged 21 commits into from
Sep 24, 2020

Conversation

plamut
Copy link
Contributor

@plamut plamut commented Sep 18, 2020

Closes #61.

Remaining things:

  • Add the UPGRADING guide to docs.
  • Fix failing tests
  • Decide what to do with v1beta1 and v1beta2 code (remove? YES!).
  • Samples tests not running on Kokoro?

@google-cla google-cla bot added the cla: yes This human has signed the Contributor License Agreement. label Sep 18, 2020
@product-auto-label product-auto-label bot added the api: bigquerystorage Issues related to the googleapis/python-bigquery-storage API. label Sep 19, 2020
@plamut plamut force-pushed the iss-61 branch 2 times, most recently from c252fba to 38e6c39 Compare September 22, 2020 13:12
@plamut plamut marked this pull request as ready for review September 22, 2020 13:49
@plamut plamut requested a review from a team September 22, 2020 13:49
@plamut plamut requested a review from a team as a code owner September 22, 2020 13:49
@plamut plamut requested review from dinagraves, busunkim96 and tswast and removed request for a team September 22, 2020 13:49
.kokoro/release/common.cfg Show resolved Hide resolved
google/cloud/bigquery/storage/__init__.py Outdated Show resolved Hide resolved
google/cloud/bigquery/storage/__init__.py Outdated Show resolved Hide resolved
# limitations under the License.
#

from .services.big_query_read import BigQueryReadClient
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's have this be the manual wrapper.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a generated file and bigquery_storage_v1/client.py imports from it. The latter assumes that the generated client is imported and wraps it, although it could also import directly from storage_v1.services.big_query_read, in principle.

Do you think it's worth maintaining additional synth replacement rules to prevent users from accessing the generated client through google.cloud.bigquery.storage_v1?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you think it's worth maintaining additional synth replacement rules to prevent users from accessing the generated client through google.cloud.bigquery.storage_v1?

I'm pretty sure that's what I did for the bigquery_storage_v1beta1 package (that is, exclude the equivalent top-level __init__.py from getting copied over.

I'm still flip-flopping about what namespace we should use for the manual clients. It's nice to avoid breaking changes, but if we're breaking anyway, maybe we should be consistent. What do other manual libs do for namespace of manual clients?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PubSub just exposes the generated clients through pubsub_v1.__init__.py import.

I don't know about other libs, but can check (I suspect they left these imports intact, though).

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Interesting! I find that it uses the google.pubsub_v1 for generated and google.cloud.pubsub_v1 for handwritten to be pretty confusing, but I guess so long as it's documented well, it'll be okay.

Copy link
Contributor Author

@plamut plamut Sep 23, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can at least mention this in the UPGRADING guide and just recommend to import from google.cloud.bigquery.storage (the samples use this path, too). This will give us more freedom to restructure the library internally in the future (in theory at least).

Update: FWIW, I found another library with hand-written layer that has already been migrated to microgenerator (AutoML) and it, too, exposes the generated client through google.cloud.automl_v1.

google/cloud/bigquery_storage_v1/types.py Show resolved Hide resolved
samples/quickstart/quickstart.py Outdated Show resolved Hide resolved
synth.py Show resolved Hide resolved
@plamut plamut requested a review from tswast September 23, 2020 13:47
docs/index.rst Outdated Show resolved Hide resolved
noxfile.py Outdated Show resolved Hide resolved
google/cloud/bigquery_storage_v1/async_client.py Outdated Show resolved Hide resolved
@plamut
Copy link
Contributor Author

plamut commented Sep 23, 2020

Addressed all comments. Once the changes are finalized and confirmed, I will follow up with the UPGRADING guide.

Update: Added the upgrade guide, as I think nothing significant will change anymore now that we discussed the import paths.

@tswast
Copy link
Contributor

tswast commented Sep 24, 2020

Regarding the Sample tests, I just sent out internal CL 333505519 to add the missing Kokoro config.

UPGRADING.md Show resolved Hide resolved
UPGRADING.md Outdated Show resolved Hide resolved
Copy link
Contributor

@tswast tswast left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM once those UPGRADING fixes are in.

@tswast tswast merged commit 0a0eb2e into googleapis:master Sep 24, 2020
@plamut plamut deleted the iss-61 branch September 24, 2020 15:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: bigquerystorage Issues related to the googleapis/python-bigquery-storage API. cla: yes This human has signed the Contributor License Agreement.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Transition the library to microgenerator
2 participants