Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: remove dependency on google-cloud-bigquery (cyclic dep) #1295

Merged
merged 8 commits into from
Sep 9, 2021

Conversation

stephaniewang526
Copy link
Contributor

Fixes #1249

@stephaniewang526 stephaniewang526 requested a review from a team September 7, 2021 20:19
@stephaniewang526 stephaniewang526 requested a review from a team as a code owner September 7, 2021 20:19
@google-cla google-cla bot added the cla: yes This human has signed the Contributor License Agreement. label Sep 7, 2021
@product-auto-label product-auto-label bot added the api: bigquerystorage Issues related to the googleapis/java-bigquerystorage API. label Sep 7, 2021
@generated-files-bot
Copy link

generated-files-bot bot commented Sep 7, 2021

Warning: This pull request is touching the following templated files:

  • .kokoro/dependencies.sh
  • samples/install-without-bom/pom.xml
  • samples/snapshot/pom.xml
  • samples/snippets/pom.xml

@@ -39,27 +37,31 @@ public static void runWriteToDefaultStream()
String projectId = "MY_PROJECT_ID";
String datasetName = "MY_DATASET_NAME";
String tableName = "MY_TABLE_NAME";

writeToDefaultStream(projectId, datasetName, tableName);
TableFieldSchema strField =
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would suggest keep the sample the way it is. Move BQV2ToBQStorageConverter.java into sample as a lib, and the sample would still do a GetTable(), use BQV2ToBQStorageConverter to convert the schema and then use JsonWriter. It doesn't need to block this PR.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure. Will make this change in a separate PR.

Copy link

@tswast tswast left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I imagine you'll want a helper in the google-cloud-bigquery client for converting to the BQ Storage TableSchema once the v1 protos are available. That should be possible without a circular dependency.

@@ -47,11 +47,11 @@ function completenessCheck() {
# This is stripped from the output as it is not present in the flattened pom.
# Only dependencies with 'compile' or 'runtime' scope are included from original dependency list.
msg "Generating dependency list using original pom..."
mvn dependency:list -f pom.xml -DincludeScope=runtime -Dsort=true | grep '\[INFO] .*:.*:.*:.*:.*' | sed -e 's/ --.*//' >.org-list.txt
mvn dependency:list -f pom.xml -DincludeScope=runtime -DexcludeArtifactIds=gson,commons-codec,commons-logging,opencensus-contrib-http-util,httpclient,httpcore -Dsort=true | grep '\[INFO] .*:.*:.*:.*:.*' | sed -e 's/ --.*//' >.org-list.txt
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where is this exclusion list coming from? A template?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes indeed. It comes from synthtool. There is a bug in maven-dependency-plugin which is causing this issue: https://issues.apache.org/jira/browse/MDEP-737
So we temporarily add this exclusion as a workaround.

@stephaniewang526 stephaniewang526 added automerge Merge the pull request once unit tests and other checks pass. kokoro:force-run Add this label to force Kokoro to re-run the tests. kokoro:run Add this label to force Kokoro to re-run the tests. and removed kokoro:force-run Add this label to force Kokoro to re-run the tests. kokoro:run Add this label to force Kokoro to re-run the tests. labels Sep 8, 2021
@yoshi-kokoro yoshi-kokoro removed the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Sep 8, 2021
@gcf-merge-on-green gcf-merge-on-green bot merged commit 7ac47de into master Sep 9, 2021
@gcf-merge-on-green gcf-merge-on-green bot deleted the refactor branch September 9, 2021 00:00
@gcf-merge-on-green gcf-merge-on-green bot removed the automerge Merge the pull request once unit tests and other checks pass. label Sep 9, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: bigquerystorage Issues related to the googleapis/java-bigquerystorage API. cla: yes This human has signed the Contributor License Agreement.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Refactor write client code to use the generated instead of the handwritten BigQuery client
4 participants