Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: v1beta2 manual client library for BigQueryWrite #714

Merged
merged 15 commits into from
Dec 9, 2020
Merged

feat: v1beta2 manual client library for BigQueryWrite #714

merged 15 commits into from
Dec 9, 2020

Conversation

yirutang
Copy link
Contributor

@yirutang yirutang commented Dec 8, 2020

Thank you for opening a Pull Request! Before submitting your PR, there are a few things you can do to make sure it goes smoothly:

  • Make sure to open an issue as a bug/issue before writing your code! That way we can discuss the change, evaluate designs, and agree on the general idea
  • Ensure the tests and linter pass
  • Code coverage does not decrease (if any source code was changed)
  • Appropriate docs were updated (if necessary)

Fixes #<issue_number_goes_here> ☕️

@yirutang yirutang requested review from a team and shollyman December 8, 2020 00:39
@product-auto-label product-auto-label bot added the api: bigquerystorage Issues related to the googleapis/java-bigquerystorage API. label Dec 8, 2020
@google-cla google-cla bot added the cla: yes This human has signed the Contributor License Agreement. label Dec 8, 2020
yirutang and others added 3 commits December 7, 2020 16:46
…gquery/storage/v1beta2/JsonStreamWriter.java

Co-authored-by: yoshi-code-bot <[email protected]>
…gquery/storage/v1beta2/OnSchemaUpdateRunnable.java

Co-authored-by: yoshi-code-bot <[email protected]>
@yirutang yirutang changed the title feat: v1beta2 manual client library for BigQueryWrite. feat: v1beta2 manual client library for BigQueryWrite Dec 8, 2020
@stephaniewang526 stephaniewang526 self-requested a review December 8, 2020 17:28
@stephaniewang526
Copy link
Contributor

@yirutang there are a number of checkstyle errors: https://github.com/googleapis/java-bigquerystorage/pull/714/checks?check_run_id=1514447927. Running mvn checkstyle:check can repro locally.

@yirutang
Copy link
Contributor Author

yirutang commented Dec 8, 2020 via email

@codecov
Copy link

codecov bot commented Dec 8, 2020

Codecov Report

Merging #714 (a9fc8dd) into master (20aef28) will increase coverage by 1.63%.
The diff coverage is 86.08%.

Impacted file tree graph

@@             Coverage Diff              @@
##             master     #714      +/-   ##
============================================
+ Coverage     78.79%   80.42%   +1.63%     
- Complexity      718      959     +241     
============================================
  Files            65       73       +8     
  Lines          4263     5334    +1071     
  Branches        253      405     +152     
============================================
+ Hits           3359     4290     +931     
- Misses          786      877      +91     
- Partials        118      167      +49     
Impacted Files Coverage Δ Complexity Δ
.../google/cloud/bigquery/storage/v1beta2/Waiter.java 61.17% <61.17%> (ø) 16.00 <16.00> (?)
...oud/bigquery/storage/v1beta2/JsonStreamWriter.java 77.19% <77.19%> (ø) 12.00 <12.00> (?)
...e/cloud/bigquery/storage/v1beta2/StreamWriter.java 84.98% <84.98%> (ø) 37.00 <37.00> (?)
...gquery/storage/v1beta2/OnSchemaUpdateRunnable.java 85.71% <85.71%> (ø) 4.00 <4.00> (?)
.../bigquery/storage/v1beta2/SchemaCompatibility.java 91.30% <91.30%> (ø) 99.00 <99.00> (?)
...bigquery/storage/v1beta2/ProtoSchemaConverter.java 96.22% <96.22%> (ø) 10.00 <10.00> (?)
...d/bigquery/storage/v1beta2/JsonToProtoMessage.java 96.96% <96.96%> (ø) 47.00 <47.00> (?)
...torage/v1beta2/BQTableSchemaToProtoDescriptor.java 98.48% <98.48%> (ø) 8.00 <8.00> (?)
... and 9 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 20aef28...a9fc8dd. Read the comment docs.

@@ -1,7 +1,6 @@
/*
* Copyright 2020 Google LLC
*
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The bot incorrectly proposed these changes. Raised this issue to track a fix.

@yirutang
Copy link
Contributor Author

yirutang commented Dec 8, 2020 via email

@stephaniewang526
Copy link
Contributor

One more IT failure:

com.google.api.gax.rpc.FailedPreconditionException: io.grpc.StatusRuntimeException: FAILED_PRECONDITION: Precondition check failed.
	at com.google.api.gax.rpc.ApiExceptionFactory.createException(ApiExceptionFactory.java:59)
	at com.google.api.gax.grpc.GrpcApiExceptionFactory.create(GrpcApiExceptionFactory.java:72)
	at com.google.api.gax.grpc.GrpcApiExceptionFactory.create(GrpcApiExceptionFactory.java:60)
	at com.google.api.gax.grpc.GrpcExceptionCallable$ExceptionTransformingFuture.onFailure(GrpcExceptionCallable.java:97)
	at com.google.api.core.ApiFutures$1.onFailure(ApiFutures.java:68)
	at com.google.common.util.concurrent.Futures$CallbackListener.run(Futures.java:1041)
	at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30)
	at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1215)
	at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983)
	at com.google.common.util.concurrent.AbstractFuture.setException(AbstractFuture.java:771)
	at io.grpc.stub.ClientCalls$GrpcFuture.setException(ClientCalls.java:563)
	at io.grpc.stub.ClientCalls$UnaryStreamToFuture.onClose(ClientCalls.java:533)
	at io.grpc.internal.ClientCallImpl.closeObserver(ClientCallImpl.java:617)
	at io.grpc.internal.ClientCallImpl.access$300(ClientCallImpl.java:70)
	at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInternal(ClientCallImpl.java:803)
	at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInContext(ClientCallImpl.java:782)
	at io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37)
	at io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:123)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
	Suppressed: com.google.api.gax.rpc.AsyncTaskException: Asynchronous task failed
		at com.google.api.gax.rpc.ApiExceptions.callAndTranslateApiException(ApiExceptions.java:57)
		at com.google.api.gax.rpc.UnaryCallable.call(UnaryCallable.java:112)
		at com.google.cloud.bigquery.storage.v1beta2.BigQueryWriteClient.flushRows(BigQueryWriteClient.java:652)
		at com.google.cloud.bigquery.storage.v1beta2.it.ITBigQueryWriteManualClientTest.testFlushRows(ITBigQueryWriteManualClientTest.java:606)
		at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
		at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
		at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
		at java.lang.reflect.Method.invoke(Method.java:498)
		at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
		at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
		at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
		at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
		at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
		at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100)
		at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366)
		at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103)
		at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63)
		at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
		at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
		at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
		at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
		at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
		at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
		at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
		at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
		at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
		at org.junit.runners.Suite.runChild(Suite.java:128)
		at org.junit.runners.Suite.runChild(Suite.java:27)
		at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
		at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
		at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
		at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
		at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
		at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
		at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
		at org.apache.maven.surefire.junitcore.JUnitCore.run(JUnitCore.java:55)
		at org.apache.maven.surefire.junitcore.JUnitCoreWrapper.createRequestAndRun(JUnitCoreWrapper.java:137)
		at org.apache.maven.surefire.junitcore.JUnitCoreWrapper.executeEager(JUnitCoreWrapper.java:107)
		at org.apache.maven.surefire.junitcore.JUnitCoreWrapper.execute(JUnitCoreWrapper.java:83)
		at org.apache.maven.surefire.junitcore.JUnitCoreWrapper.execute(JUnitCoreWrapper.java:75)
		at org.apache.maven.surefire.junitcore.JUnitCoreProvider.invoke(JUnitCoreProvider.java:158)
		at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:377)
		at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:138)
		at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:465)
		at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:451)
Caused by: io.grpc.StatusRuntimeException: FAILED_PRECONDITION: Precondition check failed.
	at io.grpc.Status.asRuntimeException(Status.java:533)
	... 14 more

Comment on lines 22 to 23
import static org.junit.Assert.assertTrue;
import static org.junit.Assert.fail;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
import static org.junit.Assert.assertTrue;
import static org.junit.Assert.fail;
import static org.junit.Assert.assertTrue;

@stephaniewang526 stephaniewang526 merged commit 53a9c3a into googleapis:master Dec 9, 2020
@yirutang yirutang deleted the beta-client branch December 10, 2020 23:28
shubhwip pushed a commit to shubhwip/java-bigquerystorage that referenced this pull request Oct 7, 2023
…onfig to v1.0.3 (googleapis#714)

[![WhiteSource Renovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com)

This PR contains the following updates:

| Package | Change | Age | Adoption | Passing | Confidence |
|---|---|---|---|---|---|
| [com.google.cloud:google-cloud-shared-config](https://togithub.com/googleapis/java-shared-config) | `1.0.2` -> `1.0.3` | [![age](https://badges.renovateapi.com/packages/maven/com.google.cloud:google-cloud-shared-config/1.0.3/age-slim)](https://docs.renovatebot.com/merge-confidence/) | [![adoption](https://badges.renovateapi.com/packages/maven/com.google.cloud:google-cloud-shared-config/1.0.3/adoption-slim)](https://docs.renovatebot.com/merge-confidence/) | [![passing](https://badges.renovateapi.com/packages/maven/com.google.cloud:google-cloud-shared-config/1.0.3/compatibility-slim/1.0.2)](https://docs.renovatebot.com/merge-confidence/) | [![confidence](https://badges.renovateapi.com/packages/maven/com.google.cloud:google-cloud-shared-config/1.0.3/confidence-slim/1.0.2)](https://docs.renovatebot.com/merge-confidence/) |

---

### Release Notes

<details>
<summary>googleapis/java-shared-config</summary>

### [`v1.0.3`](https://togithub.com/googleapis/java-shared-config/blob/master/CHANGELOG.md#&#8203;103-httpswwwgithubcomgoogleapisjava-shared-configcomparev102v103-2021-09-21)

[Compare Source](https://togithub.com/googleapis/java-shared-config/compare/v1.0.2...v1.0.3)

</details>

---

### Configuration

📅 **Schedule**: At any time (no schedule defined).

🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about this update again.

---

 - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box.

---

This PR has been generated by [WhiteSource Renovate](https://renovate.whitesourcesoftware.com). View repository job log [here](https://app.renovatebot.com/dashboard#github/googleapis/java-storage-nio).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: bigquerystorage Issues related to the googleapis/java-bigquerystorage API. cla: yes This human has signed the Contributor License Agreement.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants