Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

sync pull causes DeadlineExceededException #189

Closed
saturnism opened this issue May 16, 2020 · 7 comments · Fixed by #989
Closed

sync pull causes DeadlineExceededException #189

saturnism opened this issue May 16, 2020 · 7 comments · Fixed by #989
Assignees
Labels
api: pubsub Issues related to the googleapis/java-pubsub API. type: feature request ‘Nice-to-have’ improvement, new feature or different behavior or design.

Comments

@saturnism
Copy link

doing a sync pull on a subscription, and if there is no message, it throws DeadlineExceededException. Usually, if a pull times out, it would just return null. Also, pull wait time interval is usually configuration, how can this be configured?

Caused by: com.google.api.gax.rpc.DeadlineExceededException: io.grpc.StatusRuntimeException: DEADLINE_EXCEEDED: deadline exceeded after 4.978775875s. [buffered_nanos=291856840, buffered_nanos=331252614, remote_addr=pubsub.googleapis.com/172.217.12.138:443]
	at com.google.api.gax.rpc.ApiExceptionFactory.createException(ApiExceptionFactory.java:51)
	at com.google.api.gax.grpc.GrpcApiExceptionFactory.create(GrpcApiExceptionFactory.java:72)
	at com.google.api.gax.grpc.GrpcApiExceptionFactory.create(GrpcApiExceptionFactory.java:60)
	at com.google.api.gax.grpc.GrpcExceptionCallable$ExceptionTransformingFuture.onFailure(GrpcExceptionCallable.java:97)
	at com.google.api.core.ApiFutures$1.onFailure(ApiFutures.java:68)
	at com.google.common.util.concurrent.Futures$CallbackListener.run(Futures.java:1039)
	at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30)
	at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1165)
	at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:958)
	at com.google.common.util.concurrent.AbstractFuture.setException(AbstractFuture.java:749)
	at io.grpc.stub.ClientCalls$GrpcFuture.setException(ClientCalls.java:522)
	at io.grpc.stub.ClientCalls$UnaryStreamToFuture.onClose(ClientCalls.java:497)
	at io.grpc.internal.ClientCallImpl.closeObserver(ClientCallImpl.java:426)
	at io.grpc.internal.ClientCallImpl.access$500(ClientCallImpl.java:66)
	at io.grpc.internal.ClientCallImpl$1CloseInContext.runInContext(ClientCallImpl.java:416)
	at io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37)
	at io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:123)
	... 6 more
@product-auto-label product-auto-label bot added the api: pubsub Issues related to the googleapis/java-pubsub API. label May 16, 2020
@yoshi-automation yoshi-automation added the triage me I really want to be triaged. label May 17, 2020
@hannahrogers-google hannahrogers-google added the type: question Request for information or clarification. Not an issue. label May 19, 2020
@yoshi-automation yoshi-automation removed the triage me I really want to be triaged. label May 19, 2020
@kamalaboulhosn
Copy link
Contributor

When you say "usually, if a pull times out, it would just return null," what do you mean? As in, that used to be the behavior or that is the behavior you expect?

@saturnism
Copy link
Author

When you say "usually, if a pull times out, it would just return null," what do you mean? As in, that used to be the behavior or that is the behavior you expect?

IIRC, it was the old behavior when the client library had the idiomatic APIs. It's also the behavior for most of the other messaging APIs other than Pub/Sub.

@elefeint
Copy link

We have a user issue that is related to this -- Pub/Sub timing out on a health check, which performs a synchronous pull with returnImmediately=true: spring-attic/spring-cloud-gcp#2628.

What should be done here -- should the user open a support ticket for abnormally high pull latency, or is this a normal scenario, and we should adjust the health check indicator to ignore DEADLINE_EXCEEDED? It feels wrong to ignore, since a too-long latency is a valid indicator of a service problem.

@meredithslota
Copy link

I'm going to assign this to @anguillanneuf to discuss with Elena and see if we can figure out next steps.

@brachipa
Copy link

For me it happens when there is no messages available (v1.90.0)

@anguillanneuf anguillanneuf removed the type: question Request for information or clarification. Not an issue. label Jan 27, 2022
@anguillanneuf
Copy link
Contributor

Both gcloud and Python will just return nothing or null if there are no messages. The Java client library behaves somewhat differently. If we do want to change it back, this would be a feature request. FYI - @mmicatka

@yoshi-automation yoshi-automation added triage me I really want to be triaged. 🚨 This issue needs some love. labels Jan 27, 2022
@meredithslota meredithslota added type: feature request ‘Nice-to-have’ improvement, new feature or different behavior or design. and removed 🚨 This issue needs some love. triage me I really want to be triaged. labels Jan 28, 2022
@anguillanneuf
Copy link
Contributor

anguillanneuf commented Feb 3, 2022

I gave the current sample a try. When no messages are pulled, it no longer returns DeadlineExceededException. It seems to return immediately and returns an empty list. Updating the sample to handle the no ackId case.

Exception in thread "main" com.google.api.gax.rpc.InvalidArgumentException: io.grpc.StatusRuntimeException: INVALID_ARGUMENT: You have not specified an ack ID in the request.
	at com.google.api.gax.rpc.ApiExceptionFactory.createException(ApiExceptionFactory.java:47)
	at com.google.api.gax.grpc.GrpcApiExceptionFactory.create(GrpcApiExceptionFactory.java:72)
	at com.google.api.gax.grpc.GrpcApiExceptionFactory.create(GrpcApiExceptionFactory.java:60)
	at com.google.api.gax.grpc.GrpcExceptionCallable$ExceptionTransformingFuture.onFailure(GrpcExceptionCallable.java:97)
	at com.google.api.core.ApiFutures$1.onFailure(ApiFutures.java:68)
	at com.google.common.util.concurrent.Futures$CallbackListener.run(Futures.java:1133)
	at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:31)
	at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1277)
	at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:1038)
	at com.google.common.util.concurrent.AbstractFuture.setException(AbstractFuture.java:808)
	at io.grpc.stub.ClientCalls$GrpcFuture.setException(ClientCalls.java:564)
	at io.grpc.stub.ClientCalls$UnaryStreamToFuture.onClose(ClientCalls.java:534)
	at io.grpc.internal.ClientCallImpl.closeObserver(ClientCallImpl.java:562)
	at io.grpc.internal.ClientCallImpl.access$300(ClientCallImpl.java:70)
	at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInternal(ClientCallImpl.java:743)
	at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInContext(ClientCallImpl.java:722)
	at io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37)
	at io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:133)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:829)
	Suppressed: com.google.api.gax.rpc.AsyncTaskException: Asynchronous task failed
		at com.google.api.gax.rpc.ApiExceptions.callAndTranslateApiException(ApiExceptions.java:57)
		at com.google.api.gax.rpc.UnaryCallable.call(UnaryCallable.java:112)
		at pubsub.SubscribeSyncExample.subscribeSyncExample(SubscribeSyncExample.java:80)
		at pubsub.SubscribeSyncExample.main(SubscribeSyncExample.java:43)
Caused by: io.grpc.StatusRuntimeException: INVALID_ARGUMENT: You have not specified an ack ID in the request.
	at io.grpc.Status.asRuntimeException(Status.java:535)
	... 10 more

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: pubsub Issues related to the googleapis/java-pubsub API. type: feature request ‘Nice-to-have’ improvement, new feature or different behavior or design.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

8 participants