Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

samples.snippets.subscriber_test: test_update_dead_letter_policy failed #321

Closed
flaky-bot bot opened this issue Mar 11, 2021 · 2 comments · Fixed by #322 or #329
Closed

samples.snippets.subscriber_test: test_update_dead_letter_policy failed #321

flaky-bot bot opened this issue Mar 11, 2021 · 2 comments · Fixed by #322 or #329
Labels
api: pubsub Issues related to the googleapis/python-pubsub API. flakybot: flaky Tells the Flaky Bot not to close or comment on this issue. flakybot: issue An issue filed by the Flaky Bot. Should not be added manually. priority: p1 Important issue which blocks shipping the next release. Will be fixed prior to next release. samples Issues that are directly related to samples. type: bug Error or flaw in code with unintended results or allowing sub-optimal usage patterns.

Comments

@flaky-bot
Copy link

flaky-bot bot commented Mar 11, 2021

This test failed!

To configure my behavior, see the Flaky Bot documentation.

If I'm commenting on this issue too often, add the flakybot: quiet label and
I will stop commenting.


commit: 4dc89fc
buildURL: Build Status, Sponge
status: failed

Test output
args = (subscription {
  name: "projects/python-docs-samples-tests/subscriptions/subscription-test-subscription-dlq-3.6-2ed44...-2ed446af9a224d22abf7364d88a983ae"
    max_delivery_attempts: 20
  }
}
update_mask {
  paths: "dead_letter_policy"
}
,)
kwargs = {'metadata': [('x-goog-request-params', 'subscription.name=projects/python-docs-samples-tests/subscriptions/subscripti...q-3.6-2ed446af9a224d22abf7364d88a983ae'), ('x-goog-api-client', 'gl-python/3.6.13 grpc/1.36.1 gax/1.26.1 gccl/2.4.0')]}
@six.wraps(callable_)
def error_remapped_callable(*args, **kwargs):
    try:
      return callable_(*args, **kwargs)

.nox/py-3-6/lib/python3.6/site-packages/google/api_core/grpc_helpers.py:73:


self = <grpc._channel._UnaryUnaryMultiCallable object at 0x7f5f9ab37c88>
request = subscription {
name: "projects/python-docs-samples-tests/subscriptions/subscription-test-subscription-dlq-3.6-2ed446....6-2ed446af9a224d22abf7364d88a983ae"
max_delivery_attempts: 20
}
}
update_mask {
paths: "dead_letter_policy"
}

timeout = None
metadata = [('x-goog-request-params', 'subscription.name=projects/python-docs-samples-tests/subscriptions/subscription-test-subsc...lq-3.6-2ed446af9a224d22abf7364d88a983ae'), ('x-goog-api-client', 'gl-python/3.6.13 grpc/1.36.1 gax/1.26.1 gccl/2.4.0')]
credentials = None, wait_for_ready = None, compression = None

def __call__(self,
             request,
             timeout=None,
             metadata=None,
             credentials=None,
             wait_for_ready=None,
             compression=None):
    state, call, = self._blocking(request, timeout, metadata, credentials,
                                  wait_for_ready, compression)
  return _end_unary_response_blocking(state, call, False, None)

.nox/py-3-6/lib/python3.6/site-packages/grpc/_channel.py:923:


state = <grpc._channel._RPCState object at 0x7f5f9ab37550>
call = <grpc._cython.cygrpc.SegregatedCall object at 0x7f5f9abe1d88>
with_call = False, deadline = None

def _end_unary_response_blocking(state, call, with_call, deadline):
    if state.code is grpc.StatusCode.OK:
        if with_call:
            rendezvous = _MultiThreadedRendezvous(state, call, None, deadline)
            return state.response, rendezvous
        else:
            return state.response
    else:
      raise _InactiveRpcError(state)

E grpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that terminated with:
E status = StatusCode.INTERNAL
E details = "A service error has occurred. Please retry your request. If the error persists, please report it. [code=e8c0]"
E debug_error_string = "{"created":"@1615457112.769783514","description":"Error received from peer ipv4:173.194.202.95:443","file":"src/core/lib/surface/call.cc","file_line":1067,"grpc_message":"A service error has occurred. Please retry your request. If the error persists, please report it. [code=e8c0]","grpc_status":13}"
E >

.nox/py-3-6/lib/python3.6/site-packages/grpc/_channel.py:826: _InactiveRpcError

The above exception was the direct cause of the following exception:

subscription_dlq = 'projects/python-docs-samples-tests/subscriptions/subscription-test-subscription-dlq-3.6-2ed446af9a224d22abf7364d88a983ae'
dead_letter_topic = 'projects/python-docs-samples-tests/topics/subscription-test-dead-letter-topic-3.6-2ed446af9a224d22abf7364d88a983ae'
capsys = <_pytest.capture.CaptureFixture object at 0x7f5f9ab29860>

def test_update_dead_letter_policy(subscription_dlq, dead_letter_topic, capsys):
    _ = subscriber.update_subscription_with_dead_letter_policy(
        PROJECT_ID,
        TOPIC,
        SUBSCRIPTION_DLQ,
        DEAD_LETTER_TOPIC,
      UPDATED_MAX_DELIVERY_ATTEMPTS,
    )

subscriber_test.py:249:


subscriber.py:317: in update_subscription_with_dead_letter_policy
request={"subscription": subscription, "update_mask": update_mask}
../../google/cloud/pubsub_v1/_gapic.py:40: in
fx = lambda self, *a, **kw: wrapped_fx(self.api, *a, **kw) # noqa
../../google/pubsub_v1/services/subscriber/client.py:655: in update_subscription
response = rpc(request, retry=retry, timeout=timeout, metadata=metadata,)
.nox/py-3-6/lib/python3.6/site-packages/google/api_core/gapic_v1/method.py:145: in call
return wrapped_func(*args, **kwargs)
.nox/py-3-6/lib/python3.6/site-packages/google/api_core/retry.py:286: in retry_wrapped_func
on_error=on_error,
.nox/py-3-6/lib/python3.6/site-packages/google/api_core/retry.py:184: in retry_target
return target()
.nox/py-3-6/lib/python3.6/site-packages/google/api_core/grpc_helpers.py:75: in error_remapped_callable
six.raise_from(exceptions.from_grpc_error(exc), exc)


value = None
from_value = <_InactiveRpcError of RPC that terminated with:
status = StatusCode.INTERNAL
details = "A service error has occurred...ror has occurred. Please retry your request. If the error persists, please report it. [code=e8c0]","grpc_status":13}"

???
E google.api_core.exceptions.InternalServerError: 500 A service error has occurred. Please retry your request. If the error persists, please report it. [code=e8c0]

:3: InternalServerError

@flaky-bot flaky-bot bot added flakybot: issue An issue filed by the Flaky Bot. Should not be added manually. priority: p1 Important issue which blocks shipping the next release. Will be fixed prior to next release. type: bug Error or flaw in code with unintended results or allowing sub-optimal usage patterns. labels Mar 11, 2021
@product-auto-label product-auto-label bot added api: pubsub Issues related to the googleapis/python-pubsub API. samples Issues that are directly related to samples. labels Mar 11, 2021
@flaky-bot
Copy link
Author

flaky-bot bot commented Mar 11, 2021

Looks like this issue is flaky. 😟

I'm going to leave this open and stop commenting.

A human should fix and close this.


When run at the same commit (4dc89fc), this test passed in one build (Build Status, Sponge) and failed in another build (Build Status, Sponge).

@flaky-bot flaky-bot bot added the flakybot: flaky Tells the Flaky Bot not to close or comment on this issue. label Mar 11, 2021
@flaky-bot
Copy link
Author

flaky-bot bot commented Mar 14, 2021

Oops! Looks like this issue is still flaky. It failed again. 😬

I reopened the issue, but a human will need to close it again.


commit: 5a97ef1
buildURL: Build Status, Sponge
status: failed

Test output
args = (subscription {
  name: "projects/python-docs-samples-tests/subscriptions/subscription-test-subscription-dlq-3.8-8f115...-8f115fc03a8247b3907526819f42e20b"
    max_delivery_attempts: 20
  }
}
update_mask {
  paths: "dead_letter_policy"
}
,)
kwargs = {'metadata': [('x-goog-request-params', 'subscription.name=projects/python-docs-samples-tests/subscriptions/subscripti...lq-3.8-8f115fc03a8247b3907526819f42e20b'), ('x-goog-api-client', 'gl-python/3.8.8 grpc/1.36.1 gax/1.26.1 gccl/2.4.0')]}
@six.wraps(callable_)
def error_remapped_callable(*args, **kwargs):
    try:
      return callable_(*args, **kwargs)

.nox/py-3-8/lib/python3.8/site-packages/google/api_core/grpc_helpers.py:73:


self = <grpc._channel._UnaryUnaryMultiCallable object at 0x7f1be575eeb0>
request = subscription {
name: "projects/python-docs-samples-tests/subscriptions/subscription-test-subscription-dlq-3.8-8f115f....8-8f115fc03a8247b3907526819f42e20b"
max_delivery_attempts: 20
}
}
update_mask {
paths: "dead_letter_policy"
}

timeout = None
metadata = [('x-goog-request-params', 'subscription.name=projects/python-docs-samples-tests/subscriptions/subscription-test-subsc...dlq-3.8-8f115fc03a8247b3907526819f42e20b'), ('x-goog-api-client', 'gl-python/3.8.8 grpc/1.36.1 gax/1.26.1 gccl/2.4.0')]
credentials = None, wait_for_ready = None, compression = None

def __call__(self,
             request,
             timeout=None,
             metadata=None,
             credentials=None,
             wait_for_ready=None,
             compression=None):
    state, call, = self._blocking(request, timeout, metadata, credentials,
                                  wait_for_ready, compression)
  return _end_unary_response_blocking(state, call, False, None)

.nox/py-3-8/lib/python3.8/site-packages/grpc/_channel.py:923:


state = <grpc._channel._RPCState object at 0x7f1be5750dc0>
call = <grpc._cython.cygrpc.SegregatedCall object at 0x7f1be5771780>
with_call = False, deadline = None

def _end_unary_response_blocking(state, call, with_call, deadline):
    if state.code is grpc.StatusCode.OK:
        if with_call:
            rendezvous = _MultiThreadedRendezvous(state, call, None, deadline)
            return state.response, rendezvous
        else:
            return state.response
    else:
      raise _InactiveRpcError(state)

E grpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that terminated with:
E status = StatusCode.INTERNAL
E details = "A service error has occurred. Please retry your request. If the error persists, please report it. [code=e8c0]"
E debug_error_string = "{"created":"@1615719918.990875736","description":"Error received from peer ipv4:142.250.99.95:443","file":"src/core/lib/surface/call.cc","file_line":1067,"grpc_message":"A service error has occurred. Please retry your request. If the error persists, please report it. [code=e8c0]","grpc_status":13}"
E >

.nox/py-3-8/lib/python3.8/site-packages/grpc/_channel.py:826: _InactiveRpcError

The above exception was the direct cause of the following exception:

subscription_dlq = 'projects/python-docs-samples-tests/subscriptions/subscription-test-subscription-dlq-3.8-8f115fc03a8247b3907526819f42e20b'
dead_letter_topic = 'projects/python-docs-samples-tests/topics/subscription-test-dead-letter-topic-3.8-8f115fc03a8247b3907526819f42e20b'
capsys = <_pytest.capture.CaptureFixture object at 0x7f1be91232e0>

def test_update_dead_letter_policy(subscription_dlq, dead_letter_topic, capsys):
  _ = subscriber.update_subscription_with_dead_letter_policy(
        PROJECT_ID,
        TOPIC,
        SUBSCRIPTION_DLQ,
        DEAD_LETTER_TOPIC,
        UPDATED_MAX_DELIVERY_ATTEMPTS,
    )

subscriber_test.py:244:


subscriber.py:316: in update_subscription_with_dead_letter_policy
subscription_after_update = subscriber.update_subscription(
../../google/cloud/pubsub_v1/_gapic.py:40: in
fx = lambda self, *a, **kw: wrapped_fx(self.api, *a, **kw) # noqa
../../google/pubsub_v1/services/subscriber/client.py:655: in update_subscription
response = rpc(request, retry=retry, timeout=timeout, metadata=metadata,)
.nox/py-3-8/lib/python3.8/site-packages/google/api_core/gapic_v1/method.py:145: in call
return wrapped_func(*args, **kwargs)
.nox/py-3-8/lib/python3.8/site-packages/google/api_core/retry.py:281: in retry_wrapped_func
return retry_target(
.nox/py-3-8/lib/python3.8/site-packages/google/api_core/retry.py:184: in retry_target
return target()
.nox/py-3-8/lib/python3.8/site-packages/google/api_core/grpc_helpers.py:75: in error_remapped_callable
six.raise_from(exceptions.from_grpc_error(exc), exc)


value = None
from_value = <_InactiveRpcError of RPC that terminated with:
status = StatusCode.INTERNAL
details = "A service error has occurred...ror has occurred. Please retry your request. If the error persists, please report it. [code=e8c0]","grpc_status":13}"

???
E google.api_core.exceptions.InternalServerError: 500 A service error has occurred. Please retry your request. If the error persists, please report it. [code=e8c0]

:3: InternalServerError

@flaky-bot flaky-bot bot reopened this Mar 14, 2021
tmatsuo pushed a commit to tmatsuo/python-pubsub that referenced this issue Mar 15, 2021
anguillanneuf pushed a commit that referenced this issue Mar 15, 2021
anguillanneuf pushed a commit that referenced this issue Mar 16, 2021
anguillanneuf added a commit that referenced this issue Mar 30, 2021
…#324)

* Remove EXPERIMENTAL tag for ordering keys in publisher/client.py

* fix(sample): mark test flaky (#326)

* fix(sample): retry InternalServerError (#329)

fixes #321

Co-authored-by: Tianzi Cai <[email protected]>
Co-authored-by: Takashi Matsuo <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: pubsub Issues related to the googleapis/python-pubsub API. flakybot: flaky Tells the Flaky Bot not to close or comment on this issue. flakybot: issue An issue filed by the Flaky Bot. Should not be added manually. priority: p1 Important issue which blocks shipping the next release. Will be fixed prior to next release. samples Issues that are directly related to samples. type: bug Error or flaw in code with unintended results or allowing sub-optimal usage patterns.
Projects
None yet
0 participants