Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BadLogLines Unexpected error seastar::nested_exception in e2e_shadow_indexing_test.ShadowIndexingManyPartitionsTest.test_many_partitions_recovery #10706

Closed
bharathv opened this issue May 11, 2023 · 2 comments · Fixed by #10909
Assignees
Labels
area/cloud-storage Shadow indexing subsystem ci-failure kind/bug Something isn't working sev/low Bugs which are non-functional paper cuts, e.g. typos, issues in log messages

Comments

@bharathv
Copy link
Contributor

https://buildkite.com/redpanda/redpanda/builds/29014#01880cbb-6dc6-425d-8460-9e3fb83b362d

test_id:    rptest.tests.e2e_shadow_indexing_test.ShadowIndexingManyPartitionsTest.test_many_partitions_recovery
status:     FAIL
run time:   1 minute 26.304 seconds


    <BadLogLines nodes=docker-rp-6(1) example="ERROR 2023-05-11 22:30:33,807 [shard 0] s3 - util.cc:71 - Unexpected error seastar::nested_exception: seastar::abort_requested_exception (abort requested) (while cleaning up after seastar::abort_requested_exception (abort requested))">
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/dist-packages/ducktape/tests/runner_client.py", line 135, in run
    data = self.run_test()
  File "/usr/local/lib/python3.10/dist-packages/ducktape/tests/runner_client.py", line 227, in run_test
    return self.test_context.function(self.test)
  File "/root/tests/rptest/utils/mode_checks.py", line 63, in f
    return func(*args, **kwargs)
  File "/root/tests/rptest/services/cluster.py", line 87, in wrapped
    redpanda.raise_on_bad_logs(
  File "/root/tests/rptest/services/redpanda.py", line 2274, in raise_on_bad_logs
    raise BadLogLines(bad_lines)
rptest.services.utils.BadLogLines: <BadLogLines nodes=docker-rp-6(1) example="ERROR 2023-05-11 22:30:33,807 [shard 0] s3 - util.cc:71 - Unexpected error seastar::nested_exception: seastar::abort_requested_exception (abort requested) (while cleaning up after seastar::abort_requested_exception (abort requested))">
@bharathv bharathv added kind/bug Something isn't working ci-failure area/cloud-storage Shadow indexing subsystem labels May 11, 2023
@dotnwat dotnwat added the sev/high loss of availability, pathological performance degradation, recoverable corruption label May 12, 2023
@abhijat abhijat self-assigned this May 22, 2023
@abhijat
Copy link
Contributor

abhijat commented May 22, 2023

@dotnwat, after looking at logs this issue seems more like sev/low to me. This is an abort request exception wrapped in another abort requested exception caught when redpanda was shutting down.

Ordinarily it would have been logged below error level, but our exception handling mechanism tripped over the nested exception. Let me know if you have issues with changing the severity here.

I have raised PR #10909 to handle nested exceptions such as this one.

@dotnwat
Copy link
Member

dotnwat commented May 23, 2023

@abhijat thanks that makes sense! changed to sev/low

@dotnwat dotnwat added sev/low Bugs which are non-functional paper cuts, e.g. typos, issues in log messages and removed sev/high loss of availability, pathological performance degradation, recoverable corruption labels May 23, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/cloud-storage Shadow indexing subsystem ci-failure kind/bug Something isn't working sev/low Bugs which are non-functional paper cuts, e.g. typos, issues in log messages
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants