CI Failure (timeout waiting for _topic_remote_deleted) in TopicDeleteCloudStorageTest.topic_delete_unavailable_test

[mmaslankaprv]

https://buildkite.com/redpanda/redpanda/builds/22844#01863582-a12f-4883-a955-081d10603729

Module: rptest.tests.topic_delete_test 
Class:  TopicDeleteCloudStorageTest 
Method: topic_delete_unavailable_test

====================================================================================================
test_id:    rptest.tests.topic_delete_test.TopicDeleteCloudStorageTest.topic_delete_unavailable_test
status:     FAIL
run time:   3 minutes 44.867 seconds


    TimeoutError('')
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/dist-packages/ducktape/tests/runner_client.py", line 135, in run
    data = self.run_test()
  File "/usr/local/lib/python3.10/dist-packages/ducktape/tests/runner_client.py", line 227, in run_test
    return self.test_context.function(self.test)
  File "/root/tests/rptest/services/cluster.py", line 35, in wrapped
    r = f(self, *args, **kwargs)
  File "/root/tests/rptest/tests/topic_delete_test.py", line 330, in topic_delete_unavailable_test
    wait_until(lambda: self._topic_remote_deleted(next_topic),
  File "/usr/local/lib/python3.10/dist-packages/ducktape/utils/util.py", line 57, in wait_until
    raise TimeoutError(err_msg() if callable(err_msg) else err_msg) from last_exception
ducktape.errors.TimeoutError

[jcsp]

This may be a duplicate of #8496, needs logs checked.

[dotnwat]

https://buildkite.com/redpanda/redpanda/builds/22881#0186373b-6243-4cc3-87ae-95f029b24e44

[michael-redpanda]

https://buildkite.com/redpanda/redpanda/builds/23384#01865aad-431a-4616-85d8-cda7558d442e

[ZeDRoman]

https://buildkite.com/redpanda/redpanda/builds/23367#018659fc-6e61-44f2-893f-bdb156b4c075

[bharathv]

Here are a couple of failures with the fix.

https://buildkite.com/redpanda/redpanda/builds/23654#018674d3-2175-4907-9d9a-b8e1f8abb960
https://buildkite.com/redpanda/redpanda/builds/23654#018674d3-2177-4195-b810-0a381eda5e73

INFO  2023-02-21 18:02:39,023 [shard 0] main - application.cc:324 - Redpanda v23.1.1-rc6-44-gf7b4556af - f7b4556afef2ff139e6f909ad0a68900725a9b0f-dirty

FAIL test: TopicDeleteCloudStorageTest.topic_delete_unavailable_test.cloud_storage_type=CloudStorageType.ABS (2/78 runs)
  failure at 2023-02-21T18:07:51.099Z: TimeoutError('')
      on (amd64, container) in job https://buildkite.com/redpanda/redpanda/builds/23654#018674d3-2175-4907-9d9a-b8e1f8abb960
FAIL test: TopicDeleteCloudStorageTest.topic_delete_unavailable_test.cloud_storage_type=CloudStorageType.S3 (2/78 runs)
  failure at 2023-02-21T18:06:23.550Z: TimeoutError('')
      on (amd64, container) in job https://buildkite.com/redpanda/redpanda/builds/23654#018674d3-2177-4195-b810-0a381eda5e73

[jcsp]

This was fixed in #9018, which skips the offending test in debug mode (because the test relies on a prompt leadership transfer to avoid leaving orphan objects behind)