CI Failure (PipeTimeout()) in ShadowIndexingManyPartitionsTest.test_many_partitions_shutdown

[r-vasquez]

https://buildkite.com/redpanda/redpanda/builds/31981#0188f955-b8cd-4aa3-b3a9-3f9120fe8c19/6-5544

Module: rptest.tests.e2e_shadow_indexing_test
Class:  ShadowIndexingManyPartitionsTest
Method: test_many_partitions_shutdown

====================================================================================================
test_id:    rptest.tests.e2e_shadow_indexing_test.ShadowIndexingManyPartitionsTest.test_many_partitions_shutdown
status:     FAIL
run time:   6 minutes 23.222 seconds


    TimeoutError()
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/dist-packages/paramiko/channel.py", line 699, in recv
    out = self.in_buffer.read(nbytes, self.timeout)
  File "/usr/local/lib/python3.10/dist-packages/paramiko/buffered_pipe.py", line 164, in read
    raise PipeTimeout()
paramiko.buffered_pipe.PipeTimeout

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/dist-packages/ducktape/tests/runner_client.py", line 135, in run
    data = self.run_test()
  File "/usr/local/lib/python3.10/dist-packages/ducktape/tests/runner_client.py", line 227, in run_test
    return self.test_context.function(self.test)
  File "/root/tests/rptest/utils/mode_checks.py", line 63, in f
    return func(*args, **kwargs)
  File "/root/tests/rptest/services/cluster.py", line 155, in wrapped
    self.redpanda.stop_and_scrub_object_storage()
  File "/root/tests/rptest/services/redpanda.py", line 3628, in stop_and_scrub_object_storage
    output = node.account.ssh_output(
  File "/usr/local/lib/python3.10/dist-packages/ducktape/cluster/remoteaccount.py", line 372, in ssh_output
    stdoutdata = stdout.read()
  File "/usr/local/lib/python3.10/dist-packages/paramiko/file.py", line 200, in read
    new_data = self._read(self._DEFAULT_BUFSIZE)
  File "/usr/local/lib/python3.10/dist-packages/paramiko/channel.py", line 1361, in _read
    return self.channel.recv(size)
  File "/usr/local/lib/python3.10/dist-packages/paramiko/channel.py", line 701, in recv
    raise socket.timeout()
TimeoutError

[bharathv]

https://buildkite.com/redpanda/redpanda/builds/32852#01894085-e863-4e04-9b8e-94c69b2b26ae

[abhijat]

https://buildkite.com/redpanda/redpanda/builds/32924#01894437-1a9e-4bc3-a758-10387205583c

[andijcr]

this should be fixed by #12047

[r-vasquez]

I saw this again in a v23.2.x branch, but it seems like the commit is already in that branch:

Build: https://buildkite.com/redpanda/redpanda/builds/37801#018ada0e-267b-4aef-92d4-493fc17c1b52/6-3940

Last commit of the PR: e998005