AzureBlobContainerRetriesTests testRetryUntilFail fails

[mayya-sharipova]

Build scan:
https://gradle-enterprise.elastic.co/s/yb7lph3epqlqy

Repro line:
./gradlew ':plugins:repository-azure:test' --tests "org.elasticsearch.repositories.azure.AzureBlobContainerRetriesTests.testRetryUntilFail"
-Dtests.seed=E146AF3527EDBEB
-Dtests.security.manager=true
-Dbuild.snapshot=false
-Dtests.jvm.argline="-Dbuild.snapshot=false"
-Dtests.locale=en-IE
-Dtests.timezone=America/Fort_Wayne
-Druntime.java=8

Reproduces locally?:
No

Applicable branches:
7.11, 7.x, master

Failure history:
9 failures from Dec 22 to Dec 29

Failure excerpt:

org.elasticsearch.repositories.azure.AzureBlobContainerRetriesTests > testRetryUntilFail FAILED
09:10:38     java.lang.AssertionError: 
09:10:38     Expected: <3>
09:10:38          but: was <2>
09:10:38         at __randomizedtesting.SeedInfo.seed([E146AF3527EDBEB:E214676353BDDD3B]:0)
09:10:38         at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:18)
09:10:38         at org.junit.Assert.assertThat(Assert.java:956)
09:10:38         at org.junit.Assert.assertThat(Assert.java:923)
09:10:38         at org.elasticsearch.repositories.azure.AzureBlobContainerRetriesTests.testRetryUntilFail(AzureBlobContainerRetriesTests.java:440)

Possibly related to #66531

[elasticmachine]

Pinging @elastic/es-distributed (Team:Distributed)

[dimitris-athanasiou]

While the original issue was fixed, it seems this test may still fail: https://gradle-enterprise.elastic.co/s/3vbyqe7zniqii

Failure:

com.carrotsearch.randomizedtesting.RandomizedRunner$QueueUncaughtExceptionsHandler uncaughtException |  
-- | --
  | WARNING: Uncaught exception in thread: Thread[Thread-18,5,TGRP-AzureBlobContainerRetriesTests] |  
  | java.lang.AssertionError: |  
  | Expected: "connection closed before all data received" |  
  | but: was "Stream closed" |  
  | at __randomizedtesting.SeedInfo.seed([C84F4316D3721809]:0) |  
  | at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:18) |  
  | at org.junit.Assert.assertThat(Assert.java:956) |  
  | at org.junit.Assert.assertThat(Assert.java:923) |  
  | at org.elasticsearch.repositories.azure.AzureBlobContainerRetriesTests.lambda$testRetryUntilFail$7(AzureBlobContainerRetriesTests.java:415) |  
  | at com.sun.net.httpserver.Filter$Chain.doFilter(Filter.java:79) |  
  | at sun.net.httpserver.AuthFilter.doFilter(AuthFilter.java:83) |  
  | at com.sun.net.httpserver.Filter$Chain.doFilter(Filter.java:82) |  
  | at sun.net.httpserver.ServerImpl$Exchange$LinkHandler.handle(ServerImpl.java:675) |  
  | at com.sun.net.httpserver.Filter$Chain.doFilter(Filter.java:79) |  
  | at sun.net.httpserver.ServerImpl$Exchange.run(ServerImpl.java:647) |  
  | at sun.net.httpserver.ServerImpl$DefaultExecutor.execute(ServerImpl.java:158) |  
  | at sun.net.httpserver.ServerImpl$Dispatcher.handle(ServerImpl.java:431) |  
  | at sun.net.httpserver.ServerImpl$Dispatcher.run(ServerImpl.java:396) |  
  | at java.lang.Thread.run(Thread.java:748)