[BUG] Cluster manager bootstrap takes time causing intermittent failures in integration tests (o.o.c.ClusterHealthIT.testHealthOnClusterManagerFailover)

[dreamer-89]

Describe the bug
A clear and concise description of what the bug is.

To Reproduce
Steps to reproduce the behavior:

1. Run o.o.cluster.ClusterHealthIT.testHealthOnMasterFailover and enable index creation.
2. Reduce the master node timeout to <10 seconds.
3. Run test multiple times. It fails with >90% when 1 second timeout is used.

Expected behavior
Master node boot up time should stay less than < 1 minut.

Host/Environment (please complete the following information):

• OS: iOS

[dblock]

Failure in #1874 (comment) looks the same

> Task :server:internalClusterTest

REPRODUCE WITH: ./gradlew ':server:internalClusterTest' --tests "org.opensearch.cluster.ClusterHealthIT.testHealthOnMasterFailover" -Dtests.seed=2391EC7752804595 -Dtests.security.manager=true -Dtests.jvm.argline="-XX:TieredStopAtLevel=1 -XX:ReservedCodeCacheSize=64m" -Dtests.locale=es-EC -Dtests.timezone=Etc/Greenwich -Druntime.java=17

org.opensearch.cluster.ClusterHealthIT > testHealthOnMasterFailover FAILED
    java.lang.AssertionError: expected same:<RED> was not:<GREEN>
        at __randomizedtesting.SeedInfo.seed([2391EC7752804595:BD7EC35A516A5973]:0)
        at org.junit.Assert.fail(Assert.java:89)
        at org.junit.Assert.failNotSame(Assert.java:829)
        at org.junit.Assert.assertSame(Assert.java:772)
        at org.junit.Assert.assertSame(Assert.java:783)
        at org.opensearch.cluster.ClusterHealthIT.testHealthOnMasterFailover(ClusterHealthIT.java:393)

[saratvemulapalli]

Similar failure: #2037 #2047

[tlfeng]

For detail, please see issue #1693 for the error message of MasterNotDiscoveredException

[dblock]

Another one in #5354 (comment)

[rahulkarajgikar]

checking

[rahulkarajgikar]

5k runs on linux machine with 2 minutes, was able to see 5 failures.

5k runs on linux machine with 3 minutes, did not see any failures.

[rahulkarajgikar]

Raised PR to increase timeout: #13505