rolling upgrade should use a replica to prevent relocations while run…

…ning a scroll

A scroll holds a reference to the shard store. If the cluster is moving shards
around that reference can prevent a shard from relocating back to node it used
to be on, causing test failures.

Closes #31827

x-pack/qa/rolling-upgrade/src/test/resources/rest-api-spec/test/mixed_cluster/10_basic.yml

@@ -6,7 +6,10 @@
         wait_for_active_shards: all
         body:
           settings:
-            number_of_replicas: 0
+# we use 1 replica to make sure we don't have shards relocating. Relocating a shard with
+# a scroll on it prevents shards from moving back into a where a scroll is running (it holds the shard lock)
+# see https://github.com/elastic/elasticsearch/issues/31827
+            number_of_replicas: 1
             index.routing.allocation.include.upgraded: true
 
   - do: