Reduce overhead of large numbers of snapshots in a single repository

[dakrone]

We currently recommend SLM limit the number of snapshots to < 200 in a single repository, however, with searchable snapshots we are likely to exceed this number quickly. Thankfully searchable snapshot snapshots are of a single index and therefore less overhead in the repository.

Regardless, we should attempt to switch our overhead from something like:

O(snapshots * average_number_indices_per_snapshots)

to

O(indices_count)

As this would allow us to better support clusters with 10,000+ searchable snapshot backed indices.

[elasticmachine]

Pinging @elastic/es-distributed (Team:Distributed)

[matschaffer]

For context, this may have a relation to https://cloud-status.elastic.co/incidents/wkctl07nqv80

[original-brownbear]

This is being worked on in #74350 now and considerable progress has been made already, closing this as a duplicate.