SLM Throwing IllegalArgumentException during snapshot retention

[LukeEvans]

Elasticsearch version (bin/elasticsearch --version):
Found in 7.7.1

Description of the problem including expected versus actual behavior:

It seems while SLM is checking for eligible snapshots to remove, if the number successful snapshots is less than the minimumSnapshotCount, the following stack trace is seen:

[instance-0000000039] error during snapshot retention task
java.lang.IllegalArgumentException: -5
	at java.util.stream.ReferencePipeline.limit(ReferencePipeline.java:469) ~[?:?]
	at org.elasticsearch.xpack.core.slm.SnapshotRetentionConfiguration.lambda$getSnapshotDeletionPredicate$6(SnapshotRetentionConfiguration.java:195) ~[?:?]
	at org.elasticsearch.xpack.slm.SnapshotRetentionTask.snapshotEligibleForDeletion(SnapshotRetentionTask.java:245) ~[?:?]
	at org.elasticsearch.xpack.slm.SnapshotRetentionTask$1.lambda$onResponse$0(SnapshotRetentionTask.java:163) ~[?:?]
	at java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:176) ~[?:?]
	at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1624) ~[?:?]
	at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484) ~[?:?]
	at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474) ~[?:?]
	at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:913) ~[?:?]
	at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) ~[?:?]
	at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:578) ~[?:?]
	at org.elasticsearch.xpack.slm.SnapshotRetentionTask$1.lambda$onResponse$1(SnapshotRetentionTask.java:164) ~[?:?]
	at java.util.stream.Collectors.lambda$uniqKeysMapAccumulator$1(Collectors.java:178) ~[?:?]
	at java.util.stream.ReduceOps$3ReducingSink.accept(ReduceOps.java:169) ~[?:?]
	at java.util.concurrent.ConcurrentHashMap$EntrySpliterator.forEachRemaining(ConcurrentHashMap.java:3652) ~[?:?]
	at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484) ~[?:?]
	at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474) ~[?:?]
	at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:913) ~[?:?]
	at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) ~[?:?]
	at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:578) ~[?:?]
	at org.elasticsearch.xpack.slm.SnapshotRetentionTask$1.onResponse(SnapshotRetentionTask.java:161) ~[?:?]
	at org.elasticsearch.xpack.slm.SnapshotRetentionTask$1.onResponse(SnapshotRetentionTask.java:152) ~[?:?]
	at org.elasticsearch.xpack.slm.SnapshotRetentionTask.lambda$getAllRetainableSnapshots$9(SnapshotRetentionTask.java:267) ~[?:?]
	at org.elasticsearch.xpack.slm.SnapshotRetentionTask.lambda$getAllRetainableSnapshots$12(SnapshotRetentionTask.java:285) ~[?:?]
	at org.elasticsearch.action.ActionListener$1.onResponse(ActionListener.java:63) [elasticsearch-7.7.0.jar:7.7.0]
	at org.elasticsearch.action.support.ContextPreservingActionListener.onResponse(ContextPreservingActionListener.java:43) [elasticsearch-7.7.0.jar:7.7.0]
	at org.elasticsearch.action.support.TransportAction$1.onResponse(TransportAction.java:70) [elasticsearch-7.7.0.jar:7.7.0]
	at org.elasticsearch.action.support.TransportAction$1.onResponse(TransportAction.java:64) [elasticsearch-7.7.0.jar:7.7.0]
	at org.elasticsearch.action.support.ContextPreservingActionListener.onResponse(ContextPreservingActionListener.java:43) [elasticsearch-7.7.0.jar:7.7.0]
	at org.elasticsearch.action.ActionListener$2.onResponse(ActionListener.java:89) [elasticsearch-7.7.0.jar:7.7.0]
	at org.elasticsearch.action.admin.cluster.snapshots.get.TransportGetSnapshotsAction.masterOperation(TransportGetSnapshotsAction.java:151) [elasticsearch-7.7.0.jar:7.7.0]
	at org.elasticsearch.action.admin.cluster.snapshots.get.TransportGetSnapshotsAction.masterOperation(TransportGetSnapshotsAction.java:58) [elasticsearch-7.7.0.jar:7.7.0]
	at org.elasticsearch.action.support.master.TransportMasterNodeAction.masterOperation(TransportMasterNodeAction.java:99) [elasticsearch-7.7.0.jar:7.7.0]
	at org.elasticsearch.action.support.master.TransportMasterNodeAction$AsyncSingleAction.lambda$doStart$3(TransportMasterNodeAction.java:170) [elasticsearch-7.7.0.jar:7.7.0]
	at org.elasticsearch.action.ActionRunnable$2.doRun(ActionRunnable.java:73) [elasticsearch-7.7.0.jar:7.7.0]
	at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:692) [elasticsearch-7.7.0.jar:7.7.0]
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) [elasticsearch-7.7.0.jar:7.7.0]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) [?:?]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630) [?:?]
	at java.lang.Thread.run(Thread.java:832) [?:?]

Steps to reproduce:
Unknown

[elasticmachine]

Pinging @elastic/es-core-features (:Core/Features/ILM+SLM)

[dakrone]

I think I see how this can happen but it looks like @original-brownbear beat me to it, so I'll let him fix it :)

[original-brownbear]

@dakrone ah sorry I though I could fix it right away last night but then got distracted by other stuff and will be for the ~next two days. Please feel free to go ahead with fixing this one if you have the time. I'll reassign to you, also feel free to push it back to me if you don't have the time <= we should fix it swiftly for Cloud.

[original-brownbear]

Thanks for fixing this so quickly Lee!