Invalid Server Selection after Redis Cluster Failover

[leachdaniel-clark]

During maintenance we cycle our Redis Servers and failovers happen and cluster is becomes healthy again. However, even hours after doing an experiment where we rebooted all the servers, the ConnectionMultiplexer continues have issues with server section with a stack trace related to this. The only way to resolve the problem is to restart the service or recreate multiplexers. We've tried the using the latest client version 2.8.0 with HeartbeatConsistencyChecks enabled.

We are not using an CommandFlags to override server selection and are using the method overload:

KeyDeleteAsync(RedisKey[] keys, CommandFlags flags = CommandFlags.None);

Since Redis Cluster is not actually throwing this error the bug must be an issue with a mismatch in local state of this client triggered by server maintenance.

[leachdaniel-clark]

related to #1874 "Seamlessly handling Elasticache (AWS)" failovers but not specific to AWS

[leachdaniel-clark]