Skip to content

Commit

Permalink
release-20.2: kvserver: prevent StoreRebalancer from downreplicating
Browse files Browse the repository at this point in the history
When the replication factor is lowered and the StoreRebalancer
attempts a rebalance, it will accidentally perform a downreplication.
Since it wasn't ever supposed to do that, the downreplication is
pretty haphazard and doesn't safeguard quorum in the same way
that a "proper" downreplication likely would. Prevent if from
changing the number of voters and non-voters to avoid this issue.

Annoyingly, I [knew] about this problem, but instead of fixing it at the
source - as this commit does - I added a lower- level check that could
then not be backported to release-20.2, where we are now seeing this
problem.

[knew]: cockroachdb#54444 (comment)

Release note: None
  • Loading branch information
tbg committed May 20, 2021
1 parent 986d236 commit 7601d76
Showing 1 changed file with 6 additions and 0 deletions.
6 changes: 6 additions & 0 deletions pkg/kv/kvserver/store_rebalancer.go
Original file line number Diff line number Diff line change
Expand Up @@ -506,6 +506,12 @@ func (sr *StoreRebalancer) chooseReplicaToRebalance(
targetReplicas := make([]roachpb.ReplicaDescriptor, 0, desiredReplicas)
currentReplicas := desc.Replicas().All()

if cur := len(desc.Replicas().Voters()); desiredReplicas != cur {
log.VEventf(ctx, 3, "cannot change number of voters from %d to %d for r%d",
cur, desiredReplicas, desc.RangeID)
continue
}

// Check the range's existing diversity score, since we want to ensure we
// don't hurt locality diversity just to improve QPS.
curDiversity := rangeDiversityScore(
Expand Down

0 comments on commit 7601d76

Please sign in to comment.