Skip to content

Commit

Permalink
kvserver: resurrect using_applied_state_key in ReplicaState
Browse files Browse the repository at this point in the history
Fixes #72029.

using_applied_state_key was previously used to check whether the Range
had upgraded to begin using the RangeAppliedState key. When set to true,
replicas receiving the state (through a ReplicatedEvalResult) knew to
begin using the new key. In #58088 (in 21.1) we introduced a migration
to iterate through all ranges in the system and have them start using
the RangeAppliedState key. In 21.2, this field was always set to true --
21.2 nodes were already using the RangeAppliedState key, or receiving
messages from 21.1 nodes that were also using the RangeAppliedState key.
In 21.2 (and in 21.1 for that matter) we didn't need to trigger the "if
set to true in an incoming message, start using the RangeAppliedState
key" code path.

When looking to get rid of this field in 22.1 (#70464), we observed that
there was an unintentional read of this field in 21.2 nodes (see #72029 and
\#72222); the saga is as follows:
 - Removing this field in 22.1 meant it was set as false when received at
   21.2 nodes.
 - This should've been fine! We weren't using this field to trigger any
   upgrade code paths (like it was originally intended for).
 - It turns out that in 21.2 we were using the ReplicaState from the
   incoming snapshot to update our in-memory replica state
 - Because the proto field was being phased out, there was now a divergence
   between the on-disk state (field set to true, from earlier 21.2
   operations) and the in-memory state (field set to false, because sent
   from a version that attempted to get rid of this field).

Removing proto fields from the replica state are not possible until we stop
using the protobuf copy of the replica state when applying a snapshot
(#72222). Once that's done, we should be able to stop sending the replica
state as part of the snapshot in the subsequent release.

Release note: None
  • Loading branch information
irfansharif committed Oct 29, 2021
1 parent 60a41f6 commit 95e5b69
Show file tree
Hide file tree
Showing 5 changed files with 190 additions and 69 deletions.
209 changes: 141 additions & 68 deletions pkg/kv/kvserver/kvserverpb/state.pb.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading

0 comments on commit 95e5b69

Please sign in to comment.