Fixed Raft voter priority override with single replica topics #10800
Conversation
|
|
||
| target = random.choice(self.redpanda.nodes) | ||
|
|
||
| self._enable_maintenance(target) |
There was a problem hiding this comment.
I wonder, what will happen with the maintenance status in this case? Presumably the operator will wait for the node to become fully drained before rebooting it and this will fail because leaders for single-replica topics have nowhere to move.
There was a problem hiding this comment.
Drain manager doesn't care about leaders of single replica partitions. It is the same for the case where a maintenance mode is enabled and node wasn't restarted - single replica partition leaders will stay in place.
|
Patch looks fine but I'm wondering how the only replica ended up leaderless in the first place, we have checks against it? The sequence of actions in drain_manager is..
(2) always returns error, no? |
Node restart is critical in this case |
mmaslankaprv
force-pushed
the
fix-single-replica-maintence-mode
branch
from
87088e2
to
0bb9778
Compare
mmaslankaprv
force-pushed
the
fix-single-replica-maintence-mode
branch
from
0bb9778
to
ce6d675
Compare
mmaslankaprv
force-pushed
the
fix-single-replica-maintence-mode
branch
from
ce6d675
to
2d3474e
Compare
mmaslankaprv
force-pushed
the
fix-single-replica-maintence-mode
branch
from
2d3474e
to
46b8dc0
Compare
mmaslankaprv
force-pushed
the
fix-single-replica-maintence-mode
branch
from
46b8dc0
to
241108f
Compare
Redpanda Raft implementation exposes an API allowing to override a voter priority. This is used by the drain manager when a node is in maintenance mode. In current implementation when the only voter is in maintenance mode the Raft group is not able to elect a leader as the reported priority it to low (the priority override in maintenance is set to 0). Fixed Raft implementation to make sure that it prioritize an availability over the user priority preference. If a node is the only voter the priority override is ignored. Fixes: redpanda-data/cloudv2#6174 Signed-off-by: Michal Maslanka <[email protected]>
Signed-off-by: Michal Maslanka <[email protected]>
Since now leader is elected earlier there is a race condition in updating Health Report when a single node starts. Made the timeout longer to allow the `feature_manager` to retry activating cluster version. Signed-off-by: Michal Maslanka <[email protected]>
mmaslankaprv
force-pushed
the
fix-single-replica-maintence-mode
branch
from
241108f
to
5ce6ef8
Compare
|
ci failure: #11454 |
|
/backport v23.1.x |
|
/backport v22.3.x |
|
/backport v22.2.x |
|
Failed to run cherry-pick command. I executed the commands below: |
|
Failed to run cherry-pick command. I executed the commands below: |
Redpanda Raft implementation exposes an API allowing to override a voter
priority. This is used by the drain manager when a node is in
maintenance mode. In current implementation when the only voter is in
maintenance mode the Raft group is not able to elect a leader as the
reported priority it to low (the priority override in maintenance is set to 0).
Fixed Raft implementation to make sure that it prioritize an
availability over the user priority preference. If a node is the only
voter the priority override is ignored.
Backports Required
Release Notes
Bug Fixes