tests/integration: deflake Corruption cases

[fuweid]

If the corrupted member has been elected as leader, the memberID in alert response won't be the corrupted one. It will be a smaller follower ID since the raftCluster.Members always sorts by ID. We should check the leader ID and decide to use which memberID.

Fixes: #14823

Signed-off-by: Wei Fu [email protected]

Please read https://github.com/etcd-io/etcd/blob/main/CONTRIBUTING.md#contribution-flow.

[serathius]

Great find!

[serathius]

This flake shows that there is a issue with meaning of MemberID field. For corruption alarm it depend on which member is corrupted:

• If follower is corrupted, leader will set MemberID to corrupted follower.
• If leader is corrupted, leader will set MemberID to follower with lower ID (assuming that both followers are up during check).

Might be worth creating an issue to fix it, however I don't think this can be easily fixed.
cc @ahrtr @ptabor

At least I don't a solution now. First thoughts: validating that which member is corrupted requires separate quorum of members. In 3 node clusters you need all 3 members up (2 members for quorum + 1 corrupted member), in 5 node clusters you need (3 members for quorum and 1 corrupted member).

One possible fix would be reporting MemberId equal 0 and delaying setting true MemberId until enough members are up.

[ahrtr]

Thanks @fuweid for fixing the flaky test case, but it should be a product/etcd issue instead of a test case issue, so we should fix product code. At least, we should leave a TODO comment in the modified test case.

Actually I raised this comment when I was reviewing the code more than 4 months ago, #14120 (comment). We should depend on quorum to identify the corrupted member, let me deliver a PR for this.

[fuweid]

@ahrtr Looking forward to your pr!