Does quorum=true eliminate the problems in #741?

[justinsb]

We're doing a multi-key operation in kubernetes, and I'm not 100% clear on the guarantees we get from etcd (given #741) I was hoping to get some clarification!

If we set quorum=true, then the read goes through Raft (so is slower), but is it true that:

• We will always see a version that is up-to-date at the leader (though of course the second the response is sent the cluster can advance)
• Because we had quorum, we do not need to worry about multiple nodes that temporarily think they are the leader (as identified in 741), because only one "leader" can have quorum

On the other hand, without quorum=true, it sounds like we might read stale versions because we might read from a node that is lagging.

For our particular problem (kubernetes/kubernetes#8295), I think one answer is to do consistent reads against etcd for our multi-key operations, otherwise it gets really tricky to reason about. I think by running reads through the quorum, we get ordering guarantees (read 2 is after read 1 => read 2 sees a later version than read 1)? Would we get the same guarantee though by comparing the raft-index at read 1 and 2, and simply retrying if index1 > index2?

[xiang90]

@justinsb etcd always consistent read. #741 turns out to be a linearizable problem. Multiple independent groups/people have done linearizable testing against etcd with q = true. The result is good.

[justinsb]

Thanks! Can we get the same behaviour with q=false by making sure that our reads always see a monotonic increasing sequence of raft-indexes?

(Here's what I'm trying to do: kubernetes/kubernetes#8295 (comment) - it is non-trivial to set q=true because of how the go-etcd library works)