raft: consensus protocol design doc #100

SUMUKHA-PK · 2020-03-29T14:53:26Z

The first step towards having a consensus protocol running.
This'll serve as a spec to be followed for implementation.
Updates to this doc will be done when needed to accommodate issues.

Closes #21

Adds basic skeleton of the design doc

doc/internal/parser/scanner/Consensus-Protocol.md

tsatke · 2020-03-30T09:34:43Z

I like it! Looks good so far

Co-Authored-By: Tim Satke <[email protected]>

…ensus-protocol-Design-Doc

SUMUKHA-PK · 2020-04-10T05:24:53Z

Moved.

doc/internal/consensus/Consensus-Protocol.md

Abby3017

What about membership change i.e if node added or deleted ?

doc/internal/consensus/Consensus-Protocol.md

SUMUKHA-PK · 2020-05-06T18:26:04Z

@TimSatke @Abby3017
Your review happened on the older version, I was still writing the doc.
Please do have a look now.

tsatke

Please also check for whitespace errors in the lower part of the document.
Other than that, this is a design document, but instead of designing the system, it is basically just a checklist what needs to be done.
Please add an architecture. This means:

include, which components will be implemented
how will they interact with each other
how will they fit into the existing system
what will the APIs look like
Maybe even draft some Go interface types that show the intended usage of the raft package.

doc/internal/consensus/Consensus-Protocol.md

tsatke · 2020-05-07T17:00:51Z

Also important, but optional for this PR, specify the protobuf messages that need to be used.
@SUMUKHA-PK you can do this in here if you like, otherwise, we can get in touch, so that I know what requirements are there, and what needs to be communicated.
A protocol draft would be nice however (what messages are sent (election, appendentries, initial handshake, goodbye, requests))

SUMUKHA-PK · 2020-05-07T17:44:38Z

Looks like a lot of work here.

So I'd better get into this before I even think of implementing.

Co-authored-by: Tim Satke <[email protected]>

SUMUKHA-PK · 2020-05-08T04:19:22Z

What about membership change i.e if node added or deleted ?

@Abby3017

Currently, doesn't fall into the domain of what we want to implement. It's kind of an extended issue, we'll tackle that if and when needed.

SUMUKHA-PK · 2020-05-08T04:53:22Z

Please also check for whitespace errors in the lower part of the document.
Other than that, this is a design document, but instead of designing the system, it is basically just a checklist what needs to be done.
Please add an architecture. This means:
* include, which components will be implemented

* how will they interact with each other

* how will they fit into the existing system

* what will the APIs look like
  Maybe even draft some Go interface types that show the intended usage of the `raft` package.

Following is how each issue is tackled:

The main modules represent the components that will be implemented.
A separate module interaction section is added.
They'll fit in as a separate module on internal dir; mentioned in the beginning.

As far as the API goes, I think it'll need more time.
I wish to begin an implementation and keep adding small patches to this because I think that'd be better.

Let me know if I misinterpreted something or anything else needs to be added.

tsatke · 2020-05-08T07:35:02Z

doc/internal/consensus/Consensus-Protocol.md

 * Security: Access control mechanisms need to be in place to decide on access to functions in the servers based on their state (leader, follower, candidate)
-* Routing to leader: One of the issues with a varying leader is for the clients to know which IP address to contact for the service. We can solve this problem by advertising any/all IPs of the cluster and simply forward this request to the current leader; OR have a proxy that can forward the request to the current leader wheneve the requests come in. (Section client interaction of post has another approach which works too)
+* Routing to leader: One of the issues with a varying leader is for the clients to know which IP address to contact for the service. We can solve this problem by advertising any/all IPs of the cluster and the client returns the IP of the leader if its not the leader.


I don't understand. Any node knows all IPs and can answer that question. Why should follower nodes only respond with the leader IP?

At any point of time when raft is in the "working phase" (which is the leader is up, no election is happenning), there are only 2 kinds of nodes; the leader and the follower. So if the client doesn't hit the leader, it means it hit the follower.

doc/internal/consensus/Consensus-Protocol.md

Co-authored-by: Tim Satke <[email protected]>

tsatke · 2020-05-17T13:36:56Z

doc/internal/consensus/Consensus-Protocol.md

+* A raft server is implemented as:
+```
+type simpleServer struct {
+	node          *Node
+	cluster       Cluster
+	onReplication ReplicationHandler
+	log           zerolog.Logger
+}
+
+type Node struct {
+	State string
+
+	PersistentState     *PersistentState
+	VolatileState       *VolatileState
+	VolatileStateLeader *VolatileStateLeader
+}
+


why is there implementation in a design doc?

Well if I add the API, they struct's would be a good reference to have, was my thought.

Keep in mind that you are writing a design doc, not an implementation guide. Such a concrete struct may have heavy implications on the rest of the implementation. The point of a design doc is, to just outline the components and how they interact, and leave the concrete implementation to the developer.

tsatke · 2020-05-17T13:37:30Z

doc/internal/consensus/Consensus-Protocol.md

@@ -64,7 +138,7 @@ A detailed description of all the modules and their implementation follow:
 * A committed entry: When a leader decides that the log entry is safe to apply to other state machines, that entry is called committed. All committed entries are durable and _will eventually be executed_ by all state machines.
 * An entry -> Committed entry: A log entry is called committed once its replicated on the majority of the servers in the cluster. Once an entry is committed, it commits all the previous entries in the leaders log, including the entries created by the previous leaders.
 * The  leader keeps track of the highest known index that it knows is committed and it is included in all the future `AppendEntriesRPC` (including heartbeats) to inform other servers.
-* Theres some issue about log committing - "A log entry is committed once the leader that createdthe entry has replicated it on a majority of the servers" and " Once a follower learns that a log entry is committed, it applies theentry to its local state machine (in log order)." are not clear whether replicating and applying to state machine are the same. If they are its kind of a contradiction, else "application" can mean executing the STMT in the DB in our case.
+* Theres some issue about log committing - "A log entry is committed once the leader that created the entry has replicated it on a majority of the servers" and " Once a follower learns that a log entry is committed, it applies theentry to its local state machine (in log order)." are not clear whether replicating and applying to state machine are the same. If they are its kind of a contradiction, else "application" can mean executing the STMT in the DB in our case.


resolve the issue and add one or more solutions to this doc

SUMUKHA-PK added 2 commits March 29, 2020 16:59

Create Consensus-Protocol.md

5a7a27b

Adds basic skeleton of the design doc

Update Consensus-Protocol.md

df185c3

tsatke reviewed Mar 30, 2020

View reviewed changes

doc/internal/parser/scanner/Consensus-Protocol.md Outdated Show resolved Hide resolved

Merge branch 'master' into Consensus-protocol-Design-Doc

9657559

SUMUKHA-PK and others added 9 commits March 30, 2020 15:05

Update doc/internal/parser/scanner/Consensus-Protocol.md

099498a

Co-Authored-By: Tim Satke <[email protected]>

Update Consensus-Protocol.md

43e600e

Update Consensus-Protocol.md

bd90b1f

Update Consensus-Protocol.md

7d26490

Update Consensus-Protocol.md

5923f4f

Update Consensus-Protocol.md

02239c8

Update Consensus-Protocol.md

2b900c0

Merge branch 'master' of https://github.com/tomarrell/lbadd into Cons…

8527df5

…ensus-protocol-Design-Doc

Moved doc to appropriate folder

b18332a

SUMUKHA-PK added 2 commits May 1, 2020 18:28

Update Consensus-Protocol.md

b11486e

Merge branch 'master' into Consensus-protocol-Design-Doc

4e559a5

tsatke reviewed May 6, 2020

View reviewed changes

doc/internal/consensus/Consensus-Protocol.md Outdated Show resolved Hide resolved

tsatke reviewed May 6, 2020

View reviewed changes

doc/internal/consensus/Consensus-Protocol.md Outdated Show resolved Hide resolved

Abby3017 reviewed May 6, 2020

View reviewed changes

doc/internal/consensus/Consensus-Protocol.md Show resolved Hide resolved

Abby3017 reviewed May 6, 2020

View reviewed changes

Update Consensus-Protocol.md

7eef110

Abby3017 reviewed May 6, 2020

View reviewed changes

doc/internal/consensus/Consensus-Protocol.md Outdated Show resolved Hide resolved

Merge branch 'master' into Consensus-protocol-Design-Doc

9b209f1

SUMUKHA-PK marked this pull request as ready for review May 6, 2020 18:31

SUMUKHA-PK requested a review from tomarrell as a code owner May 6, 2020 18:31

SUMUKHA-PK requested review from tsatke and removed request for tomarrell May 7, 2020 05:35

Merge branch 'master' into Consensus-protocol-Design-Doc

17a8e1a

Merge branch 'master' into Consensus-protocol-Design-Doc

1cb7555

tsatke suggested changes May 7, 2020

View reviewed changes

SUMUKHA-PK and others added 6 commits May 7, 2020 23:15

Update doc/internal/consensus/Consensus-Protocol.md

1209205

Co-authored-by: Tim Satke <[email protected]>

Apply suggestions from code review

8d25041

Co-authored-by: Tim Satke <[email protected]>

Apply suggestions from code review

01d0c32

Co-authored-by: Tim Satke <[email protected]>

Apply suggestions from code review

d354ab9

Co-authored-by: Tim Satke <[email protected]>

Merge branch 'master' into Consensus-protocol-Design-Doc

2e71890

Update Consensus-Protocol.md

2a8c4d9

SUMUKHA-PK added 2 commits May 8, 2020 10:23

Update Consensus-Protocol.md

c32828c

Update Consensus-Protocol.md

6815e8c

tsatke suggested changes May 8, 2020

View reviewed changes

SUMUKHA-PK and others added 5 commits May 8, 2020 13:41

Update doc/internal/consensus/Consensus-Protocol.md

281906e

Co-authored-by: Tim Satke <[email protected]>

Update doc/internal/consensus/Consensus-Protocol.md

b180d33

Co-authored-by: Tim Satke <[email protected]>

Merge branch 'master' into Consensus-protocol-Design-Doc

981d873

Update Consensus-Protocol.md

fd1f272

Update Consensus-Protocol.md

a16b909

tsatke suggested changes May 17, 2020

View reviewed changes

tsatke changed the title ~~Consensus protocol design doc~~ raft: consensus protocol design doc May 25, 2020

tsatke added this to the v0.0.1 milestone May 25, 2020

tsatke assigned Abby3017 and SUMUKHA-PK Jul 19, 2020

Merge branch 'master' into Consensus-protocol-Design-Doc

c312812

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

raft: consensus protocol design doc #100

raft: consensus protocol design doc #100

SUMUKHA-PK commented Mar 29, 2020 •

edited by tsatke

Loading

tsatke commented Mar 30, 2020

SUMUKHA-PK commented Apr 10, 2020

Abby3017 left a comment

SUMUKHA-PK commented May 6, 2020

tsatke left a comment

tsatke commented May 7, 2020

SUMUKHA-PK commented May 7, 2020

SUMUKHA-PK commented May 8, 2020 •

edited

Loading

SUMUKHA-PK commented May 8, 2020

tsatke May 8, 2020

SUMUKHA-PK May 8, 2020

tsatke May 17, 2020

SUMUKHA-PK May 17, 2020

tsatke May 18, 2020

tsatke May 17, 2020

raft: consensus protocol design doc #100

Are you sure you want to change the base?

raft: consensus protocol design doc #100

Conversation

SUMUKHA-PK commented Mar 29, 2020 • edited by tsatke Loading

tsatke commented Mar 30, 2020

SUMUKHA-PK commented Apr 10, 2020

Abby3017 left a comment

Choose a reason for hiding this comment

SUMUKHA-PK commented May 6, 2020

tsatke left a comment

Choose a reason for hiding this comment

tsatke commented May 7, 2020

SUMUKHA-PK commented May 7, 2020

SUMUKHA-PK commented May 8, 2020 • edited Loading

SUMUKHA-PK commented May 8, 2020

tsatke May 8, 2020

Choose a reason for hiding this comment

SUMUKHA-PK May 8, 2020

Choose a reason for hiding this comment

tsatke May 17, 2020

Choose a reason for hiding this comment

SUMUKHA-PK May 17, 2020

Choose a reason for hiding this comment

tsatke May 18, 2020

Choose a reason for hiding this comment

tsatke May 17, 2020

Choose a reason for hiding this comment

SUMUKHA-PK commented Mar 29, 2020 •

edited by tsatke

Loading

SUMUKHA-PK commented May 8, 2020 •

edited

Loading