From 4370d21910e9f365ef7967c2f80ff1c75b956ff4 Mon Sep 17 00:00:00 2001 From: Anatoly Yakovenko Date: Fri, 1 Feb 2019 20:35:14 -0800 Subject: [PATCH 1/7] fork design book --- book/src/SUMMARY.md | 1 + book/src/fork-checkpoints.md | 82 ++++++++++++++++++++++++++++++++++++ 2 files changed, 83 insertions(+) create mode 100644 book/src/fork-checkpoints.md diff --git a/book/src/SUMMARY.md b/book/src/SUMMARY.md index 0944c0e889a235..5ecd80d0a8c7e5 100644 --- a/book/src/SUMMARY.md +++ b/book/src/SUMMARY.md @@ -30,6 +30,7 @@ - [Blocktree](blocktree.md) - [Data Plane Fanout](data-plane-fanout.md) - [Reliable Vote Transmission](reliable-vote-transmission.md) + - [Fork Deltas](fork-deltas.md) - [Cluster Economics](ed_overview.md) - [Validation-client Economics](ed_validation_client_economics.md) - [State-validation Protocol-based Rewards](ed_vce_state_validation_protocol_based_rewards.md) diff --git a/book/src/fork-checkpoints.md b/book/src/fork-checkpoints.md new file mode 100644 index 00000000000000..9667d32221942b --- /dev/null +++ b/book/src/fork-checkpoints.md @@ -0,0 +1,82 @@ +# Fork Checkpoints + +This design describes the software architecture for fork check pointing. It addresses the following challenges: + +* Forks are created at every block. +* Forks can be based on any previously produced block. +* Forks are eventually fully committed such that rollback is impossible. +* Unreachable forks need to be pruned. + +## Architecture + +The basic design idea is to maintain a DAG of forks. Each fork points back to a single ancestor. The DAG is initialized with a root. Each subsequent fork must descend from the root. + +## Active Chains + +An *active chain* is a direct list of connected forks that descend from the current root to a specific fork without any descendants. + +For example: +``` + 1 + / \ + 2 \ + /| | + / | | + 4 | | + 5 /\ + 6 \ + 7 +``` + +The following *acitve chains* are in the checkpoints DAG + +* 4,2,1 +* 5,2,1 +* 6,1 +* 7,1 + +## Merging into root + +A validator votes for a finalized fork. The *active chain* connecting the fork to the root is merged. If the *active chain* is longer than `Forks::ROLLBACK_DEPTH` the oldest two forks are merged. The oldest fork in the *active chain* is the current root, so the second oldest is a direct descendant of the root fork. Once merged, the current root is updated to the root descendant. Any forks that are not descendants from the new root are pruned since they are no longer reachable. + +For example: +``` + 1 + / \ + 2 \ + /| | + / | | + 4 | | + 5 /\ + 6 \ + 7 +``` + +* ROLLBACK\_DEPTH=2, vote=5, *active chain*={5,2,1} + +``` + 2 + /| + / | + 4 | + 5 + +``` + +The new root is 2, and any active chains that are not descendants from 2 are pruned. + +* ROLLBACK\_DEPTH=2, vote=6, *active chain*={6,1} + +``` + 1 + / \ + 2 \ + /| | + / | | + 4 | | + 5 /\ + 6 \ + 7 +``` + +The tree remains with `root=1`, since the *active chain* starting at 6 is only 2 forks long. From b85b9e0b3416e6a6072da3fb8fe85a646539754f Mon Sep 17 00:00:00 2001 From: Greg Fitzgerald Date: Wed, 6 Feb 2019 16:45:23 -0700 Subject: [PATCH 2/7] Cleanup --- book/src/fork-checkpoints.md | 82 ------------------------------------ book/src/fork-deltas.md | 82 ++++++++++++++++++++++++++++++++++++ 2 files changed, 82 insertions(+), 82 deletions(-) delete mode 100644 book/src/fork-checkpoints.md create mode 100644 book/src/fork-deltas.md diff --git a/book/src/fork-checkpoints.md b/book/src/fork-checkpoints.md deleted file mode 100644 index 9667d32221942b..00000000000000 --- a/book/src/fork-checkpoints.md +++ /dev/null @@ -1,82 +0,0 @@ -# Fork Checkpoints - -This design describes the software architecture for fork check pointing. It addresses the following challenges: - -* Forks are created at every block. -* Forks can be based on any previously produced block. -* Forks are eventually fully committed such that rollback is impossible. -* Unreachable forks need to be pruned. - -## Architecture - -The basic design idea is to maintain a DAG of forks. Each fork points back to a single ancestor. The DAG is initialized with a root. Each subsequent fork must descend from the root. - -## Active Chains - -An *active chain* is a direct list of connected forks that descend from the current root to a specific fork without any descendants. - -For example: -``` - 1 - / \ - 2 \ - /| | - / | | - 4 | | - 5 /\ - 6 \ - 7 -``` - -The following *acitve chains* are in the checkpoints DAG - -* 4,2,1 -* 5,2,1 -* 6,1 -* 7,1 - -## Merging into root - -A validator votes for a finalized fork. The *active chain* connecting the fork to the root is merged. If the *active chain* is longer than `Forks::ROLLBACK_DEPTH` the oldest two forks are merged. The oldest fork in the *active chain* is the current root, so the second oldest is a direct descendant of the root fork. Once merged, the current root is updated to the root descendant. Any forks that are not descendants from the new root are pruned since they are no longer reachable. - -For example: -``` - 1 - / \ - 2 \ - /| | - / | | - 4 | | - 5 /\ - 6 \ - 7 -``` - -* ROLLBACK\_DEPTH=2, vote=5, *active chain*={5,2,1} - -``` - 2 - /| - / | - 4 | - 5 - -``` - -The new root is 2, and any active chains that are not descendants from 2 are pruned. - -* ROLLBACK\_DEPTH=2, vote=6, *active chain*={6,1} - -``` - 1 - / \ - 2 \ - /| | - / | | - 4 | | - 5 /\ - 6 \ - 7 -``` - -The tree remains with `root=1`, since the *active chain* starting at 6 is only 2 forks long. diff --git a/book/src/fork-deltas.md b/book/src/fork-deltas.md new file mode 100644 index 00000000000000..d1e897a6a3fe06 --- /dev/null +++ b/book/src/fork-deltas.md @@ -0,0 +1,82 @@ +# Fork Deltas + +This design describes a way to checkpoint the bank state such that it can track multiple forks without duplicating data. It addresses the following challenges: + +* Forks are potentially created at every slot boundary. +* Forks can be based on any previously produced block. +* Forks are eventually finalized such that rollback is impossible. +* Unreachable forks need to be pruned. + +## Architecture + +The basic design idea is to maintain a DAG of forks. Each fork points back to a single ancestor. The DAG is initialized with a root. Each subsequent fork must descend from the root. + +## Active Forks + +An *active fork* is a direct list of connected forks that descend from the current root to a specific fork without any descendants. + +For example: +``` + 1 + / \ + 2 \ + /| | + / | | + 4 | | + 5 /\ + 6 \ + 7 +``` + +The following *active forks* are in the deltas DAG + +* 4,2,1 +* 5,2,1 +* 6,1 +* 7,1 + +## Merging into root + +A validator votes for a finalized fork. The *active fork* connecting the fork to the root is merged. If the *active fork* is longer than `Forks::ROLLBACK_DEPTH` the oldest two forks are merged. The oldest fork in the *active fork* is the current root, so the second oldest is a direct descendant of the root fork. Once merged, the current root is updated to the root descendant. Any forks that are not descendants from the new root are pruned since they are no longer reachable. + +For example: + +``` + 1 + / \ + 2 \ + /| | + / | | + 4 | | + 5 /\ + 6 \ + 7 +``` + +* ROLLBACK\_DEPTH=2, vote=5, *active fork*={5,2,1} + +``` + 2 + /| + / | + 4 | + 5 +``` + +The new root is 2, and any active forks that are not descendants from 2 are pruned. + +* ROLLBACK\_DEPTH=2, vote=6, *active fork*={6,1} + +``` + 1 + / \ + 2 \ + /| | + / | | + 4 | | + 5 /\ + 6 \ + 7 +``` + +The tree remains with `root=1`, since the *active fork* starting at 6 is only 2 forks long. From 16139cf5d7f32bbe27c4517efa57017196709a2c Mon Sep 17 00:00:00 2001 From: Greg Fitzgerald Date: Wed, 6 Feb 2019 16:53:05 -0700 Subject: [PATCH 3/7] Fix markdown --- book/src/fork-deltas.md | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/book/src/fork-deltas.md b/book/src/fork-deltas.md index d1e897a6a3fe06..efa9a124649a50 100644 --- a/book/src/fork-deltas.md +++ b/book/src/fork-deltas.md @@ -16,7 +16,8 @@ The basic design idea is to maintain a DAG of forks. Each fork points back to a An *active fork* is a direct list of connected forks that descend from the current root to a specific fork without any descendants. For example: -``` + +```art 1 / \ 2 \ @@ -41,7 +42,7 @@ A validator votes for a finalized fork. The *active fork* connecting the fork t For example: -``` +```art 1 / \ 2 \ @@ -55,7 +56,7 @@ For example: * ROLLBACK\_DEPTH=2, vote=5, *active fork*={5,2,1} -``` +```art 2 /| / | @@ -67,7 +68,7 @@ The new root is 2, and any active forks that are not descendants from 2 are prun * ROLLBACK\_DEPTH=2, vote=6, *active fork*={6,1} -``` +```art 1 / \ 2 \ From 342c783c19793a1cc16412b0830727c94d441937 Mon Sep 17 00:00:00 2001 From: Greg Fitzgerald Date: Thu, 7 Feb 2019 08:41:12 -0700 Subject: [PATCH 4/7] Render ASCII art --- book/art/forks-pruned.bob | 5 +++++ book/art/forks.bob | 9 ++++++++ book/src/fork-deltas.md | 44 ++++----------------------------------- 3 files changed, 18 insertions(+), 40 deletions(-) create mode 100644 book/art/forks-pruned.bob create mode 100644 book/art/forks.bob diff --git a/book/art/forks-pruned.bob b/book/art/forks-pruned.bob new file mode 100644 index 00000000000000..103f075fd3ffaf --- /dev/null +++ b/book/art/forks-pruned.bob @@ -0,0 +1,5 @@ + 2 + /| + / | + 4 | + 5 diff --git a/book/art/forks.bob b/book/art/forks.bob new file mode 100644 index 00000000000000..2ebb960054eaf0 --- /dev/null +++ b/book/art/forks.bob @@ -0,0 +1,9 @@ + 1 + / \ + 2 \ + /| | + / | | + 4 | | + 5 /\ + 6 \ + 7 diff --git a/book/src/fork-deltas.md b/book/src/fork-deltas.md index efa9a124649a50..ac97e6b12898f0 100644 --- a/book/src/fork-deltas.md +++ b/book/src/fork-deltas.md @@ -17,17 +17,7 @@ An *active fork* is a direct list of connected forks that descend from the curre For example: -```art - 1 - / \ - 2 \ - /| | - / | | - 4 | | - 5 /\ - 6 \ - 7 -``` +Forks The following *active forks* are in the deltas DAG @@ -42,42 +32,16 @@ A validator votes for a finalized fork. The *active fork* connecting the fork t For example: -```art - 1 - / \ - 2 \ - /| | - / | | - 4 | | - 5 /\ - 6 \ - 7 -``` +Forks * ROLLBACK\_DEPTH=2, vote=5, *active fork*={5,2,1} -```art - 2 - /| - / | - 4 | - 5 -``` +Forks after pruning The new root is 2, and any active forks that are not descendants from 2 are pruned. * ROLLBACK\_DEPTH=2, vote=6, *active fork*={6,1} -```art - 1 - / \ - 2 \ - /| | - / | | - 4 | | - 5 /\ - 6 \ - 7 -``` +Forks The tree remains with `root=1`, since the *active fork* starting at 6 is only 2 forks long. From 46086840fcc861f85b883e55345e2db01e2259db Mon Sep 17 00:00:00 2001 From: Greg Fitzgerald Date: Thu, 7 Feb 2019 08:42:01 -0700 Subject: [PATCH 5/7] Move to 80-char lines --- book/src/fork-deltas.md | 25 +++++++++++++++++++------ 1 file changed, 19 insertions(+), 6 deletions(-) diff --git a/book/src/fork-deltas.md b/book/src/fork-deltas.md index ac97e6b12898f0..0b67cf2ca7a14f 100644 --- a/book/src/fork-deltas.md +++ b/book/src/fork-deltas.md @@ -1,6 +1,8 @@ # Fork Deltas -This design describes a way to checkpoint the bank state such that it can track multiple forks without duplicating data. It addresses the following challenges: +This design describes a way to checkpoint the bank state such that it can track +multiple forks without duplicating data. It addresses the following +challenges: * Forks are potentially created at every slot boundary. * Forks can be based on any previously produced block. @@ -9,11 +11,14 @@ This design describes a way to checkpoint the bank state such that it can track ## Architecture -The basic design idea is to maintain a DAG of forks. Each fork points back to a single ancestor. The DAG is initialized with a root. Each subsequent fork must descend from the root. +The basic design idea is to maintain a DAG of forks. Each fork points back to +a single ancestor. The DAG is initialized with a root. Each subsequent fork +must descend from the root. ## Active Forks -An *active fork* is a direct list of connected forks that descend from the current root to a specific fork without any descendants. +An *active fork* is a direct list of connected forks that descend from the +current root to a specific fork without any descendants. For example: @@ -28,7 +33,13 @@ The following *active forks* are in the deltas DAG ## Merging into root -A validator votes for a finalized fork. The *active fork* connecting the fork to the root is merged. If the *active fork* is longer than `Forks::ROLLBACK_DEPTH` the oldest two forks are merged. The oldest fork in the *active fork* is the current root, so the second oldest is a direct descendant of the root fork. Once merged, the current root is updated to the root descendant. Any forks that are not descendants from the new root are pruned since they are no longer reachable. +A validator votes for a finalized fork. The *active fork* connecting the fork +to the root is merged. If the *active fork* is longer than +`Forks::ROLLBACK_DEPTH` the oldest two forks are merged. The oldest fork in +the *active fork* is the current root, so the second oldest is a direct +descendant of the root fork. Once merged, the current root is updated to the +root descendant. Any forks that are not descendants from the new root are +pruned since they are no longer reachable. For example: @@ -38,10 +49,12 @@ For example: Forks after pruning -The new root is 2, and any active forks that are not descendants from 2 are pruned. +The new root is 2, and any active forks that are not descendants from 2 are +pruned. * ROLLBACK\_DEPTH=2, vote=6, *active fork*={6,1} Forks -The tree remains with `root=1`, since the *active fork* starting at 6 is only 2 forks long. +The tree remains with `root=1`, since the *active fork* starting at 6 is only 2 +forks long. From ab825bd0ef48acacfd083cf1c8014f433cd0e3e0 Mon Sep 17 00:00:00 2001 From: Greg Fitzgerald Date: Thu, 7 Feb 2019 10:28:34 -0700 Subject: [PATCH 6/7] finalized -> frozen --- book/src/fork-deltas.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/book/src/fork-deltas.md b/book/src/fork-deltas.md index 0b67cf2ca7a14f..246fc249824dd9 100644 --- a/book/src/fork-deltas.md +++ b/book/src/fork-deltas.md @@ -33,7 +33,7 @@ The following *active forks* are in the deltas DAG ## Merging into root -A validator votes for a finalized fork. The *active fork* connecting the fork +A validator votes for a frozen fork. The *active fork* connecting the fork to the root is merged. If the *active fork* is longer than `Forks::ROLLBACK_DEPTH` the oldest two forks are merged. The oldest fork in the *active fork* is the current root, so the second oldest is a direct From 83b9e6b13ad13b6cfa29f46fd059d779b9c09286 Mon Sep 17 00:00:00 2001 From: Greg Fitzgerald Date: Thu, 7 Feb 2019 15:11:00 -0700 Subject: [PATCH 7/7] Rename, purge use of term delta This would be a fine document to introduce the term delta, but it looks like the content flows just fine without it. --- book/src/SUMMARY.md | 2 +- book/src/{fork-deltas.md => bank-forks.md} | 9 ++++----- 2 files changed, 5 insertions(+), 6 deletions(-) rename book/src/{fork-deltas.md => bank-forks.md} (86%) diff --git a/book/src/SUMMARY.md b/book/src/SUMMARY.md index 5ecd80d0a8c7e5..fb5a71d04cf41d 100644 --- a/book/src/SUMMARY.md +++ b/book/src/SUMMARY.md @@ -30,7 +30,7 @@ - [Blocktree](blocktree.md) - [Data Plane Fanout](data-plane-fanout.md) - [Reliable Vote Transmission](reliable-vote-transmission.md) - - [Fork Deltas](fork-deltas.md) + - [Bank Forks](bank-forks.md) - [Cluster Economics](ed_overview.md) - [Validation-client Economics](ed_validation_client_economics.md) - [State-validation Protocol-based Rewards](ed_vce_state_validation_protocol_based_rewards.md) diff --git a/book/src/fork-deltas.md b/book/src/bank-forks.md similarity index 86% rename from book/src/fork-deltas.md rename to book/src/bank-forks.md index 246fc249824dd9..5bed9fd2a9187d 100644 --- a/book/src/fork-deltas.md +++ b/book/src/bank-forks.md @@ -1,4 +1,4 @@ -# Fork Deltas +# Bank Fork This design describes a way to checkpoint the bank state such that it can track multiple forks without duplicating data. It addresses the following @@ -11,9 +11,8 @@ challenges: ## Architecture -The basic design idea is to maintain a DAG of forks. Each fork points back to -a single ancestor. The DAG is initialized with a root. Each subsequent fork -must descend from the root. +The basic design idea is to maintain a DAG of forks. The DAG is initialized with +a *root*. Each subsequent fork must descend from the root. ## Active Forks @@ -24,7 +23,7 @@ For example: Forks -The following *active forks* are in the deltas DAG +The following *active forks* are in the forks DAG * 4,2,1 * 5,2,1