add 6 federation support with 3 sentry 3 operator nodes #216

README.md

@@ -1,4 +1,5 @@
 # Archipel
+
 ![stability-wip](https://img.shields.io/badge/stability-work_in_progress-lightgrey.svg)
 [![Website archipel.id](https://img.shields.io/badge/Website-archipel.id-brightgreen.svg)](https://archipel.id/)
 [![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://opensource.org/licenses/Apache-2.0)
@@ -7,47 +8,53 @@
 Welcome to **Archipel** monorepository!
 
 ## What is Archipel ?
-Archipel is a high availability solution for blockchain services. 
+
+Archipel is a high availability solution for blockchain services.
 
 ## Services Supported
-* Polkadot Validator Node
+
+- Polkadot Validator Node
 
 ## Why Archipel ?
+
 Nowadays, many blockchain services are centralized on cloud infrastructure. For instance, around [70% of Ethereum nodes are in VPC](https://twitter.com/DAppNode/status/1108693643320270848?s=20) and 63% of Ethereum Dapps use Infura Provider as [this survey](https://medium.com/fluence-network/dapp-survey-results-2019-a04373db6452) shows. We can imagine that in a few years some of blockchain services can be banned from public cloud providers.
 
-The solution is to have a decentralized infrastructure at home. The problem is that it is challenging to maintain a good quality of service at home. 
+The solution is to have a decentralized infrastructure at home. The problem is that it is challenging to maintain a good quality of service at home.
 You can have internet connection or power cuts. As a result, it is very unsafe to install a Proof-of-Stake validator at home. Your validator must be always up (24/7) and ready to execute its duty. If not, you will be slashed by the network and lose your money.
 To solve this problem, we are creating a solution to provide high availability for blockchain services. The first service that we target is Polkadot PoS Validator.
 
 ## How it works ?
+
 The idea behind Archipel is federating some nodes between friends and family.
-We are using [DAppNode](https://dappnode.io/) as the infrastructure layer for our solution. 
-With DAppNode, you can launch a blockchain node (Bitcoin, Ethereum ...) or any P2P software. 
+We are using [DAppNode](https://dappnode.io/) as the infrastructure layer for our solution.
+With DAppNode, you can launch a blockchain node (Bitcoin, Ethereum ...) or any P2P software.
 We would like to thank DAppNode Team for their amazing work.
 To achieve the high availability of services at a DAppNode, we are adding Archipel on top of it.
 
 ## Archipel Components
-| Component | Description |
-| --- | --- |
-| [Chain](chain/) | Chain component is responsible for Archipel state synchronization between participants |
-| [Orchestrator](orchestrator/) | Orchestrator is the decision-making component in Archipel federation |
-| [CLI](cli/) | CLI is a component that generates configuration and bootstraps an Archipel federation |
-| [UI](ui/) | UI is the Archipel chain state visualization component |
-| [Deployer](deployer/) | Archipel End-To-End tests, build scripts and deploy tools |
-| [DAppNodePackage](https://github.com/luguslabs/DAppNodePackage-archipel) | DAppNode package wrapping Archipel |
+
+| Component                                                                | Description                                                                            |
+| ------------------------------------------------------------------------ | -------------------------------------------------------------------------------------- |
+| [Chain](chain/)                                                          | Chain component is responsible for Archipel state synchronization between participants |
+| [Orchestrator](orchestrator/)                                            | Orchestrator is the decision-making component in Archipel federation                   |
+| [CLI](cli/)                                                              | CLI is a component that generates configuration and bootstraps an Archipel federation  |
+| [UI](ui/)                                                                | UI is the Archipel chain state visualization component                                 |
+| [Deployer](deployer/)                                                    | Archipel End-To-End tests, build scripts and deploy tools                              |
+| [DAppNodePackage](https://github.com/luguslabs/DAppNodePackage-archipel) | DAppNode package wrapping Archipel                                                     |
 
 Please refer to the README instructions in the sub-repositories for more information on building, using, and testing each software component.
 
 ## [Archipel Chain](chain/)
-To federate several nodes and have a shared state to elect a leader, we created a specific blockchain using the Substrate framework. 
 
-[Substrate](https://substrate.dev/) is a Parity framework that allows creating application-specific blockchains. 
+To federate several nodes and have a shared state to elect a leader, we created a specific blockchain using the Substrate framework.
 
-We created a Substrate runtime that collects all nodes metrics and sets federation leadership. This helps Archipel orchestrator to select the best leader appropriately in the federation. We call this specific blockchain the Archipel Substrate Chain or Archipel Chain. 
+[Substrate](https://substrate.dev/) is a Parity framework that allows creating application-specific blockchains.
 
-All nodes inside a federation, run Archipel Chain. In the current implementation, an Archipel must be composed of 3 nodes. That means that to operate, you have to set up 3 nodes. Try to set up nodes in different locations.
+We created a Substrate runtime that collects all nodes metrics and sets federation leadership. This helps Archipel orchestrator to select the best leader appropriately in the federation. We call this specific blockchain the Archipel Substrate Chain or Archipel Chain.
 
-The idea is that in the Archipel federation, all participants are trusted. They can be friends or family or other trusted social links. That allows us to have a fast chain consensus. 
+All nodes inside a federation, run Archipel Chain. In the current implementation, an Archipel must be composed of at least 3 nodes. That means that to operate, you have to set up at least 3 nodes. Try to set up nodes in different locations.
+
+The idea is that in the Archipel federation, all participants are trusted. They can be friends or family or other trusted social links. That allows us to have a fast chain consensus.
 
 More information on [chain/README.md](chain/README.md)
 
@@ -56,11 +63,43 @@ More information on [chain/README.md](chain/README.md)
 Orchestrator is the component that is responsible for decision making in an Archipel Federation.
 
 ### External services modes
-* **Active mode** - the service will be launched in active mode only on the leader node
-* **Passive mode** - the service will be launched in passive mode on all other non-leader nodes
+
+Archipel federation support only 3 nodes or 6 nodes.
+
+- **Operator Role** - the service will be launched in active mode if the node is elected or in passive mode in the others cases.
+- **Sentry Role** - the service will be launched in sentry mode and protect active and passive nodes from public exposition.
+  In 3 nodes federation, all 3 nodes are operator. Passive nodes act as sentry nodes for the current validator on a 3 nodes federation.
+  That means that at some point a validator node can be exposed if passive (sentry). To have a more secure setup use the 6 nodes federation config.
+  In 6 nodes federation, sentry nodes never because passive and active nodes. And passive and active nods never become sentry nodes during the HA orchestration.
+
+#### 3 nodes federation roles
+
+All 3 nodes are operator.
+| Node | Role |
+| ------------------------------------------------------------------- | ------------------------------------------------------------------------------- |
+| archipel-node-1-{active or passive} | operator |
+| archipel-node-2-{active or passive} | operator |
+| archipel-node-3-{active or passive} | operator |
+
+You must have the 3 operator nodes running to operate.
+
+#### 6 nodes federation roles
+
+3 nodes (1,2,3) are operator. 3 nodes ( 4,5,6) are sentry.
+| Node | Role |
+| ------------------------------------------------------------------- | ------------------------------------------------------------------------------- |
+| archipel-node-1-{active or passive} | operator |
+| archipel-node-2-{active or passive} | operator |
+| archipel-node-3-{active or passive} | operator |
+| archipel-node-4 | sentry |
+| archipel-node-5 | sentry |
+| archipel-node-6 | sentry |
+
+You must have the 3 operator nodes running to operate and at least one sentry node up. 2 sentry nodes up is better. 3 sentry nodes up is safer.
 
 ### Archipel Orchestrator Workflow
-- Launch external service in passive mode
+
+- Launch external service in passive mode or sentry node according to node role
 - Send node metrics to Archipel Chain
 - Retrieve other nodes metrics from Archipel Chain
 - Retrieve current leader from Archipel Chain and determine its availability
@@ -72,11 +111,20 @@ Orchestrator is the component that is responsible for decision making in an Arch
 
 ### Archipel Orchestrator for HA Polkadot Validator Setup
 
-The first service targeted by Archipel is Polkadot Validator. 
+The first service targeted by Archipel is Polkadot Validator.
+
+The Polkadot node can be launched :
+
+#### 3 nodes federation
 
-The Polkadot node can be launched in two modes:
-* **Active mode** - Polkadot node in with validator option
-* **Passive mode** - Polkadot node in the sync-only mode
+- **Active mode** - Polkadot node in with validator option
+- **Sentry-Passive mode** - Polkadot node in the sentry mode for the current validator
+
+#### 6 nodes federation
+
+- **Active mode** - Polkadot node in with validator option
+- **Passive mode** - Polkadot node in the sync-only mode and with reserved peers only
+- **Sentry mode** - Polkadot node in the sentry mode for passive and active others nodes.
 
 We are also planning to support other services.
 
@@ -106,7 +154,7 @@ More information on [deployer/README.md](deployer/README.md)
 
 ## [Archipel DAppNode Package](https://github.com/luguslabs/DAppNodePackage-archipel)
 
-DAppNode package wrapping Archipel stack. 
+DAppNode package wrapping Archipel stack.
 
 It allows installing Archipel from the DAppNode interface in one click.
 
@@ -115,19 +163,23 @@ More information on [DAppNodePackage-archipel/README.md](https://github.com/lugu
 ## [Additional Documentation Resources](doc/README.md)
 
 ### Keys Initialization doc
-* [Archipel Keys Initialization](doc/archipel-keys-initialization.md)
-* [Archipel Wireguard Keys Initialization](doc/wireguard-keys-initialization.md)
-* [Polkadot Keys Initialization](doc/polkadot-keys-initialization.md)
+
+- [Archipel Keys Initialization](doc/archipel-keys-initialization.md)
+- [Archipel Wireguard Keys Initialization](doc/wireguard-keys-initialization.md)
+- [Polkadot Keys Initialization](doc/polkadot-keys-initialization.md)
 
 ### Testing doc
-* [Archipel Federation Testing](doc/archipel-federation-testing.md)
-* [Archipel Test Results](doc/archipel-test-results.md)
+
+- [Archipel Federation Testing](doc/archipel-federation-testing.md)
+- [Archipel Test Results](doc/archipel-test-results.md)
 
 ### Deployment doc
-* [Archipel Deployment On DAppNodes](doc/deployment-on-dappnodes.md)
-* [Archipel Deployment With Docker](doc/deployment-with-docker.md)
+
+- [Archipel Deployment On DAppNodes](doc/deployment-on-dappnodes.md)
+- [Archipel Deployment With Docker](doc/deployment-with-docker.md)
 
 ## Acknowledgements
+
 <p align="center">
   <img src=./web3_foundation_grants_badge.svg width = 400>
 </p>

cli/src/cli.js

@@ -14,7 +14,7 @@ const runCli = async args => {
   try {
     // Set cli version
     program
-      .version('1.0.4');
+      .version('1.1.0');
 
     // Init command
     program

cli/src/generate.js

@@ -54,14 +54,15 @@ const validateConfigData = async configData => {
   isEmptyString(configData.publicIps, 'Nodes public ips were not set in archipel.json file.');
   isEmptyString(configData.service, 'Service name was not found in archipel.json file.');
   validatePublicIps(configData.publicIps);
+  validateNodesRole(configData.nodesRole);
   await validateServiceConfig(configData);
 };
 
 // Validate public ips list
 const validatePublicIps = ips => {
   const externalIPAddresses = ips.split(',');
-  if (externalIPAddresses.length !== 3) {
-    throw Error('For now Archipel supports only 3 nodes config.');
+  if (externalIPAddresses.length < 3) {
+    throw Error('Archipel must have at least 3 nodes config.');
   }
   // Check if every ip address in list is valid
   externalIPAddresses.forEach(element => {
@@ -71,6 +72,15 @@ const validatePublicIps = ips => {
   });
 };
 
+// Validate nodes role support
+const validateNodesRole = roles => {
+  roles.split(',').forEach(role => {
+    if (role != 'sentry' && role != 'operator') {
+      throw Error('Bad role node :' + role + '. Archipel node role must be sentry or operator');
+    }
+  })
+};
+
 // Generate config
 const generateConfig = async () => {
   try {
@@ -105,6 +115,8 @@ const generateConfig = async () => {
     // Add name to config
     config.name = configData.name.toLowerCase().replace(/\s/g, '-');
 
+    config.nodesRole = configData.nodesRole;
+
     // Add node number to config
     config.nodesNumber = externalIPAddresses.length;
 

cli/src/init.js

@@ -10,7 +10,8 @@ const {
 
 const configTemplate = {
   name: 'Archipel Name',
-  publicIps: '1.1.1.1,2.2.2.2,3.3.3.3'
+  publicIps: '1.1.1.1,2.2.2.2,3.3.3.3,4.4.4.4,5.5.5.5,6.6.6.6',
+  nodesRole: 'operator,operator,operator,sentry,sentry,sentry'
 };
 
 const initConfig = async service => {

deployer/archipel/start-orchestrator.sh

@@ -52,6 +52,19 @@ if [ ! -z "$CONFIG_FILE" ]; then
     fi
 
     #set variables from config file
+    #get NODES_ROLE
+    if [ -z "$NODES_ROLE" ]; then
+        NODES_ROLE=$(cat /config/config.json | jq ".nodesRole")
+        check_cmd $? 'retrieve NODES_ROLE'
+        if [ "$NODES_ROLE" == "null" ]; then
+            echo "Assure old config support. Force config NODES_ROLE to 'operator,operator,operator'"
+            NODES_ROLE="operator,operator,operator"
+        fi
+        IFS=',' read -ra rolesArray <<< "$NODES_ROLE"
+        index=$(( $NODE_ID - 1 ))
+        NODE_ROLE=${rolesArray[index]}
+    fi
+
     #get ARCHIPEL_KEY_SEED
     if [ -z "$ARCHIPEL_KEY_SEED" ]; then
         ARCHIPEL_KEY_SEED=$(cat /config/config.json | jq ".archipelNodes[$(( $NODE_ID - 1))].seed" | sed 's/\"//g')
@@ -80,9 +93,27 @@ if [ ! -z "$CONFIG_FILE" ]; then
 
 fi
 
+if [ -z "$NODES_ROLE" ]; then
+  echo "Assure old config support. Force config NODES_ROLE to 'operator,operator,operator'"
+  NODES_ROLE="operator,operator,operator"
+fi
+echo "NODES_ROLE=$NODES_ROLE"
+
+if [ -z "$NODE_ROLE" ]; then
+  echo "Assure old config support. Force config NODE_ROLE to 'operator'"
+  NODE_ROLE="operator"
+fi
+echo "NODE_ROLE=$NODE_ROLE"
+
+
+
 # Setting Archipel orchestrator variables
+NODE_ROLE=$(echo $NODE_ROLE | sed 's/\"//g')
+NODES_ROLE=$(echo $NODES_ROLE | sed 's/\"//g')
 export NODE_ENV=production
 export NODE_WS="ws://127.0.0.1:9944"
+export NODE_ROLE=$NODE_ROLE
+export NODES_ROLE=$NODES_ROLE
 export MNEMONIC="$ARCHIPEL_KEY_SEED"
 export NODES_WALLETS="$ARCHIPEL_AUTHORITIES_SR25519_LIST"
 export ARCHIPEL_NAME="$ARCHIPEL_NAME"

deployer/test/chain/keys/key4-ed25519.txt

@@ -0,0 +1,5 @@
+Secret phrase `ketchup produce seat decade denial open around hour suit benefit dream story` is account:
+  Secret seed:      0x15ec71e703a629071664c1f27d3b4e7bc078d8e26828a559766cdaa5ffcb904d
+  Public key (hex): 0x534e09369b0e2aac3ed2e16173ce1128cff14f56494e6a41ceb6e3462068a107
+  Account ID:       0x534e09369b0e2aac3ed2e16173ce1128cff14f56494e6a41ceb6e3462068a107
+  SS58 Address:     5Dww1iPtLbZTUjuXciBdki2Wtnh55M8g1FB4hx32JcSo4NMk

deployer/test/chain/keys/key4-node-key-file

@@ -0,0 +1 @@
+k��6�6!�2��B�h�^(���ș��[�@�s

deployer/test/chain/keys/key4-peer-id.txt

@@ -0,0 +1 @@
+QmS3H2mmb9bGGZo9BmRknPGJamg1DCGHbHvNhkkMw84HQ5

deployer/test/chain/keys/key4-polkadot-node-key-file

@@ -0,0 +1 @@
+��KY)g)�T�ܩ3�ր�7k��)�*

deployer/test/chain/keys/key4-polkadot-peer-id.txt

@@ -0,0 +1 @@
+QmSrQqZQp3PYFVFwM5mdFPvNVA7CqrnLhqKEzJAe1mCYwp

deployer/test/chain/keys/key4.txt

@@ -0,0 +1,5 @@
+Secret phrase `ketchup produce seat decade denial open around hour suit benefit dream story` is account:
+  Secret seed:      0x15ec71e703a629071664c1f27d3b4e7bc078d8e26828a559766cdaa5ffcb904d
+  Public key (hex): 0xc67dedb2272ff093c0c1032f5715916ddc5b6cdf4a6604be2094161d4d68aa0a
+  Account ID:       0xc67dedb2272ff093c0c1032f5715916ddc5b6cdf4a6604be2094161d4d68aa0a
+  SS58 Address:     5GYxkGrnJ9nkuhVyZ6Qf73CLwgu3tEKXP58KFRCiKjtEBHWm