Merge #335

335: Implement real vs mock protocol selection r=Jimbo4350 a=Jimbo4350 Implements the `NodeProtocolMode` type which differentiates whether the node is running a real protocol vs a mock protocol. Changes: - When running a real protocol, the topology comes from a `JSON` file which is decoded to `RealNodeTopology` which specifies the addresses the node intends to connect to. - `socket-dir` command line argument is changed to `socket-path` i.e you must specify socket paths and not dirs anymore. - `NodeId` only needs to be specified for mock protocols (https://github.com/input-output-hk/cardano-node/blob/d24cefd7a4fb1afbf13e8e7f7971e22704673f98/cardano-config/src/Cardano/Config/Protocol.hs#L129) The `cardano-node` client is now as follows (with further help for both `run` and `run-mock` via `cabal exec cardano-node run/run-mock -- --help`): ``` Usage: cardano-node (run | run-mock) [--help] Start node of the Cardano blockchain. Available options: --help Show this help text Execute node with a real protocol. run Execute node with a real protocol. Execute node with a mock protocol. run-mock Execute node with a mock protocol. ``` Relevant: #297, #318, #314 Co-authored-by: Jordan Millar <[email protected]>
IntersectMBO · Feb 3, 2020 · 448369f · 448369f
2 parents 23c0855 + caf342f
commit 448369f
Show file tree

Hide file tree

Showing 64 changed files with 692 additions and 472 deletions.
diff --git a/benchmarking/chain-sync/benchmark-chain-sync-mainnet.sh b/benchmarking/chain-sync/benchmark-chain-sync-mainnet.sh
@@ -39,7 +39,7 @@ exec ${NODE} \
   --genesis-hash "5f20df933584822601f9e3f8c024eb5eb252fe8cefb24d1317dc3d432e940ebb" \
   --config ${BASEDIR}/configuration/log-configuration.yaml \
   --database-path .//db-mainnet \
-  --socket-dir /tmp/socket-bm-chain-sync \
+  --socket-path /tmp/socket-bm-chain-sync \
   --topology ${BASEDIR}/configuration/topology-local.yaml \
   --host-addr 127.0.0.1 \
   --port 7778 \

diff --git a/benchmarking/chain-sync/configuration/log-config-ci.yaml b/benchmarking/chain-sync/configuration/log-config-ci.yaml
@@ -77,7 +77,7 @@ options:
 ##########################################################
 
 
-NodeId: 0
+NodeId:
 Protocol: RealPBFT
 NumCoreNodes: 1
 RequiresNetworkMagic: RequiresNoMagic

diff --git a/benchmarking/chain-sync/configuration/log-configuration.yaml b/benchmarking/chain-sync/configuration/log-configuration.yaml
@@ -85,7 +85,7 @@ options:
 ##########################################################
 
 
-NodeId: 0
+NodeId:
 Protocol: RealPBFT
 NumCoreNodes: 1
 RequiresNetworkMagic: RequiresNoMagic

diff --git a/benchmarking/chain-sync/configuration/topology-local.yaml b/benchmarking/chain-sync/configuration/topology-local.yaml
@@ -1 +1,9 @@
-[{"nodeAddress":{"addr":"127.0.0.1","port":3001},"nodeId":0,"producers":[{"addr":"127.0.0.1","port":7777,"valency":1}]}]
+{
+   "Producers":[
+      {
+         "addr":"127.0.0.1",
+         "port":7777,
+         "valency":1
+      }
+   ]
+}
diff --git a/benchmarking/cluster3nodes/configuration/log-config-0.yaml b/benchmarking/cluster3nodes/configuration/log-config-0.yaml
@@ -75,7 +75,7 @@ options:
 ##########################################################
 
 
-NodeId: 0
+NodeId:
 Protocol: RealPBFT
 NumCoreNodes: 1
 RequiresNetworkMagic: RequiresMagic

diff --git a/benchmarking/cluster3nodes/configuration/log-config-1.yaml b/benchmarking/cluster3nodes/configuration/log-config-1.yaml
@@ -77,7 +77,7 @@ options:
 ##########################################################
 
 
-NodeId: 1
+NodeId:
 Protocol: RealPBFT
 NumCoreNodes: 1
 RequiresNetworkMagic: RequiresMagic

diff --git a/benchmarking/cluster3nodes/configuration/log-config-2.yaml b/benchmarking/cluster3nodes/configuration/log-config-2.yaml
@@ -81,7 +81,7 @@ options:
 ##########################################################
 
 
-NodeId: 2
+NodeId:
 Protocol: RealPBFT
 NumCoreNodes: 1
 RequiresNetworkMagic: RequiresMagic

diff --git a/benchmarking/cluster3nodes/configuration/log-config-generator.yaml b/benchmarking/cluster3nodes/configuration/log-config-generator.yaml
@@ -81,7 +81,7 @@ options:
 ##########################################################
 
 
-NodeId: 99
+NodeId:
 Protocol: RealPBFT
 NumCoreNodes: 1
 RequiresNetworkMagic: RequiresMagic

diff --git a/benchmarking/cluster3nodes/configuration/log-config-genesis.yaml b/benchmarking/cluster3nodes/configuration/log-config-genesis.yaml
@@ -49,7 +49,7 @@ options:
 ##########################################################
 
 
-NodeId: 0
+NodeId:
 Protocol: RealPBFT
 NumCoreNodes: 1
 RequiresNetworkMagic: RequiresMagic

diff --git a/benchmarking/cluster3nodes/configuration/simple-topology-real-pbft-node-0.json b/benchmarking/cluster3nodes/configuration/simple-topology-real-pbft-node-0.json
@@ -0,0 +1,15 @@
+
+  {
+     "Producers":[
+        {
+           "addr":"127.0.0.1",
+           "port":3001,
+           "valency":1
+        },
+        {
+           "addr":"127.0.0.1",
+           "port":3002,
+           "valency":1
+        }
+     ]
+  }
diff --git a/benchmarking/cluster3nodes/configuration/simple-topology-real-pbft-node-1.json b/benchmarking/cluster3nodes/configuration/simple-topology-real-pbft-node-1.json
@@ -0,0 +1,14 @@
+{
+  "Producers":[
+     {
+        "addr":"127.0.0.1",
+        "port":3000,
+        "valency":1
+     },
+     {
+        "addr":"127.0.0.1",
+        "port":3002,
+        "valency":1
+     }
+  ]
+}
diff --git a/benchmarking/cluster3nodes/configuration/simple-topology-real-pbft-node-2.json b/benchmarking/cluster3nodes/configuration/simple-topology-real-pbft-node-2.json
@@ -0,0 +1,14 @@
+{
+  "Producers":[
+     {
+        "addr":"127.0.0.1",
+        "port":3000,
+        "valency":1
+     },
+     {
+        "addr":"127.0.0.1",
+        "port":3001,
+        "valency":1
+     }
+  ]
+}
diff --git a/benchmarking/cluster3nodes/run-3node-cluster.sh b/benchmarking/cluster3nodes/run-3node-cluster.sh
@@ -41,16 +41,16 @@ function dlgcert () {
         printf -- "--delegation-certificate ${genesis_root}/delegation-cert.%03d.json " "$1"
 }
 function commonargs() {
-        printf -- "--topology configuration/simple-topology.json "
-        printf -- "--database-path ./db/ "
+        printf -- "--topology configuration/simple-topology-real-pbft-node-$1.json "
+        printf -- "--database-path ./db-$1/ "
         printf -- "--genesis-file ${genesis_file} "
         printf -- "--genesis-hash ${genesis_hash} "
-        printf -- "--socket-dir /tmp/cluster3nodes-socket/ "
+        printf -- "--socket-path /tmp/cluster3nodes-socket/$1 "
 }
 
 function nodeargs () {
         local extra="$2"
-        commonargs
+        commonargs $1
         nodecfg $1
         dlgkey $1
         dlgcert $1
@@ -65,8 +65,8 @@ tmux select-pane -t 0
 
 # start nodes
 tmux select-pane -t 0
-tmux send-keys "cd '${BASEPATH}'; ${CMD} exe:cardano-node $(nodeargs 0 "$(echo -n ${EXTRA})") " C-m
+tmux send-keys "cd '${BASEPATH}'; ${CMD} exe:cardano-node run $(nodeargs 0 "$(echo -n ${EXTRA})") " C-m
 tmux select-pane -t 1
-tmux send-keys "cd '${BASEPATH}'; ${CMD} exe:cardano-node $(nodeargs 1 "$(echo -n ${EXTRA})") " C-m
+tmux send-keys "cd '${BASEPATH}'; ${CMD} exe:cardano-node run $(nodeargs 1 "$(echo -n ${EXTRA})") " C-m
 tmux select-pane -t 2
-tmux send-keys "cd '${BASEPATH}'; ${CMD} exe:cardano-node $(nodeargs 2 "$(echo -n ${EXTRA})") " C-m
+tmux send-keys "cd '${BASEPATH}'; ${CMD} exe:cardano-node run $(nodeargs 2 "$(echo -n ${EXTRA})") " C-m
diff --git a/benchmarking/cluster3nodes/run_tx_generator.sh b/benchmarking/cluster3nodes/run_tx_generator.sh
@@ -25,7 +25,7 @@ exec ${GENERATOR} \
   --delegation-certificate ${CONFIGDIR}/latest-genesis/delegation-cert.000.json \
   --genesis-file ${GENESISJSON} \
   --genesis-hash ${GENESISHASH} \
-  --socket-dir /tmp/cluster3nodes-socket \
+  --socket-path /tmp/cluster3nodes-socket/0 \
   --real-pbft \
   --num-of-txs $numtx \
   --add-tx-size $addsizetx \
@@ -36,5 +36,4 @@ exec ${GENERATOR} \
   --sig-key ${CONFIGDIR}/latest-genesis/delegate-keys.000.key \
   --sig-key ${CONFIGDIR}/latest-genesis/delegate-keys.001.key \
   --sig-key ${CONFIGDIR}/latest-genesis/delegate-keys.002.key \
-  --node-id 0 \
   ${TARGETNODES}
diff --git a/cardano-config/src/Cardano/Config/Logging.hs b/cardano-config/src/Cardano/Config/Logging.hs
@@ -22,8 +22,6 @@ module Cardano.Config.Logging
   , mkLOMeta
   , LOMeta (..)
   , LOContent (..)
-  -- CLI argument parser
-  , LoggingCLIArguments (..)
   ) where
 
 import           Cardano.Prelude hiding (trace)
@@ -60,8 +58,9 @@ import qualified Cardano.BM.Trace as Trace
 import           Cardano.Shell.Lib (GeneralException (..))
 import           Cardano.Shell.Types (CardanoFeature (..))
 
-import           Cardano.Config.Types (ConfigYamlFilePath(..), CardanoEnvironment,
-                   NodeCLI(..), NodeConfiguration(..), parseNodeConfiguration)
+import           Cardano.Config.Types (ConfigYamlFilePath (..), CardanoEnvironment,
+                                       NodeMockCLI (..), NodeProtocolMode (..),
+                                       NodeConfiguration (..), NodeCLI (..),parseNodeConfiguration)
 
 
 --------------------------------------------------------------------------------
@@ -114,11 +113,6 @@ data LoggingLayer = LoggingLayer
 -- Feature
 --------------------------------
 
--- | CLI specific data structure.
-data LoggingCLIArguments = LoggingCLIArguments
-  { logConfigFile :: !(Maybe FilePath)
-  , captureMetrics :: !Bool
-  }
 
 data LoggingFlag = LoggingEnabled | LoggingDisabled
   deriving (Eq, Show)
@@ -174,17 +168,13 @@ createLoggingFeatureCLI _ mLogConfig captureLogMetrics = do
 
 -- | Create logging feature for `cardano-node`
 createLoggingFeature
-  :: CardanoEnvironment -> NodeCLI -> IO (LoggingLayer, CardanoFeature)
-createLoggingFeature _ nCli = do
-    -- we parse any additional configuration if there is any
-    -- We don't know where the user wants to fetch the additional
-    -- configuration from, it could be from
-    -- the filesystem, so we give him the most flexible/powerful context, @IO@.
-    --
-    -- Currently we parse outside the features since we want to have a complete
-    -- parser for __every feature__.
-    nc <- parseNodeConfiguration . unConfigPath $ configFp nCli
-    let logConfigFp = if ncLoggingSwitch nc then Just . unConfigPath $ configFp nCli else Nothing
+  :: CardanoEnvironment -> NodeProtocolMode -> IO (LoggingLayer, CardanoFeature)
+createLoggingFeature _ nodeProtocolMode = do
+
+    configYamlFp <- pure $ getConfigYaml nodeProtocolMode
+
+    nc <- parseNodeConfiguration $ unConfigPath configYamlFp
+    let logConfigFp = if ncLoggingSwitch nc then Just $ unConfigPath configYamlFp else Nothing
 
     (disabled', loggingConfiguration) <- loggingCLIConfiguration
                                            logConfigFp
@@ -203,6 +193,10 @@ createLoggingFeature _ nCli = do
 
     -- we return both
     pure (loggingLayer, cardanoFeature)
+ where
+  getConfigYaml :: NodeProtocolMode -> ConfigYamlFilePath
+  getConfigYaml (RealProtocolMode (NodeCLI _ _ _ rConfigFp _ )) = rConfigFp
+  getConfigYaml (MockProtocolMode (NodeMockCLI _ _ _ mConfigFp _)) = mConfigFp
 
 -- | Initialize `LoggingCardanoFeature`
 loggingCardanoFeatureInit :: LoggingFlag -> LoggingConfiguration -> IO (LoggingLayer, LoggingLayer -> IO())

diff --git a/cardano-config/src/Cardano/Config/Protocol.hs b/cardano-config/src/Cardano/Config/Protocol.hs
@@ -131,7 +131,7 @@ fromProtocol
   -> Maybe NodeId
   -> Maybe Word64
   -- ^ Number of core nodes
-  -> GenesisFile
+  -> Maybe GenesisFile
   -> RequiresNetworkMagic
   -> Maybe Double
   -> Maybe DelegationCertFile
@@ -162,8 +162,11 @@ fromProtocol _ nId mNumCoreNodes _ _ _ _ _ _ MockPBFT =
                  , pbftSlotLength         = mockSlotLength
                  }
       cid
-fromProtocol gHash _ _ genFile nMagic sigThresh delCertFp sKeyFp update RealPBFT = do
+fromProtocol gHash _ _ mGenFile nMagic sigThresh delCertFp sKeyFp update RealPBFT = do
     let genHash = either panic identity $ decodeHash gHash
+        genFile = fromMaybe (panic $ "Cardano.Config.Protocol.fromProtocol: "
+                                   <> "Genesis file not specified"
+                            ) mGenFile
 
     gc <- firstExceptT LedgerConfigError $ Genesis.mkConfigFromFile
              nMagic

diff --git a/cardano-config/src/Cardano/Config/Topology.hs b/cardano-config/src/Cardano/Config/Topology.hs
@@ -12,6 +12,7 @@ module Cardano.Config.Topology
   , NodeAddress(..)
   , NodeHostAddress(..)
   , NodeSetup(..)
+  , RealNodeTopology(..)
   , RemoteAddress(..)
   , createNodeAddress
   , nodeAddressInfo
@@ -135,6 +136,13 @@ data NodeSetup = NodeSetup
   , producers :: ![RemoteAddress]
   } deriving Show
 
+data RealNodeTopology = RealNodeTopology { rProducers :: ![RemoteAddress] }
+
+instance FromJSON RealNodeTopology where
+  parseJSON = withObject "RealNodeTopology" $ \v ->
+    RealNodeTopology
+      <$> v .: "Producers"
+
 instance FromJSON NodeId where
   parseJSON v = CoreId . CoreNodeId <$> parseJSON v
 

diff --git a/cardano-config/src/Cardano/Config/Types.hs b/cardano-config/src/Cardano/Config/Types.hs
@@ -10,13 +10,15 @@ module Cardano.Config.Types
     , GenesisFile (..)
     , LastKnownBlockVersion (..)
     , MiscellaneousFilepaths (..)
-    , NodeCLI (..)
     , NodeConfiguration (..)
     , Protocol (..)
+    , NodeMockCLI (..)
+    , NodeCLI (..)
+    , NodeProtocolMode (..)
     , SigningKeyFile (..)
-    , SocketFile (..)
     , TopologyFile (..)
     , TraceOptions (..)
+    , SocketPath (..)
     , Update (..)
     , ViewMode (..)
     , parseNodeConfiguration
@@ -56,8 +58,21 @@ data NodeCLI = NodeCLI
     , nodeAddr :: !NodeAddress
     , configFp :: !ConfigYamlFilePath
     , validateDB :: !Bool
+    }
+
+data NodeMockCLI = NodeMockCLI
+    { mockMscFp :: !MiscellaneousFilepaths
+    , mockGenesisHash :: !Text
+    , mockNodeAddr :: !NodeAddress
+    , mockConfigFp :: !ConfigYamlFilePath
+    , mockValidateDB :: !Bool
     } deriving Show
 
+-- | Mock protocols requires different parameters to real protocols.
+-- Therefore we distinguish this at the top level on the command line.
+data NodeProtocolMode = MockProtocolMode NodeMockCLI
+                      | RealProtocolMode NodeCLI
+
 -- | Filepath of the configuration yaml file. This file determines
 -- all the configuration settings required for the cardano node
 -- (logging, tracing, protocol, slot length etc)
@@ -68,10 +83,10 @@ newtype ConfigYamlFilePath = ConfigYamlFilePath
 data MiscellaneousFilepaths = MiscellaneousFilepaths
   { topFile :: !TopologyFile
   , dBFile :: !DbFile
-  , genesisFile :: !GenesisFile
+  , genesisFile :: !(Maybe GenesisFile)
   , delegCertFile :: !(Maybe DelegationCertFile)
   , signKeyFile :: !(Maybe SigningKeyFile)
-  , socketFile :: !SocketFile
+  , socketFile :: !SocketPath
   } deriving Show
 
 newtype TopologyFile = TopologyFile
@@ -90,7 +105,7 @@ newtype DelegationCertFile = DelegationCertFile
   { unDelegationCert :: FilePath }
   deriving Show
 
-newtype SocketFile = SocketFile
+data SocketPath = SocketFile
   { unSocket :: FilePath }
   deriving Show
 
@@ -102,7 +117,7 @@ data NodeConfiguration =
     NodeConfiguration
       { ncProtocol :: Protocol
       , ncNodeId :: Maybe NodeId
-      , ncNumCoreNodes :: Maybe Int
+      , ncNumCoreNodes :: Maybe Word64
       , ncReqNetworkMagic :: RequiresNetworkMagic
       , ncPbftSignatureThresh :: Maybe Double
       , ncLoggingSwitch :: Bool

diff --git a/cardano-node/README.md b/cardano-node/README.md
@@ -47,7 +47,7 @@ The general synopsis is as follows:
 
 Usage: cardano-node --topology FILEPATH --database-path FILEPATH
                     --genesis-file FILEPATH [--delegation-certificate FILEPATH]
-                    [--signing-key FILEPATH] --socket-dir FILEPATH
+                    [--signing-key FILEPATH] --socket-path FILEPATH
                     [--host-addr HOST-NAME] --port PORT
                     --config NODE-CONFIGURATION [--help] [--help-tracing]
                     [--help-advanced]
@@ -63,7 +63,7 @@ Usage: cardano-node --topology FILEPATH --database-path FILEPATH
 
 `--signing-key` - Optional path to the signing key.
 
-`--socket-dir` - Path to the socket directory.
+`--socket-path` - Path to the socket file.
 
 `--host-addr` - Optionally specify your node's IPv4 or IPv6 address.
 
@@ -77,7 +77,7 @@ Usage: cardano-node --topology FILEPATH --database-path FILEPATH
 The `--config` flag points to a `.yaml` file that is responsible to configuring the logging & other important settings for the node.
 Some of the more important settings are as follows:
 
-`NodeId: 0`  -- Soon to be removed from config file. Used to identify nodes in local clusters.
+`NodeId: 0`  -- Used in mock protocols only to differentiate nodes.
 
 `Protocol: RealPBFT` -- Protocol the node will execute