feat: more idiomatic CLI interface for mock network (#6897)

- use kebab-case rather than snake-case
- replace mode argument with custom syntax with a pair of
  * --client-height
  * --network-height
- use convention-over-configuration for the binary name

These are mostly inconsequential cleanups which I am not feeling particularly strongly about specifically. I do feel moderately strongly about the overall point of sticking to conventions, defaults, and trying to leverage existing functionality instead of implementing custom logic :)

tools/mock_node/Cargo.toml

@@ -1,16 +1,12 @@
 [package]
-name = "mock_node"
+name = "mock-node"
 version = "0.0.0"
 authors = ["Near Inc <[email protected]>"]
 publish = false
 # Please update rust-toolchain.toml as well when changing version here:
 rust-version = "1.60.0"
 edition = "2021"
 
-[[bin]]
-path = "src/main.rs"
-name = "start_mock_node"
-
 [dependencies]
 actix = "0.13.0"
 actix-rt = "2"
@@ -21,7 +17,6 @@ flate2 = "1.0.22"
 futures = "0.3"
 rand = "0.7"
 rayon = "1.5"
-regex = "1"
 tar = "0.4.38"
 tempfile = "3"
 tracing = "0.1.13"

tools/mock_node/README.md

@@ -1,77 +1,60 @@
 # mock-node
-This crate hosts libraries to start a test env for a single node by replacing the network module with a mock network environment. 
-The mock network environment simulates the interaction that the client will usually have with other nodes by 
-responding to the client's network messages and broadcasting new blocks. The mock network reads a pre-generated chain 
+This crate hosts libraries to start a test env for a single node by replacing the network module with a mock network environment.
+The mock network environment simulates the interaction that the client will usually have with other nodes by
+responding to the client's network messages and broadcasting new blocks. The mock network reads a pre-generated chain
 history from storage.
 
-The crate has two files and a binary.
+## Quick Start
 
-## mod.rs
-Implements `ChainHistoryAccess` and `MockPeerManagerActor`, which is the main 
-components of the mock network.
-## setup.rs
-Provides functions for setting up a mock network from configs and home dirs.
-## main.rs
-A binary that starts a mock testing environment for ClientActor.
-It simulates the entire network by substituting PeerManagerActor with a mock network,
-responding to the client's network requests by reading from a pre-generated chain history
-in storage.
+```console
+$ cargo run --release -p mock-node -- ~/.near/localnet/node0
+```
 
-### Setup
-- ```bash 
-  cargo build -p mock_node (--release)
-- If you are running a mock node for mainnet or testnet, or any performance sensitive jobs, 
-  please compile with --release.
-- If you are running a mock node for mainnet or testnet on a GCP node, you want to place the new client home
-  dir on a SSD disk for optimal rocksdb performance. Note that the 
-  default booting disk of GCP notes are HDD, so you need to mount a new SSD disk on 
-  your node and put the mock node's home dir there. See https://cloud.google.com/compute/docs/disks/add-persistent-disk
-  for how to attach and mount a new disk to an existing GCP node. 
+where the `node0` directory contains some pre-generated chain history in storage.
+You can find two examples in the ./benches directory.
 
-### Usage
-```bash
-start_mock_node [FLAGS] [OPTIONS] <chain-history-home-dir> --mode <mode> [client-home-dir]
-```
-Run
-```bash
-start_mock_node --help
-```
-to see all flags and options the command supports. 
+If you are running a mock node for mainnet or testnet on a GCP node, you want to place the new client home
+dir on a SSD disk for optimal rocksdb performance. Note that the
+default booting disk of GCP notes are HDD, so you need to mount a new SSD disk on
+your node and put the mock node's home dir there. See https://cloud.google.com/compute/docs/disks/add-persistent-disk
+for how to attach and mount a new disk to an existing GCP node.
 
+See `$ cargo run -p mock-node -- --help` for the list of available options and their documentation.
 
-Example use cases:
+## Examples
 
 #### Replay localnet history
-```bash
-start_mock_node ~/.near/localnet/node0 --mode no_new_blocks
+
+```console
+$ cargo r -r -p mock-node -- ~/.near/localnet/node0
 ```
-Here we take the home dir of an existing node in a localnet as chain history home dir, 
-so the mock network will reproduce the entire history of the localnet from genesis.
-Without specified client-home-dir, the binary will create a temporary directory as home directory of the new client.
+Here we take the home dir of an existing node in a localnet as chain history home dir,
+so the mock network will reproduce the client catching up with the entire history of the localnet from genesis.
 
 #### Replay mainnet history from a certain height
+
 To replay mainnet or testnet history, in most use cases, we want to start replaying from a certain height, instead
-of from genesis block. The following comment replays mainnet history from block height 60925880 to block height 60925900. 
-The start height is specified by `-s` and the end height is specified by `-h`.
-```bash
-start_mock_node ~/.near ~/mock_node_home_dir -s 60925880 -h 60925900 --mode "no_new_blocks"
+of from genesis block. The following comment replays mainnet history from block height 60925880 to block height 60925900.
+
+```console
+$ cargo r -r -p mock-node --  ~/.near ~/mock_node_home_dir --start_height 60925880 --target-height 60925900
 ```
 
 By providing a starting height,
 the binary will set up the data dir before starting the client, by copying the state at the specified height
 and other chain info necessary for processing the blocks afterwards (such as block headers and blocks).
-This initial setup may take a long time (The exact depends on your 
+This initial setup may take a long time (The exact depends on your
 source dir, my experiment takes about an hour from a non-archival source dir. Copying from archival node source
-dir may take longer as rocksdb is slower). So we suggest specifying a client dir so you can reuse it again 
-without having to copy the state again.
+dir may take longer as rocksdb is slower). So we suggest specifying a client dir (the `~/mock_node_home_dir` argument)
+so you can reuse it again without having to copy the state again.
 
 Note that the start height must be the last block of an epoch.
 
-Once you have the source dir already set up, you can run the command without `-s`,
-```bash
-start_mock_node ~/.near ~/mock_node_home_dir -h 60926000 --mode "no_new_blocks"
+Once you have the source dir already set up, you can run the command without `--start_height`,
+
+```console
+$  cargo r -r -p mock-node --  ~/.near ~/mock_node_home_dir --target-height 60926000
 ```
-The binary will not modify the client dir and the client will start from the chain head stored in the 
+The binary will not modify the client dir and the client will start from the chain head stored in the
 client dir, which is height 60925900 in this case because that was the position of the chain head
 in the client dir storage.
-

tools/mock_node/benches/README.md

@@ -4,15 +4,15 @@ to a particular height in the chain defined by the sample home
 directory archives included here. To run all the benchmarks:
 
 ```shell
-$ cargo bench -p mock_node
+$ cargo bench -p mock-node
 ```
 
 This will take quite a while though, as each iteration of the
 benchmark `mock_node_sync_full` takes several minutes, and it's run 10
 times. To run just the quicker one:
 
 ```shell
-$ cargo bench -p mock_node -- mock_node_sync_empty
+$ cargo bench -p mock-node -- mock_node_sync_empty
 ```
 
 You can pretty easily define and run your own benchmark based on some
@@ -33,4 +33,4 @@ the code like so:
 +}
 +
 +criterion_group!(benches, sync_empty_chunks, sync_full_chunks, sync_foo_chunks);
-```
+```

tools/mock_node/benches/sync.rs

@@ -5,7 +5,7 @@ use actix::System;
 use anyhow::anyhow;
 use criterion::Criterion;
 use flate2::read::GzDecoder;
-use mock_node::setup::{setup_mock_node, MockNetworkMode};
+use mock_node::setup::setup_mock_node;
 use mock_node::GetChainTargetBlockHeight;
 use near_actix_test_utils::{block_on_interruptible, setup_actix};
 use near_chain_configs::GenesisValidationMode;
@@ -106,8 +106,8 @@ fn do_bench(c: &mut Criterion, home_archive: &str, target_height: Option<BlockHe
                     tempdir.path(),
                     home.as_path(),
                     near_config,
-                    MockNetworkMode::NoNewBlocks,
                     Duration::from_millis(100),
+                    0,
                     None,
                     target_height,
                     false,

tools/mock_node/src/lib.rs

@@ -1,3 +1,6 @@
+//! Implements `ChainHistoryAccess` and `MockPeerManagerActor`, which is the main
+//! components of the mock network.
+
 use actix::{Actor, Context, Handler, Recipient};
 use near_chain::{Block, BlockHeader, Chain, ChainStoreAccess, Error};
 use near_chain_configs::GenesisConfig;
@@ -47,7 +50,7 @@ impl MockPeerManagerActor {
         client_addr: Recipient<NetworkClientMessages>,
         genesis_config: &GenesisConfig,
         chain: Chain,
-        peers_start_height: BlockHeight,
+        network_start_height: BlockHeight,
         target_height: BlockHeight,
         block_production_delay: Duration,
         network_delay: Duration,
@@ -61,7 +64,7 @@ impl MockPeerManagerActor {
                     chain_id: genesis_config.chain_id.clone(),
                     hash: *chain.genesis().hash(),
                 },
-                height: peers_start_height,
+                height: network_start_height,
                 tracked_shards: (0..genesis_config.shard_layout.num_shards()).collect(),
                 archival: false,
             },

tools/mock_node/src/main.rs

@@ -1,9 +1,12 @@
+//! A binary that starts a mock testing environment for ClientActor. It
+//! simulates the entire network by substituting PeerManagerActor with a mock
+//! network, responding to the client's network requests by reading from a
+//! pre-generated chain history in storage.
+
 use actix::{Actor, System};
 use clap::Parser;
 use futures::{future, FutureExt};
-use std::path::Path;
-
-use mock_node::setup::{setup_mock_node, MockNetworkMode};
+use mock_node::setup::setup_mock_node;
 use mock_node::GetChainTargetBlockHeight;
 use near_actix_test_utils::run_actix;
 use near_chain_configs::GenesisValidationMode;
@@ -12,57 +15,67 @@ use near_crypto::{InMemorySigner, KeyType};
 use near_logger_utils::init_integration_logger;
 use near_network::test_utils::WaitOrTimeoutActor;
 use near_primitives::types::BlockHeight;
+use std::path::{Path, PathBuf};
 use std::time::Duration;
 
 /// Program to start a mock node, which runs a regular client in a mock network environment.
-/// The mock network simulates the entire network by reading a pre-generated chain history
-/// on storage and responds to the client's network requests.
-/// The binary runs in two modes, NoNewBlocks and ProduceNewBlocks, determined by flag `mode`.
+/// The mock network simulates the entire network by replaying a pre-generated chain history
+/// from storage and responds to the client's network requests.
 ///
-/// In NoNewBlocks mode, the mock network will not simulate block production.
-/// The simulated peers will start from the target height and no new blocks are produced.
+/// There are two ways to replay the stored history:
+///   * catchup: client is behind the network and applies the blocks as fast as possible
+///   * normal block production: client accept "new" blocks as they are produced
+///     (in reality, blocks are just fetched from the pre-generated store).
 ///
-/// In ProduceNewBlocks mode, new blocks will be produced, i.e., the client will receive new
-/// blocks from the mock network. User can provide a number in this mode, which will be the starting
-/// height the new produced blocks. If no number is provided, the mock network will start at
-/// the same height as the client, which is specified by client_start_height.
+/// This is controlled by two flags:
+///   * `--client-height` specifies the height the client starts at. Defaults to 0.
+///   * `--network-height` specifies the hight the rest of the (simulated)
+///     network starts at. Defaults to the latest recorded height.
 ///
-/// Example commands:
-/// start_mock_node ~/.near/localnet/node0 -h 100 --mode no_new_blocks
+/// As a shortcut, `--start-height` sets both.
 ///
-/// Client starts at genesis height and mock network starts at height 100. No new blocks will be produced.
-/// The simulated peers stay at height 100.
 ///
-/// start_mock_node ~/.near/localnet/node0 -s 61 -h 100 --mode produce_new_blocks
+/// Examples
 ///
-/// Both client and mock network starts at height 61, mock network will produce new blocks until height 100
-///  
-/// start_mock_node ~/.near/localnet/node0 -h 100 --mode "produce_new_blocks(20)"
+/// ```console
+/// # Pure catchup from genesis height to the end of the recorded history.
+/// $ mock-node ~/.near/localnet/node0
 ///
-/// Client starts at genesis height and mock network starts at heigh 20,
-/// mock network will produce new blocks until height 100
+/// # Pure block production starting from block height 61.
+/// $ mock-node ~/.near/localnet/node0 --start-height 61
+///
+/// # Mixed: client starts at genesis and tries to catch up with the network, which starts at height 20.
+/// $ mock-node ~/.near/localnet/node0 --network-height 20
+/// ```
 #[derive(Parser)]
 struct Cli {
     /// Existing home dir for the pre-generated chain history. For example, you can use
     /// the home dir of a near node.
     chain_history_home_dir: String,
     /// Home dir for the new client that will be started. If not specified, the binary will
     /// generate a temporary directory
-    client_home_dir: Option<String>,
+    client_home_dir: Option<PathBuf>,
     /// Simulated network delay (in ms)
     #[clap(short = 'd', long, default_value = "100")]
     network_delay: u64,
-    /// Mode of the mock network, choices: [no_new_blocks, produce_new_blocks, produce_new_blocks(u64)]
-    #[clap(short = 'M', long)]
-    mode: MockNetworkMode,
-    /// If specified, the binary will set up client home dir before starting the client node
-    /// so head of the client chain will be the specified height when the client starts.
-    /// The given height must be the last block in an epoch.
-    #[clap(short = 's', long)]
-    client_start_height: Option<BlockHeight>,
+    /// If specified, the binary will set up client home dir before starting the
+    /// client node so head of the client chain will be the specified height
+    /// when the client starts. The given height must be the last block in an
+    /// epoch.
+    #[clap(long, default_value = "0")]
+    client_height: BlockHeight,
+    /// The height at which the mock network starts. The client would have to
+    /// catch up to this height before participating in new block production.
+    ///
+    /// Defaults to the largest height in history.
+    #[clap(long)]
+    network_height: Option<BlockHeight>,
+    /// Shortcut to set both `--client-height` and `--network-height`.
+    #[clap(long, conflicts_with_all(&["client-height", "network-height"]))]
+    start_height: Option<BlockHeight>,
     /// Target height that the client should sync to before stopping. If not specified,
     /// use the height of the last block in chain history
-    #[clap(short = 'h', long)]
+    #[clap(long)]
     target_height: Option<BlockHeight>,
     /// If true, use in memory storage instead of rocksdb for the client
     #[clap(short = 'i', long)]
@@ -83,18 +96,26 @@ fn main() {
     near_config.client_config.tracked_shards =
         (0..near_config.genesis.config.shard_layout.num_shards()).collect();
 
-    let tempdir = tempfile::Builder::new().prefix("mock_node").tempdir().unwrap();
-    let client_home_dir =
-        args.client_home_dir.unwrap_or(String::from(tempdir.path().to_str().unwrap()));
+    let tempdir;
+    let client_home_dir = match &args.client_home_dir {
+        Some(it) => it.as_path(),
+        None => {
+            tempdir = tempfile::Builder::new().prefix("mock_node").tempdir().unwrap();
+            tempdir.path()
+        }
+    };
     let network_delay = Duration::from_millis(args.network_delay);
+
+    let client_height = args.start_height.unwrap_or(args.client_height);
+    let network_height = args.start_height.or(args.network_height);
     run_actix(async move {
         let (mock_network, _client, view_client, _) = setup_mock_node(
             Path::new(&client_home_dir),
             home_dir,
             near_config,
-            args.mode,
             network_delay,
-            args.client_start_height,
+            client_height,
+            network_height,
             args.target_height,
             args.in_memory_storage,
         );