The Zcash Network Stability Framework
A Ziggurat is a rectangular stepped tower that uses precise measurements to ensure that each foundation can support the layers above. This metaphor can be applied to network testing by defining three layers:
- Conformance - adhering to the network protocol
- Performance - healthy throughput under pressure
- Resistance - avoiding malicious behavior
Ziggurat is network test suite that provides zcashd and zebra devs with this reliable foundation.
Note: This project is a work in progress.
Ziggurat is written in stable Rust; you can install the Rust toolchain by following the official instructions here.
You also need to install at least one node implementation to test. Ziggurat is currently configured to test the Nu5 network protocol (version 170_100
).
zcashd
can be installed by using the official instructions for your operating system. We recommend building from source for consistency and to ensure you're using the right versions. Alternatively, you can use ECC's Debian/Ubuntu package or the binary tarball.
However, please note that Docker is not supported as it can theoretically produce unreliable test results and increases network complexity.
# After installing dependencies
$ git clone https://github.com/zcash/zcash
$ cd zcash
$ git checkout v5.2.0 # optional, or use master
$ ./zcutil/fetch-params.sh
$ ./zcutil/build.sh -j$(nproc) # or number of cores
After completing the above, you can skip the configuration steps, i.e. creating ~/.zcashd/zcash.conf
as Ziggurat will create new configuration files for each test run. Also, syncing the blockchain is not required.
zebra
can be installed from its source code on GitHub. Although a Dockerfile is provided, Docker is not supported. We suggest following the instructions below, or similar.
# After installing dependencies
$ git clone https://github.com/ZcashFoundation/zebra
$ cd zebra
# Download the parameters
$ cargo +stable run --release download
# Run the node once before testing with Ziggurat, just to make sure all is working correctly
$ cargo +stable run --release -- --verbose start
Similarly to zcashd
, configuration is not necessary since Ziggurat generates new configurations for each test run.
Ziggurat is configured via a config.toml
file in the ~/.ziggurat
directory (you'll need to create this yourself). It must contain the following fields:
kind
: one ofzebra
orzcashd
.path
: absolute path in which to run the start command.start_command
: the command used to start the node
We recommend using the following Zcashd config:
kind = "zcashd"
path = "path/to/zcash/repo"
start_command = "./src/zcashd -debug=1 -printtoconsole -logips=1 -dnsseed=0 -dns=0 -listenonion=0"
# debug=1 enables debug logging
# logips=1 adds connection IP spans to the logs
# printtoconsole logs to stdout
# dnsseed=0 disables looking for hardcoded DNS seeding nodes (we want to isolate our node to just the test)
# dns=0 disables DNS lookup
# listenonion=0 disables the Tor network
and for Zebra:
kind = "zebra"
path = "path/to/zebra/repo"
start_command = "target/release/zebrad start"
-datadir |
---|
Ziggurat uses the -datadir configuration argument internally for Zcashd nodes, to prevent corrupting the user's Zcashd cache. This option gets appended to the start command, and will override any user specified -datadir values. |
Ziggurat's documentation can be built with cargo doc --no-deps --open
.
Ziggurat currently uses rust's standard test runner, a simple cargo test -- --test-threads=1
should suffice. We use the single threaded executor as spinning up multiple test nodes isn't currently supported.
Logs are disabled by default, as they usually just add noise and slow down the test. They can be very useful for debugging and can be enabled on a test case level.
Ziggurat's SyntheticNode
supports tracing
- this can be enabled by inserting a call to synthetic_node::enable_tracing()
inside the test case.
Use the --nocapture
option combined with the RUST_LOG
environment variable to show logs from stdout
. Configure the RUST_LOG
environment variable to select the logging level. For example: RUST_LOG=trace cargo test -- --test-threads=1 --nocapture
.
The test node's stdout
and stderr
logs can be piped to stdout
by inserting a call to node.log_to_stdout(true)
before starting the node. Note that logs will need to be enabled for the node as detailed in Configuration.
let mut node = Node::new().unwrap();
node.initial_action(Action::WaitForConnection)
.log_to_stdout(true) // pipes the node's `stdout` and `stderr` to `stdout`
.start()
.await
.unwrap();
Short overview of test cases and their current status. In case of failure, the behaviour observed for zebra
and zcashd
is usually documented in the test case.
These results were obtained by running the test suite against Zcashd v5.2.0 (d6d2093) and Zebra 1.0.0-beta.14 (c3225331).
Legend | |
---|---|
✓ | pass |
✖ | fail |
- | unimplemented |
Test Case | Zcashd | Zebra | Additional Information |
---|---|---|---|
001 | ✓ | ✓ | |
002 | ✓ | ✓ | |
003 | ✓ | ✓ | |
004 | ✓ | ✓ | |
005 | ✓ | ✓ | |
006 | ✓ | ✓ | |
007 | ✓ | ✓ | |
008 | ✓ | ✖ | |
009 | ✖ | ✖ | ⚠ filters may need work (malformed), ⚠ require zcashd feedback |
010 | ✓ | ✓ | |
011 | ✖ | ✖ | ⚠ todo: mempool seeding |
012 | ✖ | ✖ | |
013 | ✖ | ✖ | ⚠ zcashd peering issues, zebra passes under certain conditions |
014 | ✖ | ✖ | ⚠ zcashd peering issues |
015 | - | - | ⚠ not yet implemented (blocked by mempool seeding) |
016 | ✖ | - | ⚠ todo: zebra block seeding |
017 | ✖ | - | ⚠ todo: zebra block seeding |
018 | ✖ | ✖ | ⚠ partially implemented (requires mempool seeding, and zebra block seeding) |
Test Case | Zcashd | Zebra | Additional Information |
---|---|---|---|
001 | ✓ | ✖ | |
002 | ✓ | ✖ |
Test Case | Zcashd | Zebra | Additional Information |
---|---|---|---|
001 | ✓ | ✓ | |
002 | ✓ | ✓ | |
003 | ✓ | ✓ | Zcashd is extremely slow |
004 | ✓ | ✓ | Zcashd is extremely slow |
005 | ✓ | ✓ | |
006 | ✓ | - |
Test Case | Zcashd | Zebra | Additional Information |
---|---|---|---|
001 | ✓ | ✓ | |
002 | ✓ | ✓ | |
003 | ✓ | ✓ | Zcashd is extremely slow |
004 | ✓ | ✓ | Zcashd is extremely slow |
005 | ✓ | ✓ | |
006 | ✓ | - |
Test Case | Zcashd | Zebra | Additional Information |
---|---|---|---|
001 | ✖ | ✖ | |
002 | ✖ | ✖ | |
003 | ✖ | ✖ | Zcashd is extremely slow |
004 | ✖ | ✖ | Zcashd is extremely slow |
005 | ✓ | ✓ | |
006 | ✓ | - |
Test Case | Zcashd | Zebra | Additional Information |
---|---|---|---|
001 | ✖ | ✓ | |
002 | ✖ | ✓ | |
003 | ✖ | ✓ | |
004 | ✖ | ✓ | |
005 | ✖ | ✖ | |
006 | ✓ | - |
Test Case | Zcashd | Zebra | Additional Information |
---|---|---|---|
001 | ✖ | ✓ | |
002 | ✖ | ✓ | |
003 | ✖ | ✓ | |
004 | ✖ | ✓ | |
005 | ✖ | ✓ | |
006 | ✓ | - |
Test Case | Zcashd | Zebra | Additional Information |
---|---|---|---|
001 | ✓ | ✓ | |
002 | ✓ | ✓ | |
003 | ✓ | ✓ | Zcashd is extremely slow |
004 | ✓ | ✓ | Zcashd is extremely slow |
005 | ✓ | ✓ | |
006 | ✓ | - |