Skip to content
This repository has been archived by the owner on Nov 15, 2023. It is now read-only.

Make zombienet tests native friendly #6588

Merged
merged 7 commits into from
Jan 28, 2023
Merged
Show file tree
Hide file tree
Changes from 6 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 20 additions & 0 deletions zombienet_tests/.set_env
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
pathprepend() {
for ((i=$#; i>0; i--));
do
ARG="$@[$i]"
if [ -d "$ARG" ] && [[ ":$PATH:" != *":$ARG:"* ]]; then
PATH="$ARG${PATH:+":$PATH"}"
fi
done
}

# paths are prepend in order, so you can manage with version will run
# by the order of this array
CUSTOM_PATHS=(
"~/polkadot/target/release"
"~/polkadot/target/testnet"
"~/cumulus/target/release"
)

pathprepend $CUSTOM_PATHS
export PATH=$PATH
43 changes: 43 additions & 0 deletions zombienet_tests/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,49 @@ _The content of this directory is meant to be used by Parity's private CI/CD inf
* [zombienet repo](https://github.com/paritytech/zombienet)
* [zombienet book](https://paritytech.github.io/zombienet/)

## Running tests locally

To run any test locally use the native provider (`zombienet test -p native ...`) you need first build the binaries. They are:

* adder-collator -> polkadot/target/testnet/adder-collator
* malus -> polkadot/target/testnet/malus
* polkadot -> polkadot/target/testnet/polkadot
* polkadot-collator -> cumulus/target/release/polkadot-parachain
* undying-collator -> polkadot/target/testnet/undying-collator

To build them use:
* adder-collator -> `cargo build --profile testnet -p test-parachain-adder-collator`
* undying-collator -> `cargo build --profile testnet -p test-parachain-undying-collator`
* malus -> cargo build --profile testnet -p polkadot-test-malus
* polkadot (in polkadot repo) and polkadot-collator (in cumulus repo) -> `cargo build --profile testnet`

One solution is to use the `.set_env` file (from this directory) and fill the `CUSTOM_PATHS` before *source* it to patch the PATH of your system to find the binaries you just built.

E.g.:
```
$ cat .set_env
(...)
# by the order of this array
CUSTOM_PATHS=(
"~/polkadot/target/release"
"~/polkadot/target/testnet"
"~/cumulus/target/release"
)
(...)
source .set_env
```

Then you have your `PATH` customized and ready to run `zombienet`.
**NOTE**: You should need to do this ones per terminal session, since we are patching the `PATH` and re-exporting. **Or** you can also `source` this file in your `.bashrc` file to get executed automatically in each new session.

Example:

You can run a test locally by executing:
```sh
zombienet test -p native 0001-parachains-pvf.zndsl
```

## Questions / permissions

Ping in element Javier (@javier:matrix.parity.io) to ask questions or grant permission to run the test from your local setup.
6 changes: 3 additions & 3 deletions zombienet_tests/functional/0001-parachains-pvf.zndsl
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,7 @@ eve: reports histogram polkadot_pvf_preparation_time has at least 1 samples in b
one: reports histogram polkadot_pvf_preparation_time has at least 1 samples in buckets ["0.1", "0.5", "1", "2", "3", "10"] within 10 seconds
two: reports histogram polkadot_pvf_preparation_time has at least 1 samples in buckets ["0.1", "0.5", "1", "2", "3", "10"] within 10 seconds

# Check all buckets >= 20.
# Check all buckets >= 20.
alice: reports histogram polkadot_pvf_preparation_time has 0 samples in buckets ["20", "30", "60", "180", "+Inf"] within 10 seconds
bob: reports histogram polkadot_pvf_preparation_time has 0 samples in buckets ["20", "30", "60", "180", "+Inf"] within 10 seconds
charlie: reports histogram polkadot_pvf_preparation_time has 0 samples in buckets ["20", "30", "60", "180", "+Inf"] within 10 seconds
Expand All @@ -78,8 +78,8 @@ two: reports histogram polkadot_pvf_preparation_time has 0 samples in buckets ["
# APPROVAL_EXECUTION_TIMEOUT(6s). Currently these are not differentiated by metrics
# because the metrics are defined in `polkadot-node-core-pvf` which is a level below
# the relevant subsystems.
# That being said, we will take the simplifying assumption of testing only the
# 2s timeout.
# That being said, we will take the simplifying assumption of testing only the
# 2s timeout.
# We do this check by ensuring all executions fall into bucket le="2" or lower.
# First, check if we have at least 1 sample, but we should have many more.
alice: reports histogram polkadot_pvf_execution_time has at least 1 samples in buckets ["0.1", "0.5", "1", "2"] within 10 seconds
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ requests = { memory = "2G", cpu = "1" }
command = "malus suggest-garbage-candidate"
args = ["-lparachain=debug,MALUS=trace"]
count = 1

{% for id in range(2000,2003) %}
[[parachains]]
id = {{id}}
Expand Down
2 changes: 1 addition & 1 deletion zombienet_tests/misc/0001-paritydb.toml
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ requests = { memory = "2G", cpu = "1" }
name = "validator"
count = 10
args = ["-lparachain=debug", "--db=paritydb"]

{% for id in range(2000,2010) %}
[[parachains]]
id = {{id}}
Expand Down
2 changes: 1 addition & 1 deletion zombienet_tests/smoke/0001-parachains-smoke-test.toml
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ addToGenesis = false
[parachains.collator]
name = "collator01"
image = "{{COL_IMAGE}}"
command = "/usr/local/bin/adder-collator"
command = "adder-collator"
args = [ "-lruntime=debug,parachain=trace" ]

[types.Header]
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -21,4 +21,3 @@ command = "polkadot"
[[relaychain.nodes]]
name = "dave"
args = [ "-lruntime=debug,parachain=trace" ]