Preview randomness

[zjma]

Description

Test Plan

[trunk-io]

⏱️ 22h 11m total CI duration on this PR

Job	Cumulative Duration	Recent Runs
rust-smoke-coverage	3h	⬜ 🟩
windows-build	2h 19m	🟩 🟩 🟩 🟩 🟩 (+2 more)
forge-framework-upgrade-test / forge	2h 13m	🟥 ⬜ ⬜ 🟥 🟥 (+1 more)
rust-unit-tests	1h 47m	🟥 🟥 🟥 🟥 🟥 (+1 more)
rust-move-tests	1h 47m	🟥 ⬜ 🟥 🟥 🟥
execution-performance / single-node-performance	1h 45m	🟩 🟩 🟥 🟩 🟩 (+1 more)
rust-move-unit-coverage	1h 27m	🟩 🟩 🟩 🟩 🟩
rust-images / rust-all	1h 19m	🟩 🟩 🟩 🟩 🟩 (+1 more)
forge-compat-test / forge	1h 13m	🟩 ⬜ ⬜ 🟩 🟩 (+1 more)
forge-e2e-test / forge	1h 8m	🟥 ⬜ ⬜ 🟥 🟥 (+1 more)
rust-smoke-tests	1h 1m	🟥 🟥 🟥 🟥 🟥
cli-e2e-tests / run-cli-tests	35m	🟩 ⬜ 🟩 🟩 🟩 (+1 more)
rust-lints	33m	🟥 🟥 🟩 🟩 🟩 (+1 more)
run-tests-main-branch	25m	🟩 🟩 🟩 🟩 🟩 (+2 more)
indexer-grpc-e2e-tests / test-indexer-grpc-docker-compose	25m	🟥 🟩 🟥 🟩 🟩 (+1 more)
check	22m	🟩 🟩 🟩 🟩 🟩 (+1 more)
rust-unit-coverage	14m	⬜ 🟥
check-dynamic-deps	13m	🟩 🟩 🟩 🟩 🟩 (+2 more)
general-lints	10m	🟩 🟩 🟩 🟩 🟩 (+1 more)
node-api-compatibility-tests / node-api-compatibility-tests	5m	🟩 🟩 🟩 🟩 🟩 (+1 more)
semgrep/ci	3m	🟩 🟩 🟩 🟩 🟩 (+2 more)
file_change_determinator	1m	🟩 🟩 🟩 🟩 🟩 (+2 more)
execution-performance / file_change_determinator	1m	🟩 🟩 🟩 🟩 🟩 (+1 more)
file_change_determinator	1m	🟩 🟩 🟩 🟩 🟩 (+1 more)
file_change_determinator	1m	🟩 🟩 🟩 🟩 🟩 (+1 more)
permission-check	35s	🟩 🟩 🟩 🟩 🟩 (+3 more)
permission-check	27s	🟩 🟩 🟩 🟩 🟩 (+3 more)
permission-check	25s	🟩 🟩 🟩 🟩 🟩 (+3 more)
permission-check	19s	🟩 🟩 🟩 🟩 🟩 (+2 more)
permission-check	16s	🟩 🟩 🟩 🟩 🟩 (+2 more)
determine-docker-build-metadata	14s	🟩 🟩 🟩 🟩 🟩 (+1 more)

🚨 2 jobs on the last run were significantly faster/slower than expected

Job	Duration	vs 7d avg	Delta
rust-move-tests	23m	18m
rust-unit-tests	38m	31m

_{settings ⋅ feedback ⋅ docs ⋅ learn more about trunk.io}

[github-actions]

❌ Forge suite realistic_env_max_load failure on 19fc1ee7f9a84c7450d0656f9937428e22664991

Forge test runner terminated:
Trailing Log Lines:
             at ./testsuite/forge/src/runner.rs:598:30
  21: forge::run_forge
             at ./testsuite/forge-cli/src/main.rs:424:11
  22: forge::main
             at ./testsuite/forge-cli/src/main.rs:350:21
  23: core::ops::function::FnOnce::call_once
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/core/src/ops/function.rs:250:5
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
{"level":"INFO","source":{"package":"aptos_forge","file":"testsuite/forge/src/backend/k8s/cluster_helper.rs:292"},"thread_name":"main","hostname":"forge-e2e-pr-12327-1709429382-19fc1ee7f9a84c7450d0656f9937428e2","timestamp":"2024-03-03T01:42:39.376529Z","message":"Deleting namespace forge-e2e-pr-12327: Some(NamespaceStatus { conditions: None, phase: Some(\"Terminating\") })"}
{"level":"INFO","source":{"package":"aptos_forge","file":"testsuite/forge/src/backend/k8s/cluster_helper.rs:400"},"thread_name":"main","hostname":"forge-e2e-pr-12327-1709429382-19fc1ee7f9a84c7450d0656f9937428e2","timestamp":"2024-03-03T01:42:39.376573Z","message":"aptos-node resources for Forge removed in namespace: forge-e2e-pr-12327"}
Debugging output:
NAME                                   READY   STATUS      RESTARTS   AGE
aptos-node-0-fullnode-eforge65-0       1/1     Running     0          11m
aptos-node-0-validator-0               1/1     Running     0          11m
aptos-node-1-fullnode-eforge65-0       1/1     Running     0          11m
aptos-node-1-validator-0               1/1     Running     0          11m
aptos-node-2-fullnode-eforge65-0       1/1     Running     0          11m
aptos-node-2-validator-0               1/1     Running     0          11m
aptos-node-3-fullnode-eforge65-0       1/1     Running     0          11m
aptos-node-3-validator-0               1/1     Running     0          11m
aptos-node-4-fullnode-eforge65-0       1/1     Running     0          11m
aptos-node-4-validator-0               1/1     Running     0          11m
aptos-node-5-validator-0               1/1     Running     0          11m
aptos-node-6-validator-0               1/1     Running     0          11m
genesis-aptos-genesis-eforge65-8mpbn   0/1     Completed   0          11m

• Grafana dashboard
• Humio Logs
• Axiom Logs
• Test runner output
• Test run is land-blocking

[github-actions]

✅ Forge suite compat success on aptos-node-v1.9.5 ==> 19fc1ee7f9a84c7450d0656f9937428e22664991

Compatibility test results for aptos-node-v1.9.5 ==> 19fc1ee7f9a84c7450d0656f9937428e22664991 (PR)
1. Check liveness of validators at old version: aptos-node-v1.9.5
compatibility::simple-validator-upgrade::liveness-check : committed: 5683 txn/s, latency: 5018 ms, (p50: 4800 ms, p90: 8700 ms, p99: 9600 ms), latency samples: 233040
2. Upgrading first Validator to new version: 19fc1ee7f9a84c7450d0656f9937428e22664991
compatibility::simple-validator-upgrade::single-validator-upgrade : committed: 1436 txn/s, latency: 18020 ms, (p50: 19600 ms, p90: 25000 ms, p99: 26400 ms), latency samples: 76160
3. Upgrading rest of first batch to new version: 19fc1ee7f9a84c7450d0656f9937428e22664991
compatibility::simple-validator-upgrade::half-validator-upgrade : committed: 671 txn/s, submitted: 705 txn/s, expired: 34 txn/s, latency: 33645 ms, (p50: 38700 ms, p90: 55200 ms, p99: 56400 ms), latency samples: 57764
4. upgrading second batch to new version: 19fc1ee7f9a84c7450d0656f9937428e22664991
compatibility::simple-validator-upgrade::rest-validator-upgrade : committed: 6371 txn/s, latency: 5223 ms, (p50: 4800 ms, p90: 9300 ms, p99: 10200 ms), latency samples: 229380
5. check swarm health
Compatibility test for aptos-node-v1.9.5 ==> 19fc1ee7f9a84c7450d0656f9937428e22664991 passed
Test Ok

• Grafana dashboard
• Humio Logs
• Axiom Logs
• Test runner output
• Test run is land-blocking

[github-actions]

❌ Forge suite framework_upgrade failure on aptos-node-v1.9.5 ==> 19fc1ee7f9a84c7450d0656f9937428e22664991

Forge test runner terminated:
Trailing Log Lines:
             at ./testsuite/forge/src/runner.rs:598:30
  22: forge::run_forge
             at ./testsuite/forge-cli/src/main.rs:424:11
  23: forge::main
             at ./testsuite/forge-cli/src/main.rs:350:21
  24: core::ops::function::FnOnce::call_once
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/core/src/ops/function.rs:250:5
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
{"level":"INFO","source":{"package":"aptos_forge","file":"testsuite/forge/src/backend/k8s/cluster_helper.rs:292"},"thread_name":"main","hostname":"forge-framework-upgrade-pr-12327-1709429375-aptos-node-v1-9-5","timestamp":"2024-03-03T02:05:31.802367Z","message":"Deleting namespace forge-framework-upgrade-pr-12327: Some(NamespaceStatus { conditions: None, phase: Some(\"Terminating\") })"}
{"level":"INFO","source":{"package":"aptos_forge","file":"testsuite/forge/src/backend/k8s/cluster_helper.rs:400"},"thread_name":"main","hostname":"forge-framework-upgrade-pr-12327-1709429375-aptos-node-v1-9-5","timestamp":"2024-03-03T02:05:31.802413Z","message":"aptos-node resources for Forge removed in namespace: forge-framework-upgrade-pr-12327"}
Debugging output:
NAME                                   READY   STATUS      RESTARTS   AGE
aptos-node-0-validator-0               1/1     Running     0          33m
aptos-node-1-validator-0               1/1     Running     0          32m
aptos-node-2-validator-0               1/1     Running     0          31m
aptos-node-3-validator-0               1/1     Running     0          31m
genesis-aptos-genesis-eforge54-2lhzs   0/1     Completed   0          35m

• Grafana dashboard
• Humio Logs
• Axiom Logs
• Test runner output
• Test run is land-blocking