[pipeline] fixes and counters #15505

zekun000 · 2024-12-05T01:59:00Z

Description

How Has This Been Tested?

Key Areas to Review

Type of Change

Which Components or Systems Does This Change Impact?

Checklist

I have read and followed the CONTRIBUTING doc
I have performed a self-review of my own code
I have commented my code, particularly in hard-to-understand areas
I identified and added all stakeholders and component owners affected by this change as reviewers
I tested both happy and unhappy path of the functionality
I have made corresponding changes to the documentation

trunk-io · 2024-12-05T01:59:04Z

⏱️ 11h 34m total CI duration on this PR

Slowest 15 Jobs	Cumulative Duration	Recent Runs
execution-performance / single-node-performance	4h 54m	🟩 🟩 🟩 🟩 🟩 (+8 more)
execution-performance / test-target-determinator	57m	🟩 🟩 🟩 🟩 🟩 (+8 more)
test-target-determinator	43m	⬜ 🟩 ⬜ 🟩 🟩 (+6 more)
check	35m	🟩 🟩 ⬜ 🟩 🟩 (+7 more)
rust-cargo-deny	19m	🟩 🟩 ⬜ 🟩 🟩 (+7 more)
fetch-last-released-docker-image-tag	16m	🟩 🟩 ⬜ 🟩 🟩 (+6 more)
forge-compat-test / forge	16m	🟩
check-dynamic-deps	15m	🟩 🟩 🟩 🟩 🟩 (+10 more)
forge-e2e-test / forge	14m	🟩
rust-move-tests	13m	🟩
rust-move-tests	13m	🟩
rust-move-tests	13m	🟩
rust-move-tests	13m	🟩
rust-move-tests	13m	🟩
rust-move-tests	13m	🟩

🚨 1 job on the last run was significantly faster/slower than expected

Job	Duration	vs 7d avg	Delta
check-dynamic-deps	2m	1m

_{settings ⋅ feedback ⋅ docs ⋅ learn more about trunk.io}

[pipeline] expect cancellation error [pipeline] block callback until notification finishes [pipeline] avoid panic when parent fut doesn't exist during epoch ending

github-actions · 2024-12-12T00:08:49Z

✅ Forge suite `compat` success on `3c6e693a27339e73520f41030dce8fc9cd504967` ==> `02e57fa4f0a6ef969e05099606f74dc6bddcbde3`

Compatibility test results for 3c6e693a27339e73520f41030dce8fc9cd504967 ==> 02e57fa4f0a6ef969e05099606f74dc6bddcbde3 (PR)
1. Check liveness of validators at old version: 3c6e693a27339e73520f41030dce8fc9cd504967
compatibility::simple-validator-upgrade::liveness-check : committed: 16189.73 txn/s, latency: 2101.67 ms, (p50: 2100 ms, p70: 2200, p90: 2400 ms, p99: 2500 ms), latency samples: 520980
2. Upgrading first Validator to new version: 02e57fa4f0a6ef969e05099606f74dc6bddcbde3
compatibility::simple-validator-upgrade::single-validator-upgrading : committed: 6694.01 txn/s, latency: 4145.58 ms, (p50: 4500 ms, p70: 4700, p90: 4900 ms, p99: 5100 ms), latency samples: 128720
compatibility::simple-validator-upgrade::single-validator-upgrade : committed: 7314.09 txn/s, latency: 4493.66 ms, (p50: 4800 ms, p70: 4900, p90: 5100 ms, p99: 5300 ms), latency samples: 240000
3. Upgrading rest of first batch to new version: 02e57fa4f0a6ef969e05099606f74dc6bddcbde3
compatibility::simple-validator-upgrade::half-validator-upgrading : committed: 6682.68 txn/s, latency: 4305.39 ms, (p50: 4800 ms, p70: 5100, p90: 5400 ms, p99: 5600 ms), latency samples: 127320
compatibility::simple-validator-upgrade::half-validator-upgrade : committed: 6890.32 txn/s, latency: 4812.15 ms, (p50: 5200 ms, p70: 5300, p90: 5400 ms, p99: 5700 ms), latency samples: 233840
4. upgrading second batch to new version: 02e57fa4f0a6ef969e05099606f74dc6bddcbde3
compatibility::simple-validator-upgrade::rest-validator-upgrading : committed: 985.13 txn/s, latency: 24078.02 ms, (p50: 24500 ms, p70: 28800, p90: 33000 ms, p99: 34100 ms), latency samples: 40320
compatibility::simple-validator-upgrade::rest-validator-upgrade : committed: 3305.95 txn/s, latency: 10111.00 ms, (p50: 8100 ms, p70: 13900, p90: 21800 ms, p99: 22500 ms), latency samples: 106660
5. check swarm health
Compatibility test for 3c6e693a27339e73520f41030dce8fc9cd504967 ==> 02e57fa4f0a6ef969e05099606f74dc6bddcbde3 passed
Test Ok

github-actions · 2024-12-12T00:09:35Z

✅ Forge suite `realistic_env_max_load` success on `02e57fa4f0a6ef969e05099606f74dc6bddcbde3`

two traffics test: inner traffic : committed: 14785.03 txn/s, latency: 2685.23 ms, (p50: 2700 ms, p70: 2700, p90: 3000 ms, p99: 3100 ms), latency samples: 5621600
two traffics test : committed: 99.95 txn/s, latency: 1448.05 ms, (p50: 1400 ms, p70: 1500, p90: 1500 ms, p99: 2700 ms), latency samples: 1840
Latency breakdown for phase 0: ["MempoolToBlockCreation: max: 1.561, avg: 1.478", "ConsensusProposalToOrdered: max: 0.315, avg: 0.289", "ConsensusOrderedToCommit: max: 0.383, avg: 0.371", "ConsensusProposalToCommit: max: 0.668, avg: 0.660"]
Max non-epoch-change gap was: 0 rounds at version 0 (avg 0.00) [limit 4], 0.86s no progress at version 8581 (avg 0.20s) [limit 15].
Max epoch-change gap was: 0 rounds at version 0 (avg 0.00) [limit 4], 0.65s no progress at version 2573330 (avg 0.65s) [limit 16].
Test Ok

github-actions · 2024-12-12T00:11:05Z

✅ Forge suite `framework_upgrade` success on `3c6e693a27339e73520f41030dce8fc9cd504967` ==> `02e57fa4f0a6ef969e05099606f74dc6bddcbde3`

Compatibility test results for 3c6e693a27339e73520f41030dce8fc9cd504967 ==> 02e57fa4f0a6ef969e05099606f74dc6bddcbde3 (PR)
Upgrade the nodes to version: 02e57fa4f0a6ef969e05099606f74dc6bddcbde3
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1457.10 txn/s, submitted: 1459.99 txn/s, failed submission: 2.89 txn/s, expired: 2.89 txn/s, latency: 2092.88 ms, (p50: 2100 ms, p70: 2100, p90: 2700 ms, p99: 4000 ms), latency samples: 131020
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1554.83 txn/s, submitted: 1556.99 txn/s, failed submission: 2.16 txn/s, expired: 2.16 txn/s, latency: 2094.79 ms, (p50: 2100 ms, p70: 2400, p90: 2700 ms, p99: 3900 ms), latency samples: 129420
5. check swarm health
Compatibility test for 3c6e693a27339e73520f41030dce8fc9cd504967 ==> 02e57fa4f0a6ef969e05099606f74dc6bddcbde3 passed
Upgrade the remaining nodes to version: 02e57fa4f0a6ef969e05099606f74dc6bddcbde3
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1524.79 txn/s, submitted: 1527.69 txn/s, failed submission: 2.90 txn/s, expired: 2.90 txn/s, latency: 2159.88 ms, (p50: 2000 ms, p70: 2700, p90: 3000 ms, p99: 4200 ms), latency samples: 126300
Test Ok

zekun000 force-pushed the zekun/pipeline branch from 034a69e to 303f957 Compare December 5, 2024 02:00

zekun000 added CICD:run-e2e-tests when this label is present github actions will run all land-blocking e2e tests from the PR CICD:build-images when this label is present github actions will start build+push rust images from the PR. labels Dec 5, 2024