Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

compare perf of runs-on runners #13698

Closed
wants to merge 1 commit into from
Closed

compare perf of runs-on runners #13698

wants to merge 1 commit into from

Conversation

geekflyer
Copy link
Contributor

No description provided.

Copy link

trunk-io bot commented Jun 14, 2024

⏱️ 16h 10m total CI duration on this PR
Job Cumulative Duration Recent Runs
rust-smoke-coverage 4h 26m 🟩🟩
rust-unit-coverage 2h 1m 🟩🟩
rust-smoke-tests 1h 41m 🟩🟩🟩
test-fuzzers 1h 11m 🟩🟩
rust-move-tests 51m 🟩🟩🟩🟩
forge-e2e-test / forge 40m 🟥🟩🟩
rust-unit-tests 38m 🟥
forge-compat-test / forge 37m 🟩🟩🟩
rust-images / rust-all 36m 🟩🟩🟩
rust-lints 24m 🟩🟩🟩🟩
rust-move-unit-coverage 22m 🟩🟩
prover-inconsistency-test 20m 🟥🟥
run-tests-main-branch 18m 🟩🟩🟩🟩
cli-e2e-tests / run-cli-tests 17m 🟩🟩🟩
execution-performance / test-target-determinator 15m 🟩🟩🟩🟩
rust-targeted-unit-tests 13m 🟩🟩🟩🟩
rust-build-cached-packages 13m 🟩🟩🟩
check 12m 🟩🟩🟩
test-target-determinator 11m 🟩🟩🟩
run-all-circuit-tests 11m 🟩🟩
general-lints 7m 🟩🟩🟩🟩
check-dynamic-deps 6m 🟩🟩🟩🟩🟩
build 5m 🟥🟥
indexer-grpc-e2e-tests / test-indexer-grpc-docker-compose 4m 🟥🟩🟩
node-api-compatibility-tests / node-api-compatibility-tests 3m 🟥🟩🟩
semgrep/ci 2m 🟩🟩🟩🟩🟩
file_change_determinator 57s 🟩🟩🟩🟩🟩
file_change_determinator 53s 🟩🟩🟩🟩🟩
file_change_determinator 40s 🟩🟩🟩🟩
execution-performance / single-node-performance 30s 🟩🟩🟩🟩
upload-to-codecov 26s 🟩🟩
permission-check 22s 🟩🟩🟩🟩🟩
permission-check 20s 🟩🟩🟩🟩🟩
permission-check 16s 🟩🟩🟩🟩
permission-check 16s 🟩🟩🟩🟩🟩
permission-check 12s 🟩🟩🟩🟩🟩
determine-docker-build-metadata 8s 🟩🟩🟩🟩

🚨 2 jobs on the last run were significantly faster/slower than expected

Job Duration vs 7d avg Delta
rust-images / rust-all 11m 14m -23%
rust-targeted-unit-tests 3m 16m -78%

settingsfeedbackdocs ⋅ learn more about trunk.io

Copy link

codecov bot commented Jun 14, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 71.2%. Comparing base (061db31) to head (d3f2226).
Report is 910 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff            @@
##             main   #13698     +/-   ##
=========================================
+ Coverage    70.8%    71.2%   +0.4%     
=========================================
  Files        2299     2299             
  Lines      451793   451793             
=========================================
+ Hits       319875   321795   +1920     
+ Misses     131918   129998   -1920     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@geekflyer geekflyer marked this pull request as ready for review June 14, 2024 04:10
@geekflyer geekflyer requested a review from a team as a code owner June 14, 2024 04:10
@geekflyer geekflyer marked this pull request as draft June 14, 2024 04:11
@geekflyer geekflyer added CICD:run-e2e-tests when this label is present github actions will run all land-blocking e2e tests from the PR CICD:run-coverage run tests with test coverage instrumentation labels Jun 14, 2024

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

@geekflyer geekflyer force-pushed the runs-on-comparison branch from 2d98ea3 to d3f2226 Compare June 18, 2024 05:29

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

@geekflyer geekflyer added the CICD:run-all-unit-tests Runs all unit tests label Jun 18, 2024

This comment has been minimized.

This comment has been minimized.

Copy link
Contributor

✅ Forge suite compat success on a68e71c05caebf01504d4499110f3fba213fb53d ==> d3f222614305518c4fafdf61acdf38d3d55c4674

Compatibility test results for a68e71c05caebf01504d4499110f3fba213fb53d ==> d3f222614305518c4fafdf61acdf38d3d55c4674 (PR)
1. Check liveness of validators at old version: a68e71c05caebf01504d4499110f3fba213fb53d
compatibility::simple-validator-upgrade::liveness-check : committed: 8002.007764792961 txn/s, latency: 3827.0372431125534 ms, (p50: 3000 ms, p90: 4200 ms, p99: 26600 ms), latency samples: 314340
2. Upgrading first Validator to new version: d3f222614305518c4fafdf61acdf38d3d55c4674
compatibility::simple-validator-upgrade::single-validator-upgrade : committed: 3066.504840380666 txn/s, latency: 10126.792478632478 ms, (p50: 9700 ms, p90: 13900 ms, p99: 14300 ms), latency samples: 128700
3. Upgrading rest of first batch to new version: d3f222614305518c4fafdf61acdf38d3d55c4674
compatibility::simple-validator-upgrade::half-validator-upgrade : committed: 3063.569841175058 txn/s, latency: 10206.143054903801 ms, (p50: 9600 ms, p90: 14000 ms, p99: 14200 ms), latency samples: 127860
4. upgrading second batch to new version: d3f222614305518c4fafdf61acdf38d3d55c4674
compatibility::simple-validator-upgrade::rest-validator-upgrade : committed: 3269.052296421205 txn/s, latency: 9758.342038753159 ms, (p50: 10000 ms, p90: 15500 ms, p99: 18000 ms), latency samples: 118700
5. check swarm health
Compatibility test for a68e71c05caebf01504d4499110f3fba213fb53d ==> d3f222614305518c4fafdf61acdf38d3d55c4674 passed
Test Ok

Copy link
Contributor

✅ Forge suite realistic_env_max_load success on d3f222614305518c4fafdf61acdf38d3d55c4674

two traffics test: inner traffic : committed: 8539.578687463061 txn/s, latency: 4590.7472107275535 ms, (p50: 4400 ms, p90: 5400 ms, p99: 10500 ms), latency samples: 3687700
two traffics test : committed: 100.04548094933187 txn/s, latency: 1972.1972222222223 ms, (p50: 1900 ms, p90: 2100 ms, p99: 3800 ms), latency samples: 1800
Latency breakdown for phase 0: ["QsBatchToPos: max: 0.216, avg: 0.209", "QsPosToProposal: max: 0.387, avg: 0.264", "ConsensusProposalToOrdered: max: 0.311, avg: 0.288", "ConsensusOrderedToCommit: max: 0.369, avg: 0.357", "ConsensusProposalToCommit: max: 0.659, avg: 0.646"]
Max round gap was 1 [limit 4] at version 1790460. Max no progress secs was 5.07094 [limit 15] at version 1790460.
Test Ok

Copy link
Contributor

github-actions bot commented Aug 3, 2024

This issue is stale because it has been open 45 days with no activity. Remove the stale label, comment or push a commit - otherwise this will be closed in 15 days.

@github-actions github-actions bot added Stale and removed Stale labels Aug 3, 2024
Copy link
Contributor

This issue is stale because it has been open 45 days with no activity. Remove the stale label, comment or push a commit - otherwise this will be closed in 15 days.

@github-actions github-actions bot added Stale and removed Stale labels Sep 18, 2024
Copy link
Contributor

github-actions bot commented Nov 3, 2024

This issue is stale because it has been open 45 days with no activity. Remove the stale label, comment or push a commit - otherwise this will be closed in 15 days.

@github-actions github-actions bot added the Stale label Nov 3, 2024
@github-actions github-actions bot closed this Nov 18, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CICD:run-all-unit-tests Runs all unit tests CICD:run-coverage run tests with test coverage instrumentation CICD:run-e2e-tests when this label is present github actions will run all land-blocking e2e tests from the PR Stale
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant