Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[forge] Framework upgrade test all proposals #14023

Merged
merged 1 commit into from
Jul 17, 2024
Merged

[forge] Framework upgrade test all proposals #14023

merged 1 commit into from
Jul 17, 2024

Conversation

perryjrandall
Copy link
Contributor

@perryjrandall perryjrandall commented Jul 16, 2024 •
edited
Loading

Add and test any proposal release yaml in the data dir of release builder

Test Plan: Build / runs on PR with label

Also a child PR with a new yaml should show running and executing that yaml

#14025

@perryjrandall perryjrandall requested a review from a team as a code owner July 16, 2024 23:55
@trunk-io Trunk.io
Copy link

trunk-io bot commented Jul 16, 2024 •
edited
Loading

⏱️ 7h 40m total CI duration on this PR
Job Cumulative Duration Recent Runs
test-fuzzers 4h 14m 🟩 🟩 🟩 🟩 🟩 (+2 more)
forge-framework-upgrade-test / forge 1h 5m 🟩 ⬜ 🟩 🟩 🟩 
test-target-determinator 25m 🟩 🟩 🟩 🟩 🟩 (+1 more)
execution-performance / single-node-performance 21m 🟩 
forge-e2e-test / forge 14m 🟩 
forge-compat-test / forge 13m 🟩 
general-lints 11m 🟩 🟩 🟩 🟩 🟩 (+1 more)
rust-cargo-deny 11m 🟩 🟩 🟩 🟩 🟩 (+1 more)
check-dynamic-deps 9m 🟩 🟩 🟩 🟩 🟩 (+2 more)
rust-move-tests 6m 🟩 
execution-performance / test-target-determinator 5m 🟩 
check 4m 🟩 
rust-move-tests 3m 🟩 
rust-move-tests 3m 🟩 
rust-move-tests 3m 🟩 
rust-move-tests 3m 🟩 
semgrep/ci 2m 🟩 🟩 🟩 🟩 🟩 (+2 more)
rust-move-tests 2m ⬜ 
file_change_determinator 1m 🟩 🟩 🟩 🟩 🟩 (+2 more)
file_change_determinator 1m 🟩 🟩 🟩 🟩 🟩 (+1 more)
file_change_determinator 1m 🟩 🟩 🟩 🟩 🟩 (+2 more)
permission-check 24s 🟩 🟩 🟩 🟩 🟩 (+2 more)
permission-check 21s 🟩 🟩 🟩 🟩 🟩 (+2 more)
permission-check 21s 🟩 🟩 🟩 🟩 🟩 (+2 more)
permission-check 18s 🟩 🟩 🟩 🟩 🟩 (+1 more)
permission-check 16s 🟩 🟩 🟩 🟩 🟩 (+2 more)
Backport PR 15s 🟥 🟥 
determine-docker-build-metadata 14s 🟩 🟩 🟩 🟩 🟩 (+1 more)
permission-check 4s 🟩 🟩 
rust-move-tests 1s ⬜ 

settings ⋅ feedback ⋅ docs ⋅ learn more about trunk.io

@perryjrandall perryjrandall marked this pull request as draft July 16, 2024 23:55
@perryjrandall perryjrandall force-pushed the perry/forge-upgrade-yamls branch from c44fd32 to 2941ff1 Compare July 16, 2024 23:58
@github-actions GitHub Actions

This comment has been minimized.

Sign in to view
@github-actions GitHub Actions

This comment has been minimized.

Sign in to view
@perryjrandall perryjrandall force-pushed the perry/forge-upgrade-yamls branch from 2941ff1 to 188e681 Compare July 17, 2024 01:02
@perryjrandall perryjrandall marked this pull request as ready for review July 17, 2024 01:02
@perryjrandall perryjrandall force-pushed the perry/forge-upgrade-yamls branch from 188e681 to 49fc643 Compare July 17, 2024 01:07
@github-actions GitHub Actions

This comment has been minimized.

Sign in to view
@github-actions GitHub Actions

This comment has been minimized.

Sign in to view
@perryjrandall
[forge] Framework upgrade test all proposals
1443055 
Add and test any proposal release yaml in the data dir of release builder

Test Plan: Build / runs on PR with label
@perryjrandall perryjrandall force-pushed the perry/forge-upgrade-yamls branch from 49fc643 to 1443055 Compare July 17, 2024 01:57
@github-actions GitHub Actions

This comment has been minimized.

Sign in to view
@github-actions GitHub Actions

This comment has been minimized.

Sign in to view
@perryjrandall perryjrandall enabled auto-merge (rebase) July 17, 2024 06:32
@github-actions GitHub Actions

This comment has been minimized.

Sign in to view
@github-actions GitHub Actions

This comment has been minimized.

Sign in to view
@github-actions GitHub Actions

This comment has been minimized.

Sign in to view
@github-actions GitHub Actions
Copy link
Contributor

✅ Forge suite compat success on 1c2ee7082d6eff8c811ee25d6f5a7d00860a75d5 ==> 144305500cda4468551b6c282699267bec25ccbd

Compatibility test results for 1c2ee7082d6eff8c811ee25d6f5a7d00860a75d5 ==> 144305500cda4468551b6c282699267bec25ccbd (PR)
1. Check liveness of validators at old version: 1c2ee7082d6eff8c811ee25d6f5a7d00860a75d5
compatibility::simple-validator-upgrade::liveness-check : committed: 7617.320168129822 txn/s, latency: 4232.920476788785 ms, (p50: 3300 ms, p90: 7200 ms, p99: 24100 ms), latency samples: 302440
2. Upgrading first Validator to new version: 144305500cda4468551b6c282699267bec25ccbd
compatibility::simple-validator-upgrade::single-validator-upgrading : committed: 6760.353303046131 txn/s, latency: 3986.8295602476583 ms, (p50: 4100 ms, p90: 5800 ms, p99: 6000 ms), latency samples: 125980
compatibility::simple-validator-upgrade::single-validator-upgrade : committed: 7194.120482880719 txn/s, latency: 4529.119446035513 ms, (p50: 4600 ms, p90: 5300 ms, p99: 5800 ms), latency samples: 244420
3. Upgrading rest of first batch to new version: 144305500cda4468551b6c282699267bec25ccbd
compatibility::simple-validator-upgrade::half-validator-upgrading : committed: 3884.215391820493 txn/s, latency: 7253.593318284425 ms, (p50: 5100 ms, p90: 12000 ms, p99: 12200 ms), latency samples: 88600
compatibility::simple-validator-upgrade::half-validator-upgrade : committed: 6653.256288293704 txn/s, latency: 4600.700832621691 ms, (p50: 4300 ms, p90: 7200 ms, p99: 7800 ms), latency samples: 234200
4. upgrading second batch to new version: 144305500cda4468551b6c282699267bec25ccbd
compatibility::simple-validator-upgrade::rest-validator-upgrading : committed: 3182.760324915736 txn/s, latency: 7193.601773981603 ms, (p50: 3900 ms, p90: 25900 ms, p99: 32500 ms), latency samples: 121760
compatibility::simple-validator-upgrade::rest-validator-upgrade : committed: 9006.483985386374 txn/s, latency: 3539.239219405071 ms, (p50: 3000 ms, p90: 6200 ms, p99: 10600 ms), latency samples: 300540
5. check swarm health
Compatibility test for 1c2ee7082d6eff8c811ee25d6f5a7d00860a75d5 ==> 144305500cda4468551b6c282699267bec25ccbd passed
Test Ok

@github-actions GitHub Actions
Copy link
Contributor

✅ Forge suite realistic_env_max_load success on 144305500cda4468551b6c282699267bec25ccbd

two traffics test: inner traffic : committed: 9372.935470270213 txn/s, latency: 4247.636265530826 ms, (p50: 4200 ms, p90: 4500 ms, p99: 11300 ms), latency samples: 3563880
two traffics test : committed: 99.97898905160322 txn/s, latency: 2091.054945054945 ms, (p50: 2100 ms, p90: 2200 ms, p99: 2700 ms), latency samples: 1820
Latency breakdown for phase 0: ["QsBatchToPos: max: 0.246, avg: 0.221", "QsPosToProposal: max: 1.880, avg: 1.841", "ConsensusProposalToOrdered: max: 0.309, avg: 0.291", "ConsensusOrderedToCommit: max: 0.400, avg: 0.386", "ConsensusProposalToCommit: max: 0.690, avg: 0.677"]
Max round gap was 1 [limit 4] at version 1914991. Max no progress secs was 5.847084 [limit 15] at version 1914991.
Test Ok

@github-actions GitHub Actions
Copy link
Contributor

✅ Forge suite framework_upgrade success on 1c2ee7082d6eff8c811ee25d6f5a7d00860a75d5 ==> 144305500cda4468551b6c282699267bec25ccbd

Compatibility test results for 1c2ee7082d6eff8c811ee25d6f5a7d00860a75d5 ==> 144305500cda4468551b6c282699267bec25ccbd (PR)
Upgrade the nodes to version: 144305500cda4468551b6c282699267bec25ccbd
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1114.401888310882 txn/s, submitted: 1116.598186868846 txn/s, failed submission: 2.1962985579638983 txn/s, expired: 2.1962985579638983 txn/s, latency: 2768.8373571147026 ms, (p50: 2100 ms, p90: 5400 ms, p99: 10900 ms), latency samples: 101480
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1099.345958084342 txn/s, submitted: 1100.2590361226182 txn/s, failed submission: 0.9130780382760317 txn/s, expired: 0.9130780382760317 txn/s, latency: 2780.2503426079734 ms, (p50: 2100 ms, p90: 4800 ms, p99: 9800 ms), latency samples: 96320
5. check swarm health
Compatibility test for 1c2ee7082d6eff8c811ee25d6f5a7d00860a75d5 ==> 144305500cda4468551b6c282699267bec25ccbd passed
Upgrade the remaining nodes to version: 144305500cda4468551b6c282699267bec25ccbd
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1033.9339182158096 txn/s, submitted: 1036.7321155587483 txn/s, failed submission: 2.798197342938592 txn/s, expired: 2.798197342938592 txn/s, latency: 2963.5998872350024 ms, (p50: 2100 ms, p90: 5700 ms, p99: 12700 ms), latency samples: 88680
Test Ok

@perryjrandall perryjrandall requested a review from brianolson July 17, 2024 16:18
@perryjrandall perryjrandall merged commit 42d561d into main Jul 17, 2024
90 checks passed
@perryjrandall perryjrandall deleted the perry/forge-upgrade-yamls branch July 17, 2024 16:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@sherry-x sherry-x sherry-x approved these changes

@zjma zjma zjma approved these changes

@runtian-zhou runtian-zhou Awaiting requested review from runtian-zhou

@rahxephon89 rahxephon89 Awaiting requested review from rahxephon89

@vgao1996 vgao1996 Awaiting requested review from vgao1996

@brianolson brianolson Awaiting requested review from brianolson

Assignees
No one assigned
Labels
CICD:run-framework-upgrade-test
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@perryjrandall @zjma @sherry-x