add check for forge images #15007

aluon · 2024-10-18T18:27:55Z

Description

Indicate which image is being used by Forge in the workflow summary. The current behavior of falling back to the latest image can be misleading

How Has This Been Tested?

https://github.com/aptos-labs/aptos-core/actions/runs/11410691190

https://github.com/aptos-labs/aptos-core/actions/runs/11410692772

Key Areas to Review

Type of Change

Which Components or Systems Does This Change Impact?

Checklist

I have read and followed the CONTRIBUTING doc
I have performed a self-review of my own code
I have commented my code, particularly in hard-to-understand areas
I identified and added all stakeholders and component owners affected by this change as reviewers
I tested both happy and unhappy path of the functionality
I have made corresponding changes to the documentation

trunk-io · 2024-10-18T18:27:59Z

⏱️ 1h 16m total CI duration on this PR

Slowest 15 Jobs	Cumulative Duration	Recent Runs
adhoc-forge-test / forge	14m	🟩
adhoc-forge-test / forge	14m	🟩
adhoc-forge-test / forge	13m	🟩
rust-cargo-deny	9m	🟩 🟩 🟩 🟩 🟩
check-dynamic-deps	6m	🟩 🟩 🟩 🟩 🟩
general-lints	2m	🟩 🟩 🟩 🟩 🟩
rust-move-tests	2m	🟩
rust-move-tests	2m	🟩
semgrep/ci	2m	🟩 🟩 🟩 🟩 🟩
adhoc-forge-test / forge	2m	🟥
adhoc-forge-test / forge	2m	🟥
rust-move-tests	2m	🟩
rust-move-tests	1m	🟩
rust-move-tests	1m	🟩
file_change_determinator	58s	🟩 🟩 🟩 🟩 🟩

_{settings ⋅ feedback ⋅ docs ⋅ learn more about trunk.io}

ibalajiarun · 2024-11-12T16:46:50Z

.github/workflows/workflow-run-forge.yaml

+                echo "✅ $image_name image found for tag: $tag" >> $GITHUB_STEP_SUMMARY
+              else
+                echo "${output_prefix}_EXISTS=false" >> $GITHUB_OUTPUT
+                echo "⚠️ No $image_name image found for tag: $tag. The latest available image will be used instead." >> $GITHUB_STEP_SUMMARY


Is it easy to print which image will be used if latest? Also, where will this be posted on a PR job?

There's code around getting the latest image in forge.py that I didn't want to duplicate here. This is mostly to prompt users to use the CICD: build-images label when running Forge tests against their PR. Users can still see the actual images being used by checking the workflow logs

The output shows up in the workflow summary https://github.com/aptos-labs/aptos-core/actions/runs/11410691190

May be we should think about unifying these logic in a future PR.

Yeah I think the logic to determine the images belongs in GHA. We should pull out the image tags to inputs/outputs to make the Forge runs reproducible. Right now if we rerun the workflow, it could result in different images being used

github-actions · 2024-11-13T20:34:16Z

✅ Forge suite `realistic_env_max_load` success on `27d5b65ff701c44badc54b55151247811e289b3a`

two traffics test: inner traffic : committed: 14329.89 txn/s, latency: 2775.65 ms, (p50: 2700 ms, p70: 2700, p90: 3000 ms, p99: 3600 ms), latency samples: 5448860
two traffics test : committed: 99.94 txn/s, latency: 1517.18 ms, (p50: 1400 ms, p70: 1500, p90: 1600 ms, p99: 8900 ms), latency samples: 1780
Latency breakdown for phase 0: ["MempoolToBlockCreation: max: 2.093, avg: 1.562", "ConsensusProposalToOrdered: max: 0.310, avg: 0.293", "ConsensusOrderedToCommit: max: 0.366, avg: 0.358", "ConsensusProposalToCommit: max: 0.658, avg: 0.651"]
Max non-epoch-change gap was: 0 rounds at version 0 (avg 0.00) [limit 4], 1.48s no progress at version 28568 (avg 0.20s) [limit 15].
Max epoch-change gap was: 0 rounds at version 0 (avg 0.00) [limit 4], 8.47s no progress at version 2797757 (avg 8.47s) [limit 15].
Test Ok

github-actions · 2024-11-13T20:35:48Z

✅ Forge suite `framework_upgrade` success on `ea6e45f0eee4b6da2ebf93b9b89d269d334fcf70` ==> `27d5b65ff701c44badc54b55151247811e289b3a`

Compatibility test results for ea6e45f0eee4b6da2ebf93b9b89d269d334fcf70 ==> 27d5b65ff701c44badc54b55151247811e289b3a (PR)
Upgrade the nodes to version: 27d5b65ff701c44badc54b55151247811e289b3a
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1352.29 txn/s, submitted: 1356.24 txn/s, failed submission: 3.95 txn/s, expired: 3.95 txn/s, latency: 2357.76 ms, (p50: 2100 ms, p70: 2400, p90: 3600 ms, p99: 5400 ms), latency samples: 123260
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1222.30 txn/s, submitted: 1225.14 txn/s, failed submission: 2.85 txn/s, expired: 2.85 txn/s, latency: 2409.10 ms, (p50: 2400 ms, p70: 2700, p90: 3600 ms, p99: 4800 ms), latency samples: 111660
5. check swarm health
Compatibility test for ea6e45f0eee4b6da2ebf93b9b89d269d334fcf70 ==> 27d5b65ff701c44badc54b55151247811e289b3a passed
Upgrade the remaining nodes to version: 27d5b65ff701c44badc54b55151247811e289b3a
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1234.96 txn/s, submitted: 1238.49 txn/s, failed submission: 3.53 txn/s, expired: 3.53 txn/s, latency: 2392.30 ms, (p50: 2400 ms, p70: 2400, p90: 3300 ms, p99: 5400 ms), latency samples: 111940
Test Ok

github-actions · 2024-11-13T20:36:02Z

✅ Forge suite `compat` success on `ea6e45f0eee4b6da2ebf93b9b89d269d334fcf70` ==> `27d5b65ff701c44badc54b55151247811e289b3a`

Compatibility test results for ea6e45f0eee4b6da2ebf93b9b89d269d334fcf70 ==> 27d5b65ff701c44badc54b55151247811e289b3a (PR)
1. Check liveness of validators at old version: ea6e45f0eee4b6da2ebf93b9b89d269d334fcf70
compatibility::simple-validator-upgrade::liveness-check : committed: 15431.16 txn/s, latency: 2209.54 ms, (p50: 2000 ms, p70: 2100, p90: 2400 ms, p99: 6400 ms), latency samples: 495840
2. Upgrading first Validator to new version: 27d5b65ff701c44badc54b55151247811e289b3a
compatibility::simple-validator-upgrade::single-validator-upgrading : committed: 7379.53 txn/s, latency: 3686.07 ms, (p50: 4200 ms, p70: 4400, p90: 5000 ms, p99: 5100 ms), latency samples: 132400
compatibility::simple-validator-upgrade::single-validator-upgrade : committed: 7521.07 txn/s, latency: 4240.51 ms, (p50: 4400 ms, p70: 4600, p90: 6100 ms, p99: 6500 ms), latency samples: 247340
3. Upgrading rest of first batch to new version: 27d5b65ff701c44badc54b55151247811e289b3a
compatibility::simple-validator-upgrade::half-validator-upgrading : committed: 6761.06 txn/s, latency: 3881.10 ms, (p50: 4400 ms, p70: 4600, p90: 4800 ms, p99: 4900 ms), latency samples: 135300
compatibility::simple-validator-upgrade::half-validator-upgrade : committed: 6403.77 txn/s, latency: 4993.88 ms, (p50: 5300 ms, p70: 5400, p90: 7300 ms, p99: 7500 ms), latency samples: 209800
4. upgrading second batch to new version: 27d5b65ff701c44badc54b55151247811e289b3a
compatibility::simple-validator-upgrade::rest-validator-upgrading : committed: 12076.04 txn/s, latency: 2288.74 ms, (p50: 2500 ms, p70: 2600, p90: 2900 ms, p99: 2900 ms), latency samples: 208040
compatibility::simple-validator-upgrade::rest-validator-upgrade : committed: 10573.02 txn/s, latency: 2945.59 ms, (p50: 2600 ms, p70: 2800, p90: 6000 ms, p99: 7000 ms), latency samples: 340640
5. check swarm health
Compatibility test for ea6e45f0eee4b6da2ebf93b9b89d269d334fcf70 ==> 27d5b65ff701c44badc54b55151247811e289b3a passed
Test Ok

aluon force-pushed the aluon/push-sotppkqtpxws branch 3 times, most recently from c3d6ca7 to 96a93a3 Compare October 18, 2024 22:14

aluon requested review from rustielin, hariria and a team October 18, 2024 22:29

aluon marked this pull request as ready for review October 18, 2024 22:29

aluon requested a review from a team as a code owner October 18, 2024 22:29

add check for forge images

1ef2b66

aluon force-pushed the aluon/push-sotppkqtpxws branch from 96a93a3 to 1ef2b66 Compare October 18, 2024 22:30

sionescu approved these changes Oct 18, 2024

View reviewed changes

ibalajiarun reviewed Nov 12, 2024

View reviewed changes

ibalajiarun approved these changes Nov 12, 2024

View reviewed changes

Merge branch 'main' into aluon/push-sotppkqtpxws

27d5b65

aluon enabled auto-merge (squash) November 13, 2024 20:06

This comment has been minimized.

Sign in to view

aluon merged commit 1142552 into main Nov 13, 2024
92 checks passed

aluon deleted the aluon/push-sotppkqtpxws branch November 13, 2024 20:36

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

add check for forge images #15007

add check for forge images #15007

aluon commented Oct 18, 2024 •

edited

Loading

trunk-io bot commented Oct 18, 2024 •

edited

Loading

ibalajiarun Nov 12, 2024 •

edited

Loading

aluon Nov 12, 2024

ibalajiarun Nov 12, 2024

aluon Nov 12, 2024

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

github-actions bot commented Nov 13, 2024

github-actions bot commented Nov 13, 2024

github-actions bot commented Nov 13, 2024

add check for forge images #15007

add check for forge images #15007

Conversation

aluon commented Oct 18, 2024 • edited Loading

Description

How Has This Been Tested?

Key Areas to Review

Type of Change

Which Components or Systems Does This Change Impact?

Checklist

trunk-io bot commented Oct 18, 2024 • edited Loading

ibalajiarun Nov 12, 2024 • edited Loading

Choose a reason for hiding this comment

aluon Nov 12, 2024

Choose a reason for hiding this comment

ibalajiarun Nov 12, 2024

Choose a reason for hiding this comment

aluon Nov 12, 2024

Choose a reason for hiding this comment

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

github-actions bot commented Nov 13, 2024

✅ Forge suite realistic_env_max_load success on 27d5b65ff701c44badc54b55151247811e289b3a

github-actions bot commented Nov 13, 2024

✅ Forge suite framework_upgrade success on ea6e45f0eee4b6da2ebf93b9b89d269d334fcf70 ==> 27d5b65ff701c44badc54b55151247811e289b3a

github-actions bot commented Nov 13, 2024

✅ Forge suite compat success on ea6e45f0eee4b6da2ebf93b9b89d269d334fcf70 ==> 27d5b65ff701c44badc54b55151247811e289b3a

aluon commented Oct 18, 2024 •

edited

Loading

trunk-io bot commented Oct 18, 2024 •

edited

Loading

ibalajiarun Nov 12, 2024 •

edited

Loading

✅ Forge suite `realistic_env_max_load` success on `27d5b65ff701c44badc54b55151247811e289b3a`

✅ Forge suite `framework_upgrade` success on `ea6e45f0eee4b6da2ebf93b9b89d269d334fcf70` ==> `27d5b65ff701c44badc54b55151247811e289b3a`

✅ Forge suite `compat` success on `ea6e45f0eee4b6da2ebf93b9b89d269d334fcf70` ==> `27d5b65ff701c44badc54b55151247811e289b3a`