Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fix keyless test flakiness #13764

Merged
merged 1 commit into from
Jun 20, 2024
Merged

fix keyless test flakiness #13764

merged 1 commit into from
Jun 20, 2024

Conversation

zjma
Copy link
Contributor

@zjma zjma commented Jun 20, 2024

Description

Fix keyless test flakiness by explicitly specifying max gas.

Type of Change

  • Bug fix
  • Tests

Which Components or Systems Does This Change Impact?

n/a

How Has This Been Tested?

n/a

Key Areas to Review

n/a

Checklist

  • I have read and followed the CONTRIBUTING doc
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I identified and added all stakeholders and component owners affected by this change as reviewers
  • I tested both happy and unhappy path of the functionality
  • I have made corresponding changes to the documentation

@trunk-io Trunk.io
Copy link

trunk-io bot commented Jun 20, 2024 •
edited
Loading

⏱️ 3h 4m total CI duration on this PR
Job Cumulative Duration Recent Runs
test-fuzzers 1h 9m 🟩 🟩 
rust-smoke-tests 32m 🟩 
forge-e2e-test / forge 13m 🟩 
forge-compat-test / forge 13m 🟩 
rust-images / rust-all 12m 🟩 
cli-e2e-tests / run-cli-tests 6m 🟩 
rust-lints 5m 🟩 
run-tests-main-branch 4m 🟩 
test-target-determinator 4m 🟩 
rust-build-cached-packages 4m 🟩 
execution-performance / test-target-determinator 4m 🟩 
check 4m 🟩 
rust-move-tests 3m ⬜ 🟩 
rust-targeted-unit-tests 3m 🟩 
check-dynamic-deps 2m 🟩 🟩 
general-lints 2m 🟩 
node-api-compatibility-tests / node-api-compatibility-tests 1m 🟩 
semgrep/ci 40s 🟩 🟩 
file_change_determinator 22s ⬜ 🟩 
file_change_determinator 11s 🟩 
file_change_determinator 10s 🟩 
execution-performance / single-node-performance 8s 🟩 
permission-check 7s 🟩 🟩 
forge-framework-upgrade-test / forge 7s 🟩 
permission-check 6s 🟩 🟩 
permission-check 6s 🟩 🟩 
permission-check 4s 🟩 🟩 
permission-check 2s 🟩 
determine-docker-build-metadata 2s 🟩 

🚨 3 jobs on the last run were significantly faster/slower than expected

Job Duration vs 7d avg Delta
rust-build-cached-packages 4m 5m -23%
rust-targeted-unit-tests 3m 14m -79%
forge-framework-upgrade-test / forge 7s 14m -99%

settings ⋅ feedback ⋅ docs ⋅ learn more about trunk.io

@zjma zjma marked this pull request as ready for review June 20, 2024 05:19
@zjma zjma enabled auto-merge (squash) June 20, 2024 05:20
geekflyer
geekflyer reviewed Jun 20, 2024
View reviewed changes
testsuite/smoke-test/src/keyless.rs
@@ -495,7 +495,15 @@ fun main(core_resources: &signer) {{
"#,
KEYLESS_ACCOUNT_MODULE_NAME, KEYLESS_ACCOUNT_MODULE_NAME
);
let txn_summary = cli.run_script(root_idx, &script).await.unwrap();
let gas_options = GasOptions {
Copy link
Contributor

@geekflyer geekflyer Jun 20, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we make this more DRY instead of copy pasting the same code into 3 places?

@github-actions GitHub Actions

This comment has been minimized.

Sign in to view
@github-actions GitHub Actions

This comment has been minimized.

Sign in to view
@github-actions GitHub Actions
Copy link
Contributor

✅ Forge suite compat success on a68e71c05caebf01504d4499110f3fba213fb53d ==> c9ad9ccb0691b8713ecfb840da897580c2f49f2b

Compatibility test results for a68e71c05caebf01504d4499110f3fba213fb53d ==> c9ad9ccb0691b8713ecfb840da897580c2f49f2b (PR)
1. Check liveness of validators at old version: a68e71c05caebf01504d4499110f3fba213fb53d
compatibility::simple-validator-upgrade::liveness-check : committed: 8728.066460633027 txn/s, latency: 3748.4182656708795 ms, (p50: 2700 ms, p90: 6600 ms, p99: 25300 ms), latency samples: 313320
2. Upgrading first Validator to new version: c9ad9ccb0691b8713ecfb840da897580c2f49f2b
compatibility::simple-validator-upgrade::single-validator-upgrade : committed: 3244.195014295774 txn/s, latency: 9606.119992581602 ms, (p50: 10200 ms, p90: 14000 ms, p99: 14400 ms), latency samples: 134800
3. Upgrading rest of first batch to new version: c9ad9ccb0691b8713ecfb840da897580c2f49f2b
compatibility::simple-validator-upgrade::half-validator-upgrade : committed: 3329.2810469365027 txn/s, latency: 9365.668441146809 ms, (p50: 9400 ms, p90: 14200 ms, p99: 14500 ms), latency samples: 138820
4. upgrading second batch to new version: c9ad9ccb0691b8713ecfb840da897580c2f49f2b
compatibility::simple-validator-upgrade::rest-validator-upgrade : committed: 4736.574900658582 txn/s, latency: 6853.551956991694 ms, (p50: 5100 ms, p90: 11100 ms, p99: 15900 ms), latency samples: 175780
5. check swarm health
Compatibility test for a68e71c05caebf01504d4499110f3fba213fb53d ==> c9ad9ccb0691b8713ecfb840da897580c2f49f2b passed
Test Ok

@github-actions GitHub Actions
Copy link
Contributor

✅ Forge suite realistic_env_max_load success on c9ad9ccb0691b8713ecfb840da897580c2f49f2b

two traffics test: inner traffic : committed: 8660.012594391135 txn/s, latency: 4529.737442916263 ms, (p50: 4500 ms, p90: 5400 ms, p99: 9200 ms), latency samples: 3735740
two traffics test : committed: 100.03248506111906 txn/s, latency: 2064.9533333333334 ms, (p50: 2000 ms, p90: 2200 ms, p99: 6800 ms), latency samples: 1800
Latency breakdown for phase 0: ["QsBatchToPos: max: 0.220, avg: 0.212", "QsPosToProposal: max: 0.251, avg: 0.239", "ConsensusProposalToOrdered: max: 0.303, avg: 0.289", "ConsensusOrderedToCommit: max: 0.368, avg: 0.358", "ConsensusProposalToCommit: max: 0.654, avg: 0.647"]
Max round gap was 1 [limit 4] at version 1781585. Max no progress secs was 4.819482 [limit 15] at version 1781585.
Test Ok

@zjma zjma merged commit b25016a into main Jun 20, 2024
80 of 88 checks passed
@zjma zjma deleted the zjma/fix_flaky_keyless_tests branch June 20, 2024 17:04
perryjrandall pushed a commit that referenced this pull request Jun 21, 2024
@zjma @perryjrandall
fix keyless test flakiness (#13764)
14682de
sherry-x pushed a commit that referenced this pull request Jun 22, 2024
@zjma @sherry-x
fix keyless test flakiness (#13764)
10b7395
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@geekflyer geekflyer geekflyer approved these changes

@alinush alinush alinush approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@zjma @geekflyer @alinush