Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add workflow to test fuzzers #13710

Merged
merged 3 commits into from
Jun 17, 2024
Merged

add workflow to test fuzzers #13710

merged 3 commits into from
Jun 17, 2024

Conversation

gedigi
Copy link
Contributor

@gedigi gedigi commented Jun 14, 2024

Description

This PR adds a GitHub workflow to test fuzzers work as expected.

Type of Change

  • New feature
  • Bug fix
  • Breaking change
  • Performance improvement
  • Refactoring
  • Dependency update
  • Documentation update
  • Tests

Which Components or Systems Does This Change Impact?

  • Validator Node
  • Full Node (API, Indexer, etc.)
  • Move/Aptos Virtual Machine
  • Aptos Framework
  • Aptos CLI/SDK
  • Developer Infrastructure
  • Other (specify)

How Has This Been Tested?

Key Areas to Review

n/a

Checklist

  • I have read and followed the CONTRIBUTING doc
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I identified and added all stakeholders and component owners affected by this change as reviewers
  • I tested both happy and unhappy path of the functionality
  • I have made corresponding changes to the documentation

Copy link

trunk-io bot commented Jun 14, 2024

⏱️ 6h 34m total CI duration on this PR
Job Cumulative Duration Recent Runs
test-fuzzers 2h 5m 🟥🟩🟩🟩
test-fuzzer 42m 🟥🟥🟩
rust-smoke-tests 40m 🟩
run-tests-main-branch 27m 🟩🟩🟩🟩 (+2 more)
rust-lints 26m 🟩🟩🟩 (+2 more)
rust-move-tests 23m 🟩🟩🟩 (+2 more)
rust-images / rust-all 16m 🟩
rust-targeted-unit-tests 15m 🟩🟩🟩 (+2 more)
forge-e2e-test / forge 14m 🟩
forge-compat-test / forge 13m 🟩
check-dynamic-deps 10m 🟩🟩🟩🟩🟩 (+2 more)
general-lints 10m 🟩🟩🟩🟩 (+2 more)
cli-e2e-tests / run-cli-tests 9m 🟩
rust-build-cached-packages 5m 🟩
test-target-determinator 4m 🟩
execution-performance / test-target-determinator 4m 🟩
check 4m 🟩
semgrep/ci 2m 🟩🟩🟩🟩🟩 (+2 more)
file_change_determinator 1m 🟩🟩🟩🟩🟩 (+2 more)
file_change_determinator 1m 🟩🟩🟩🟩🟩 (+2 more)
node-api-compatibility-tests / node-api-compatibility-tests 1m 🟩
permission-check 22s 🟩🟩🟩🟩🟩 (+2 more)
permission-check 22s 🟩🟩🟩🟩🟩 (+2 more)
permission-check 20s 🟩🟩🟩🟩🟩 (+2 more)
permission-check 15s 🟩🟩🟩🟩🟩 (+2 more)
file_change_determinator 11s 🟩
execution-performance / single-node-performance 7s 🟩
forge-framework-upgrade-test / forge 6s 🟩
permission-check 2s 🟩
determine-docker-build-metadata 2s 🟩

🚨 4 jobs on the last run were significantly faster/slower than expected

Job Duration vs 7d avg Delta
cli-e2e-tests / run-cli-tests 9m 6m +39%
rust-targeted-unit-tests 4m 15m -77%
forge-framework-upgrade-test / forge 6s 12m -99%
execution-performance / single-node-performance 7s 14m -99%

settingsfeedbackdocs ⋅ learn more about trunk.io

@gedigi gedigi force-pushed the gdg/test_fuzzer branch 2 times, most recently from ab1d8f5 to de08db2 Compare June 14, 2024 22:47
@gedigi gedigi force-pushed the gdg/test_fuzzer branch from de08db2 to f6c1f9f Compare June 15, 2024 02:19
@gedigi gedigi marked this pull request as ready for review June 17, 2024 19:33
@gedigi gedigi requested a review from a team as a code owner June 17, 2024 19:33
@gedigi gedigi requested review from zi0Black and GotenJBZ June 17, 2024 19:33
@gedigi gedigi changed the title add fuzzer-test.yaml add workflow to test fuzzers Jun 17, 2024
@gedigi gedigi enabled auto-merge (squash) June 17, 2024 19:37

This comment has been minimized.

This comment has been minimized.

Copy link
Contributor

✅ Forge suite compat success on a68e71c05caebf01504d4499110f3fba213fb53d ==> 524beb854cc26d683199f6fef0ab1f3805f7f2dd

Compatibility test results for a68e71c05caebf01504d4499110f3fba213fb53d ==> 524beb854cc26d683199f6fef0ab1f3805f7f2dd (PR)
1. Check liveness of validators at old version: a68e71c05caebf01504d4499110f3fba213fb53d
compatibility::simple-validator-upgrade::liveness-check : committed: 9169.52304808467 txn/s, latency: 3639.2604061205707 ms, (p50: 2700 ms, p90: 6300 ms, p99: 28800 ms), latency samples: 347680
2. Upgrading first Validator to new version: 524beb854cc26d683199f6fef0ab1f3805f7f2dd
compatibility::simple-validator-upgrade::single-validator-upgrade : committed: 3334.870282611824 txn/s, latency: 9323.971169182569 ms, (p50: 9500 ms, p90: 13900 ms, p99: 14400 ms), latency samples: 140440
3. Upgrading rest of first batch to new version: 524beb854cc26d683199f6fef0ab1f3805f7f2dd
compatibility::simple-validator-upgrade::half-validator-upgrade : committed: 3339.7369216797088 txn/s, latency: 9053.983874210224 ms, (p50: 8500 ms, p90: 14100 ms, p99: 14500 ms), latency samples: 139280
4. upgrading second batch to new version: 524beb854cc26d683199f6fef0ab1f3805f7f2dd
compatibility::simple-validator-upgrade::rest-validator-upgrade : committed: 4470.308379333658 txn/s, latency: 7157.588827661379 ms, (p50: 6900 ms, p90: 10000 ms, p99: 14200 ms), latency samples: 165140
5. check swarm health
Compatibility test for a68e71c05caebf01504d4499110f3fba213fb53d ==> 524beb854cc26d683199f6fef0ab1f3805f7f2dd passed
Test Ok

Copy link
Contributor

✅ Forge suite realistic_env_max_load success on 524beb854cc26d683199f6fef0ab1f3805f7f2dd

two traffics test: inner traffic : committed: 8702.045904289163 txn/s, latency: 4502.008982067763 ms, (p50: 4200 ms, p90: 5100 ms, p99: 10500 ms), latency samples: 3761940
two traffics test : committed: 100.05236534063144 txn/s, latency: 1982.9384615384615 ms, (p50: 1900 ms, p90: 2200 ms, p99: 4600 ms), latency samples: 1820
Latency breakdown for phase 0: ["QsBatchToPos: max: 0.210, avg: 0.206", "QsPosToProposal: max: 0.418, avg: 0.259", "ConsensusProposalToOrdered: max: 0.306, avg: 0.285", "ConsensusOrderedToCommit: max: 0.357, avg: 0.345", "ConsensusProposalToCommit: max: 0.639, avg: 0.631"]
Max round gap was 1 [limit 4] at version 1714295. Max no progress secs was 5.185059 [limit 15] at version 1714295.
Test Ok

@gedigi gedigi merged commit 712d1eb into main Jun 17, 2024
85 of 88 checks passed
@gedigi gedigi deleted the gdg/test_fuzzer branch June 17, 2024 20:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants