Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[1.23] Temporarily loosen backpressures, until module loading improvement speeds up blocks #15368

Merged
merged 1 commit into from
Nov 23, 2024

Conversation

igor-aptos
Copy link
Contributor

Description

How Has This Been Tested?

Key Areas to Review

Type of Change

  • New feature
  • Bug fix
  • Breaking change
  • Performance improvement
  • Refactoring
  • Dependency update
  • Documentation update
  • Tests

Which Components or Systems Does This Change Impact?

  • Validator Node
  • Full Node (API, Indexer, etc.)
  • Move/Aptos Virtual Machine
  • Aptos Framework
  • Aptos CLI/SDK
  • Developer Infrastructure
  • Move Compiler
  • Other (specify)

Checklist

  • I have read and followed the CONTRIBUTING doc
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I identified and added all stakeholders and component owners affected by this change as reviewers
  • I tested both happy and unhappy path of the functionality
  • I have made corresponding changes to the documentation

Copy link

trunk-io bot commented Nov 22, 2024

⏱️ 4h 41m total CI duration on this PR
Slowest 15 Jobs Cumulative Duration Recent Runs
execution-performance / single-node-performance 1h 17m 🟩🟩🟩
rust-unit-tests 25m 🟩
rust-unit-tests 24m 🟩
rust-unit-tests 23m 🟥
rust-unit-tests 21m 🟩
execution-performance / test-target-determinator 13m 🟩🟩🟩
test-target-determinator 13m 🟩🟩🟩
check 11m 🟩🟩🟩
rust-move-tests 10m 🟩
rust-move-tests 9m 🟩
rust-move-tests 9m 🟩
rust-move-tests 8m 🟩
rust-cargo-deny 7m 🟩🟩🟩🟩
check-dynamic-deps 6m 🟩🟩🟩🟩
rust-doc-tests 5m 🟩

🚨 1 job on the last run was significantly faster/slower than expected

Job Duration vs 7d avg Delta
execution-performance / single-node-performance 27m 17m +55%

settingsfeedbackdocs ⋅ learn more about trunk.io

Copy link
Contributor

@sitalkedia sitalkedia left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As discussed offline, let's validate this change by deploying to few of our nodes first.

Copy link

trunk-io bot commented Nov 22, 2024

Static BadgeStatic BadgeStatic BadgeStatic Badge

View Full Report ↗︎Docs

@sherry-x sherry-x enabled auto-merge (rebase) November 23, 2024 18:41

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

@sherry-x sherry-x force-pushed the igor/temporarily_loosen_backpressure branch from ee7057f to 594c2b7 Compare November 23, 2024 19:50

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

@sherry-x sherry-x merged commit c7fe444 into aptos-release-v1.23 Nov 23, 2024
48 checks passed
@sherry-x sherry-x deleted the igor/temporarily_loosen_backpressure branch November 23, 2024 20:26
@JoshLind JoshLind added the v1.24 label Nov 25, 2024
Copy link
Contributor

💚 All backports created successfully

Status Branch Result
aptos-release-v1.24

Questions ?

Please refer to the Backport tool documentation and see the Github Action logs for details

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

Copy link
Contributor

💔 Some backports could not be created

Status Branch Result
aptos-release-v1.24 Cherrypick failed because the selected commit (c7fe444) is empty. Did you already backport this commit?
aptos-release-v1.25

Manual backport

To create the backport manually run:

backport --pr 15368

Questions ?

Please refer to the Backport tool documentation and see the Github Action logs for details

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

Copy link
Contributor

✅ Forge suite realistic_env_max_load success on 594c2b7ee549d80f128b6d2f71d2dac8938a8511

two traffics test: inner traffic : committed: 14146.27 txn/s, latency: 2806.50 ms, (p50: 2700 ms, p70: 2700, p90: 3000 ms, p99: 5700 ms), latency samples: 5378800
two traffics test : committed: 99.93 txn/s, latency: 1833.62 ms, (p50: 1300 ms, p70: 1400, p90: 1700 ms, p99: 15000 ms), latency samples: 1760
Latency breakdown for phase 0: ["MempoolToBlockCreation: max: 2.584, avg: 1.617", "ConsensusProposalToOrdered: max: 0.313, avg: 0.292", "ConsensusOrderedToCommit: max: 0.361, avg: 0.353", "ConsensusProposalToCommit: max: 0.655, avg: 0.645"]
Max non-epoch-change gap was: 0 rounds at version 0 (avg 0.00) [limit 4], 1.72s no progress at version 30778 (avg 0.20s) [limit 15].
Max epoch-change gap was: 0 rounds at version 0 (avg 0.00) [limit 4], 15.57s no progress at version 2722449 (avg 15.57s) [limit 16].
Test Ok

Copy link
Contributor

✅ Forge suite compat success on afa8d75c467820b7d96bbe13ab0d011c12f8b3a3 ==> 594c2b7ee549d80f128b6d2f71d2dac8938a8511

Compatibility test results for afa8d75c467820b7d96bbe13ab0d011c12f8b3a3 ==> 594c2b7ee549d80f128b6d2f71d2dac8938a8511 (PR)
1. Check liveness of validators at old version: afa8d75c467820b7d96bbe13ab0d011c12f8b3a3
compatibility::simple-validator-upgrade::liveness-check : committed: 17127.36 txn/s, latency: 1994.83 ms, (p50: 2100 ms, p70: 2100, p90: 2200 ms, p99: 2400 ms), latency samples: 549820
2. Upgrading first Validator to new version: 594c2b7ee549d80f128b6d2f71d2dac8938a8511
compatibility::simple-validator-upgrade::single-validator-upgrading : committed: 7003.75 txn/s, latency: 3854.56 ms, (p50: 4100 ms, p70: 4400, p90: 5400 ms, p99: 5600 ms), latency samples: 127400
compatibility::simple-validator-upgrade::single-validator-upgrade : committed: 7761.62 txn/s, latency: 4140.49 ms, (p50: 4300 ms, p70: 4400, p90: 6100 ms, p99: 6300 ms), latency samples: 254200
3. Upgrading rest of first batch to new version: 594c2b7ee549d80f128b6d2f71d2dac8938a8511
compatibility::simple-validator-upgrade::half-validator-upgrading : committed: 7604.61 txn/s, latency: 3780.49 ms, (p50: 4200 ms, p70: 4300, p90: 4400 ms, p99: 4500 ms), latency samples: 144620
compatibility::simple-validator-upgrade::half-validator-upgrade : committed: 6979.55 txn/s, latency: 4264.77 ms, (p50: 4500 ms, p70: 4500, p90: 4600 ms, p99: 6100 ms), latency samples: 262220
4. upgrading second batch to new version: 594c2b7ee549d80f128b6d2f71d2dac8938a8511
compatibility::simple-validator-upgrade::rest-validator-upgrading : committed: 11341.78 txn/s, latency: 2439.31 ms, (p50: 2100 ms, p70: 2700, p90: 4500 ms, p99: 4900 ms), latency samples: 216080
compatibility::simple-validator-upgrade::rest-validator-upgrade : committed: 11544.98 txn/s, latency: 2662.69 ms, (p50: 2100 ms, p70: 2600, p90: 5000 ms, p99: 7800 ms), latency samples: 376240
5. check swarm health
Compatibility test for afa8d75c467820b7d96bbe13ab0d011c12f8b3a3 ==> 594c2b7ee549d80f128b6d2f71d2dac8938a8511 passed
Test Ok

Copy link
Contributor

✅ Forge suite framework_upgrade success on afa8d75c467820b7d96bbe13ab0d011c12f8b3a3 ==> 594c2b7ee549d80f128b6d2f71d2dac8938a8511

Compatibility test results for afa8d75c467820b7d96bbe13ab0d011c12f8b3a3 ==> 594c2b7ee549d80f128b6d2f71d2dac8938a8511 (PR)
Upgrade the nodes to version: 594c2b7ee549d80f128b6d2f71d2dac8938a8511
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1232.70 txn/s, submitted: 1234.26 txn/s, failed submission: 1.56 txn/s, expired: 1.56 txn/s, latency: 2460.01 ms, (p50: 2400 ms, p70: 2600, p90: 3700 ms, p99: 5300 ms), latency samples: 110480
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1345.56 txn/s, submitted: 1348.63 txn/s, failed submission: 3.07 txn/s, expired: 3.07 txn/s, latency: 2105.02 ms, (p50: 1800 ms, p70: 2100, p90: 3000 ms, p99: 5000 ms), latency samples: 122720
5. check swarm health
Compatibility test for afa8d75c467820b7d96bbe13ab0d011c12f8b3a3 ==> 594c2b7ee549d80f128b6d2f71d2dac8938a8511 passed
Upgrade the remaining nodes to version: 594c2b7ee549d80f128b6d2f71d2dac8938a8511
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1410.80 txn/s, submitted: 1413.38 txn/s, failed submission: 2.58 txn/s, expired: 2.58 txn/s, latency: 2149.00 ms, (p50: 1800 ms, p70: 2400, p90: 3300 ms, p99: 5100 ms), latency samples: 120520
Test Ok

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants