Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add validation for internal indexer data #14840

Merged
merged 1 commit into from
Oct 26, 2024

Conversation

areshand
Copy link
Contributor

@areshand areshand commented Oct 2, 2024

Description

one time tool, save in case we may want to use

How Has This Been Tested?

Key Areas to Review

Type of Change

  • New feature
  • Bug fix
  • Breaking change
  • Performance improvement
  • Refactoring
  • Dependency update
  • Documentation update
  • Tests

Which Components or Systems Does This Change Impact?

  • Validator Node
  • Full Node (API, Indexer, etc.)
  • Move/Aptos Virtual Machine
  • Aptos Framework
  • Aptos CLI/SDK
  • Developer Infrastructure
  • Move Compiler
  • Other (specify)

Checklist

  • I have read and followed the CONTRIBUTING doc
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I identified and added all stakeholders and component owners affected by this change as reviewers
  • I tested both happy and unhappy path of the functionality
  • I have made corresponding changes to the documentation

Copy link

trunk-io bot commented Oct 2, 2024

⏱️ 25m total CI duration on this PR
Job Cumulative Duration Recent Runs
rust-cargo-deny 7m 🟩🟩🟩🟩
check-dynamic-deps 6m 🟩🟩🟩🟩
rust-move-tests 2m 🟩
general-lints 2m 🟩🟩🟩🟩
rust-move-tests 2m 🟩
rust-move-tests 2m 🟩
rust-move-tests 2m 🟩
semgrep/ci 1m 🟩🟩🟩🟩
file_change_determinator 44s 🟩🟩🟩🟩
permission-check 19s 🟩🟩🟩🟩
permission-check 14s 🟩🟩🟩🟩
permission-check 4s 🟩

🚨 1 job on the last run was significantly faster/slower than expected

Job Duration vs 7d avg Delta
check-dynamic-deps 2m 1m +99%

settingsfeedbackdocs ⋅ learn more about trunk.io

@areshand areshand force-pushed the internal_indexer_data_validation branch 3 times, most recently from 722ace0 to 644eaf8 Compare October 7, 2024 16:51
@areshand areshand marked this pull request as ready for review October 10, 2024 23:08
@areshand areshand force-pushed the internal_indexer_data_validation branch 2 times, most recently from cb6d4ae to bb21237 Compare October 16, 2024 22:56
@areshand areshand enabled auto-merge (rebase) October 24, 2024 17:54

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

@areshand areshand force-pushed the internal_indexer_data_validation branch 2 times, most recently from 8be6523 to bc413c3 Compare October 26, 2024 03:57
@areshand areshand force-pushed the internal_indexer_data_validation branch from bc413c3 to 0b0de09 Compare October 26, 2024 04:30

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

Copy link
Contributor

✅ Forge suite realistic_env_max_load success on 0b0de09892f23f3484d5befcff211309056b2ec7

two traffics test: inner traffic : committed: 14341.83 txn/s, latency: 2768.38 ms, (p50: 2700 ms, p70: 2700, p90: 3000 ms, p99: 3200 ms), latency samples: 5453480
two traffics test : committed: 100.09 txn/s, latency: 1443.69 ms, (p50: 1400 ms, p70: 1500, p90: 1600 ms, p99: 1700 ms), latency samples: 1740
Latency breakdown for phase 0: ["MempoolToBlockCreation: max: 1.941, avg: 1.526", "ConsensusProposalToOrdered: max: 0.320, avg: 0.294", "ConsensusOrderedToCommit: max: 0.378, avg: 0.367", "ConsensusProposalToCommit: max: 0.668, avg: 0.661"]
Max non-epoch-change gap was: 0 rounds at version 0 (avg 0.00) [limit 4], 0.87s no progress at version 2494918 (avg 0.20s) [limit 15].
Max epoch-change gap was: 0 rounds at version 0 (avg 0.00) [limit 4], 8.54s no progress at version 2494916 (avg 8.54s) [limit 15].
Test Ok

Copy link
Contributor

✅ Forge suite compat success on f38ec72e975a4dff4e9919e7a1d8118a75858bab ==> 0b0de09892f23f3484d5befcff211309056b2ec7

Compatibility test results for f38ec72e975a4dff4e9919e7a1d8118a75858bab ==> 0b0de09892f23f3484d5befcff211309056b2ec7 (PR)
1. Check liveness of validators at old version: f38ec72e975a4dff4e9919e7a1d8118a75858bab
compatibility::simple-validator-upgrade::liveness-check : committed: 14466.27 txn/s, latency: 2394.34 ms, (p50: 1700 ms, p70: 1900, p90: 3300 ms, p99: 16700 ms), latency samples: 546600
2. Upgrading first Validator to new version: 0b0de09892f23f3484d5befcff211309056b2ec7
compatibility::simple-validator-upgrade::single-validator-upgrading : committed: 5062.21 txn/s, latency: 5594.58 ms, (p50: 5800 ms, p70: 6000, p90: 8100 ms, p99: 8500 ms), latency samples: 92760
compatibility::simple-validator-upgrade::single-validator-upgrade : committed: 5492.02 txn/s, latency: 5462.32 ms, (p50: 5900 ms, p70: 6000, p90: 6100 ms, p99: 7400 ms), latency samples: 206580
3. Upgrading rest of first batch to new version: 0b0de09892f23f3484d5befcff211309056b2ec7
compatibility::simple-validator-upgrade::half-validator-upgrading : committed: 6534.21 txn/s, latency: 4061.10 ms, (p50: 4600 ms, p70: 4700, p90: 4800 ms, p99: 5000 ms), latency samples: 130240
compatibility::simple-validator-upgrade::half-validator-upgrade : committed: 7216.29 txn/s, latency: 4405.69 ms, (p50: 4600 ms, p70: 4700, p90: 6600 ms, p99: 6800 ms), latency samples: 242140
4. upgrading second batch to new version: 0b0de09892f23f3484d5befcff211309056b2ec7
compatibility::simple-validator-upgrade::rest-validator-upgrading : committed: 9333.04 txn/s, latency: 2952.23 ms, (p50: 3100 ms, p70: 3700, p90: 4000 ms, p99: 4200 ms), latency samples: 164920
compatibility::simple-validator-upgrade::rest-validator-upgrade : committed: 8934.74 txn/s, latency: 3513.66 ms, (p50: 3500 ms, p70: 3700, p90: 5400 ms, p99: 5700 ms), latency samples: 297780
5. check swarm health
Compatibility test for f38ec72e975a4dff4e9919e7a1d8118a75858bab ==> 0b0de09892f23f3484d5befcff211309056b2ec7 passed
Test Ok

Copy link
Contributor

✅ Forge suite framework_upgrade success on f38ec72e975a4dff4e9919e7a1d8118a75858bab ==> 0b0de09892f23f3484d5befcff211309056b2ec7

Compatibility test results for f38ec72e975a4dff4e9919e7a1d8118a75858bab ==> 0b0de09892f23f3484d5befcff211309056b2ec7 (PR)
Upgrade the nodes to version: 0b0de09892f23f3484d5befcff211309056b2ec7
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1227.05 txn/s, submitted: 1229.80 txn/s, failed submission: 2.75 txn/s, expired: 2.75 txn/s, latency: 2554.75 ms, (p50: 2400 ms, p70: 2700, p90: 3900 ms, p99: 5200 ms), latency samples: 107120
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1214.83 txn/s, submitted: 1217.51 txn/s, failed submission: 2.68 txn/s, expired: 2.68 txn/s, latency: 2481.94 ms, (p50: 2400 ms, p70: 2400, p90: 3900 ms, p99: 6700 ms), latency samples: 108900
5. check swarm health
Compatibility test for f38ec72e975a4dff4e9919e7a1d8118a75858bab ==> 0b0de09892f23f3484d5befcff211309056b2ec7 passed
Upgrade the remaining nodes to version: 0b0de09892f23f3484d5befcff211309056b2ec7
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1055.32 txn/s, submitted: 1058.45 txn/s, failed submission: 3.12 txn/s, expired: 3.12 txn/s, latency: 2859.67 ms, (p50: 2700 ms, p70: 3000, p90: 3900 ms, p99: 5700 ms), latency samples: 94600
Test Ok

@areshand areshand merged commit d4c5493 into main Oct 26, 2024
48 checks passed
@areshand areshand deleted the internal_indexer_data_validation branch October 26, 2024 05:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants