Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[data service] Add support for configuring in memory cache size #13570

Merged
merged 1 commit into from
Jun 5, 2024

Conversation

banool
Copy link
Contributor

@banool banool commented Jun 5, 2024

Description

We want to be able to tune the value of the in memory cache for data service. This PR adds support for that.

Type of Change

  • New feature
  • Bug fix
  • Breaking change
  • Performance improvement
  • Refactoring
  • Dependency update
  • Documentation update
  • Tests

Which Components or Systems Does This Change Impact?

  • Validator Node
  • Full Node (API, Indexer, etc.)
  • Move/Aptos Virtual Machine
  • Aptos Framework
  • Aptos CLI/SDK
  • Developer Infrastructure
  • Other (specify)

How Has This Been Tested?

For now, just see that the existing tests pass:

cd ecosystem/indexer-grpc/indexer-grpc-utils/
cargo test

Key Areas to Review

Make sure I'm not missing any spot we might need to change.

Checklist

  • I have read and followed the CONTRIBUTING doc
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I identified and added all stakeholders and component owners affected by this change as reviewers
  • I tested both happy and unhappy path of the functionality
  • I have made corresponding changes to the documentation

Copy link

trunk-io bot commented Jun 5, 2024

⏱️ 4h 8m total CI duration on this PR
Job Cumulative Duration Recent Runs
windows-build 44m 🟩
rust-smoke-tests 35m 🟩
run-tests-main-branch 22m 🟩🟩🟩🟩🟩
rust-targeted-unit-tests 21m 🟩🟩
forge-framework-upgrade-test / forge 15m 🟩
forge-e2e-test / forge 14m 🟩
rust-lints 14m 🟩🟩
rust-move-tests 14m 🟩🟩
rust-images / rust-all 13m 🟩
forge-compat-test / forge 12m 🟩
cli-e2e-tests / run-cli-tests 9m 🟩
check-dynamic-deps 7m 🟩🟩🟩🟩🟩
general-lints 6m 🟩🟩🟩
rust-build-cached-packages 4m 🟩
check 4m 🟩
test-target-determinator 4m 🟩
execution-performance / test-target-determinator 3m 🟩
semgrep/ci 2m 🟩🟩🟩🟩🟩
node-api-compatibility-tests / node-api-compatibility-tests 1m 🟩
file_change_determinator 53s 🟩🟩🟩🟩🟩
file_change_determinator 43s 🟩🟩🟩🟩
permission-check 21s 🟩🟩🟩🟩🟩
permission-check 12s 🟩🟩🟩🟩🟩
permission-check 12s 🟩🟩🟩🟩🟩
file_change_determinator 11s 🟩
permission-check 10s 🟩🟩🟩🟩
execution-performance / single-node-performance 8s 🟩
permission-check 3s 🟩
determine-docker-build-metadata 2s 🟩

🚨 4 jobs on the last run were significantly faster/slower than expected

Job Duration vs 7d avg Delta
cli-e2e-tests / run-cli-tests 9m 7m +34%
rust-targeted-unit-tests 9m 19m -49%
rust-move-tests 3m 9m -64%
execution-performance / single-node-performance 8s 21m -99%

settingsfeedbackdocs ⋅ learn more about trunk.io

@banool banool force-pushed the banool/data-service-cache-size branch 2 times, most recently from cbc2c93 to 044f189 Compare June 5, 2024 13:47
@banool banool force-pushed the banool/data-service-cache-size branch from 044f189 to 2ad040c Compare June 5, 2024 13:47
@banool banool enabled auto-merge (squash) June 5, 2024 17:04

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

Copy link
Contributor

github-actions bot commented Jun 5, 2024

✅ Forge suite compat success on a68e71c05caebf01504d4499110f3fba213fb53d ==> 2ad040cc7aae957f82a256302618fa0215e048b2

Compatibility test results for a68e71c05caebf01504d4499110f3fba213fb53d ==> 2ad040cc7aae957f82a256302618fa0215e048b2 (PR)
1. Check liveness of validators at old version: a68e71c05caebf01504d4499110f3fba213fb53d
compatibility::simple-validator-upgrade::liveness-check : committed: 10469.483861183695 txn/s, latency: 3014.0274786324785 ms, (p50: 2700 ms, p90: 4800 ms, p99: 7200 ms), latency samples: 374400
2. Upgrading first Validator to new version: 2ad040cc7aae957f82a256302618fa0215e048b2
compatibility::simple-validator-upgrade::single-validator-upgrade : committed: 3350.831921262083 txn/s, latency: 9293.450904977375 ms, (p50: 9400 ms, p90: 13900 ms, p99: 14200 ms), latency samples: 137020
3. Upgrading rest of first batch to new version: 2ad040cc7aae957f82a256302618fa0215e048b2
compatibility::simple-validator-upgrade::half-validator-upgrade : committed: 3411.2588756668547 txn/s, latency: 9165.645631563593 ms, (p50: 9500 ms, p90: 14100 ms, p99: 14500 ms), latency samples: 137120
4. upgrading second batch to new version: 2ad040cc7aae957f82a256302618fa0215e048b2
compatibility::simple-validator-upgrade::rest-validator-upgrade : committed: 6383.812758684274 txn/s, latency: 5098.179511903746 ms, (p50: 4800 ms, p90: 8400 ms, p99: 9400 ms), latency samples: 234380
5. check swarm health
Compatibility test for a68e71c05caebf01504d4499110f3fba213fb53d ==> 2ad040cc7aae957f82a256302618fa0215e048b2 passed
Test Ok

Copy link
Contributor

github-actions bot commented Jun 5, 2024

✅ Forge suite realistic_env_max_load success on 2ad040cc7aae957f82a256302618fa0215e048b2

two traffics test: inner traffic : committed: 8551.652778614249 txn/s, latency: 4579.76161822696 ms, (p50: 4500 ms, p90: 5400 ms, p99: 9600 ms), latency samples: 3697380
two traffics test : committed: 99.91829960453138 txn/s, latency: 1971.3863636363637 ms, (p50: 2000 ms, p90: 2200 ms, p99: 2600 ms), latency samples: 1760
Latency breakdown for phase 0: ["QsBatchToPos: max: 0.227, avg: 0.207", "QsPosToProposal: max: 0.220, avg: 0.201", "ConsensusProposalToOrdered: max: 0.426, avg: 0.375", "ConsensusOrderedToCommit: max: 0.364, avg: 0.355", "ConsensusProposalToCommit: max: 0.737, avg: 0.729"]
Max round gap was 1 [limit 4] at version 1801066. Max no progress secs was 4.993101 [limit 15] at version 1801066.
Test Ok

Copy link
Contributor

github-actions bot commented Jun 5, 2024

✅ Forge suite framework_upgrade success on a68e71c05caebf01504d4499110f3fba213fb53d ==> 2ad040cc7aae957f82a256302618fa0215e048b2

Compatibility test results for a68e71c05caebf01504d4499110f3fba213fb53d ==> 2ad040cc7aae957f82a256302618fa0215e048b2 (PR)
Upgrade the nodes to version: 2ad040cc7aae957f82a256302618fa0215e048b2
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1041.7856480105818 txn/s, submitted: 1043.7687425182162 txn/s, failed submission: 1.983094507634356 txn/s, expired: 1.983094507634356 txn/s, latency: 2917.6897948392557 ms, (p50: 2100 ms, p90: 5700 ms, p99: 9500 ms), latency samples: 94560
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1131.5081850988993 txn/s, submitted: 1133.3035174036972 txn/s, failed submission: 1.7953323047979362 txn/s, expired: 1.7953323047979362 txn/s, latency: 2644.623353827846 ms, (p50: 2100 ms, p90: 4700 ms, p99: 7900 ms), latency samples: 100840
5. check swarm health
Compatibility test for a68e71c05caebf01504d4499110f3fba213fb53d ==> 2ad040cc7aae957f82a256302618fa0215e048b2 passed
Upgrade the remaining nodes to version: 2ad040cc7aae957f82a256302618fa0215e048b2
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1252.8694939249413 txn/s, submitted: 1254.6859049346187 txn/s, failed submission: 1.8164110096773343 txn/s, expired: 1.8164110096773343 txn/s, latency: 2644.8420985864445 ms, (p50: 2100 ms, p90: 4500 ms, p99: 7800 ms), latency samples: 110360
Test Ok

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants