Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[aggregator] Fix simulation & view for delayed fields #12063

Merged
merged 4 commits into from
Feb 16, 2024

Conversation

georgemitenkov
Copy link
Contributor

Description

Previously, we have been tagging aggregator fields if the feature flag is enabled. This is not correct because it can be enabled but we run under non-capable context (i.e., outside of Block-STM). This PR fixes that by explicitly providing a boolean whether type layouts need to be tagged inside a VM.

Test Plan

  • Test for view function
  • Test for simulation

Copy link

trunk-io bot commented Feb 16, 2024

⏱️ 22h 12m total CI duration on this PR
Job Cumulative Duration Recent Runs
rust-unit-coverage 4h 14m 🟩
rust-smoke-coverage 3h 28m 🟩
rust-unit-tests 2h 🟩🟩🟩🟩
rust-smoke-tests 1h 59m 🟩🟩🟩🟩
rust-move-unit-coverage 1h 40m 🟩🟩🟩
windows-build 1h 32m 🟩🟩🟩🟩🟩
execution-performance / single-node-performance 1h 19m 🟩🟩🟩🟩
forge-e2e-test / forge 56m 🟩🟩🟩🟩
rust-move-tests 53m 🟩🟩🟩
rust-images / rust-all 52m 🟥🟩🟩🟩🟩
forge-compat-test / forge 48m 🟥🟥🟥🟩
cli-e2e-tests / run-cli-tests 36m 🟥🟥🟥🟥
run-tests-main-branch 31m 🟥🟥🟥🟥🟥 (+1 more)
rust-lints 28m 🟩🟩🟩🟩
check 17m 🟩🟩🟩🟩
check-dynamic-deps 13m 🟩🟩🟩🟩🟩
general-lints 9m 🟩🟩🟩🟩
indexer-grpc-e2e-tests / test-indexer-grpc-docker-compose 7m 🟥🟩🟩🟩
node-api-compatibility-tests / node-api-compatibility-tests 3m 🟩🟩🟩🟩
semgrep/ci 2m 🟩🟩🟩🟩🟩
file_change_determinator 51s 🟩🟩🟩🟩🟩
file_change_determinator 45s 🟩🟩🟩🟩🟩
file_change_determinator 43s 🟩🟩🟩🟩
execution-performance / file_change_determinator 38s 🟩🟩🟩🟩
permission-check 15s 🟩🟩🟩🟩🟩
permission-check 13s 🟩🟩🟩🟩🟩
upload-to-codecov 13s 🟩
permission-check 13s 🟩🟩🟩🟩🟩
permission-check 12s 🟩🟩🟩🟩
determine-docker-build-metadata 12s 🟩🟩🟩🟩
permission-check 10s 🟩🟩🟩🟩🟩

🚨 3 jobs on the last run were significantly faster/slower than expected

Job Duration vs 7d avg Delta
cli-e2e-tests / run-cli-tests 9m 6m +41%
windows-build 25m 18m +36%
rust-images / rust-all 17m 12m +35%

settingsfeedbackdocs ⋅ learn more about trunk.io

@georgemitenkov georgemitenkov added the CICD:run-e2e-tests when this label is present github actions will run all land-blocking e2e tests from the PR label Feb 16, 2024
Copy link

codecov bot commented Feb 16, 2024

Codecov Report

Attention: 16 lines in your changes are missing coverage. Please review.

Comparison is base (ea133dc) 71.4% compared to head (2485b09) 71.4%.

Files Patch % Lines
aptos-move/e2e-tests/src/executor.rs 30.7% 9 Missing ⚠️
aptos-move/aptos-vm/src/aptos_vm.rs 81.8% 4 Missing ⚠️
aptos-move/aptos-vm/src/move_vm_ext/vm.rs 60.0% 2 Missing ⚠️
aptos-move/vm-genesis/src/lib.rs 50.0% 1 Missing ⚠️
Additional details and impacted files
@@           Coverage Diff           @@
##             main   #12063   +/-   ##
=======================================
  Coverage    71.4%    71.4%           
=======================================
  Files         802      802           
  Lines      184373   184411   +38     
=======================================
+ Hits       131737   131826   +89     
+ Misses      52636    52585   -51     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

Copy link
Contributor

@ziaptos ziaptos left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please address the comments before landing


fn new_impl(
resolver: &impl AptosMoveResolver,
is_delayed_field_optimization_capable: bool,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Too many constructors, we can have:
fn new(resolver: &..., maybe_override_delay_field_flag: Option) and cover all the cases.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point!

}

pub(crate) fn new_with_delayed_fields(resolver: &impl AptosMoveResolver) -> Self {
Self::new_impl(resolver, true)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we assert resolver.is_delayed_field_optimization_capable())

Copy link
Contributor Author

@georgemitenkov georgemitenkov Feb 16, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Used a single constructor instead

@@ -82,3 +86,57 @@ async fn test_simulate_transaction_with_insufficient_balance() {
let resp = simulate_aptos_transfer(&mut context, false, LARGE_TRANSFER_AMOUNT, 200).await;
assert!(!resp[0]["success"].as_bool().is_some_and(|v| v));
}

#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we know what is this worker_threads = 2 achieving?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This makes sure your runtime is configured as multi-threaded with 2 threads to run async

@@ -0,0 +1 @@
["10"]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

newline

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

golden tests compare the full file so new line breaks them

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

let vm = AptosVM::new(&resolver);
let vm = AptosVM::new(
&resolver,
/*override_is_delayed_field_optimization_capable=*/ Some(false),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we need to disable aggregators here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This runs outside of Block-STM context

This comment has been minimized.

This comment has been minimized.

Copy link
Contributor

✅ Forge suite compat success on testnet ==> 2485b09cbf42f03cdad5a6f1b7ff51dd9be52ff3

Compatibility test results for testnet ==> 2485b09cbf42f03cdad5a6f1b7ff51dd9be52ff3 (PR)
1. Check liveness of validators at old version: testnet
compatibility::simple-validator-upgrade::liveness-check : committed: 6571 txn/s, latency: 4913 ms, (p50: 4800 ms, p90: 5400 ms, p99: 8400 ms), latency samples: 249720
2. Upgrading first Validator to new version: 2485b09cbf42f03cdad5a6f1b7ff51dd9be52ff3
compatibility::simple-validator-upgrade::single-validator-upgrade : committed: 668 txn/s, latency: 35555 ms, (p50: 36900 ms, p90: 55900 ms, p99: 57700 ms), latency samples: 56160
3. Upgrading rest of first batch to new version: 2485b09cbf42f03cdad5a6f1b7ff51dd9be52ff3
compatibility::simple-validator-upgrade::half-validator-upgrade : committed: 395 txn/s, submitted: 635 txn/s, expired: 240 txn/s, latency: 48030 ms, (p50: 56200 ms, p90: 57600 ms, p99: 58400 ms), latency samples: 26085
4. upgrading second batch to new version: 2485b09cbf42f03cdad5a6f1b7ff51dd9be52ff3
compatibility::simple-validator-upgrade::rest-validator-upgrade : committed: 2597 txn/s, latency: 10681 ms, (p50: 12800 ms, p90: 15300 ms, p99: 20300 ms), latency samples: 129880
5. check swarm health
Compatibility test for testnet ==> 2485b09cbf42f03cdad5a6f1b7ff51dd9be52ff3 passed
Test Ok

Copy link
Contributor

✅ Forge suite realistic_env_max_load success on 2485b09cbf42f03cdad5a6f1b7ff51dd9be52ff3

two traffics test: inner traffic : committed: 7145 txn/s, latency: 5342 ms, (p50: 4900 ms, p90: 6600 ms, p99: 14100 ms), latency samples: 3086860
two traffics test : committed: 100 txn/s, latency: 2326 ms, (p50: 2100 ms, p90: 2700 ms, p99: 12800 ms), latency samples: 1820
Latency breakdown for phase 0: ["QsBatchToPos: max: 0.290, avg: 0.217", "QsPosToProposal: max: 0.279, avg: 0.179", "ConsensusProposalToOrdered: max: 0.598, avg: 0.560", "ConsensusOrderedToCommit: max: 0.491, avg: 0.465", "ConsensusProposalToCommit: max: 1.050, avg: 1.025"]
Max round gap was 2 [limit 4] at version 1345240. Max no progress secs was 10.083244 [limit 15] at version 1345240.
Test Ok

@georgemitenkov georgemitenkov merged commit a27c520 into main Feb 16, 2024
44 of 45 checks passed
@georgemitenkov georgemitenkov deleted the george/simulation-view-fix branch February 16, 2024 19:59
@@ -0,0 +1,26 @@
module addr::counter {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

license headers?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need them for test sources as well?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CICD:run-e2e-tests when this label is present github actions will run all land-blocking e2e tests from the PR
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants