Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[compiler v2] Fix equalities, FreezeRef conversion #12162

Merged
merged 8 commits into from
Feb 27, 2024
Merged

[compiler v2] Fix equalities, FreezeRef conversion #12162

merged 8 commits into from
Feb 27, 2024

Conversation

wrwg
Copy link
Contributor

@wrwg wrwg commented Feb 22, 2024

This enables the comparison of references with mixed mutability (&x == &mut y), and introduces generation of the FreezeRef operation when an argument is widned from mutable to immutable references. The later applies for any kind of function call, not only equalities. Without freeze, certain scenarios produce borrow errors in reference safety and/or the bytecode verifier.

Note that this PR is intended to work together with the new reference safety analysis, so some tests do not yet work without it.

Fixes #12151
Fixes #11738
Fixes #11434

Copy link

trunk-io bot commented Feb 22, 2024

⏱️ 28h 39m total CI duration on this PR
Job Cumulative Duration Recent Runs
rust-unit-tests 4h 56m 🟥🟥🟩 (+5 more)
rust-unit-coverage 4h 29m 🟩
rust-smoke-coverage 3h 1m 🟩
windows-build 3h 1m 🟩🟩🟩🟩🟩 (+5 more)
rust-smoke-tests 2h 13m 🟥🟩🟩🟩
rust-move-tests 2h 8m 🟥🟩🟩🟩 (+3 more)
rust-move-unit-coverage 2h 5m 🟩🟩🟩🟩 (+3 more)
execution-performance / single-node-performance 1h 12m 🟩🟩🟩🟩
rust-images / rust-all 56m 🟩🟩🟩
run-tests-main-branch 52m 🟥🟥🟥🟥🟥 (+3 more)
rust-lints 51m 🟩🟩🟩🟩 (+3 more)
forge-e2e-test / forge 40m 🟩🟩🟩
forge-compat-test / forge 40m 🟩🟩🟩
check 33m 🟩🟩🟩🟩🟩 (+4 more)
cli-e2e-tests / run-cli-tests 19m 🟩🟩🟩
check-dynamic-deps 17m 🟩🟩🟩🟩🟩 (+4 more)
general-lints 15m 🟩🟩🟩🟩 (+3 more)
semgrep/ci 3m 🟩🟩🟩🟩🟩 (+4 more)
node-api-compatibility-tests / node-api-compatibility-tests 3m 🟩🟩🟩
file_change_determinator 1m 🟩🟩🟩🟩🟩 (+3 more)
file_change_determinator 1m 🟩🟩🟩🟩🟩 (+4 more)
execution-performance / file_change_determinator 41s 🟩🟩🟩🟩
file_change_determinator 39s 🟩🟩🟩🟩
permission-check 34s 🟩🟩🟩🟩🟩 (+4 more)
permission-check 25s 🟩🟩🟩🟩🟩 (+4 more)
permission-check 23s 🟩🟩🟩🟩🟩 (+4 more)
permission-check 21s 🟩🟩🟩🟩🟩 (+4 more)
permission-check 11s 🟩🟩🟩🟩
upload-to-codecov 11s 🟩
determine-docker-build-metadata 10s 🟩🟩🟩🟩

🚨 2 jobs on the last run were significantly faster/slower than expected

Job Duration vs 7d avg Delta
run-tests-main-branch 6m 4m +65%
rust-move-unit-coverage 16m 30m -48%

settingsfeedbackdocs ⋅ learn more about trunk.io

Copy link

codecov bot commented Feb 22, 2024

Codecov Report

Attention: Patch coverage is 94.63087% with 8 lines in your changes are missing coverage. Please review.

Project coverage is 64.1%. Comparing base (11e3752) to head (8c6cf4e).
Report is 4 commits behind head on main.

Files Patch % Lines
...d_party/move/move-model/src/builder/exp_builder.rs 88.8% 8 Missing ⚠️
Additional details and impacted files
@@           Coverage Diff           @@
##             main   #12162   +/-   ##
=======================================
  Coverage    64.1%    64.1%           
=======================================
  Files         792      792           
  Lines      175986   176059   +73     
=======================================
+ Hits       112916   112994   +78     
+ Misses      63070    63065    -5     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link
Contributor

@vineethk vineethk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, assuming the commented out code is taken care of.

@@ -2672,6 +2672,7 @@ impl<'env, 'translator, 'module_translator> ExpTranslator<'env, 'translator, 'mo
}
let mut success = true;
for (i, arg_ty) in arg_types.iter().enumerate() {
/*
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Delete commented out code?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

Copy link
Contributor

@rahxephon89 rahxephon89 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks!

Copy link
Contributor

@brmataptos brmataptos left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Solves a bunch of issues, but we seem to have lost a few errors.

error: cannot mutably borrow local `x` since other references exists
┌─ tests/reference-safety/eq_ref.move:16:19
16 │ &mut x == &mut x; // error expected, will be fixed with new reference safety
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we have an issue to track this problem?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually no error expected here since both args are frozen and then it is allowed.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

Copy link
Contributor

@brmataptos brmataptos left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. See one potential loop optimization.

fn check_borrow_safety(&mut self, temps: &BTreeSet<TempIndex>) {
fn check_borrow_safety(&mut self, temps_vec: &[TempIndex]) {
// First check direct duplicates
for (i, temp) in temps_vec.iter().enumerate() {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You could probably do something like:

let temps = BTreeSet<_>::new();
for temp in temps_vec.iter().rev() {
    if temps.contains(temp) {
        self.exclusive_access_direct_dup_error(temp.clone())
   } else {
        temps.insert(temp.clone());
   }
}

and reduce this from O(N^2) to O(N log N) or so.

Or we can save that for later optimization if needed.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry I saw only after merge.

I'm certain there is a non-quadratic version, but I can't quite see what you mean, this seems to generate an error for every iteration?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, only if we've already seen temp does it yield an error. And we're processing iter().rev() so it's going backwards over the vec just as before. Though actually, now that I think about it, that might not matter unless we are showing the location.

wrwg added 4 commits February 26, 2024 17:53
This refactors instrumentation of copy and drop instructions, and checking of abilities in general, closing #11223, which lead us to generate unnecessary copies. Copy instrumentation is now moved after reference safety as borrow information is needed to compute it. The existing ability checking and explicit drop processors have been marged into a new `ability_processor` which deals with instrumenting copies and drops as well as checking ability conformance.
wrwg added 4 commits February 26, 2024 17:53
This enables the comparison of references with mixed mutability (`&x == &mut y`), and introduces generation of the `FreezeRef` operation when an argument is widned from mutable to immutable references. The later applies for any kind of function call, not only equalities. Without freeze, certain scenarios produce borrow errors in reference safety and/or the bytecode verifier.

Note that this PR is intended to work together with the new reference safety analysis, so some tests do not yet work without it.

Fixes #12151
Fixes #11738
Fixes #11434

This comment has been minimized.

This comment has been minimized.

Copy link
Contributor

✅ Forge suite compat success on aptos-node-v1.9.5 ==> 8c6cf4e0875c53a306e91023a53e74023199c98c

Compatibility test results for aptos-node-v1.9.5 ==> 8c6cf4e0875c53a306e91023a53e74023199c98c (PR)
1. Check liveness of validators at old version: aptos-node-v1.9.5
compatibility::simple-validator-upgrade::liveness-check : committed: 6774 txn/s, latency: 4867 ms, (p50: 4800 ms, p90: 8100 ms, p99: 9300 ms), latency samples: 237100
2. Upgrading first Validator to new version: 8c6cf4e0875c53a306e91023a53e74023199c98c
compatibility::simple-validator-upgrade::single-validator-upgrade : committed: 717 txn/s, latency: 34352 ms, (p50: 36700 ms, p90: 55100 ms, p99: 56800 ms), latency samples: 56700
3. Upgrading rest of first batch to new version: 8c6cf4e0875c53a306e91023a53e74023199c98c
compatibility::simple-validator-upgrade::half-validator-upgrade : committed: 14 txn/s, submitted: 384 txn/s, expired: 370 txn/s, latency: 59226 ms, (p50: 66600 ms, p90: 74800 ms, p99: 74800 ms), latency samples: 1517
4. upgrading second batch to new version: 8c6cf4e0875c53a306e91023a53e74023199c98c
compatibility::simple-validator-upgrade::rest-validator-upgrade : committed: 2393 txn/s, latency: 12631 ms, (p50: 13800 ms, p90: 15300 ms, p99: 16300 ms), latency samples: 105300
5. check swarm health
Compatibility test for aptos-node-v1.9.5 ==> 8c6cf4e0875c53a306e91023a53e74023199c98c passed
Test Ok

Copy link
Contributor

✅ Forge suite realistic_env_max_load success on 8c6cf4e0875c53a306e91023a53e74023199c98c

two traffics test: inner traffic : committed: 8327 txn/s, latency: 4709 ms, (p50: 4500 ms, p90: 5600 ms, p99: 9900 ms), latency samples: 3597300
two traffics test : committed: 100 txn/s, latency: 1826 ms, (p50: 1800 ms, p90: 2000 ms, p99: 3900 ms), latency samples: 1740
Latency breakdown for phase 0: ["QsBatchToPos: max: 0.231, avg: 0.205", "QsPosToProposal: max: 0.247, avg: 0.231", "ConsensusProposalToOrdered: max: 0.464, avg: 0.421", "ConsensusOrderedToCommit: max: 0.289, avg: 0.278", "ConsensusProposalToCommit: max: 0.711, avg: 0.699"]
Max round gap was 1 [limit 4] at version 1566974. Max no progress secs was 4.327317 [limit 15] at version 1566974.
Test Ok

@wrwg wrwg merged commit 997d925 into main Feb 27, 2024
46 checks passed
@wrwg wrwg deleted the wg-freeze branch February 27, 2024 02:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
4 participants