Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[move-compiler-v2] Add all remaining V1 tests to V2, except evm/async #14578

Merged
merged 8 commits into from
Sep 19, 2024

Conversation

brmataptos
Copy link
Contributor

@brmataptos brmataptos commented Sep 10, 2024

Description

Fixes #13731.
Fixes #9003.

Add the remaining tests from V1 to V2, except for flavors (evm and async). Most are added in subdirs of a new top-level dir more-v1/, but a few are in checking-lang-v1/, deprecated/, and unused-assignment/v1-locals/.

To maximize comparability, I've also moved around some tests and/or split/duplicated them to check whether V2 catches the same errors as V1, and added bytecode generation to one set of tests, that previously stopped just before that, since it is needed by V2 to issue some errors.

See below for a link to a description of remaining test differences.

How Has This Been Tested?

Running the tests.

Key Areas to Review

  1. Consider .rs files:

    • testsuite.rs:127: some errors in acquires tests are caught in the BytecodeGen phase, so that is added.
    • testsuite.rs:200: most new tests are added under /more-v1, which runs most passes
    • move-compiler-v2/tools/testdiff/src/main.rs: We add files from V1's "expansion" and "dependencies" test dirs.
  2. Consider changes to v1.matched and v1.unmatched:

    • many files are removed from .unmatched and moved to .matched
    • some test files are moved, e.g.: from move-compiler-v2/tests/checking/ to move-compiler-v2/tests/ability-check/typing, to enable checks which are needed to find all errors/warnings in those tests.
    • some test files are duplicated, e.g., to /eq_invalid.exp we add /eq_invalid2.exp. These generally comment out parts of the original tests to avoid earlier errors hiding later errors in V2.
  3. Look at the changes to V1 test files files move-compiler/.../*.move, verify that some files were moved around and/or fixed, but nothing was removed:

    • unused_ref.move removes an unrelated error (too many params to borrow)
    • Various test files *2.move such as use_nested_self_as_invalid2.move duplicate use_nested_self_as_invalid.move but with some errrors commented out or functions split to avoid shadowing in V2.
    • acquires_error_msg_inline.move is an added test for the inlining case
    • resources_invalid.move was expanded into 3 tests to explore the original situation as well as 2 others.
    • address_too_long*.move tests were corrected to actually have addresses too long, and new tests for the boundary cases were added.
    • ability_constraint_*_invalid.move tests split a function in two to avoid duplicate error removal in V2.
    • no_receiver_calls.move has some edits to remove unrelated errors checked elsewhere.
  4. Look at the changes to V2 test files move-compiler-v2/.../*.move, verify that (1) no files are removed, and (2) nothing significant is deleted, but (3) some new tests are added.

  5. Look at the following diffs, which compare the differing .move files corresponding to comparable .exp files in v1.matched. Note that 51 files differ a little bit, typically due to changes moving them to V2.

  6. If you're bored, look at the output diffs below to see what is different in outputs. Or view the spreadsheet for a summary.

Analysis enabled by this PR

This PR's landing should be independent of the following, but is enabled by the PR so I mention it here:

Look at the differences in test outputs:

A summary of significant changes in the test outputs is provided here:
https://docs.google.com/spreadsheets/d/12lWYfk5wIUbK4i9jXBirwv1OU7UYmdYjmwySGGThEwI/edit?usp=sharing

Type of Change

  • New feature
  • Bug fix
  • Breaking change
  • Performance improvement
  • Refactoring
  • Dependency update
  • Documentation update
  • Tests

Which Components or Systems Does This Change Impact?

  • Validator Node
  • Full Node (API, Indexer, etc.)
  • Move/Aptos Virtual Machine
  • Aptos Framework
  • Aptos CLI/SDK
  • Developer Infrastructure
  • Other (specify)

Checklist

  • I have read and followed the CONTRIBUTING doc
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I identified and added all stakeholders and component owners affected by this change as reviewers
  • I tested both happy and unhappy path of the functionality
  • I have made corresponding changes to the documentation

Copy link

trunk-io bot commented Sep 10, 2024

⏱️ 2h 28m total CI duration on this PR
Job Cumulative Duration Recent Runs
rust-move-unit-coverage 18m 🟩
rust-move-unit-coverage 16m 🟩
rust-move-unit-coverage 16m 🟩
rust-move-unit-coverage 13m 🟩
rust-move-unit-coverage 11m 🟩
rust-move-tests 10m 🟩
rust-move-tests 10m 🟩
rust-cargo-deny 9m 🟩🟩🟩🟩🟩
rust-move-tests 9m 🟩
rust-move-tests 9m 🟩
rust-move-tests 9m 🟩
general-lints 9m 🟩🟩🟩🟩🟩
check-dynamic-deps 5m 🟩🟩🟩🟩🟩
semgrep/ci 2m 🟩🟩🟩🟩🟩
file_change_determinator 58s 🟩🟩🟩🟩🟩
file_change_determinator 55s 🟩🟩🟩🟩🟩
permission-check 17s 🟩🟩🟩🟩🟩
permission-check 16s 🟩🟩🟩🟩🟩
permission-check 14s 🟩🟩🟩🟩🟩
permission-check 10s 🟩🟩🟩🟩🟩

settingsfeedbackdocs ⋅ learn more about trunk.io

@brmataptos brmataptos changed the title Add all remaining V1 tests to V2, except flavors [move-compiler-v2] Add all remaining V1 tests to V2, except flavors Sep 10, 2024
@brmataptos brmataptos changed the title [move-compiler-v2] Add all remaining V1 tests to V2, except flavors [move-compiler-v2] Add all remaining V1 tests to V2, except evm/async Sep 10, 2024
Copy link

codecov bot commented Sep 10, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 59.8%. Comparing base (eab8d27) to head (40ac663).
Report is 1 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main   #14578   +/-   ##
=======================================
  Coverage    59.8%    59.8%           
=======================================
  Files         851      851           
  Lines      207246   207246           
=======================================
+ Hits       124004   124038   +34     
+ Misses      83242    83208   -34     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link
Contributor

@vineethk vineethk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left one comment. Also, a bunch of baseline files need updating as the tests are failing on CI. Approving assuming these are fixed.

This must have been an incredibly arduous task. Thanks for doing this!

fun receiver(self: &T, _x: u64) { abort 1 }

fun call_receiver(t: T) {
t.receiver(1);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should these be run with language version 1?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I put a copy in move-compiler-v2/tests/checking-lang-v1/. That one looks more like the V1 test.

@brmataptos brmataptos enabled auto-merge (squash) September 19, 2024 16:29

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

Copy link
Contributor

✅ Forge suite realistic_env_max_load success on 40ac6630e7266e81703831a71d0c474ea52e3b5c

two traffics test: inner traffic : committed: 14198.19 txn/s, latency: 2798.04 ms, (p50: 2700 ms, p70: 2700, p90: 3000 ms, p99: 3300 ms), latency samples: 5398420
two traffics test : committed: 100.05 txn/s, latency: 1729.08 ms, (p50: 1500 ms, p70: 1600, p90: 1700 ms, p99: 9800 ms), latency samples: 1780
Latency breakdown for phase 0: ["QsBatchToPos: max: 0.244, avg: 0.229", "QsPosToProposal: max: 1.103, avg: 1.070", "ConsensusProposalToOrdered: max: 0.312, avg: 0.296", "ConsensusOrderedToCommit: max: 0.445, avg: 0.432", "ConsensusProposalToCommit: max: 0.745, avg: 0.728"]
Max non-epoch-change gap was: 0 rounds at version 0 (avg 0.00) [limit 4], 0.92s no progress at version 2677363 (avg 0.20s) [limit 15].
Max epoch-change gap was: 0 rounds at version 0 (avg 0.00) [limit 4], 8.22s no progress at version 2677361 (avg 8.22s) [limit 15].
Test Ok

Copy link
Contributor

✅ Forge suite framework_upgrade success on 25a081116546670e62ca927ba90478de78557056 ==> 40ac6630e7266e81703831a71d0c474ea52e3b5c

Compatibility test results for 25a081116546670e62ca927ba90478de78557056 ==> 40ac6630e7266e81703831a71d0c474ea52e3b5c (PR)
Upgrade the nodes to version: 40ac6630e7266e81703831a71d0c474ea52e3b5c
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1341.41 txn/s, submitted: 1343.64 txn/s, failed submission: 2.23 txn/s, expired: 2.23 txn/s, latency: 2377.57 ms, (p50: 2100 ms, p70: 2400, p90: 3900 ms, p99: 4800 ms), latency samples: 120440
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1244.94 txn/s, submitted: 1247.20 txn/s, failed submission: 2.26 txn/s, expired: 2.26 txn/s, latency: 2403.71 ms, (p50: 2100 ms, p70: 2400, p90: 3800 ms, p99: 5100 ms), latency samples: 109940
5. check swarm health
Compatibility test for 25a081116546670e62ca927ba90478de78557056 ==> 40ac6630e7266e81703831a71d0c474ea52e3b5c passed
Upgrade the remaining nodes to version: 40ac6630e7266e81703831a71d0c474ea52e3b5c
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1509.15 txn/s, submitted: 1510.70 txn/s, failed submission: 1.55 txn/s, expired: 1.55 txn/s, latency: 2330.78 ms, (p50: 2100 ms, p70: 2400, p90: 3600 ms, p99: 4700 ms), latency samples: 116840
Test Ok

Copy link
Contributor

✅ Forge suite compat success on 25a081116546670e62ca927ba90478de78557056 ==> 40ac6630e7266e81703831a71d0c474ea52e3b5c

Compatibility test results for 25a081116546670e62ca927ba90478de78557056 ==> 40ac6630e7266e81703831a71d0c474ea52e3b5c (PR)
1. Check liveness of validators at old version: 25a081116546670e62ca927ba90478de78557056
compatibility::simple-validator-upgrade::liveness-check : committed: 14048.56 txn/s, latency: 2260.46 ms, (p50: 2100 ms, p70: 2200, p90: 2500 ms, p99: 7700 ms), latency samples: 532120
2. Upgrading first Validator to new version: 40ac6630e7266e81703831a71d0c474ea52e3b5c
compatibility::simple-validator-upgrade::single-validator-upgrading : committed: 7476.98 txn/s, latency: 3784.62 ms, (p50: 4200 ms, p70: 4500, p90: 4600 ms, p99: 4700 ms), latency samples: 136460
compatibility::simple-validator-upgrade::single-validator-upgrade : committed: 6882.17 txn/s, latency: 4624.92 ms, (p50: 4600 ms, p70: 4600, p90: 6900 ms, p99: 7100 ms), latency samples: 231600
3. Upgrading rest of first batch to new version: 40ac6630e7266e81703831a71d0c474ea52e3b5c
compatibility::simple-validator-upgrade::half-validator-upgrading : committed: 7711.00 txn/s, latency: 3717.17 ms, (p50: 4200 ms, p70: 4300, p90: 4400 ms, p99: 4600 ms), latency samples: 142580
compatibility::simple-validator-upgrade::half-validator-upgrade : committed: 6727.66 txn/s, latency: 4396.84 ms, (p50: 4500 ms, p70: 4600, p90: 4700 ms, p99: 6500 ms), latency samples: 256440
4. upgrading second batch to new version: 40ac6630e7266e81703831a71d0c474ea52e3b5c
compatibility::simple-validator-upgrade::rest-validator-upgrading : committed: 11310.04 txn/s, latency: 2377.44 ms, (p50: 2400 ms, p70: 2500, p90: 3500 ms, p99: 3700 ms), latency samples: 199060
compatibility::simple-validator-upgrade::rest-validator-upgrade : committed: 10920.00 txn/s, latency: 2789.03 ms, (p50: 2500 ms, p70: 2700, p90: 5100 ms, p99: 6700 ms), latency samples: 357240
5. check swarm health
Compatibility test for 25a081116546670e62ca927ba90478de78557056 ==> 40ac6630e7266e81703831a71d0c474ea52e3b5c passed
Test Ok

@brmataptos brmataptos merged commit 2b208e5 into main Sep 19, 2024
94 of 95 checks passed
@brmataptos brmataptos deleted the brm-issue-13731 branch September 19, 2024 17:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
3 participants