Astifier fixes #15026

wrwg · 2024-10-21T05:52:20Z

Description

This PR implements some major improvements to the astifier:

Finally (I think) cracked the way how to do correct topological sorting and managing nested blocks. More complicated examples now produce more reasonable results.
Rewrote AST transformations for if-then-else and loops.
Fixed issues with the fixpoint analysis on the AST
Did some minor fixes in the sourcifier

The generated source is still not correct in some cases, but much better then before.

How Has This Been Tested?

Baseline tests

Type of Change

Which Components or Systems Does This Change Impact?

Checklist

I have read and followed the CONTRIBUTING doc
I have performed a self-review of my own code
I have commented my code, particularly in hard-to-understand areas
I identified and added all stakeholders and component owners affected by this change as reviewers
I tested both happy and unhappy path of the functionality
I have made corresponding changes to the documentation

trunk-io · 2024-10-21T05:52:24Z

⏱️ 27m total CI duration on this PR

Job	Cumulative Duration	Recent Runs
rust-move-tests	9m	🟩
rust-move-tests	9m	🟩
rust-cargo-deny	4m	🟩 🟩
check-dynamic-deps	3m	🟩 🟩
general-lints	54s	🟩 🟩
semgrep/ci	40s	🟩 🟩
file_change_determinator	20s	🟩 🟩
permission-check	5s	🟩 🟩
permission-check	4s	🟩 🟩

_{settings ⋅ feedback ⋅ docs ⋅ learn more about trunk.io}

wrwg · 2024-10-21T05:52:37Z

Astifier fixes #15026 👈
[move-decompiler] Model import from binary and basic decompiler setup #14964
main

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @wrwg and the rest of your teammates on Graphite

rahxephon89

Still need more time to understand the astifier part but the generated code has been improved a lot! Thanks

third_party/move/tools/move-decompiler/src/lib.rs

third_party/move/move-model/src/sourcifier.rs

vineethk

Some minor comments, but otherwise LGTM.

third_party/move/move-model/bytecode/src/astifier.rs

third_party/move/move-model/src/sourcifier.rs

third_party/move/move-compiler-v2/tests/checking/typing/loop_result_type.exp

brmataptos

A few nits, one duplicated operation.

third_party/move/tools/move-decompiler/tests/vector.exp

third_party/move/move-model/bytecode/src/astifier.rs

wrwg

Thanks for the reviews!

third_party/move/tools/move-decompiler/src/lib.rs

third_party/move/move-model/src/sourcifier.rs

third_party/move/move-model/bytecode/src/astifier.rs

third_party/move/tools/move-decompiler/tests/vector.exp

third_party/move/move-model/bytecode/src/astifier.rs

This PR implements some major improvements to the astifier: - Finally (I think) cracked the way how to do correct topological sorting and managing nested blocks. More complicated examples now produce more reasonable results. - Rewrote AST transformations for if-then-else and loops. - Fixed issues with the fixpoint analysis on the AST - Did some minor fixes in the sourcifier The generated source is still not correct in some cases, but much better then before.

github-actions · 2024-11-03T19:22:13Z

✅ Forge suite `realistic_env_max_load` success on `c6bb3ad84de216817a5e5cc1874db38932ec8efe`

two traffics test: inner traffic : committed: 14176.92 txn/s, latency: 2801.87 ms, (p50: 2700 ms, p70: 2700, p90: 3000 ms, p99: 3600 ms), latency samples: 5390480
two traffics test : committed: 99.95 txn/s, latency: 1482.41 ms, (p50: 1400 ms, p70: 1400, p90: 1500 ms, p99: 7600 ms), latency samples: 1760
Latency breakdown for phase 0: ["MempoolToBlockCreation: max: 2.123, avg: 1.617", "ConsensusProposalToOrdered: max: 0.327, avg: 0.299", "ConsensusOrderedToCommit: max: 0.363, avg: 0.351", "ConsensusProposalToCommit: max: 0.659, avg: 0.651"]
Max non-epoch-change gap was: 0 rounds at version 0 (avg 0.00) [limit 4], 1.04s no progress at version 1886596 (avg 0.20s) [limit 15].
Max epoch-change gap was: 0 rounds at version 0 (avg 0.00) [limit 4], 8.47s no progress at version 1886594 (avg 8.23s) [limit 15].
Test Ok

github-actions · 2024-11-03T19:23:26Z

✅ Forge suite `framework_upgrade` success on `1086a5e00d773704731ab84fb4ed3538613b2250` ==> `c6bb3ad84de216817a5e5cc1874db38932ec8efe`

Compatibility test results for 1086a5e00d773704731ab84fb4ed3538613b2250 ==> c6bb3ad84de216817a5e5cc1874db38932ec8efe (PR)
Upgrade the nodes to version: c6bb3ad84de216817a5e5cc1874db38932ec8efe
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1348.64 txn/s, submitted: 1352.21 txn/s, failed submission: 3.58 txn/s, expired: 3.58 txn/s, latency: 2236.66 ms, (p50: 2100 ms, p70: 2400, p90: 3100 ms, p99: 4800 ms), latency samples: 120680
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1416.05 txn/s, submitted: 1419.36 txn/s, failed submission: 3.31 txn/s, expired: 3.31 txn/s, latency: 2105.00 ms, (p50: 1800 ms, p70: 2100, p90: 3300 ms, p99: 4400 ms), latency samples: 128180
5. check swarm health
Compatibility test for 1086a5e00d773704731ab84fb4ed3538613b2250 ==> c6bb3ad84de216817a5e5cc1874db38932ec8efe passed
Upgrade the remaining nodes to version: c6bb3ad84de216817a5e5cc1874db38932ec8efe
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1340.30 txn/s, submitted: 1343.40 txn/s, failed submission: 3.10 txn/s, expired: 3.10 txn/s, latency: 2427.50 ms, (p50: 2100 ms, p70: 2600, p90: 3600 ms, p99: 5500 ms), latency samples: 120900
Test Ok

github-actions · 2024-11-03T19:25:38Z

✅ Forge suite `compat` success on `1086a5e00d773704731ab84fb4ed3538613b2250` ==> `c6bb3ad84de216817a5e5cc1874db38932ec8efe`

Compatibility test results for 1086a5e00d773704731ab84fb4ed3538613b2250 ==> c6bb3ad84de216817a5e5cc1874db38932ec8efe (PR)
1. Check liveness of validators at old version: 1086a5e00d773704731ab84fb4ed3538613b2250
compatibility::simple-validator-upgrade::liveness-check : committed: 14073.32 txn/s, latency: 2412.73 ms, (p50: 1700 ms, p70: 1900, p90: 4200 ms, p99: 13900 ms), latency samples: 501700
2. Upgrading first Validator to new version: c6bb3ad84de216817a5e5cc1874db38932ec8efe
compatibility::simple-validator-upgrade::single-validator-upgrading : committed: 6082.58 txn/s, latency: 4671.86 ms, (p50: 5200 ms, p70: 5500, p90: 5800 ms, p99: 5900 ms), latency samples: 113060
compatibility::simple-validator-upgrade::single-validator-upgrade : committed: 6553.97 txn/s, latency: 4926.20 ms, (p50: 5300 ms, p70: 5500, p90: 6500 ms, p99: 6800 ms), latency samples: 216980
3. Upgrading rest of first batch to new version: c6bb3ad84de216817a5e5cc1874db38932ec8efe
compatibility::simple-validator-upgrade::half-validator-upgrading : committed: 6045.55 txn/s, latency: 4676.63 ms, (p50: 5200 ms, p70: 5400, p90: 6000 ms, p99: 6200 ms), latency samples: 113820
compatibility::simple-validator-upgrade::half-validator-upgrade : committed: 6461.68 txn/s, latency: 5020.05 ms, (p50: 5400 ms, p70: 5500, p90: 6900 ms, p99: 7300 ms), latency samples: 220880
4. upgrading second batch to new version: c6bb3ad84de216817a5e5cc1874db38932ec8efe
compatibility::simple-validator-upgrade::rest-validator-upgrading : committed: 8403.26 txn/s, latency: 3305.84 ms, (p50: 3100 ms, p70: 3300, p90: 5500 ms, p99: 5800 ms), latency samples: 150700
compatibility::simple-validator-upgrade::rest-validator-upgrade : committed: 7421.23 txn/s, latency: 4234.65 ms, (p50: 3200 ms, p70: 6100, p90: 7000 ms, p99: 9100 ms), latency samples: 244700
5. check swarm health
Compatibility test for 1086a5e00d773704731ab84fb4ed3538613b2250 ==> c6bb3ad84de216817a5e5cc1874db38932ec8efe passed
Test Ok

wrwg mentioned this pull request Oct 21, 2024

[move-decompiler] Model import from binary and basic decompiler setup #14964

Merged

16 tasks

wrwg force-pushed the wrwg/decompiler branch from e95cddc to 944da2d Compare October 27, 2024 03:53

wrwg force-pushed the wrwg/asti2 branch from 68e9a5e to e0a66b6 Compare October 27, 2024 03:54

wrwg force-pushed the wrwg/decompiler branch from 7c93f0f to 0f82561 Compare October 29, 2024 03:21

Base automatically changed from wrwg/decompiler to main October 29, 2024 03:53

wrwg force-pushed the wrwg/asti2 branch from e0a66b6 to 67cdc1a Compare November 1, 2024 00:00

wrwg requested review from brmataptos, fEst1ck, rahxephon89 and vineethk November 1, 2024 00:02

wrwg marked this pull request as ready for review November 1, 2024 00:03

wrwg force-pushed the wrwg/asti2 branch from 67cdc1a to 7d4dbf7 Compare November 1, 2024 05:48

rahxephon89 approved these changes Nov 1, 2024

View reviewed changes

third_party/move/tools/move-decompiler/src/lib.rs Show resolved Hide resolved

third_party/move/move-model/src/sourcifier.rs Outdated Show resolved Hide resolved

vineethk approved these changes Nov 1, 2024

View reviewed changes

brmataptos reviewed Nov 1, 2024

View reviewed changes

wrwg force-pushed the wrwg/asti2 branch from 1b96a96 to 1805efa Compare November 3, 2024 17:52

wrwg commented Nov 3, 2024

View reviewed changes

wrwg force-pushed the wrwg/asti2 branch from 1805efa to c6bb3ad Compare November 3, 2024 18:52

wrwg requested a review from brmataptos November 3, 2024 18:52

wrwg enabled auto-merge (squash) November 3, 2024 18:52

This comment has been minimized.

Sign in to view

wrwg merged commit 9baf39b into main Nov 3, 2024
87 of 93 checks passed

wrwg deleted the wrwg/asti2 branch November 3, 2024 19:25

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Astifier fixes #15026

Astifier fixes #15026

wrwg commented Oct 21, 2024 •

edited

Loading

trunk-io bot commented Oct 21, 2024 •

edited

Loading

wrwg commented Oct 21, 2024 •

edited

Loading

rahxephon89 left a comment

vineethk left a comment

brmataptos left a comment

wrwg left a comment

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

github-actions bot commented Nov 3, 2024

github-actions bot commented Nov 3, 2024

github-actions bot commented Nov 3, 2024

Astifier fixes #15026

Astifier fixes #15026

Conversation

wrwg commented Oct 21, 2024 • edited Loading

Description

How Has This Been Tested?

Type of Change

Which Components or Systems Does This Change Impact?

Checklist

trunk-io bot commented Oct 21, 2024 • edited Loading

wrwg commented Oct 21, 2024 • edited Loading

rahxephon89 left a comment

Choose a reason for hiding this comment

vineethk left a comment

Choose a reason for hiding this comment

brmataptos left a comment

Choose a reason for hiding this comment

wrwg left a comment

Choose a reason for hiding this comment

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

github-actions bot commented Nov 3, 2024

✅ Forge suite realistic_env_max_load success on c6bb3ad84de216817a5e5cc1874db38932ec8efe

github-actions bot commented Nov 3, 2024

✅ Forge suite framework_upgrade success on 1086a5e00d773704731ab84fb4ed3538613b2250 ==> c6bb3ad84de216817a5e5cc1874db38932ec8efe

github-actions bot commented Nov 3, 2024

✅ Forge suite compat success on 1086a5e00d773704731ab84fb4ed3538613b2250 ==> c6bb3ad84de216817a5e5cc1874db38932ec8efe

wrwg commented Oct 21, 2024 •

edited

Loading

trunk-io bot commented Oct 21, 2024 •

edited

Loading

wrwg commented Oct 21, 2024 •

edited

Loading

✅ Forge suite `realistic_env_max_load` success on `c6bb3ad84de216817a5e5cc1874db38932ec8efe`

✅ Forge suite `framework_upgrade` success on `1086a5e00d773704731ab84fb4ed3538613b2250` ==> `c6bb3ad84de216817a5e5cc1874db38932ec8efe`

✅ Forge suite `compat` success on `1086a5e00d773704731ab84fb4ed3538613b2250` ==> `c6bb3ad84de216817a5e5cc1874db38932ec8efe`