Skip to content

Commit

Permalink
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
[compiler-v2] Ast generator from stackless bytecode, Source gen from Ast
Browse files Browse the repository at this point in the history
Converter from stackless bytecode into Model AST ('astifier'). (We already have one from binary format to stackless.) Also adds a converter from AST to source ('sourcifier').

This adds the major pieces for a working decompiler. However, the pieces are not fully connected to a tool yet, as more sidework needs to be done first. Specifically, this PR adds breaks/continues to labeled outer loops, which are not yet supported by the compiler.

There are some tests in a new test crate `ast-generator-tests`. These call the v2 compiler to compile up to file format, then decompile from there into stackless bytecode, into AST, and via sourcifier back to source. However, desired execution roundtrip comparison tests can only be implemented once the full toolchain is ready.

Another set of tests for the 'sourcifier' is via compiler-v2 baseline files. Where AST dump is requested in those tests, we dump now also the sourcified AST. This can live side-by-side for a while until we may decide to remove one of the outputs.
wrwg committed Oct 8, 2024
1 parent 67f7ee6 commit d470c70
Showing 414 changed files with 20,090 additions and 2,420 deletions.
20 changes: 20 additions & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 2 additions & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
@@ -221,6 +221,7 @@ members = [
"third_party/move/move-model",
"third_party/move/move-model/bytecode",
"third_party/move/move-model/bytecode-test-utils",
"third_party/move/move-model/bytecode/ast-generator-tests",
"third_party/move/move-prover",
"third_party/move/move-prover/boogie-backend",
"third_party/move/move-prover/bytecode-pipeline",
@@ -791,6 +792,7 @@ tonic = { version = "0.11.0", features = [
"zstd",
] }
tonic-reflection = "0.11.0"
topological-sort = "0.2.2"
triomphe = "0.1.9"
tui = "0.19.0"
typed-arena = "2.0.2"
5 changes: 4 additions & 1 deletion third_party/move/move-compiler-v2/src/bytecode_generator.rs
Original file line number Diff line number Diff line change
@@ -447,7 +447,7 @@ impl<'env> Generator<'env> {
self.emit_with(*id, |attr| Bytecode::Jump(attr, continue_label));
self.emit_with(*id, |attr| Bytecode::Label(attr, break_label));
},
ExpData::LoopCont(id, do_continue) => {
ExpData::LoopCont(id, 0, do_continue) => {
if let Some(LoopContext {
continue_label,
break_label,
@@ -463,6 +463,9 @@ impl<'env> Generator<'env> {
self.error(*id, "missing enclosing loop statement")
}
},
ExpData::LoopCont(_, _, _) => {
unimplemented!("continue/break with nesting")

Check warning on line 467 in third_party/move/move-compiler-v2/src/bytecode_generator.rs

Codecov / codecov/patch

third_party/move/move-compiler-v2/src/bytecode_generator.rs#L467

Added line #L467 was not covered by tests
},
ExpData::SpecBlock(id, spec) => {
// Map locals in spec to assigned temporaries.
let mut replacer = |id, target| {
4 changes: 2 additions & 2 deletions third_party/move/move-compiler-v2/src/env_pipeline/inliner.rs
Original file line number Diff line number Diff line change
@@ -800,7 +800,7 @@ impl<'env, 'rewriter> InlinedRewriter<'env, 'rewriter> {
(lambda expressions)",
)
},
ExpData::LoopCont(node_id, is_continue) if !post && in_loop == 0 => {
ExpData::LoopCont(node_id, _, is_continue) if !post && in_loop == 0 => {
let node_loc = env.get_node_loc(*node_id);
env.error(
&node_loc,
@@ -1046,7 +1046,7 @@ impl<'env, 'rewriter> ExpRewriterFunctions for InlinedRewriter<'env, 'rewriter>
self.in_loop += 1;
true
},
ExpData::LoopCont(node_id, is_continue) if self.in_loop == 0 => {
ExpData::LoopCont(node_id, _, is_continue) if self.in_loop == 0 => {
let node_loc = self.env.get_node_loc(*node_id);
self.env.error(
&node_loc,
Original file line number Diff line number Diff line change
@@ -1,20 +1,20 @@

Diagnostics:
error: local `x` of type `ability::Impotent` does not have the `copy` ability
error: local `x` of type `Impotent` does not have the `copy` ability
┌─ tests/ability-check/ability_violation.move:7:10
7 │ (x, x);
│ ^ - used here
│ │
│ copy needed here because value is still in use

error: value of type `ability::Impotent` does not have the `drop` ability
error: value of type `Impotent` does not have the `drop` ability
┌─ tests/ability-check/ability_violation.move:7:10
7 │ (x, x);
│ ^ implicitly dropped here since it is no longer used

error: value of type `ability::Impotent` does not have the `drop` ability
error: value of type `Impotent` does not have the `drop` ability
┌─ tests/ability-check/ability_violation.move:7:13
7 │ (x, x);
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@

Diagnostics:
error: local `x` of type `test::Impotent` does not have the `drop` ability
error: local `x` of type `Impotent` does not have the `drop` ability
┌─ tests/ability-check/alive_since_borrowed.move:7:17
7 │ let y = &x;
│ ^^ still borrowed but will be implicitly dropped later since it is no longer used

error: local `x` of type `test::S` does not have the `drop` ability
error: local `x` of type `S` does not have the `drop` ability
┌─ tests/ability-check/alive_since_borrowed.move:21:9
21 │ x.g.h
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@

Diagnostics:
error: local `s` of type `assign::S` does not have the `drop` ability
error: local `s` of type `S` does not have the `drop` ability
┌─ tests/ability-check/assign.move:17:9
17 │ *s = S { f: 42, g: T { h: 42 } };
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@

Diagnostics:
error: local `x1` of type `test::S2` does not have the `copy` ability
error: local `x1` of type `S2` does not have the `copy` ability
┌─ tests/ability-check/bug_14189.move:34:18
34 │ let x2 = S3 { x: x1, y: x0, z: x1 };
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@

Diagnostics:
error: local `_x` of type `Module0::S` does not have the `drop` ability
error: local `_x` of type `S` does not have the `drop` ability
┌─ tests/ability-check/bug_14223_unused_non_droppable_no_abort.move:5:18
5 │ let _x = S {};
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@

Diagnostics:
error: local `_common_fields` of type `m::CommonFields` does not have the `copy` ability
error: local `_common_fields` of type `CommonFields` does not have the `copy` ability
┌─ tests/ability-check/bug_14227.move:21:16
21 │ y: vector[_common_fields]
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@

Diagnostics:
error: local `x` of type `m::R` does not have the `copy` ability
error: local `x` of type `R` does not have the `copy` ability
┌─ tests/ability-check/explicit_move.move:12:9
12 │ some(x);
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@

Diagnostics:
error: value of type `test::Y<test::X<bool>>` does not have the `copy` ability
error: value of type `Y<X<bool>>` does not have the `copy` ability
┌─ tests/ability-check/index_ability_err.move:12:17
12 │ let _ = Y<X<bool>>[addr];
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@

Diagnostics:
error: local `_x` of type `Test::Impotent` does not have the `drop` ability
error: local `_x` of type `Impotent` does not have the `drop` ability
┌─ tests/ability-check/loop_abort.move:11:18
11 │ let _x = Impotent {};
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@

Diagnostics:
error: value of type `M::R` does not have the `drop` ability
error: value of type `R` does not have the `drop` ability
┌─ tests/ability-check/typing/assign_unpack_references.move:17:33
17 │ R { s1: S { f }, s2 } = &R { s1: S{f: 0}, s2: S{f: 1} }; f; s2;
│ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ still borrowed but will be implicitly dropped later since it is no longer used

error: value of type `M::R` does not have the `drop` ability
error: value of type `R` does not have the `drop` ability
┌─ tests/ability-check/typing/assign_unpack_references.move:27:33
27 │ R { s1: S { f }, s2 } = &mut R { s1: S{f: 0}, s2: S{f: 1} }; f; s2;
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@

Diagnostics:
error: value of type `M::R` does not have the `drop` ability
error: value of type `R` does not have the `drop` ability
┌─ tests/ability-check/typing/bind_unpack_references.move:13:41
13 │ let R { s1: S { f }, s2 }: &R = &R { s1: S{f: 0}, s2: S{f: 1} }; f; s2;
│ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ still borrowed but will be implicitly dropped later since it is no longer used

error: value of type `M::R` does not have the `drop` ability
error: value of type `R` does not have the `drop` ability
┌─ tests/ability-check/typing/bind_unpack_references.move:20:45
20 │ let R { s1: S { f }, s2 }: &mut R = &mut R { s1: S{f: 0}, s2: S{f: 1} }; f; s2;
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@

Diagnostics:
error: value of type `M::R` does not have the `drop` ability
error: value of type `R` does not have the `drop` ability
┌─ tests/ability-check/typing/borrow_local_temp_resource.move:6:9
6 │ &R{};
│ ^^^^ implicitly dropped here since it is no longer used

error: value of type `M::R` does not have the `drop` ability
error: value of type `R` does not have the `drop` ability
┌─ tests/ability-check/typing/borrow_local_temp_resource.move:7:9
7 │ &mut R{};
Original file line number Diff line number Diff line change
@@ -1,42 +1,42 @@

Diagnostics:
error: local `r` of type `M::R` does not have the `copy` ability
error: local `r` of type `R` does not have the `copy` ability
┌─ tests/ability-check/typing/derefrence_reference.move:6:16
6 │ R {} = *r;
│ ^^ reference content copied here

error: local `b` of type `M::B` does not have the `copy` ability
error: local `b` of type `B` does not have the `copy` ability
┌─ tests/ability-check/typing/derefrence_reference.move:7:24
7 │ B { r: R{} } = *b;
│ ^^ reference content copied here

error: value of type `M::R` does not have the `copy` ability
error: value of type `R` does not have the `copy` ability
┌─ tests/ability-check/typing/derefrence_reference.move:8:17
8 │ R{} = *&b.r;
│ ^^^ reference content copied here

error: local `r` of type `M::R` does not have the `copy` ability
error: local `r` of type `R` does not have the `copy` ability
┌─ tests/ability-check/typing/derefrence_reference.move:12:16
12 │ R {} = *r;
│ ^^ reference content copied here

error: local `b` of type `M::B` does not have the `copy` ability
error: local `b` of type `B` does not have the `copy` ability
┌─ tests/ability-check/typing/derefrence_reference.move:13:24
13 │ B { r: R{} } = *b;
│ ^^ reference content copied here

error: value of type `M::R` does not have the `copy` ability
error: value of type `R` does not have the `copy` ability
┌─ tests/ability-check/typing/derefrence_reference.move:14:17
14 │ R{} = *&b.r;
│ ^^^ reference content copied here

error: value of type `M::R` does not have the `copy` ability
error: value of type `R` does not have the `copy` ability
┌─ tests/ability-check/typing/derefrence_reference.move:15:21
15 │ R{} = *&mut b.r;
Original file line number Diff line number Diff line change
@@ -1,18 +1,18 @@

Diagnostics:
error: local `r` of type `M::R` does not have the `copy` ability
error: local `r` of type `R` does not have the `copy` ability
┌─ tests/ability-check/typing/eq_invalid2.move:11:9
11 │ r == r;
│ ^^^^^^ copy needed here because value is still in use

error: local `r` of type `M::R` does not have the `drop` ability
error: local `r` of type `R` does not have the `drop` ability
┌─ tests/ability-check/typing/eq_invalid2.move:11:9
11 │ r == r;
│ ^^^^^^ operator drops value here (consider borrowing the argument)

error: value of type `M::G1<T>` does not have the `drop` ability
error: value of type `G1<T>` does not have the `drop` ability
┌─ tests/ability-check/typing/eq_invalid2.move:15:9
15 │ G1{ f: t } == G1{ f: t };
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@

Diagnostics:
error: local `r` of type `M::R` does not have the `drop` ability
error: local `r` of type `R` does not have the `drop` ability
┌─ tests/ability-check/typing/mutate_resource.move:5:9
5 │ *r = R {};
Original file line number Diff line number Diff line change
@@ -1,36 +1,36 @@

Diagnostics:
error: local `s2` of type `M::S` does not have the `drop` ability
error: local `s2` of type `S` does not have the `drop` ability
┌─ tests/ability-check/typing/neq_invalid2.move:17:9
17 │ s != s2;
│ ^^^^^^^ operator drops value here (consider borrowing the argument)

error: local `s` of type `M::S` does not have the `drop` ability
error: local `s` of type `S` does not have the `drop` ability
┌─ tests/ability-check/typing/neq_invalid2.move:17:9
17 │ s != s2;
│ ^^^^^^^ operator drops value here (consider borrowing the argument)

error: local `r1` of type `M::R` does not have the `drop` ability
error: local `r1` of type `R` does not have the `drop` ability
┌─ tests/ability-check/typing/neq_invalid2.move:22:9
22 │ r1 != r2;
│ ^^^^^^^^ operator drops value here (consider borrowing the argument)

error: local `r2` of type `M::R` does not have the `drop` ability
error: local `r2` of type `R` does not have the `drop` ability
┌─ tests/ability-check/typing/neq_invalid2.move:22:9
22 │ r1 != r2;
│ ^^^^^^^^ operator drops value here (consider borrowing the argument)

error: value of type `M::G1<M::Key>` does not have the `drop` ability
error: value of type `G1<Key>` does not have the `drop` ability
┌─ tests/ability-check/typing/neq_invalid2.move:27:9
27 │ G1<Key>{} != G1<Key>{};
│ ^^^^^^^^^^^^^^^^^^^^^^ operator drops value here (consider borrowing the argument)

error: value of type `M::G2<M::Key>` does not have the `drop` ability
error: value of type `G2<Key>` does not have the `drop` ability
┌─ tests/ability-check/typing/neq_invalid2.move:28:9
28 │ G2<Key>{} != G2<Key>{};
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@

Diagnostics:
error: local `s` of type `M::S` does not have the `copy` ability
error: local `s` of type `S` does not have the `copy` ability
┌─ tests/ability-check/typing/pack.move:14:29
14 │ let n2 = Nat { f: *&s };
Original file line number Diff line number Diff line change
@@ -1,26 +1,26 @@

Diagnostics:
error: local `ref` of type `M::HasDrop<M::NoAbilities, M::NoAbilities>` does not have the `drop` ability
error: local `ref` of type `HasDrop<NoAbilities, NoAbilities>` does not have the `drop` ability
┌─ tests/ability-check/typing/phantom_param_op_abilities_invalid2.move:11:9
11 │ *ref = HasDrop<NoAbilities, NoAbilities> { a: NoAbilities { } };
│ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ reference content dropped here

error: value of type `M::HasDrop<M::NoAbilities, M::NoAbilities>` does not have the `drop` ability
error: value of type `HasDrop<NoAbilities, NoAbilities>` does not have the `drop` ability
┌─ tests/ability-check/typing/phantom_param_op_abilities_invalid2.move:16:9
16 │ _ = HasDrop<NoAbilities, NoAbilities> { a: NoAbilities { } };
│ ^ implicitly dropped here since it is no longer used

error: local `_x` of type `M::HasDrop<M::NoAbilities, M::NoAbilities>` does not have the `drop` ability
error: local `_x` of type `HasDrop<NoAbilities, NoAbilities>` does not have the `drop` ability
┌─ tests/ability-check/typing/phantom_param_op_abilities_invalid2.move:20:51
20 │ fun f3(_x: HasDrop<NoAbilities, NoAbilities>) {
│ ╭───────────────────────────────────────────────────^
21 │ │ }
│ ╰─────^ implicitly dropped here since it is no longer used

error: local `x` of type `M::HasCopy<M::NoAbilities, M::NoAbilities>` does not have the `copy` ability
error: local `x` of type `HasCopy<NoAbilities, NoAbilities>` does not have the `copy` ability
┌─ tests/ability-check/typing/phantom_param_op_abilities_invalid2.move:25:10
25 │ (copy x, x)
Loading

0 comments on commit d470c70

Please sign in to comment.