Refactoring Speedy

[nickchapman-da]

This issue collects various refactorings for speedy compiler and runtime engine.

• Some refactoring are purely to improve clarity.
• Some are intended as enablers for more substantive changes, such as the stack-safety work.
• Some refactoring are steps towards improving efficiency.

Planned:

• Distinguish expression types for different phases of the speedy compilation pipeline & expressions which run on the speedy machine. Remove those expression forms which are not required/allowed at each phase. Specifically we want:

  • SExpr0 : as first converted from LF
  • SExpr1 : after closure conversion
  • SExpr : after ANF, and running on speedy machine

• Split compilation phases into separate files

• Move code which is logically part of the phase1 compilation (LF Expr -> SExpr0) out of SExpr classes and into Compiler.scala

• Stop manufacturing expressions at runtime. Instead use new continuation forms (Kont) to implement the functionality.

• Avoid construction of SEApp / SEAppGeneral application nodes, to allow this expression form to be removed from SExpr. (runtime expressions).

• Other refactorings necessary to enable all speedy compiler phases to be recoded for stack-safety. [Engine] Stack safety in Daml Engine (Speedy) #11561

• Simplify SEBuiltinRecursiveDefinition to its one case EqualList, and treat as a normal builtin

Streach:

• Fix/re-order speedy compiler phases. Specifically:

  • Avoid early conversion to de-Bruijn indexes (done in phase 1) which causes subsequent compiler phases to be way more complicated that otherwise.

  • Re-order compilation phases: closure-conversion and ANF-transformation so that closure-conversion is done later. See notes below.

Closure-conversion should come later because it's essential purpose is to map from variables to locations. i.e. (In what location will the value bound to each variable be found at runtime? The answer: on the stack, in the args array, or in the surrounding closure's free array). Before closure-conversion, the compiler phases can work with the much simpler concept of variables. Currently the ANF transformation is done before closure-conversion and consequently has to jump through complicated hoops to track a mapping between locations before/after the transformation step.

[nickchapman-da]

29/3/22
Added tick boxes to the issue description to reflect status