Project Dallas

[nojaf]

The F# compiler syntax tree. It will always have a special place in my heart and all that but in a lot of cases, it is not the tree Fantomas truly needs to restore the source code.

A couple of shortcomings (from Fantomas' point of view) come to mind:

• Code comments are floating in the tree. We still need to do our own detection to which nodes the comment belongs. (the concept of Trivia)
• There is a lot of stuff in the current tree we don't really use. This is why we so heavily use partial active patterns (in SourceParser.fs) to get around that. Think of DebugPoint, SynValData and others.
• The way some nodes are grouped isn't typically useful for Fantomas. Recursive bindings or types for example. We consider the nested children to be on the same level, while they are not in the F# AST.
• Something joining two nodes is more convenient to work it. For example, named computation expressions.

These shortcomings lead to a more complex story than necessary for newcomers.
Our guide on missing code comments is a good example of that. You really need to jump through a lot of hoops to fix this problem.

So, after carefully considering this, we think the situation could be improved by having our own syntax tree. In this PR we are laying the foundation for this. I believe it will unlock some more opportunities in the future due to the new approach.

In short, the premise goes a bit like this:

• All new type nodes in the new Tree implement the Node interface. A Node can store TriviaNodes before and after. (See SyntaxOak.fs)
• In Fangorn.fs we map the F# compiler AST to this new Node model.
• In Flowering.fs we insert the found trivia (comments, newline, directives) into the existing tree.
• And in CodePrinter2.fs we traverse the new tree to collect the WriterEvents

At the time of writing, 83% of the unit tests are passing. So, there still is some work left to do.
From a style point of view, we try to keep everything exactly as it was. But due to the size of this refactor, we don't necessarily guarantee that nothing will change. Of course, we will respect the style guide at all times. Some things might just change slightly due to convenience.

From an end-user point of view, this won't really affect the usage of the tool. Some performance will be gained, but I don't expect anything significant.

Some todo items:

• Fix all remaining unit tests
• Restore format selection
• Verify everything works for existing codebases (Fantomas, FsAutocomplete, F# compiler to name a few)
• Rename the new files and type names.
• Add signature files where it makes sense.
• Update the online tool to have a viewer for the new tree.
• Update documentation

[nojaf]

90% of Fantomas.Core.Tests are passing 🎉

[nojaf]

93%

[auduchinok]

In Fangorn.fs we map the F# compiler AST to this new Node model.
In Flowering.fs we insert the found trivia (comments, newline, directives) into the existing tree.

The names are interesting, but maybe it would be better to use self-evident names to make it easier for other, especially new, contributors? 🙂

[nojaf]

@auduchinok yup, already part of my to-do list (see above) 😊.

[nojaf]

96%

[nojaf]

98%

[nojaf]

Good morning all 😊, I was able to restore all unit tests of Fantomas.Core!
Next, I started looking at formatting selections. I think we will be able to pull this off with the new tree. I have one example working, the rest is still a work in progress.
(So at 78bbac7 not all tests of Fantomas.Core.Tests are passing, the FormatSelectionOnly will fail.)

I suspect I'll have the format selection wrapped up by the end of the week.

[nojaf]

Works for Fantomas, FsAutocomplete & dotnet/fsharp.