Problems with referencing Microsoft.CodeAnalysis.Workspaces from an analyzer

[HellBrick]

I have an analyzer that uses Formatter class from Microsoft.CodeAnalysis.Workspaces that's installed into a project via a nuget package. Even though it works when editing code in Visual studio (I get the warning squiggles and the associated code fix is triggered by Ctrl+.), whenever I build the project, I get the following warning:

Analyzer 'HellBrick.Diagnostics.Formatting.FormattingAnalyzer' threw an exception of type 'System.IO.FileNotFoundException' with message 'Could not load file or assembly 'Microsoft.CodeAnalysis.Workspaces, Version=1.1.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35' or one of its dependencies. The system cannot find the file specified.'.

First I thought it was a version mismatch issue that occurred after Update 1, but trying 3 different stable package versions (1.0.0, 1.1.1 and 1.1.0) didn't change anything.

Then I tried including Microsoft.CodeAnalysis.Workspaces dll into the nuget package and it seemed to have some effect (the dll the warning was about changed to something else, one of dependencies of Microsoft.CodeAnalysis.Workspaces, I guess), but it screwed up the code fix (VS started throwing 'One or more error occurred' pop-up on Ctrl+.).

Am I doing something wrong here or is it an actual bug? And how does the analyzer work if VS claims it can't load the assembly it requires?

[jmarolf]

1. Why does the analyzer need the Formatter class?
2. What is happening is that the commandline compiler is loading your analyzer and the commandline compiler has no knowledge of workspaces. Even if you include it as part of your nuget package, the compiler doesn't know how to create all the workspace services, only Visual Studio knows that.

[HellBrick]

1. Formatter is the easiest way (that I've found) to check if the code is formatted properly. I do Formatter.GetFormattedTextChanges( root, new AdhocWorkspace(), properFormattingOptions, context.CancellationToken ) in a syntax tree action and get all the info I need to report diagnostics. Finding all formatting issues manually would be a lot more difficult.
2. So there's basically no way to use AdhocWorkspace and therefore Formatter in an analyzer?

[jmarolf]

Finding all formatting issues manually would be a lot more difficult.

True, but it will hurt performance for a large project to format every document every time.The StyleCopAnalyzers project have implemented very targeted rules for formatting, but perhaps there is an easier way to do what you want. Are you trying to detect if the user is not using some specified formatting options? You could just write a Visual Studio extension that enforces a set of options, or write a commandline tool that uses the AdHocWorkspace.

So there's basically no way to use AdhocWorkspace and therefore Formatter in an analyzer?

The compiler only knows about things that happen at a project level and analyzers can only know what the compiler knows. We've had some discussions around designing an API that could allow you to analyze a workspace, but we haven't figured out a way to do it that wouldn't hurt performance.

[HellBrick]

You could just write a Visual Studio extension that enforces a set of options

There are two problems with that. First, my practice shows that sometimes enforcing VS formatting options is not enough. Unfortunately, VS auto-formatting experience is not perfect and sometimes leaves blocks of code formatted incorrectly until user re-types the closing brace or reformats the document manually, which is something people tend to forget to do all the time. And second, extension can't be enforced on user by the project referencing a nuget package, thus making it useless for hypothetical open source scenarios.

or write a commandline tool that uses the AdHocWorkspace

This solution basically shares the same problems. It's theoretically possible to distribute this tool as a nuget package that edits the project to insert a pre-build step that launches the tool, but it sounds a lot more awkward than just running the analyzer during the build (I'm basically re-inventing the way you're running analyzers during build). And the VS experience would obviously be worse than with a proper analyzer.

True, but it will hurt performance for a large project to format every document every time.

Will it really be worse than detecting all the possible formatting issues manually? I'll need to rewrite a lot of Formatter logic, and I'm not sure I can do it in a way that's as correct as Formatter but works faster. (Not to self: it would be interesting to measure how much of a performance hit we're talking about here. Judging by VS experience of working with this analyzer, I don't see any degradation from it.)

The compiler only knows about things that happen at a project level and analyzers can only know what the compiler knows. We've had some discussions around designing an API that could allow you to analyze a workspace, but we haven't figured out a way to do it that wouldn't hurt performance.

But I don't really want to analyze the whole workspace (at least not in this topic ;)). I'm perfectly fine with doing things at a project level; actually, for this analyzer to work there's no need for anything higher than a syntax tree. What does Formatter really need Workspace for anyway? If accessing AdhocWorkspace during build is a problem, is it possible to refactor some basic formatting API that works on just a syntax tree and formatting settings?

[jmarolf]

Looking over the Formatter code I think that we could refactor it to be used without a workspace. At think point I'd recommend filing a bug to that effect so we can prioritize it and get the API changes in.

[HellBrick]

I've opened #8268 to discuss the suggested API changes, so I'm closing this one.