FSharp.Compiler.Service dependency on MSBuild

[nightroman]

As far as I know, project related tools moved to FSharp.Compiler.Service.ProjectCracker. At the same time, FSharp.Compiler.Service still depends on MSBuild and cannot be loaded and used without MSBuild installed.

ILSpy shows the following references:

Microsoft.Build.Framework, Version=12.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a
Microsoft.Build.Tasks.v12.0, Version=12.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a
Microsoft.Build.Utilities.v12.0, Version=12.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a

I wonder if this dependency is by design or not. In the latter case, it would be nice if this dependency is removed. FSharp.Compiler.Service is useful on its own without MSBuild related stuff.

[tpetricek]

I was just going to report this too - Fable, which uses FCS 6.0.2 still does not work on machine with just VS 2015 (you need to install MSBUILD 12 to be able to run it).

[tpetricek]

There was earlier discussion about this, but apparently, this is still an issue: #337

[7sharp9]

Isn't msbuild used for reference resolution too, except in the case of mono which uses simple resolution.

[nosami]

Yeah it's used for reference resolution. Would really like to drop the
MSBuild deps though if possible. Ideally, the reference resolution should
be done before any FCS interactions.

On 11 Sep 2016 7:22 p.m., "Dave Thomas" [email protected] wrote:

Isn't msbuild used for reference resolution too, except in the case of
mono which uses simple resolution.

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
#631 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAouOiA8s_jDTtRX7ufCvVIMcipKiE1Hks5qpDi6gaJpZM4J5yZq
.

[tpetricek]

It would be great to solve this - as more and more projects use FSC and not many people use VS 2013, this is a growing issue. If we don't solve this before Progressive F# tutorials in London, I bet many people coming to Suave and Fable tutorials will be affected by this (with Suave, FSC can provide nice live-reloading experience and Fable relies on this directly).

What is a minimal viable solution to make sure we do not give really crappy experience? Should I recover my PR to bundle the DLLs as part of the package?

[7sharp9]

I wonder why its so hard to rip out, desperately need a #NO_MSBUILD flag to purge it so reference behavior is mono stylee.

[tpetricek]

(more discussion about the earlier PR is here #338 - yes, it's not a nice solution, but I think we need some solution)

[dsyme]

Let's just factor the dependency into another DLL (so you can opt in to Visual F#-Tools compatible MSBuild support).

[dsyme]

@nightroman @tpetricek @7sharp9 @nosami See #649, please take a look over this.

[nightroman]

@dsyme, it looks very promising, thank you very much! This approach, with some minor adjustments on my side, should work for FSharpFar. This module supports two configuration types, .fsproj and .fs.ini. The latter is effectively just arguments of fsc and fsi in a friendly format with some helpers like environment variables expansion. As far as understand, .fs.ini scenario will be able to work without MSBuild. Developers of F# scripts for Far Manager may still use .fsproj with MSBuild / Visual Studio / VS Code. But F# scripts released for Far Manager users come with .fs.ini, so that users do not have to install MSBuild. This is exactly what I needed.

[tpetricek]

@dsyme Thank you for looking into this!! Solving this will be fantastic. I should be able to look once I'm done with the training I'm doing over the next two days!

[nightroman]

@tpetricek, it would be interesting to know if it works for you. So I far I cannot make it working and I am not sure if it is my fault or not. Microsoft.Build.Framework.dll is loaded even if I opt out, if it is present, and it fails if it is not.

I am trying the latest NuGet package 8.0.0 with FSharpChecker(msbuildEnabled=false) and FsiInteractiveSession(msbuildEnabled=false)

[dsyme]

@nightroman Could you submit repro steps? In particular, which tool is hosting FSHarp.Compiler.Service?

[nightroman]

@dsyme The tool is FSharpFar and the FCS scenarios that it uses are briefly
described four comments above.

Let me put the latest changes of FSharpFar in order and release its version
that uses FCS 8.0.0 and opts out of MSBuild in cases of .fs.ini configuration.
Then I will describe how to get Far Manager, FarNet, and FSharpFar and repro steps.

Here is the summary of the used scenario. When FSharpFar is told to open an
interactive session or perform F# checks with .fs.ini configuration then it uses
FsiInteractiveSession(msbuildEnabled=false) or FSharpChecker(msbuildEnabled=false).
It takes arguments for fsc from the .fs.ini file and does not call the project cracker.
Nevertheless, Microsoft.Build.Framework.dll is loaded in this scenario.

[nightroman]

@dsyme

I am trying to investigate. Here is an interesting result.

I added this brutal check and exception:

FSharp.Compiler.Service\src\fsharp\SimulatedMSBuildReferenceResolver.fs(168):

    let internal GetBestAvailableResolver(msbuildEnabled: bool) =
        if msbuildEnabled then invalidOp "unexpected msbuildEnabled"

Then I ran my case with msbuildEnabled = false.
I got the exception which I kind of expected. Here is the stack:

    TypeInitializationException:
    The type initializer for 'Microsoft.FSharp.Compiler.SourceCodeServices.FSharpChecker' threw an exception.

    TypeInitializationException:
    The type initializer for '<StartupCode$FSharp-Compiler-Service>.$Service' threw an exception.

    InvalidOperationException:
    unexpected msbuildEnabled

    System.TypeInitializationException: The type initializer for 'Microsoft.FSharp.Compiler.SourceCodeServices.FSharpChecker' threw an exception. ---> System.TypeInitializationException: The type initializer for '<StartupCode$FSharp-Compiler-Service>.$Service' threw an exception. ---> System.InvalidOperationException: unexpected msbuildEnabled
       at Microsoft.FSharp.Compiler.SimulatedMSBuildReferenceResolver.GetBestAvailableResolver(Boolean msbuildEnabled)
       at Microsoft.FSharp.Compiler.SourceCodeServices.FSharpChecker.Create(FSharpOption`1 projectCacheSize, FSharpOption`1 keepAssemblyContents, FSharpOption`1 keepAllBackgroundResolutions, FSharpOption`1 msbuildEnabled)
       at <StartupCode$FSharp-Compiler-Service>.$Service..cctor()
       --- End of inner exception stack trace ---
       at Microsoft.FSharp.Compiler.SourceCodeServices.FSharpChecker..cctor()
       --- End of inner exception stack trace ---
       at Microsoft.FSharp.Compiler.SourceCodeServices.FSharpChecker.Create(FSharpOption`1 projectCacheSize, FSharpOption`1 keepAssemblyContents, FSharpOption`1 keepAllBackgroundResolutions, FSharpOption`1 msbuildEnabled)
       at Microsoft.FSharp.Compiler.Interactive.Shell.FsiEvaluationSession..ctor(FsiEvaluationSessionHostConfig fsi, String[] argv, TextReader inReader, TextWriter outWriter, TextWriter errorWriter, Boolean fsiCollectible, Boolean msbuildEnabled)
       at Microsoft.FSharp.Compiler.Interactive.Shell.FsiEvaluationSession.Create(FsiEvaluationSessionHostConfig fsiConfig, String[] argv, TextReader inReader, TextWriter outWriter, TextWriter errorWriter, FSharpOption`1 collectible, FSharpOption`1 msbuildEnabled)
       at FSharpFar.Session.Session..ctor(String configFile)
       at FSharpFar.Session.Session.FindOrCreate(String path)
       at FSharpFar.FarCommand.Invoke(Object sender, ModuleCommandEventArgs e)
       at FarNet.Far0.InvokeCommand(Char* command, Boolean isMacro)
       at FarNet.Far0.AsOpen(OpenInfo* info)
       at OpenW(OpenInfo* info)

The static initializer

FSharp.Compiler.Service\src\fsharp\vs\service.fs(2721):

    static let globalInstance = FSharpChecker.Create()

calls Create with the default parameters, no matter what I actually provide in my other calls.

As a result, the available MSBuild gets loaded even though I opt out.
I am not telling that it is used for reference resolution but it is loaded.

Before I go further, I would like to know if this is by design or not.

The above does not mean that it all is not working on a machine without MSBuild.
Why it fails on my machine without MSBuild is yet to be found, please ignore for now.

[nightroman]

I cannot tell exactly what is wrong on a machine without MSBuild. But according
to procmon, in spite of msbuildEnabled = false, Microsoft.Build.Framework
is searched in the registry and GAC.

It fails in try/with here

FSharp.Compiler.Service\src\fsharp\fsi\fsi.fs:

    /// The single, global interactive checker that can be safely used in conjunction with other operations
    /// on the FsiEvaluationSession. 
    let checker = FSharpChecker.Create()

    let (tcGlobals,frameworkTcImports,nonFrameworkResolutions,unresolvedReferences) =
        try
            let tcConfig = tcConfigP.Get()
            checker.FrameworkImportsCache.Get tcConfig
        with e ->
            stopProcessingRecovery e range0; failwithf "Error creating evaluation session: %A" e

I also tried to buils and test with my change of the global checker Create

    let checker = FSharpChecker.Create(msbuildEnabled = false)

The result/exception is the same.

Now it is getting too cryptic for me. I can make some changes in FCS, build,
and try on a "clean" test machine (I have just one). If you want me to make
some diagnostic changes and test them, please let me know.

[dsyme]

@nightroman Thanks for the update.

On the first question: using the global FSharpChecker.Instance is deprecated. You should be receiving an "Obsolete" warning when using that property. Please create an FSharpChecker choosing whether to enable msbuild (meaning enable-if-installed) or not

On the second question. If you are not using the ProjectCracker, and you are using msbuildEnabled = false, then you should never be resolving Microsoft.Build.Framework (unless you are actually referencing that in your F# compilation that you are requesting, in which case we will search on disk for that DLL).

Any chance you can list repro steps? I'd be happy to dig into it to help land this cleanly.

Are you using the ProjectCracker?

[nightroman]

@dsyme On the first question: please note that I do not use deprecated FSharpChecker.Instance. But it is always created by the FCS library itself in the static initializer of FSharpChecker. Thus MSBuild is always loaded if it is present, even if I opt out.

The static initializer

FSharp.Compiler.Service\src\fsharp\vs\service.fs(2721):

    static let globalInstance = FSharpChecker.Create()

calls Create with the default parameters, no matter what I actually provide in my other calls.

Are you using the ProjectCracker?

No. I have the dependency but I do not call it in scenarios related to this topic. MSBuild should not be loaded in my scenarios, per my understanding of 8.0.0 and my use of msbuildEnabled = false. But it happens to be loaded.

[dsyme]

But it is always created by the FCS library itself in the static initializer of FSharpChecker. Thus MSBuild is always loaded if it is present, even if I opt out.

Ah, I see. On a machine without MSBuild this attempted load should not cause a failure since it gets caught. So some other exception is happening. Can you share the stack trace and details of the exception?

[nightroman]

It does not fail in Create(). As I wrote above, it fails in try/with here

FSharp.Compiler.Service\src\fsharp\fsi\fsi.fs:

    /// The single, global interactive checker that can be safely used in conjunction with other operations
    /// on the FsiEvaluationSession. 
    let checker = FSharpChecker.Create()

    let (tcGlobals,frameworkTcImports,nonFrameworkResolutions,unresolvedReferences) =
        try
            let tcConfig = tcConfigP.Get()
            checker.FrameworkImportsCache.Get tcConfig
        with e ->
            stopProcessingRecovery e range0; failwithf "Error creating evaluation session: %A" e

That is all I can currently get, not much, unfortunately.

[nightroman]

I will remove try/with, build, and try again on my the only "clean" machine. Hopefully, we will get the stack. I am sorry that my ways to diagnose this issue are very limited.

[dsyme]

ok thanks

[nightroman]

Here is the call stack

StopProcessingExn:
Exception of type 'Microsoft.FSharp.Compiler.ErrorLogger+StopProcessingExn' was thrown.

Microsoft.FSharp.Compiler.ErrorLogger+StopProcessingExn: Exception of type 'Microsoft.FSharp.Compiler.ErrorLogger+StopProcessingExn' was thrown.
   at Microsoft.FSharp.Compiler.Interactive.Shell.ErrorLoggerThatStopsOnFirstError.ErrorSinkHelper[a](PhasedError err) in C:\-\GIT\fs-proj\FSharp.Compiler.Service\src\fsharp\fsi\fsi.fs:line 465
   at Microsoft.FSharp.Compiler.Interactive.Shell.ErrorLoggerThatStopsOnFirstError.ErrorSinkImpl(PhasedError err) in C:\-\GIT\fs-proj\FSharp.Compiler.Service\src\fsharp\fsi\fsi.fs:line 479
   at Microsoft.FSharp.Compiler.ErrorLogger.ErrorLoggerExtensions.ErrorLogger.Error[b](ErrorLogger x, Exception exn) in C:\-\GIT\fs-proj\FSharp.Compiler.Service\src\fsharp\ErrorLogger.fs:line 326
   at Microsoft.FSharp.Compiler.CompileOps.TcImports.BuildFrameworkTcImports(TcConfigProvider tcConfigP, FSharpList`1 frameworkDLLs, FSharpList`1 nonFrameworkDLLs) in C:\-\GIT\fs-proj\FSharp.Compiler.Service\src\fsharp\CompileOps.fs:line 4598
   at Microsoft.FSharp.Compiler.FrameworkImportsCache.Get(TcConfig tcConfig) in C:\-\GIT\fs-proj\FSharp.Compiler.Service\src\fsharp\vs\IncrementalBuild.fs:line 1115
   at Microsoft.FSharp.Compiler.Interactive.Shell.FsiEvaluationSession..ctor(FsiEvaluationSessionHostConfig fsi, String[] argv, TextReader inReader, TextWriter outWriter, TextWriter errorWriter, Boolean fsiCollectible, Boolean msbuildEnabled) in C:\-\GIT\fs-proj\FSharp.Compiler.Service\src\fsharp\fsi\fsi.fs:line 2462
   at Microsoft.FSharp.Compiler.Interactive.Shell.FsiEvaluationSession.Create(FsiEvaluationSessionHostConfig fsiConfig, String[] argv, TextReader inReader, TextWriter outWriter, TextWriter errorWriter, FSharpOption`1 collectible, FSharpOption`1 msbuildEnabled) in C:\-\GIT\fs-proj\FSharp.Compiler.Service\src\fsharp\fsi\fsi.fs:line 2725
   at FSharpFar.Session.Session..ctor(String configFile)
   at FSharpFar.Session.Session.FindOrCreate(String path)
   at FSharpFar.FarCommand.Invoke(Object sender, ModuleCommandEventArgs e)
   at FarNet.Works.ProxyCommand.Invoke(Object sender, ModuleCommandEventArgs e)
   at FarNet.Far0.InvokeCommand(Char* command, Boolean isMacro)
   at FarNet.Far0.AsOpen(OpenInfo* info)

[nightroman]

One more thing. The machine does not have F# runtime installed. I provide
FSharp.Core.dll with my package and non-FCS related F# code works fine.
Is it perhaps mandatory for FCS that F# runtime is installed?

[nightroman]

So here is the best I can get, some obscure error thrown at

src\fsharp\CompileOps.fs:line 4598
    error(InternalError("BuildFrameworkTcImports: no successful import of "+coreLibraryResolution.resolvedPath,coreLibraryResolution.originalReference.Range))

Please let me know how I can change something to improve diagnostics.

I will be able to publish my changes for 8.0.0 only next week. Then I may explain the steps to install and repro. But a clean machine is needed, too, I presume.

[nightroman]

Investigated. My remaining issue was not related to MSBuild, it was about
missing FSharp.Core.optdata and FSharp.Core.sigdata.

Thus, all works fine if MSBuild is not present and the issue may be closed.
My module FSharpFar with F# interactive and editor services may work in Far
Manager without installing anything but the module itself. This is great!

There is one minor imperfection though, as shown before. If MSBuild is present
then it is loaded even if I opt out. This is happening due to two global calls
to FSharpChecker.Create() with default parameters:

• src/fsharp/fsi/fsi.fs
• src/fsharp/vs/service.fs

As far as I opt out, MSBuild is not used for resolution, hopefully, it just
gets loaded. Should this be filed as a separate issue perhaps? Or is it not an
issue at all?

[dsyme]

@nightroman Super, thanks. See #657 for the fix to the FSharpChecker.Create() calls.

[nightroman]

I think the issue is resolved completely in 9.0.0. I am closing it. Thank you!