Reference assemblies are located in the wrong folder on Windows #2638

nojaf · 2022-01-25T13:13:54Z

Description

After upgrading to 5.21.0, I can no longer run my FAKE script on Windows.

There was a problem while setting up the environment:
-> Could not find referenced assemblies in path: 'C:\Users\nojaf\AppData\Local\Microsoft\dotnet\packs\Microsoft.NETCore.App.Ref\6.0.0\ref\net6.0', please check installed SDK and runtime versions

Repro steps

dotnet fake -v build

C:\Users\nojaf\Projects\Datto\network-scanner>dotnet fake -v build
runOrBuild ({ Script = None
  ScriptArguments = []
  FsiArgLine = []
  Debug = false
  NoCache = false
  RestoreOnlyGroup = false
  VerboseLevel = Verbose
  IsBuild = true })
FAKE 5 - F# Make (5.21.0) (this line is written to standard error, see https://github.com/fsharp/FAKE/issues/2066)
prepareAndRunScriptRedirect(Script: C:\Users\nojaf\Projects\Datto\network-scanner\build.fsx, fsiOptions: "")
Writing 'C:\Users\nojaf\Projects\Datto\network-scanner\.fake\build.fsx\intellisense.fsx'
Restoring with paket...
The last restore is still up to date. Nothing left to do.
Retrieving the assemblies (rid: 'win10-x64')...
Using .Net 6 assemblies
Trying to resolve runtime version from network..
Calculating the runtime graph...
Loaded runtime json from: microsoft.netcore.platforms-5.0.3
resolved runtime version: 6.0.0
Resolved referenced SDK path: C:\Users\nojaf\AppData\Local\Microsoft\dotnet\packs\Microsoft.NETCore.App.Ref\6.0.0\ref\net6.0
Performance:
 - Cli parsing: 132 milliseconds
 - Packages: 1 second
   - Creating Runtime Graph: 40 milliseconds
   - Retrieve Assembly List: 1 second
 - Runtime: 2 seconds
There was a problem while setting up the environment, see standard error for details.
There was a problem while setting up the environment:
-> Could not find referenced assemblies in path: 'C:\Users\nojaf\AppData\Local\Microsoft\dotnet\packs\Microsoft.NETCore.App.Ref\6.0.0\ref\net6.0', please check installed SDK and runtime versions
   StackTrace:
        at Microsoft.FSharp.Core.PrintfModule.PrintFormatToStringThenFail@1439.Invoke(String message) in D:\workspace\_work\1\s\src\fsharp\FSharp.Core\printf.fs:line 1439
        at Fake.Runtime.FakeRuntime.retrieveInfosUncached@114(String cacheDir, Lazy`1 paketDependenciesFile, VerboseLevel logLevel, GroupName groupName, SdkAssemblyResolver sdkAssemblyResolver, FrameworkIdentifier framework, Rid rid, Rid ridNotVersionSpecific, Lazy`1 lockFile, Lazy`1 cache, Unit unitVar0) in D:\
a\FAKE\FAKE\src\app\Fake.Runtime\FakeRuntime.fs:line 202
        at [email protected](Unit unitVar0) in D:\a\FAKE\FAKE\src\app\Fake.Runtime\FakeRuntime.fs:line 312
        at Fake.Runtime.CoreCache.getCached[a](FSharpFunc`2 getUncached, FSharpFunc`2 readFromCache, FSharpFunc`2 writeToCache, FSharpFunc`2 checkCacheUpToDate) in D:\a\FAKE\FAKE\src\app\Fake.Runtime\CoreCache.fs:line 41
        at Fake.Runtime.FakeRuntime.getKnownDependencies@310(String cacheDir, Lazy`1 paketDependenciesFile, VerboseLevel logLevel, GroupName groupName, FileInfo lockFilePath, String dependencyCacheHashFile, String dependencyCacheFile, SdkAssemblyResolver sdkAssemblyResolver, FrameworkIdentifier framework, Rid ri
d, Rid ridNotVersionSpecific, Lazy`1 lockFile, Lazy`1 cache, Lazy`1 writeIntellisenseTask, FSharpFunc`2 readFromCache, FSharpFunc`2 writeToCache, Unit unitVar0) in D:\a\FAKE\FAKE\src\app\Fake.Runtime\FakeRuntime.fs:line 311
        at [email protected](Unit unitVar) in D:\a\FAKE\FAKE\src\app\Fake.Runtime\FakeRuntime.fs:line 320
        at System.Lazy`1.ViaFactory(LazyThreadSafetyMode mode)
        at System.Lazy`1.ExecutionAndPublication(LazyHelper executionAndPublication, Boolean useDefaultConstructor)
        at System.Lazy`1.CreateValue()
        at System.Lazy`1.get_Value()
        at Fake.Runtime.FakeRuntime.paketCachingProvider(FakeConfig config, String cacheDir, Dependencies paketApi, Lazy`1 paketDependenciesFile, FSharpOption`1 group) in D:\a\FAKE\FAKE\src\app\Fake.Runtime\FakeRuntime.fs:line 323
        at Fake.Runtime.FakeRuntime.runScript(PrepareInfo preparedScript) in D:\a\FAKE\FAKE\src\app\Fake.Runtime\FakeRuntime.fs:line 579
        at Program.runOrBuild(RunArguments args) in D:\a\FAKE\FAKE\src\app\Fake.netcore\Program.fs:line 156

Expected behavior

FAKE script runs.

Actual behavior

The reference assemblies are being located from C:\Users\<user>\AppData\Local\Microsoft\dotnet\packs\Microsoft.NETCore.App.Ref, on my machine they are stored on C:\Program Files\dotnet\packs\Microsoft.NETCore.App.Ref\6.0.0.

Known workarounds

Copying the 6.0.0 folder from Program Files to AppData\Local solves the problem.

Related information

Operating system: Windows 11
Branch: default branch I think
.NET Runtime, CoreCLR or Mono Version: .NET 6
Indications of severity: FAKE is unusable.
Version of FAKE (4.X, 5.X): 5.21

The text was updated successfully, but these errors were encountered:

nojaf · 2022-01-25T14:19:49Z

Workaround

PS> $env:FAKE_SDK_RESOLVER_CUSTOM_DOTNET_PATH = "C:\Program Files\dotnet" ; dotnet fake build

TheAngryByrd · 2022-01-25T14:27:11Z

Is code here seems to be the issue:

FAKE/src/app/Fake.Runtime/SdkAssemblyResolver.fs

Lines 126 to 161 in 9dac475

    
           member this.SdkReferenceAssemblies() = 
        
               let dotnetHost = 
        
                   match Environment.isUnix with 
        
                   | true -> "dotnet" 
        
                   | false -> "dotnet.exe" 
        
               let userInstallDir = DotNet.defaultUserInstallDir 
        
               let systemInstallDir = DotNet.defaultSystemInstallDir 
        
               let dotnetHostPath = 
        
                   if not(String.isNullOrEmpty CustomDotNetHostPath) 
        
                   then CustomDotNetHostPath 
        
                   else 
        
                       match File.Exists(userInstallDir </> dotnetHost) with 
        
                       | true -> userInstallDir 
        
                       | false -> systemInstallDir 
        
               let referenceAssembliesPath = 
        
                   dotnetHostPath 
        
                   </> "packs" 
        
                   </> "Microsoft.NETCore.App.Ref" 
        
                   </> this.ResolveSdkRuntimeVersion() 
        
                   </> "ref" 
        
                   </> "net" + this.SdkVersionRaw 
        
               if this.LogLevel.PrintVerbose then 
        
                   Trace.tracefn $"Resolved referenced SDK path: {referenceAssembliesPath}" 
        
               match Directory.Exists referenceAssembliesPath with 
        
               | true -> 
        
                   Directory.GetFiles( 
        
                       referenceAssembliesPath, 
        
                       "*.dll" 
        
                   ) 
        
                   |> Seq.toList 
        
               | false -> 
        
                   failwithf "Could not find referenced assemblies in path: '%s', please check installed SDK and runtime versions" referenceAssembliesPath

Instead of setting dotnetHostPath once, it might be worth always trying all three potential paths set (ENV_VAR, User, then System), every time we go to execute something. Or maybe there's a better/safer lookup for this? cc @baronfel

@nojaf, I just want to confirm but I think you mentioned in slack that C:\Users\<user>\AppData\Local\Microsoft\dotnet\dotnet.exe did exist on your machine. Could you also show what other sdks are installed in the user path? (I'm more just curious here)

baronfel · 2022-01-25T14:37:31Z

I think the custom/user/system probing is slightly off there - the way to do that probing in a way that's consistent with the way other tooling expects is to

Check for the DOTNET_HOST_PATH env variable(which points to the dotnet binary) and use the parent dir if present,
Check for the DOTNET_ROOT env variable and use that directory if present,
probe the PATH for the dotnet binary and use it's parent directory as the root
Then finally default to the default user/system level probing

This is what we do in proj-info, and that's modeled off of the lookup logic in the .NET runtime/SDK itself

nojaf · 2022-01-25T15:31:44Z

@TheAngryByrd yes, "C:\Users\nojaf\AppData\Local\Microsoft\dotnet\dotnet.exe" does exists.
I do have this folder as well: C:\Users\nojaf\AppData\Local\Microsoft\dotnet\packs\Microsoft.NETCore.App.Ref\3.1.0

SteveGilham · 2022-01-25T17:03:48Z

I'm not sure if this is some legacy install thing, as the files I have in the AppData\Local\Microsoft\dotnet tree are all from SDK 2.1, more than two years old, even after having uninstalled everything before 3.1 to tidy up after the net6.0 release.

Question -- is is safe just to delete the dotnet.exe from this (or even the whole tree)?

baronfel · 2022-01-25T17:05:03Z

if those directories aren't in your PATH and aren't shown when you run dotnet --list-sdks, then yes you can purge with abandon.

yazeedobaid · 2022-01-25T17:09:12Z

The Sdk resolver tries first to get user-installed locations before the default dotnet installation. So it seems that you have dotnet installed in a specific location and in default location. So things got mixed up. I think that net6 is installed in the default location but you have an installation that is in your AppData\Local directory. And resolver combined them as
C:\Users\nojaf\AppData\Local\Microsoft\dotnet\packs\Microsoft.NETCore.App.Ref\6.0.0\ref\net6.0

can you please provide output for dotnet --info

yazeedobaid · 2022-01-25T17:13:06Z

@TheAngryByrd yes, "C:\Users\nojaf\AppData\Local\Microsoft\dotnet\dotnet.exe" does exists.
I do have this folder as well: C:\Users\nojaf\AppData\Local\Microsoft\dotnet\packs\Microsoft.NETCore.App.Ref\3.1.0

So yeah you have dotnet 3.1 in user specific location and net6 in default installation. And SDK resolver got mixed them.

I think we should try both; user-specific and the default location for resolved reference assemblies version. First, user-installed and then default.

So in your case it will be we try first if the path exists:
C:\Users\nojaf\AppData\Local\Microsoft\dotnet\packs\Microsoft.NETCore.App.Ref\6.0.0\ref\net6.0
And if not (which is your case), we try
C:\Program Files\dotnet\packs\Microsoft.NETCore.App.Ref\6.0.0

baronfel · 2022-01-25T17:14:18Z

I disagree, this logic is not aligned with how any other tooling works. it doesn't match how FSI typechecks the scripts, for example. the algorithm I linked above is the supported way to find the SDK directories and you really really should align with that.

yazeedobaid · 2022-01-25T17:25:00Z

@baronfel Thanks! If that is the correct algorithm and covers all the cases then we should definitely use it in FAKE.
Does anyone want to help with this?

Thanks

baronfel · 2022-01-25T17:31:41Z

@TheAngryByrd said I should extract and publish this, but you can essentially take the code from ionide/proj-info wholesale to help here: https://github.com/ionide/proj-info/blob/main/src/Ionide.ProjInfo/Utils.fs#L7-L87

You can use the Paths.dotnetRoot lazy to get the path to the dotnet directory you should use as your dotnetHostPath in the code that @TheAngryByrd linked above, but then the rest of the code you have looks good to me.

nojaf · 2022-01-25T18:57:38Z

I want to take a stab at this, feels like an easy steal of Chet's code, so I'm game 😅.

mk185147 · 2022-01-26T16:20:17Z

I also see related issue. I have a custom dotnet version locally installed/copied by build pipeline and I start dotnet fake command with this version then I expect (and was legacy behavior) that DotNet.exec within fake script will execute this custom dotnet version. This was broken with those changes.

mattiasdrp · 2022-01-29T12:23:06Z

I guess my problem is the same as this one (I'm following the install instructions from here

❯ dotnet fake -v build
runOrBuild ({ Script = None
  ScriptArguments = []
  FsiArgLine = []
  Debug = false
  NoCache = false
  RestoreOnlyGroup = false
  VerboseLevel = Verbose
  IsBuild = true })
FAKE 5 - F# Make (5.21.0-alpha004) (this line is written to standard error, see https://github.com/fsharp/FAKE/issues/2066)
prepareAndRunScriptRedirect(Script: /home/mattias/fsharp/FsAutoComplete/build.fsx, fsiOptions: "")
Writing '/home/mattias/fsharp/FsAutoComplete/.fake/build.fsx/intellisense.fsx'
Restoring with paket...
The last restore is still up to date. Nothing left to do.
Retrieving the assemblies (rid: 'ubuntu.20.04-x64')...
Calculating the runtime graph...
Loaded runtime json from: microsoft.netcore.platforms-6.0.1
Performance:
 - Cli parsing: 223 milliseconds
 - Packages: 2 seconds
   - Creating Runtime Graph: 49 milliseconds
   - Retrieve Assembly List: 2 seconds
 - Runtime: 2 seconds
There was a problem while setting up the environment, see standard error for details.
There was a problem while setting up the environment:
-> Could not find referenced assemblies in path: '/usr/local/share/dotnet/packs/Microsoft.NETCore.App.Ref/6.0.0/ref/net6.0', please check installed SDK and runtime versions
   StackTrace:
        at Microsoft.FSharp.Core.PrintfModule.PrintFormatToStringThenFail@1439.Invoke(String message) in D:\workspace\_work\1\s\src\fsharp\FSharp.Core\printf.fs:line 1439
        at Fake.Runtime.FakeRuntime.retrieveInfosUncached@114(String cacheDir, Lazy`1 paketDependenciesFile, VerboseLevel logLevel, GroupName groupName, SdkAssemblyResolver sdkAssemblyResolver, FrameworkIdentifier framework, Rid rid, Rid ridNotVersionSpecific, Lazy`1 lockFile, Lazy`1 cache, Unit unitVar0) in D:\a\FAKE\FAKE\src\app\Fake.Runtime\FakeRuntime.fs:line 206
        at [email protected](Unit unitVar0) in D:\a\FAKE\FAKE\src\app\Fake.Runtime\FakeRuntime.fs:line 312
        at Fake.Runtime.CoreCache.getCached[a](FSharpFunc`2 getUncached, FSharpFunc`2 readFromCache, FSharpFunc`2 writeToCache, FSharpFunc`2 checkCacheUpToDate) in D:\a\FAKE\FAKE\src\app\Fake.Runtime\CoreCache.fs:line 41
        at Fake.Runtime.FakeRuntime.getKnownDependencies@310(String cacheDir, Lazy`1 paketDependenciesFile, VerboseLevel logLevel, GroupName groupName, FileInfo lockFilePath, String dependencyCacheHashFile, String dependencyCacheFile, SdkAssemblyResolver sdkAssemblyResolver, FrameworkIdentifier framework, Rid rid, Rid ridNotVersionSpecific, Lazy`1 lockFile, Lazy`1 cache, Lazy`1 writeIntellisenseTask, FSharpFunc`2 readFromCache, FSharpFunc`2 writeToCache, Unit unitVar0) in D:\a\FAKE\FAKE\src\app\Fake.Runtime\FakeRuntime.fs:line 311
        at [email protected](Unit unitVar) in D:\a\FAKE\FAKE\src\app\Fake.Runtime\FakeRuntime.fs:line 320
        at System.Lazy`1.ViaFactory(LazyThreadSafetyMode mode)
        at System.Lazy`1.ExecutionAndPublication(LazyHelper executionAndPublication, Boolean useDefaultConstructor)
        at System.Lazy`1.CreateValue()
        at System.Lazy`1.get_Value()
        at Fake.Runtime.FakeRuntime.paketCachingProvider(FakeConfig config, String cacheDir, Dependencies paketApi, Lazy`1 paketDependenciesFile, FSharpOption`1 group) in D:\a\FAKE\FAKE\src\app\Fake.Runtime\FakeRuntime.fs:line 323
        at Fake.Runtime.FakeRuntime.runScript(PrepareInfo preparedScript) in D:\a\FAKE\FAKE\src\app\Fake.Runtime\FakeRuntime.fs:line 579
        at Program.runOrBuild(RunArguments args) in D:\a\FAKE\FAKE\src\app\Fake.netcore\Program.fs:line 156
Hint: If you just upgraded the fake-runner you can try to remove the .fake directory and try again.

I'm just starting with dotnet so I have no idea what I should do to be able to install it

nojaf · 2022-01-29T13:39:02Z

I think you would need a slight variation of $env:FAKE_SDK_RESOLVER_CUSTOM_DOTNET_PATH = "C:\Program Files\dotnet".
Your dotnet will be located somewhere else on Unix but setting the FAKE_SDK_RESOLVER_CUSTOM_DOTNET_PATH environment variable might work out for you.

yazeedobaid · 2022-01-30T14:31:37Z

Release 5.21.1 has been published,
Thanks @nojaf

JohnTheGr8 · 2022-01-30T15:42:43Z

Unfortunately, I am still encountering this issue in Fake 5.21.1.

In my case, I am running Fake using the dotnet/sdk:6.0.101 docker image. (This all worked fine up to Fake 5.20.4)

The error I am getting is this:

There was a problem while setting up the environment:
-> Could not find referenced assemblies in path: '/usr/bin/packs/Microsoft.NETCore.App.Ref/6.0.1/ref/net6.0', please check installed SDK and runtime versions

A workaround I've found, similar to what @nojaf initially recommended, is to set FAKE_SDK_RESOLVER_CUSTOM_DOTNET_PATH to /usr/share/dotnet. It works with both Fake 5.21.0 and 5.21.1.

(please let me know if I should open a new issue)

mattiasdrp · 2022-01-30T18:19:15Z

@nojaf My issue is that my dotnet version is 6.0.101 so the issue is that there is no /usr/local/share/dotnet/packs/Microsoft.NETCore.App.Ref/6.0.0/ref/net6.0. I do have /usr/local/share/dotnet/packs/Microsoft.NETCore.App.Ref/6.0.1/ref/net6.0/ but I don't know how to tell FAKE to look for it.

I tried adding the proper path in FAKE_SDK_RESOLVER_CUSTOM_DOTNET_PATH (with FAKE_SDK_RESOLVER_CUSTOM_DOTNET_PATH="/usr/share/dotnet:$FAKE_SDK_RESOLVER_CUSTOM_DOTNET_PATH") and even ln -s /usr/share/dotnet /usr/local/share/dotnet but it doesn't work.

yazeedobaid · 2022-01-31T09:58:41Z

@JohnTheGr8 & @mattiasdrp could you please open new issues with your case to keep track of it. Thanks

mattiasdrp · 2022-01-31T19:16:55Z

@yazeedobaid Done in #2648

nojaf mentioned this issue Jan 26, 2022

Update logic of resolving SdkReferenceAssemblies. #2639

Merged

yazeedobaid mentioned this issue Jan 30, 2022

Add Algorithm for Resolving DotNet host path to Fake.DotNet.Cli from SdkAssemblyResolver #2644

Open

yazeedobaid closed this as completed in #2639 Jan 30, 2022

mattiasdrp mentioned this issue Jan 31, 2022

There was a problem while setting up the environment #2648

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Reference assemblies are located in the wrong folder on Windows #2638

Reference assemblies are located in the wrong folder on Windows #2638

nojaf commented Jan 25, 2022

nojaf commented Jan 25, 2022

TheAngryByrd commented Jan 25, 2022

baronfel commented Jan 25, 2022

nojaf commented Jan 25, 2022

SteveGilham commented Jan 25, 2022

baronfel commented Jan 25, 2022

yazeedobaid commented Jan 25, 2022

yazeedobaid commented Jan 25, 2022

baronfel commented Jan 25, 2022

yazeedobaid commented Jan 25, 2022

baronfel commented Jan 25, 2022

nojaf commented Jan 25, 2022

mk185147 commented Jan 26, 2022

mattiasdrp commented Jan 29, 2022

nojaf commented Jan 29, 2022

yazeedobaid commented Jan 30, 2022

JohnTheGr8 commented Jan 30, 2022

mattiasdrp commented Jan 30, 2022

yazeedobaid commented Jan 31, 2022

mattiasdrp commented Jan 31, 2022

Reference assemblies are located in the wrong folder on Windows #2638

Reference assemblies are located in the wrong folder on Windows #2638

Comments

nojaf commented Jan 25, 2022

Description

Repro steps

Expected behavior

Actual behavior

Known workarounds

Related information

nojaf commented Jan 25, 2022

Workaround

TheAngryByrd commented Jan 25, 2022

baronfel commented Jan 25, 2022

nojaf commented Jan 25, 2022

SteveGilham commented Jan 25, 2022

baronfel commented Jan 25, 2022

yazeedobaid commented Jan 25, 2022

yazeedobaid commented Jan 25, 2022

baronfel commented Jan 25, 2022

yazeedobaid commented Jan 25, 2022

baronfel commented Jan 25, 2022

nojaf commented Jan 25, 2022

mk185147 commented Jan 26, 2022

mattiasdrp commented Jan 29, 2022

nojaf commented Jan 29, 2022

yazeedobaid commented Jan 30, 2022

JohnTheGr8 commented Jan 30, 2022

mattiasdrp commented Jan 30, 2022

yazeedobaid commented Jan 31, 2022

mattiasdrp commented Jan 31, 2022