Intermittent problem, nugetorgadd.trk being used by another process

[cclauson]

Currently trying to troubleshoot an issue which is intermittently happening on our continuous integration server. I can't consistently repro, but our build fails with the following error:
error : C:\Program Files\dotnet\sdk\2.1.401\Sdks\MSBuild.Sdk.Extras\Sdk not found. Check that a recent enough .NET Core SDK is installed and/or increase the version specified in global.json.
shortly after this warning:
The SDK resolver "NuGetSdkResolver" failed to run. The process cannot access the file 'C:\Users\cloud-user\AppData\Roaming\NuGet\nugetorgadd.trk' because it is being used by another process.

Discovered where in the nuget code for this lives, also posted on NuGet github forum:
NuGet/Home#7503

Was wondering if anyone here might have any ideas, though (e.g., is it possible this code is being executed simultaneously on two different MSBuild threads, why would that happen?). I can run this build using MSBuild in a debugger and set breakpoints if it would help, but I can't repro the above problem this way.

Thanks so much

[rainersigwald]

I think NuGet is the right home for this bug. MSBuild has code to run resolvers only once in a build, so I suspect the problem is sharing that file between builds, or something maintaining a long-lived lock on the file. If NuGet's investigation shows that they're not at fault we can reopen this.

[cclauson]

@rainersigwald
Would a resolver be invoked again if MSBuild launches another build using the <msbuild ... /> task? That's happening quite a bit with the target in question, which is the target we see this happening with, not any others.

[rainersigwald]

@jeffkl ☝ pretty sure the answer is "yes, that's totally fine" right?

[jeffkl]

The resolvers are invoked in order every time a project file is evaluated and it comes across an SDK that hasn't been resolved yet. The results are cached in the main node process, and <MSBuild /> task invocations re-use the cache and resolvers. It would appear there's a bug in NuGet's logic to update its "track" file:

https://github.com/NuGet/NuGet.Client/blob/c93928b4a2d7b90ed7a1d6f8bd5f9cbaa2c7b2ef/src/NuGet.Core/NuGet.Configuration/Settings/Settings.cs#L455

[cclauson]

We're thinking right now that if we just place an empty track file there, it will be a short term solution. In any case, maybe I can change the package reference in msbuild to a project reference and debug the whole thing end to end, and see if the track file code is being hit multiple times.

[johncrim]

This issue is pretty active as of .NET Core 6.0.200. See NuGet/Home#7503 for more info.

[64J0]

It is happening for me! Anyone knows some workaround for it?