Microsoft.DotNet.Arcade.Sdk only extracted on first use of per version of installed .NET

[TheJayMann]

Describe the Bug

When attempting to build .NET 8 Preview 4 using an existing build of .NET 8 Preview 3, as well as when attempting to build .NET 8 Preview 4 using the previous build of .NET 8 Preview 4, the first time the build is run, the Microsoft.DotNet.Arcade.Sdk is properly extracted to prereqs/packages/restored/ArcadeBootstrapPackage/microsoft.dotnet.arcade.sdk/, and the build continues. However, after this first build run, if the entire source directory is deleted, re-extracted, and the build is rerun, the init-build.proj project fails when attempting to import Sdk.props from Microsoft.NET.Sdk SDK, as nothing has been extracted to prereqs/packages/restored/ArcadeBootstrapPackage/microsoft.dotnet.arcade.sdk/. However, it appears that the process which extracts the package is one of the prerequisite targets of the target attempted to be run when building the init-build.proj. Thus, there appears to be a dependency cycle, where the packages needs to be already extracted in order to run the build target in init-build.prog which will then extract the package. I do not know why this works the first time a particular version of .NET is used to build. The first time I used .NET Preview 3 to build, the package was properly extracted, every time after when starting with a clean source, it failed to extract. After this, the first time I use .NET Preview 4 to build it, it extracts the package, but every time afterwards, it fails to extract.

Steps to Reproduce

I have not gone through the process of reproducing the issue, outside of seeing identical behavior when using .NET Preview 3 and .NET Preview 4. But the following are steps I've taken to get to this point.

• Download and extract the published source code asset from the Preview 3 release on the VMR
• Use the prep.sh and build.sh scripts to make a bootstrap build
• Extract the produced SDK to ~/dotnet, and save the source build archives
• Download and extract the built in source code asset from the Preview 4 release on the VMR, and download the release manifest in the extracted folder as release.json
• Copy or link the previous source build archives from Preview 3 into prereqs/packages/archive/
• Build using ./build.sh --clean-while-building --with-sdk ~/dotnet/ --release-manifest release.json -- /p:SkipPortableRuntimeBuild=true
• When the build is cancelled or stops due to an error (I've not had the chance to see what happens when the build completes successfully), delete the entire directory created when extracting the source code, extract the source code again, and add in the release manifest
• Attempt to build using the same build command again

Other Information

The error message when attempting to build after the first attempt is as follows.

"~/src/dotnet/dotnet-8.0.0-preview.4.23259.5/eng/tools/init-build.proj" (PrepareOfflineLocalTools target) (1) ->
  ~/src/dotnet/dotnet-8.0.0-preview.4.23259.5/eng/tools/init-build.proj(2,31): error MSB4242: SDK Resolver Failure: "The SDK resolver "SourceBuiltSdkResolver" failed while attempting to resolve the SDK "Microsoft.NET.Sdk". Exception: "System.IO.DirectoryNotFoundException: Could not find a part of the path '~/src/dotnet/dotnet-8.0.0-preview.4.23259.5/prereqs/packages/restored/ArcadeBootstrapPackage/microsoft.dotnet.arcade.sdk/8.0.0-beta.23168.1'.
  ~/src/dotnet/dotnet-8.0.0-preview.4.23259.5/eng/tools/init-build.proj(2,31): error MSB4242:    at System.IO.Enumeration.FileSystemEnumerator`1.CreateDirectoryHandle(String path, Boolean ignoreNotFound)
  ~/src/dotnet/dotnet-8.0.0-preview.4.23259.5/eng/tools/init-build.proj(2,31): error MSB4242:    at System.IO.Enumeration.FileSystemEnumerator`1.Init()
  ~/src/dotnet/dotnet-8.0.0-preview.4.23259.5/eng/tools/init-build.proj(2,31): error MSB4242:    at System.IO.Enumeration.FileSystemEnumerable`1..ctor(String directory, FindTransform transform, EnumerationOptions options, Boolean isNormalized)
  ~/src/dotnet/dotnet-8.0.0-preview.4.23259.5/eng/tools/init-build.proj(2,31): error MSB4242:    at System.IO.Enumeration.FileSystemEnumerableFactory.UserDirectories(String directory, String expression, EnumerationOptions options)
  ~/src/dotnet/dotnet-8.0.0-preview.4.23259.5/eng/tools/init-build.proj(2,31): error MSB4242:    at System.IO.Directory.InternalEnumeratePaths(String path, String searchPattern, SearchTarget searchTarget, EnumerationOptions options)
  ~/src/dotnet/dotnet-8.0.0-preview.4.23259.5/eng/tools/init-build.proj(2,31): error MSB4242:    at Microsoft.DotNet.SourceBuild.Tasks.SourceBuiltSdkResolver.SourceBuiltSdkOverride.Create(DictionaryEntry entry)
  ~/src/dotnet/dotnet-8.0.0-preview.4.23259.5/eng/tools/init-build.proj(2,31): error MSB4242:    at System.Linq.Enumerable.SelectEnumerableIterator`2.MoveNext()
  ~/src/dotnet/dotnet-8.0.0-preview.4.23259.5/eng/tools/init-build.proj(2,31): error MSB4242:    at System.Linq.Enumerable.WhereEnumerableIterator`1.ToArray()
  ~/src/dotnet/dotnet-8.0.0-preview.4.23259.5/eng/tools/init-build.proj(2,31): error MSB4242:    at Microsoft.DotNet.SourceBuild.Tasks.SourceBuiltSdkResolver.Resolve(SdkReference sdkReference, SdkResolverContext resolverContext, SdkResultFactory factory)
  ~/src/dotnet/dotnet-8.0.0-preview.4.23259.5/eng/tools/init-build.proj(2,31): error MSB4242:    at Microsoft.Build.BackEnd.SdkResolution.SdkResolverService.TryResolveSdkUsingSpecifiedResolvers(IList`1 resolvers, Int32 submissionId, SdkReference sdk, LoggingContext loggingContext, ElementLocation sdkReferenceLocation, String solutionPath, String projectPath, Boolean interactive, Boolean isRunningInVisualStudio, SdkResult& sdkResult, IEnumerable`1& errors, IEnumerable`1& warnings)""

I'm uncertain of why this works the first attempt to build, but doesn't on subsequent attempts. I'm assuming some setting is being set somewhere after the first run which makes further attempts fail, and that the setting is scoped per version, but I've attempted to delete various files and directories that I would believe such a setting could exist to no effect.

As a workaround, I've added to my build script wrapper that it extracts the package from the previous source build archive and then extract the contents of the package into the appropriate folder before running build.sh.

[dotnet-issue-labeler]

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

[MichaelSimons]

This appears related to #1532.

[TheJayMann]

Upon deleting the SourceBuild.MSBuildSdkResolver directory from SdkResolvers and attempting a new clean build, it appears to work properly. Thus, this does not appear to be just related, but rather, a duplicate.

A workaround mentioned was to copy the dotnet installation folder locally as .dotnet, but I might try to see if it is simple enough to set up an overlayfs mount with the upper in tmpfs.