Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

vstest: build for .NET 5 #1834

Merged
merged 9 commits into from
Nov 6, 2020
Merged

vstest: build for .NET 5 #1834

merged 9 commits into from
Nov 6, 2020

Conversation

tmds
Copy link
Member

@tmds tmds commented Oct 23, 2020

attempt to eliminate .NET 2.1 host packages used by vstest.

cc @dagood @crummel @dseefeld @omajid

@tmds
Copy link
Member Author

tmds commented Oct 26, 2020

vstest compilation failed:

2020-10-23T14:06:07.9734529Z   /src/artifacts/src/vstest.0c0fafa64de5c73981e7c40a99d7363f57635f60/src/vstest.console/CommandLine/CommandLineOptions.cs(305,17): error CA2200: Re-throwing caught exception changes stack information [/src/artifacts/src/vstest.0c0fafa64de5c73981e7c40a99d7363f57635f60/src/vstest.console/vstest.console.csproj]

@tmds
Copy link
Member Author

tmds commented Oct 26, 2020

CI now fails with:

2020-10-26T14:37:16.5805711Z /src/repos/Directory.Build.targets(553,5): error : 53 new packages used not in baseline! See report at /src/artifacts/prebuilt-report/baseline-comparison.xml for more information. Package IDs are: [/src/repos/known-good.proj]
2020-10-26T14:37:16.5807507Z /src/repos/Directory.Build.targets(553,5): error : Libuv.1.9.1 [/src/repos/known-good.proj]
2020-10-26T14:37:16.5808524Z /src/repos/Directory.Build.targets(553,5): error : Microsoft.CodeAnalysis.Analyzers.1.1.0 [/src/repos/known-good.proj]
2020-10-26T14:37:16.5809532Z /src/repos/Directory.Build.targets(553,5): error : Microsoft.CodeAnalysis.Common.1.3.0 [/src/repos/known-good.proj]
2020-10-26T14:37:16.5810514Z /src/repos/Directory.Build.targets(553,5): error : Microsoft.CodeAnalysis.CSharp.1.3.0 [/src/repos/known-good.proj]
2020-10-26T14:37:16.5811453Z /src/repos/Directory.Build.targets(553,5): error : Microsoft.CodeAnalysis.VisualBasic.1.3.0 [/src/repos/known-good.proj]
2020-10-26T14:37:16.5812400Z /src/repos/Directory.Build.targets(553,5): error : Microsoft.NETCore.App.1.0.5 [/src/repos/known-good.proj]
2020-10-26T14:37:16.5813322Z /src/repos/Directory.Build.targets(553,5): error : Microsoft.NETCore.DotNetHost.1.0.1 [/src/repos/known-good.proj]
2020-10-26T14:37:16.5814285Z /src/repos/Directory.Build.targets(553,5): error : Microsoft.NETCore.DotNetHostPolicy.1.0.5 [/src/repos/known-good.proj]
2020-10-26T14:37:16.5815231Z /src/repos/Directory.Build.targets(553,5): error : Microsoft.NETCore.DotNetHostResolver.1.0.1 [/src/repos/known-good.proj]
2020-10-26T14:37:16.5816172Z /src/repos/Directory.Build.targets(553,5): error : Microsoft.NETCore.Jit.1.0.7 [/src/repos/known-good.proj]
2020-10-26T14:37:16.5817081Z /src/repos/Directory.Build.targets(553,5): error : Microsoft.NETCore.Platforms.1.0.2 [/src/repos/known-good.proj]
2020-10-26T14:37:16.5833023Z /src/repos/Directory.Build.targets(553,5): error : Microsoft.NETCore.Runtime.CoreCLR.1.0.7 [/src/repos/known-good.proj]
2020-10-26T14:37:16.5833994Z /src/repos/Directory.Build.targets(553,5): error : Microsoft.NETCore.Targets.1.0.3 [/src/repos/known-good.proj]
2020-10-26T14:37:16.5834972Z /src/repos/Directory.Build.targets(553,5): error : Microsoft.NETCore.Windows.ApiSets.1.0.1 [/src/repos/known-good.proj]
2020-10-26T14:37:16.5835962Z /src/repos/Directory.Build.targets(553,5): error : Microsoft.VisualBasic.10.0.1 [/src/repos/known-good.proj]
2020-10-26T14:37:16.5836951Z /src/repos/Directory.Build.targets(553,5): error : runtime.placeholder-rid.System.Collections.4.0.11 [/src/repos/known-good.proj]
2020-10-26T14:37:16.5837939Z /src/repos/Directory.Build.targets(553,5): error : runtime.placeholder-rid.System.Diagnostics.Tools.4.0.1 [/src/repos/known-good.proj]
2020-10-26T14:37:16.5838961Z /src/repos/Directory.Build.targets(553,5): error : runtime.placeholder-rid.System.Globalization.4.0.11 [/src/repos/known-good.proj]
2020-10-26T14:37:16.5841319Z /src/repos/Directory.Build.targets(553,5): error : runtime.placeholder-rid.System.IO.4.1.0 [/src/repos/known-good.proj]
2020-10-26T14:37:16.5842572Z /src/repos/Directory.Build.targets(553,5): error : runtime.placeholder-rid.System.Reflection.4.1.0 [/src/repos/known-good.proj]
2020-10-26T14:37:16.5844009Z /src/repos/Directory.Build.targets(553,5): error : runtime.placeholder-rid.System.Reflection.Extensions.4.0.1 [/src/repos/known-good.proj]
2020-10-26T14:37:16.5845147Z /src/repos/Directory.Build.targets(553,5): error : runtime.placeholder-rid.System.Reflection.Primitives.4.0.1 [/src/repos/known-good.proj]
2020-10-26T14:37:16.5846281Z /src/repos/Directory.Build.targets(553,5): error : runtime.placeholder-rid.System.Resources.ResourceManager.4.0.1 [/src/repos/known-good.proj]
2020-10-26T14:37:16.5847360Z /src/repos/Directory.Build.targets(553,5): error : runtime.placeholder-rid.System.Runtime.4.1.0 [/src/repos/known-good.proj]
2020-10-26T14:37:16.5848657Z /src/repos/Directory.Build.targets(553,5): error : runtime.placeholder-rid.System.Runtime.Handles.4.0.1 [/src/repos/known-good.proj]
2020-10-26T14:37:16.5849774Z /src/repos/Directory.Build.targets(553,5): error : runtime.placeholder-rid.System.Runtime.InteropServices.4.1.0 [/src/repos/known-good.proj]
2020-10-26T14:37:16.5850901Z /src/repos/Directory.Build.targets(553,5): error : runtime.placeholder-rid.System.Text.Encoding.4.0.11 [/src/repos/known-good.proj]
2020-10-26T14:37:16.5852013Z /src/repos/Directory.Build.targets(553,5): error : runtime.placeholder-rid.System.Text.Encoding.Extensions.4.0.11 [/src/repos/known-good.proj]
2020-10-26T14:37:16.5853133Z /src/repos/Directory.Build.targets(553,5): error : runtime.placeholder-rid.System.Threading.Tasks.4.0.11 [/src/repos/known-good.proj]
2020-10-26T14:37:16.5854442Z /src/repos/Directory.Build.targets(553,5): error : runtime.native.System.Net.Security.4.0.1 [/src/repos/known-good.proj]
2020-10-26T14:37:16.5855454Z /src/repos/Directory.Build.targets(553,5): error : runtime.native.System.Security.Cryptography.4.0.1 [/src/repos/known-good.proj]
2020-10-26T14:37:16.5856467Z /src/repos/Directory.Build.targets(553,5): error : runtime.placeholder-rid.System.Diagnostics.Debug.4.0.11 [/src/repos/known-good.proj]
2020-10-26T14:37:16.5858064Z /src/repos/Directory.Build.targets(553,5): error : runtime.placeholder-rid.System.IO.FileSystem.4.0.1 [/src/repos/known-good.proj]
2020-10-26T14:37:16.5860908Z /src/repos/Directory.Build.targets(553,5): error : runtime.placeholder-rid.System.Runtime.Extensions.4.1.0 [/src/repos/known-good.proj]
2020-10-26T14:37:16.5862026Z /src/repos/Directory.Build.targets(553,5): error : runtime.placeholder-rid.System.Private.Uri.4.0.1 [/src/repos/known-good.proj]
2020-10-26T14:37:16.5862992Z /src/repos/Directory.Build.targets(553,5): error : System.ComponentModel.4.0.1 [/src/repos/known-good.proj]
2020-10-26T14:37:16.5864211Z /src/repos/Directory.Build.targets(553,5): error : System.ComponentModel.Annotations.4.1.0 [/src/repos/known-good.proj]
2020-10-26T14:37:16.5865365Z /src/repos/Directory.Build.targets(553,5): error : System.Diagnostics.DiagnosticSource.4.0.0 [/src/repos/known-good.proj]
2020-10-26T14:37:16.5866687Z /src/repos/Directory.Build.targets(553,5): error : System.Diagnostics.StackTrace.4.0.1 [/src/repos/known-good.proj]
2020-10-26T14:37:16.5867610Z /src/repos/Directory.Build.targets(553,5): error : System.IO.FileSystem.Watcher.4.0.0 [/src/repos/known-good.proj]
2020-10-26T14:37:16.5868888Z /src/repos/Directory.Build.targets(553,5): error : System.IO.MemoryMappedFiles.4.0.0 [/src/repos/known-good.proj]
2020-10-26T14:37:16.5870163Z /src/repos/Directory.Build.targets(553,5): error : System.IO.UnmanagedMemoryStream.4.0.1 [/src/repos/known-good.proj]
2020-10-26T14:37:16.5871202Z /src/repos/Directory.Build.targets(553,5): error : System.Linq.Expressions.4.1.1 [/src/repos/known-good.proj]
2020-10-26T14:37:16.5872187Z /src/repos/Directory.Build.targets(553,5): error : System.Linq.Queryable.4.0.1 [/src/repos/known-good.proj]
2020-10-26T14:37:16.5873467Z /src/repos/Directory.Build.targets(553,5): error : System.Net.Http.4.1.2 [/src/repos/known-good.proj]
2020-10-26T14:37:16.5874382Z /src/repos/Directory.Build.targets(553,5): error : System.Net.NameResolution.4.0.0 [/src/repos/known-good.proj]
2020-10-26T14:37:16.5875505Z /src/repos/Directory.Build.targets(553,5): error : System.Net.Security.4.0.1 [/src/repos/known-good.proj]
2020-10-26T14:37:16.5876425Z /src/repos/Directory.Build.targets(553,5): error : System.Numerics.Vectors.4.1.1 [/src/repos/known-good.proj]
2020-10-26T14:37:16.5877352Z /src/repos/Directory.Build.targets(553,5): error : System.Private.Uri.4.0.1 [/src/repos/known-good.proj]
2020-10-26T14:37:16.5878282Z /src/repos/Directory.Build.targets(553,5): error : System.Reflection.DispatchProxy.4.0.1 [/src/repos/known-good.proj]
2020-10-26T14:37:16.5879253Z /src/repos/Directory.Build.targets(553,5): error : System.Threading.Overlapped.4.0.1 [/src/repos/known-good.proj]
2020-10-26T14:37:16.5880380Z /src/repos/Directory.Build.targets(553,5): error : System.Threading.Tasks.Parallel.4.0.1 [/src/repos/known-good.proj]
2020-10-26T14:37:16.5881376Z /src/repos/Directory.Build.targets(553,5): error : System.Xml.XPath.XDocument.4.0.1 [/src/repos/known-good.proj]
2020-10-26T14:37:16.6155021Z   89 packages in baseline weren't used!
2020-10-26T14:37:16.6162928Z   498 packages used as expected in the baseline.
2020-10-26T14:37:16.6347826Z /src/repos/Directory.Build.targets(553,5): warning : Prebuilt usages are different from the baseline. If detected changes are acceptable, update baseline with: [/src/repos/known-good.proj]
2020-10-26T14:37:16.6349502Z /src/repos/Directory.Build.targets(553,5): warning : cp '/src/artifacts/prebuilt-report/generated-new-baseline.xml' '/src/tools-local/prebuilt-baseline-online.xml' [/src/repos/known-good.proj]
2020-10-26T14:37:16.6456113Z 
2020-10-26T14:37:16.6457107Z Build FAILED.
2020-10-26T14:37:16.6457430Z 
2020-10-26T14:37:16.6459224Z /src/packages/restored/microsoft.dotnet.arcade.sdk/5.0.0-beta.20426.4/tools/ProjectLayout.props(8,3): warning MSB4011: "/src/packages/restored/microsoft.dotnet.arcade.sdk/5.0.0-beta.20426.4/tools/RepoLayout.props" cannot be imported again. It was already imported at "/src/packages/restored/microsoft.dotnet.arcade.sdk/5.0.0-beta.20426.4/tools/Build.proj (49,3)". This is most likely a build authoring error. This subsequent import will be ignored. 
2020-10-26T14:37:16.6461475Z /src/tools-local/tasks/Microsoft.DotNet.SourceBuild.Tasks.XPlat/DownloadFileSB.cs(397,35): warning CS0108: 'DownloadFileSB.CustomHttpRequestException.StatusCode' hides inherited member 'HttpRequestException.StatusCode'. Use the new keyword if hiding was intended. [/src/tools-local/tasks/Microsoft.DotNet.SourceBuild.Tasks.XPlat/Microsoft.DotNet.SourceBuild.Tasks.XPlat.csproj]
2020-10-26T14:37:16.6462430Z EXEC : warning : eng/build.sh has type 100755, expected 100644 [/src/repos/fsharp.proj]
2020-10-26T14:37:16.6463845Z /src/repos/Directory.Build.targets(553,5): warning : Prebuilt usages are different from the baseline. If detected changes are acceptable, update baseline with: [/src/repos/known-good.proj]
2020-10-26T14:37:16.6465361Z /src/repos/Directory.Build.targets(553,5): warning : cp '/src/artifacts/prebuilt-report/generated-new-baseline.xml' '/src/tools-local/prebuilt-baseline-online.xml' [/src/repos/known-good.proj]
2020-10-26T14:37:16.6466799Z /src/repos/Directory.Build.targets(553,5): error : 53 new packages used not in baseline! See report at /src/artifacts/prebuilt-report/baseline-comparison.xml for more information. Package IDs are: [/src/repos/known-good.proj]
2020-10-26T14:37:16.6472337Z /src/repos/Directory.Build.targets(553,5): error : Libuv.1.9.1 [/src/repos/known-good.proj]
2020-10-26T14:37:16.6478040Z /src/repos/Directory.Build.targets(553,5): error : Microsoft.CodeAnalysis.Analyzers.1.1.0 [/src/repos/known-good.proj]
2020-10-26T14:37:16.6480235Z /src/repos/Directory.Build.targets(553,5): error : Microsoft.CodeAnalysis.Common.1.3.0 [/src/repos/known-good.proj]
2020-10-26T14:37:16.6481792Z /src/repos/Directory.Build.targets(553,5): error : Microsoft.CodeAnalysis.CSharp.1.3.0 [/src/repos/known-good.proj]
2020-10-26T14:37:16.6483272Z /src/repos/Directory.Build.targets(553,5): error : Microsoft.CodeAnalysis.VisualBasic.1.3.0 [/src/repos/known-good.proj]
2020-10-26T14:37:16.6509780Z /src/repos/Directory.Build.targets(553,5): error : Microsoft.NETCore.App.1.0.5 [/src/repos/known-good.proj]
2020-10-26T14:37:16.6511312Z /src/repos/Directory.Build.targets(553,5): error : Microsoft.NETCore.DotNetHost.1.0.1 [/src/repos/known-good.proj]
2020-10-26T14:37:16.6512314Z /src/repos/Directory.Build.targets(553,5): error : Microsoft.NETCore.DotNetHostPolicy.1.0.5 [/src/repos/known-good.proj]
2020-10-26T14:37:16.6515597Z /src/repos/Directory.Build.targets(553,5): error : Microsoft.NETCore.DotNetHostResolver.1.0.1 [/src/repos/known-good.proj]
2020-10-26T14:37:16.6516693Z /src/repos/Directory.Build.targets(553,5): error : Microsoft.NETCore.Jit.1.0.7 [/src/repos/known-good.proj]
2020-10-26T14:37:16.6518107Z /src/repos/Directory.Build.targets(553,5): error : Microsoft.NETCore.Platforms.1.0.2 [/src/repos/known-good.proj]
2020-10-26T14:37:16.6519255Z /src/repos/Directory.Build.targets(553,5): error : Microsoft.NETCore.Runtime.CoreCLR.1.0.7 [/src/repos/known-good.proj]
2020-10-26T14:37:16.6520252Z /src/repos/Directory.Build.targets(553,5): error : Microsoft.NETCore.Targets.1.0.3 [/src/repos/known-good.proj]
2020-10-26T14:37:16.6521221Z /src/repos/Directory.Build.targets(553,5): error : Microsoft.NETCore.Windows.ApiSets.1.0.1 [/src/repos/known-good.proj]
2020-10-26T14:37:16.6522214Z /src/repos/Directory.Build.targets(553,5): error : Microsoft.VisualBasic.10.0.1 [/src/repos/known-good.proj]
2020-10-26T14:37:16.6523209Z /src/repos/Directory.Build.targets(553,5): error : runtime.placeholder-rid.System.Collections.4.0.11 [/src/repos/known-good.proj]
2020-10-26T14:37:16.6524273Z /src/repos/Directory.Build.targets(553,5): error : runtime.placeholder-rid.System.Diagnostics.Tools.4.0.1 [/src/repos/known-good.proj]
2020-10-26T14:37:16.6525318Z /src/repos/Directory.Build.targets(553,5): error : runtime.placeholder-rid.System.Globalization.4.0.11 [/src/repos/known-good.proj]
2020-10-26T14:37:16.6526381Z /src/repos/Directory.Build.targets(553,5): error : runtime.placeholder-rid.System.IO.4.1.0 [/src/repos/known-good.proj]
2020-10-26T14:37:16.6527400Z /src/repos/Directory.Build.targets(553,5): error : runtime.placeholder-rid.System.Reflection.4.1.0 [/src/repos/known-good.proj]
2020-10-26T14:37:16.6528469Z /src/repos/Directory.Build.targets(553,5): error : runtime.placeholder-rid.System.Reflection.Extensions.4.0.1 [/src/repos/known-good.proj]
2020-10-26T14:37:16.6529529Z /src/repos/Directory.Build.targets(553,5): error : runtime.placeholder-rid.System.Reflection.Primitives.4.0.1 [/src/repos/known-good.proj]
2020-10-26T14:37:16.6530890Z /src/repos/Directory.Build.targets(553,5): error : runtime.placeholder-rid.System.Resources.ResourceManager.4.0.1 [/src/repos/known-good.proj]
2020-10-26T14:37:16.6531917Z /src/repos/Directory.Build.targets(553,5): error : runtime.placeholder-rid.System.Runtime.4.1.0 [/src/repos/known-good.proj]
2020-10-26T14:37:16.6532939Z /src/repos/Directory.Build.targets(553,5): error : runtime.placeholder-rid.System.Runtime.Handles.4.0.1 [/src/repos/known-good.proj]
2020-10-26T14:37:16.6533992Z /src/repos/Directory.Build.targets(553,5): error : runtime.placeholder-rid.System.Runtime.InteropServices.4.1.0 [/src/repos/known-good.proj]
2020-10-26T14:37:16.6535009Z /src/repos/Directory.Build.targets(553,5): error : runtime.placeholder-rid.System.Text.Encoding.4.0.11 [/src/repos/known-good.proj]
2020-10-26T14:37:16.6536048Z /src/repos/Directory.Build.targets(553,5): error : runtime.placeholder-rid.System.Text.Encoding.Extensions.4.0.11 [/src/repos/known-good.proj]
2020-10-26T14:37:16.6537070Z /src/repos/Directory.Build.targets(553,5): error : runtime.placeholder-rid.System.Threading.Tasks.4.0.11 [/src/repos/known-good.proj]
2020-10-26T14:37:16.6538744Z /src/repos/Directory.Build.targets(553,5): error : runtime.native.System.Net.Security.4.0.1 [/src/repos/known-good.proj]
2020-10-26T14:37:16.6539977Z /src/repos/Directory.Build.targets(553,5): error : runtime.native.System.Security.Cryptography.4.0.1 [/src/repos/known-good.proj]
2020-10-26T14:37:16.6541026Z /src/repos/Directory.Build.targets(553,5): error : runtime.placeholder-rid.System.Diagnostics.Debug.4.0.11 [/src/repos/known-good.proj]
2020-10-26T14:37:16.6542213Z /src/repos/Directory.Build.targets(553,5): error : runtime.placeholder-rid.System.IO.FileSystem.4.0.1 [/src/repos/known-good.proj]
2020-10-26T14:37:16.6543249Z /src/repos/Directory.Build.targets(553,5): error : runtime.placeholder-rid.System.Runtime.Extensions.4.1.0 [/src/repos/known-good.proj]
2020-10-26T14:37:16.6544238Z /src/repos/Directory.Build.targets(553,5): error : runtime.placeholder-rid.System.Private.Uri.4.0.1 [/src/repos/known-good.proj]
2020-10-26T14:37:16.6545231Z /src/repos/Directory.Build.targets(553,5): error : System.ComponentModel.4.0.1 [/src/repos/known-good.proj]
2020-10-26T14:37:16.6546372Z /src/repos/Directory.Build.targets(553,5): error : System.ComponentModel.Annotations.4.1.0 [/src/repos/known-good.proj]
2020-10-26T14:37:16.6547369Z /src/repos/Directory.Build.targets(553,5): error : System.Diagnostics.DiagnosticSource.4.0.0 [/src/repos/known-good.proj]
2020-10-26T14:37:16.6548318Z /src/repos/Directory.Build.targets(553,5): error : System.Diagnostics.StackTrace.4.0.1 [/src/repos/known-good.proj]
2020-10-26T14:37:16.6549553Z /src/repos/Directory.Build.targets(553,5): error : System.IO.FileSystem.Watcher.4.0.0 [/src/repos/known-good.proj]
2020-10-26T14:37:16.6550547Z /src/repos/Directory.Build.targets(553,5): error : System.IO.MemoryMappedFiles.4.0.0 [/src/repos/known-good.proj]
2020-10-26T14:37:16.6551710Z /src/repos/Directory.Build.targets(553,5): error : System.IO.UnmanagedMemoryStream.4.0.1 [/src/repos/known-good.proj]
2020-10-26T14:37:16.6552974Z /src/repos/Directory.Build.targets(553,5): error : System.Linq.Expressions.4.1.1 [/src/repos/known-good.proj]
2020-10-26T14:37:16.6553902Z /src/repos/Directory.Build.targets(553,5): error : System.Linq.Queryable.4.0.1 [/src/repos/known-good.proj]
2020-10-26T14:37:16.6554765Z /src/repos/Directory.Build.targets(553,5): error : System.Net.Http.4.1.2 [/src/repos/known-good.proj]
2020-10-26T14:37:16.6555668Z /src/repos/Directory.Build.targets(553,5): error : System.Net.NameResolution.4.0.0 [/src/repos/known-good.proj]
2020-10-26T14:37:16.6556548Z /src/repos/Directory.Build.targets(553,5): error : System.Net.Security.4.0.1 [/src/repos/known-good.proj]
2020-10-26T14:37:16.6557446Z /src/repos/Directory.Build.targets(553,5): error : System.Numerics.Vectors.4.1.1 [/src/repos/known-good.proj]
2020-10-26T14:37:16.6558314Z /src/repos/Directory.Build.targets(553,5): error : System.Private.Uri.4.0.1 [/src/repos/known-good.proj]
2020-10-26T14:37:16.6559231Z /src/repos/Directory.Build.targets(553,5): error : System.Reflection.DispatchProxy.4.0.1 [/src/repos/known-good.proj]
2020-10-26T14:37:16.6560525Z /src/repos/Directory.Build.targets(553,5): error : System.Threading.Overlapped.4.0.1 [/src/repos/known-good.proj]
2020-10-26T14:37:16.6561461Z /src/repos/Directory.Build.targets(553,5): error : System.Threading.Tasks.Parallel.4.0.1 [/src/repos/known-good.proj]
2020-10-26T14:37:16.6562754Z /src/repos/Directory.Build.targets(553,5): error : System.Xml.XPath.XDocument.4.0.1 [/src/repos/known-good.proj]
2020-10-26T14:37:16.6563615Z     4 Warning(s)
2020-10-26T14:37:16.6564111Z     1 Error(s)

@tmds
Copy link
Member Author

tmds commented Oct 27, 2020

CI now fails with:

@dagood @dseefeld I guess the failure may indicating some prebuilts have been removed/changed? What should I do about it?

@dagood
Copy link
Member

dagood commented Oct 27, 2020

When you get new prebuilts, you either fix them, or add them to the baseline to defer fixing them by copying the generated baseline over the corresponding checked-in baseline.

But there's another pressing problem here, smoke-test is failing (in the tarball builds, the prod build doesn't fail on baseline failure, so you can see both issues):

      running test
  /src/testing-smoke/builtCli/sdk/5.0.100-rc.1.20452.10/MSBuild.dll -nologo -maxcpucount -nodereuse:false -restore -target:VSTest -verbosity:m /bl:/src/testing-smoke/C#_xunit_local_nohttps_test.binlog ./C#_xunit.csproj
    Determining projects to restore...
    All projects are up-to-date for restore.
    You are using a preview version of .NET. See: https://aka.ms/dotnet-core-preview
    C#_xunit -> /src/testing-smoke/C#_xunit/bin/Debug/net5.0/C#_xunit.dll
  Test run for /src/testing-smoke/C#_xunit/bin/Debug/net5.0/C#_xunit.dll (.NETCoreApp,Version=v5.0)
  The application to execute does not exist: '/src/testing-smoke/builtCli/sdk/5.0.100-rc.1.20452.10/vstest.console.dll'
/src/testing-smoke/builtCli/sdk/5.0.100-rc.1.20452.10/Microsoft.TestPlatform.targets(32,5): error MSB4181: The "Microsoft.TestPlatform.Build.Tasks.VSTestTask" task returned false but did not log an error. [/src/testing-smoke/C#_xunit/C#_xunit.csproj] [/src/build.proj]

Binlogs: testing-smoke-binlogs-1834.zip

Failed task:

Microsoft.TestPlatform.Build.Tasks.VSTestTask
    Assembly = /src/testing-smoke/builtCli/sdk/5.0.100-rc.1.20452.10/Microsoft.TestPlatform.Build.dll
    Parameters
        TestFileFullPath = /src/testing-smoke/C#_xunit/bin/Debug/net5.0/C#_xunit.dll
        VSTestTestAdapterPath = /src/testing-smoke/packages/coverlet.collector/1.3.0/build/netstandard1.0/
        VSTestFramework = .NETCoreApp,Version=v5.0
        VSTestConsolePath = /src/testing-smoke/builtCli/sdk/5.0.100-rc.1.20452.10/vstest.console.dll
        VSTestTraceDataCollectorDirectoryPath = /src/testing-smoke/packages/microsoft.codecoverage/16.7.1/build/netstandard1.0/
    Errors
        /src/testing-smoke/builtCli/sdk/5.0.100-rc.1.20452.10/Microsoft.TestPlatform.targets(32,5): error MSB4181: The "Microsoft.TestPlatform.Build.Tasks.VSTestTask" task returned false but did not log an error. [/src/testing-smoke/C#_xunit/C#_xunit.csproj]

@crummel
Copy link
Contributor

crummel commented Oct 27, 2020

I'm taking a look at this now. DanS suggested we might have better luck adding the new Microsoft.NETCore.App version to source-build-reference-packages.

@dagood
Copy link
Member

dagood commented Oct 27, 2020

I think this is the key error (in order to continue switching over to net5.0:
The application to execute does not exist: '/src/testing-smoke/builtCli/sdk/5.0.100-rc.1.20452.10/vstest.console.dll'

Maybe the place to look is how exactly vstest.console.dll is being placed into the SDK dir normally. Maybe some repo downstream of vstest assumes that vstest built as netcoreapp2.1, and now that repo fails to find the net5.0 DLL. It might be package content resolver with the TFM not matching (something running equivalent to dotnet publish -f netcoreapp2.1), or maybe just an MSBuild wildcard looking at package contents directly.

@tmds
Copy link
Member Author

tmds commented Oct 28, 2020

CI fail when building installer:

2020-10-28T14:52:22.4596891Z     Restored /src/artifacts/src/installer.473d1b592e8c281f03dcc4024c04c7c188e99c03/src/redist/targets/DownloadPackage.csproj (in 3 ms).
2020-10-28T14:52:22.4599538Z   /src/artifacts/src/installer.473d1b592e8c281f03dcc4024c04c7c188e99c03/src/redist/targets/Crossgen.targets(137,5): error : Error compiling /src/artifacts/src/installer.473d1b592e8c281f03dcc4024c04c7c188e99c03/artifacts/bin/redist/Release/dotnet/sdk/5.0.100-rc.1.20452.10/net5.0/ref/Microsoft.TestPlatform.PlatformAbstractions.dll: Reference assemblies should not be loaded for execution.  They can only be loaded in the Reflection-only loader context. (0x80131058) [/src/artifacts/src/installer.473d1b592e8c281f03dcc4024c04c7c188e99c03/src/redist/redist.csproj]
2020-10-28T14:52:22.4687653Z ##[error]/src/artifacts/src/installer.473d1b592e8c281f03dcc4024c04c7c188e99c03/src/redist/targets/Crossgen.targets(137,5): error : (NETCORE_ENGINEERING_TELEMETRY=Build) Error compiling 

@dagood
Copy link
Member

dagood commented Oct 28, 2020

Hmm. The node from the binlog (Build (31).zip):

Crossgen
    Assembly = /src/artifacts/src/installer.473d1b592e8c281f03dcc4024c04c7c188e99c03/artifacts/tasks/bin/core-sdk-tasks/Release/netcoreapp5.0/core-sdk-tasks.dll
    CommandLineArguments = > /src/packages/restored//microsoft.netcore.app.runtime.centos.7-x64/5.0.0-rc.1.20451.14/tools/crossgen -readytorun -MissingDependenciesOK -in "/src/artifacts/src/installer.473d1b592e8c281f03dcc4024c04c7c188e99c03/artifacts/bin/redist/Release/dotnet/sdk/5.0.100-rc.1.20452.10/net5.0/ref/Microsoft.TestPlatform.PlatformAbstractions.dll" -out "/tmp/14e3gjol.jpy/Microsoft.TestPlatform.PlatformAbstractions.dll"  -platform_assemblies_paths /src/artifacts/src/installer.473d1b592e8c281f03dcc4024c04c7c188e99c03/artifacts/bin/redist/Release/dotnet/sdk/5.0.100-rc.1.20452.10/Extensions:/src/artifacts/src/installer.473d1b592e8c281f03dcc4024c04c7c188e99c03/artifacts/bin/redist/Release/dotnet/sdk/5.0.100-rc.1.20452.10/Microsoft/Microsoft.NET.Build.Extensions/tools/net5.0:/src/artifacts/src/installer.473d1b592e8c281f03dcc4024c04c7c188e99c03/artifacts/bin/redist/Release/dotnet/sdk/5.0.100-rc.1.20452.10/net5.0:/src/artifacts/src/installer.473d1b592e8c281f03dcc4024c04c7c188e99c03/artifacts/bin/redist/Release/dotnet/sdk/5.0.100-rc.1.20452.10/net5.0/ref:/src/artifacts/src/installer.473d1b592e8c281f03dcc4024c04c7c188e99c03/artifacts/bin/redist/Release/dotnet/sdk/5.0.100-rc.1.20452.10/Roslyn:/src/artifacts/src/installer.473d1b592e8c281f03dcc4024c04c7c188e99c03/artifacts/bin/redist/Release/dotnet/sdk/5.0.100-rc.1.20452.10/runtimes/unix/lib/netcoreapp3.0:/src/artifacts/src/installer.473d1b592e8c281f03dcc4024c04c7c188e99c03/artifacts/bin/redist/Release/dotnet/sdk/5.0.100-rc.1.20452.10/runtimes/win/lib/netcoreapp3.0:/src/artifacts/src/installer.473d1b592e8c281f03dcc4024c04c7c188e99c03/artifacts/bin/redist/Release/dotnet/sdk/5.0.100-rc.1.20452.10/Sdks/NuGet.Build.Tasks.Pack/CoreCLR:/src/artifacts/src/installer.473d1b592e8c281f03dcc4024c04c7c188e99c03/artifacts/bin/redist/Release/dotnet/sdk/5.0.100-rc.1.20452.10:/src/artifacts/src/installer.473d1b592e8c281f03dcc4024c04c7c188e99c03/artifacts/bin/redist/Release/dotnet/shared/Microsoft.NETCore.App/5.0.0-rc.1.20451.14 -JITPath /src/packages/restored//microsoft.netcore.app.runtime.centos.7-x64/5.0.0-rc.1.20451.14/runtimes/centos.7-x64/native/libclrjit.so 
    Parameters
        CrossgenPath = /src/packages/restored//microsoft.netcore.app.runtime.centos.7-x64/5.0.0-rc.1.20451.14/tools/crossgen
        JITPath = /src/packages/restored//microsoft.netcore.app.runtime.centos.7-x64/5.0.0-rc.1.20451.14/runtimes/centos.7-x64/native/libclrjit.so
        PlatformAssemblyPaths
            /src/artifacts/src/installer.473d1b592e8c281f03dcc4024c04c7c188e99c03/artifacts/bin/redist/Release/dotnet/sdk/5.0.100-rc.1.20452.10/Extensions
            /src/artifacts/src/installer.473d1b592e8c281f03dcc4024c04c7c188e99c03/artifacts/bin/redist/Release/dotnet/sdk/5.0.100-rc.1.20452.10/Microsoft/Microsoft.NET.Build.Extensions/tools/net5.0
            /src/artifacts/src/installer.473d1b592e8c281f03dcc4024c04c7c188e99c03/artifacts/bin/redist/Release/dotnet/sdk/5.0.100-rc.1.20452.10/net5.0
            /src/artifacts/src/installer.473d1b592e8c281f03dcc4024c04c7c188e99c03/artifacts/bin/redist/Release/dotnet/sdk/5.0.100-rc.1.20452.10/net5.0/ref
            /src/artifacts/src/installer.473d1b592e8c281f03dcc4024c04c7c188e99c03/artifacts/bin/redist/Release/dotnet/sdk/5.0.100-rc.1.20452.10/Roslyn
            /src/artifacts/src/installer.473d1b592e8c281f03dcc4024c04c7c188e99c03/artifacts/bin/redist/Release/dotnet/sdk/5.0.100-rc.1.20452.10/runtimes/unix/lib/netcoreapp3.0
            /src/artifacts/src/installer.473d1b592e8c281f03dcc4024c04c7c188e99c03/artifacts/bin/redist/Release/dotnet/sdk/5.0.100-rc.1.20452.10/runtimes/win/lib/netcoreapp3.0
            /src/artifacts/src/installer.473d1b592e8c281f03dcc4024c04c7c188e99c03/artifacts/bin/redist/Release/dotnet/sdk/5.0.100-rc.1.20452.10/Sdks/NuGet.Build.Tasks.Pack/CoreCLR
            /src/artifacts/src/installer.473d1b592e8c281f03dcc4024c04c7c188e99c03/artifacts/bin/redist/Release/dotnet/sdk/5.0.100-rc.1.20452.10
            /src/artifacts/src/installer.473d1b592e8c281f03dcc4024c04c7c188e99c03/artifacts/bin/redist/Release/dotnet/shared/Microsoft.NETCore.App/5.0.0-rc.1.20451.14
        SourceAssembly = /src/artifacts/src/installer.473d1b592e8c281f03dcc4024c04c7c188e99c03/artifacts/bin/redist/Release/dotnet/sdk/5.0.100-rc.1.20452.10/net5.0/ref/Microsoft.TestPlatform.PlatformAbstractions.dll
        DestinationPath = /src/artifacts/src/installer.473d1b592e8c281f03dcc4024c04c7c188e99c03/artifacts/bin/redist/Release/dotnet/sdk/5.0.100-rc.1.20452.10/net5.0/ref/Microsoft.TestPlatform.PlatformAbstractions.dll
        ReadyToRun = True
        CreateSymbols = True
    Environment Variables passed to tool:
      COMPlus_PartialNGen=0
    Errors
        /src/artifacts/src/installer.473d1b592e8c281f03dcc4024c04c7c188e99c03/src/redist/targets/Crossgen.targets(137,5): Error compiling /src/artifacts/src/installer.473d1b592e8c281f03dcc4024c04c7c188e99c03/artifacts/bin/redist/Release/dotnet/sdk/5.0.100-rc.1.20452.10/net5.0/ref/Microsoft.TestPlatform.PlatformAbstractions.dll: Reference assemblies should not be loaded for execution.  They can only be loaded in the Reflection-only loader context. (0x80131058) [/src/artifacts/src/installer.473d1b592e8c281f03dcc4024c04c7c188e99c03/src/redist/redist.csproj]
        /src/artifacts/src/installer.473d1b592e8c281f03dcc4024c04c7c188e99c03/src/redist/targets/Crossgen.targets(137,5): Error: compilation failed for "/src/artifacts/src/installer.473d1b592e8c281f03dcc4024c04c7c188e99c03/artifacts/bin/redist/Release/dotnet/sdk/5.0.100-rc.1.20452.10/net5.0/ref/Microsoft.TestPlatform.PlatformAbstractions.dll" (0x80131058) [/src/artifacts/src/installer.473d1b592e8c281f03dcc4024c04c7c188e99c03/src/redist/redist.csproj]
    Microsoft (R) CoreCLR Native Image Generator - Version 5.0.0-rc.1.20451.14
    Copyright (c) Microsoft Corporation.  All rights reserved.
    The command exited with code 88.

Doing a general binlog search for that net5.0/ref dir, I see that the DLL name/identity exists in two of those search paths:

/src/artifacts/src/installer.473d1b592e8c281f03dcc4024c04c7c188e99c03/artifacts/bin/redist/Release/dotnet/sdk/5.0.100-rc.1.20452.10/net5.0/Microsoft.TestPlatform.PlatformAbstractions.dll
/src/artifacts/src/installer.473d1b592e8c281f03dcc4024c04c7c188e99c03/artifacts/bin/redist/Release/dotnet/sdk/5.0.100-rc.1.20452.10/net5.0/ref/Microsoft.TestPlatform.PlatformAbstractions.dll

I guess the wrong one is being found? I'm not sure what exactly is the diff from this to a working build. (E.g. does the ref/ dir normally exist, net5.0/ref?)


I see this interesting looking bit in /src/artifacts/src/installer.473d1b592e8c281f03dcc4024c04c7c188e99c03/src/redist/targets/Crossgen.targets:73 though:

      <!-- Don't crossgen reference assemblies redisted with msbuild for RoslynCodeTaskFactory -->
      <RemainingFiles Remove="$(SdkOutputDirectory)ref\*.dll" />

Maybe adding <RemainingFiles Remove="$(SdkOutputDirectory)net5.0\ref\*.dll" /> there is reasonable? This seems like a potentially bad band-aid but I don't understand what's going on with this dir existing in the first place. 😕

@tmds
Copy link
Member Author

tmds commented Oct 28, 2020

It looks like this reference assembly is built by vstest:

$ tree vstest.0c0fafa64de5c73981e7c40a99d7363f57635f60/artifacts/Release/net5.0/
vstest.0c0fafa64de5c73981e7c40a99d7363f57635f60/artifacts/Release/net5.0/
├── datacollector
├── datacollector.deps.json
├── datacollector.dll
├── datacollector.dll.config
├── datacollector.pdb
├── datacollector.runtimeconfig.json
├── Extensions
│   ├── Microsoft.TestPlatform.TestHostRuntimeProvider.dll
│   ├── Microsoft.TestPlatform.TestHostRuntimeProvider.pdb
│   ├── Microsoft.VisualStudio.TestPlatform.Extensions.Html.TestLogger.dll
│   ├── Microsoft.VisualStudio.TestPlatform.Extensions.Html.TestLogger.pdb
│   ├── Microsoft.VisualStudio.TestPlatform.Extensions.Trx.TestLogger.dll
│   └── Microsoft.VisualStudio.TestPlatform.Extensions.Trx.TestLogger.pdb
├── Microsoft.Extensions.DependencyModel.dll
├── Microsoft.Extensions.FileSystemGlobbing.dll
├── Microsoft.TestPlatform.CommunicationUtilities.dll
├── Microsoft.TestPlatform.CommunicationUtilities.pdb
├── Microsoft.TestPlatform.CoreUtilities.dll
├── Microsoft.TestPlatform.CoreUtilities.pdb
├── Microsoft.TestPlatform.CrossPlatEngine.dll
├── Microsoft.TestPlatform.CrossPlatEngine.pdb
├── Microsoft.TestPlatform.PlatformAbstractions.dll
├── Microsoft.TestPlatform.PlatformAbstractions.pdb
├── Microsoft.TestPlatform.Utilities.dll
├── Microsoft.TestPlatform.Utilities.pdb
├── Microsoft.TestPlatform.VsTestConsole.TranslationLayer.dll
├── Microsoft.TestPlatform.VsTestConsole.TranslationLayer.pdb
├── Microsoft.VisualStudio.TestPlatform.Client.dll
├── Microsoft.VisualStudio.TestPlatform.Client.pdb
├── Microsoft.VisualStudio.TestPlatform.Common.dll
├── Microsoft.VisualStudio.TestPlatform.Common.pdb
├── Microsoft.VisualStudio.TestPlatform.ObjectModel.dll
├── Microsoft.VisualStudio.TestPlatform.ObjectModel.pdb
├── net5.0
│   ├── Microsoft.TestPlatform.PlatformAbstractions.deps.json
│   ├── Microsoft.TestPlatform.PlatformAbstractions.dll
│   ├── Microsoft.TestPlatform.PlatformAbstractions.pdb
│   └── ref
│       └── Microsoft.TestPlatform.PlatformAbstractions.dll
├── Newtonsoft.Json.dll
├── NuGet.Frameworks.dll
├── package.deps.json
├── TestHost
├── vstest.console
├── vstest.console.deps.json
├── vstest.console.dll
├── vstest.console.dll.config
├── vstest.console.pdb
└── vstest.console.runtimeconfig.json

@tmds
Copy link
Member Author

tmds commented Oct 28, 2020

I think these files may be the result of https://github.com/microsoft/vstest/blob/e3c7e106fd7fe3671768e1998718fffd3f1a920b/scripts/build.sh#L344. I'll look into it further tomorrow.

@tmds
Copy link
Member Author

tmds commented Oct 29, 2020

90 packages in baseline weren't used!

Great!

Unexpected, needs more investigation:

/src/repos/Directory.Build.targets(553,5): error : 53 new packages used not in baseline! See report at /src/artifacts/prebuilt-report/baseline-comparison.xml for more information. Package IDs are: [/src/repos/known-good.proj]
/src/repos/Directory.Build.targets(553,5): error : Libuv.1.9.1 [/src/repos/known-good.proj]
/src/repos/Directory.Build.targets(553,5): error : Microsoft.CodeAnalysis.Analyzers.1.1.0 [/src/repos/known-good.proj]
/src/repos/Directory.Build.targets(553,5): error : Microsoft.CodeAnalysis.Common.1.3.0 [/src/repos/known-good.proj]
/src/repos/Directory.Build.targets(553,5): error : Microsoft.CodeAnalysis.CSharp.1.3.0 [/src/repos/known-good.proj]
/src/repos/Directory.Build.targets(553,5): error : Microsoft.CodeAnalysis.VisualBasic.1.3.0 [/src/repos/known-good.proj]
/src/repos/Directory.Build.targets(553,5): error : Microsoft.NETCore.App.1.0.5 [/src/repos/known-good.proj]
/src/repos/Directory.Build.targets(553,5): error : Microsoft.NETCore.DotNetHost.1.0.1 [/src/repos/known-good.proj]
/src/repos/Directory.Build.targets(553,5): error : Microsoft.NETCore.DotNetHostPolicy.1.0.5 [/src/repos/known-good.proj]
/src/repos/Directory.Build.targets(553,5): error : Microsoft.NETCore.DotNetHostResolver.1.0.1 [/src/repos/known-good.proj]
/src/repos/Directory.Build.targets(553,5): error : Microsoft.NETCore.Jit.1.0.7 [/src/repos/known-good.proj]
/src/repos/Directory.Build.targets(553,5): error : Microsoft.NETCore.Platforms.1.0.2 [/src/repos/known-good.proj]
/src/repos/Directory.Build.targets(553,5): error : Microsoft.NETCore.Runtime.CoreCLR.1.0.7 [/src/repos/known-good.proj]
/src/repos/Directory.Build.targets(553,5): error : Microsoft.NETCore.Targets.1.0.3 [/src/repos/known-good.proj]
/src/repos/Directory.Build.targets(553,5): error : Microsoft.NETCore.Windows.ApiSets.1.0.1 [/src/repos/known-good.proj]
/src/repos/Directory.Build.targets(553,5): error : Microsoft.VisualBasic.10.0.1 [/src/repos/known-good.proj]
/src/repos/Directory.Build.targets(553,5): error : runtime.placeholder-rid.System.Collections.4.0.11 [/src/repos/known-good.proj]
/src/repos/Directory.Build.targets(553,5): error : runtime.placeholder-rid.System.Diagnostics.Tools.4.0.1 [/src/repos/known-good.proj]
/src/repos/Directory.Build.targets(553,5): error : runtime.placeholder-rid.System.Globalization.4.0.11 [/src/repos/known-good.proj]
/src/repos/Directory.Build.targets(553,5): error : runtime.placeholder-rid.System.IO.4.1.0 [/src/repos/known-good.proj]
/src/repos/Directory.Build.targets(553,5): error : runtime.placeholder-rid.System.Reflection.4.1.0 [/src/repos/known-good.proj]
/src/repos/Directory.Build.targets(553,5): error : runtime.placeholder-rid.System.Reflection.Extensions.4.0.1 [/src/repos/known-good.proj]
/src/repos/Directory.Build.targets(553,5): error : runtime.placeholder-rid.System.Reflection.Primitives.4.0.1 [/src/repos/known-good.proj]
/src/repos/Directory.Build.targets(553,5): error : runtime.placeholder-rid.System.Resources.ResourceManager.4.0.1 [/src/repos/known-good.proj]
/src/repos/Directory.Build.targets(553,5): error : runtime.placeholder-rid.System.Runtime.4.1.0 [/src/repos/known-good.proj]
/src/repos/Directory.Build.targets(553,5): error : runtime.placeholder-rid.System.Runtime.Handles.4.0.1 [/src/repos/known-good.proj]
/src/repos/Directory.Build.targets(553,5): error : runtime.placeholder-rid.System.Runtime.InteropServices.4.1.0 [/src/repos/known-good.proj]
/src/repos/Directory.Build.targets(553,5): error : runtime.placeholder-rid.System.Text.Encoding.4.0.11 [/src/repos/known-good.proj]
/src/repos/Directory.Build.targets(553,5): error : runtime.placeholder-rid.System.Text.Encoding.Extensions.4.0.11 [/src/repos/known-good.proj]
/src/repos/Directory.Build.targets(553,5): error : runtime.placeholder-rid.System.Threading.Tasks.4.0.11 [/src/repos/known-good.proj]
/src/repos/Directory.Build.targets(553,5): error : runtime.native.System.Net.Security.4.0.1 [/src/repos/known-good.proj]

@tmds
Copy link
Member Author

tmds commented Oct 29, 2020

CI jobs aren't starting for my latest commit.

@azure-pipelines

This comment has been minimized.

@dagood
Copy link
Member

dagood commented Oct 29, 2020

Not sure why that happened. 😕 (/azp run sorted it out.)

@dagood
Copy link
Member

dagood commented Oct 29, 2020

I'm taking a look at this now. DanS suggested we might have better luck adding the new Microsoft.NETCore.App version to source-build-reference-packages.

This is getting started with dotnet/source-build-reference-packages#171, so it might be better to stick with that given the additional prebuilts this seems to have brought in. (If they can be fixed, that's good, but at that point the shift to net5.0 might be wasted effort.)

@tmds
Copy link
Member Author

tmds commented Oct 30, 2020

it might be better to stick with that given the additional prebuilts this seems to have brought in.

I'm surprised by these prebuilts, I have no clue how I'm introducing them.

@dagood
Copy link
Member

dagood commented Oct 30, 2020

If you go to the annotated-usage.xml (I'm looking at the "tarball" one from the centos7 offline leg) you can find the project.assets.json files like this:

<AnnotatedUsage Id="Microsoft.CodeAnalysis.Analyzers" Version="1.1.0" File="src/vstest.0c0fafa64de5c73981e7c40a99d7363f57635f60/artifacts/Release/obj/project.assets.json" Project="src/vstest.0c0fafa64de5c73981e7c40a99d7363f57635f60/" />
<AnnotatedUsage Id="Microsoft.CodeAnalysis.Common" Version="1.3.0" File="src/vstest.0c0fafa64de5c73981e7c40a99d7363f57635f60/artifacts/Release/obj/project.assets.json" Project="src/vstest.0c0fafa64de5c73981e7c40a99d7363f57635f60/" SourceBuildPackageIdCreator="roslyn MicrosoftCodeAnalysisCommonPackageVersion/3.8.0-dev.20452.3" />
...
<AnnotatedUsage Id="Microsoft.NETCore.App" Version="1.0.5" File="src/vstest.0c0fafa64de5c73981e7c40a99d7363f57635f60/artifacts/Release/obj/project.assets.json" IsDirectDependency="true" IsAutoReferenced="true" Project="src/vstest.0c0fafa64de5c73981e7c40a99d7363f57635f60/" SourceBuildPackageIdCreator="runtime MicrosoftNETCoreAppPackageVersion/5.0.0-rc.1.20451.14" />

In that file, project -> frameworks has a netcoreapp1.0 entry with just the one Microsoft.NETCore.App/1.0.5 dependency, but that explodes into a ton of dependencies because of the way 1.0 was trying to be piecemeal with its nupkgs.

It also tells you where the project is: "projectPath": "/tb/tarball_870257/src/vstest.0c0fafa64de5c73981e7c40a99d7363f57635f60/artifacts/Release/Microsoft.TestPlatform.Build.csproj",

(The artifacts/Release/obj/project.assets.json path is oddly ambiguous, usually it's in a dir named similarly to the project. This could be related to whatever's happening.)

I don't see any obvious reason for this to happen, would probably need to build locally to poke around efficiently.

@tmds
Copy link
Member Author

tmds commented Nov 3, 2020

The artifacts/Release/obj/project.assets.json path is oddly ambiguous, usually it's in a dir named similarly to the project.

It's due to some manual copying that happens in the build.sh script here:

https://github.com/microsoft/vstest/blob/a297851001d432ee8ce0655fcd35fb1d991cc3b7/scripts/build.sh#L375-L394

@tmds
Copy link
Member Author

tmds commented Nov 3, 2020

Looking better:

2020-11-03T11:44:15.9363158Z   157 packages in baseline weren't used!



2020-11-03T11:44:15.9331887Z /tb/tarball_873964/repos/Directory.Build.targets(553,5): error : 20 new packages used not in baseline! See report at /tb/tarball_873964/artifacts/prebuilt-report/baseline-comparison.xml for more information. Package IDs are: [/tb/tarball_873964/repos/known-good.proj]
2020-11-03T11:44:15.9333764Z /tb/tarball_873964/repos/Directory.Build.targets(553,5): error : runtime.placeholder-rid.System.Collections.4.0.11 [/tb/tarball_873964/repos/known-good.proj]
2020-11-03T11:44:15.9334881Z /tb/tarball_873964/repos/Directory.Build.targets(553,5): error : runtime.placeholder-rid.System.Diagnostics.Tools.4.0.1 [/tb/tarball_873964/repos/known-good.proj]
2020-11-03T11:44:15.9335977Z /tb/tarball_873964/repos/Directory.Build.targets(553,5): error : runtime.placeholder-rid.System.Globalization.4.0.11 [/tb/tarball_873964/repos/known-good.proj]
2020-11-03T11:44:15.9336993Z /tb/tarball_873964/repos/Directory.Build.targets(553,5): error : runtime.placeholder-rid.System.IO.4.1.0 [/tb/tarball_873964/repos/known-good.proj]
2020-11-03T11:44:15.9338033Z /tb/tarball_873964/repos/Directory.Build.targets(553,5): error : runtime.placeholder-rid.System.Reflection.4.1.0 [/tb/tarball_873964/repos/known-good.proj]
2020-11-03T11:44:15.9339104Z /tb/tarball_873964/repos/Directory.Build.targets(553,5): error : runtime.placeholder-rid.System.Reflection.Extensions.4.0.1 [/tb/tarball_873964/repos/known-good.proj]
2020-11-03T11:44:15.9340347Z /tb/tarball_873964/repos/Directory.Build.targets(553,5): error : runtime.placeholder-rid.System.Reflection.Primitives.4.0.1 [/tb/tarball_873964/repos/known-good.proj]
2020-11-03T11:44:15.9341723Z /tb/tarball_873964/repos/Directory.Build.targets(553,5): error : runtime.placeholder-rid.System.Resources.ResourceManager.4.0.1 [/tb/tarball_873964/repos/known-good.proj]
2020-11-03T11:44:15.9342909Z /tb/tarball_873964/repos/Directory.Build.targets(553,5): error : runtime.placeholder-rid.System.Runtime.4.1.0 [/tb/tarball_873964/repos/known-good.proj]
2020-11-03T11:44:15.9349931Z /tb/tarball_873964/repos/Directory.Build.targets(553,5): error : runtime.placeholder-rid.System.Runtime.Handles.4.0.1 [/tb/tarball_873964/repos/known-good.proj]
2020-11-03T11:44:15.9351058Z /tb/tarball_873964/repos/Directory.Build.targets(553,5): error : runtime.placeholder-rid.System.Runtime.InteropServices.4.1.0 [/tb/tarball_873964/repos/known-good.proj]
2020-11-03T11:44:15.9352152Z /tb/tarball_873964/repos/Directory.Build.targets(553,5): error : runtime.placeholder-rid.System.Text.Encoding.4.0.11 [/tb/tarball_873964/repos/known-good.proj]
2020-11-03T11:44:15.9353668Z /tb/tarball_873964/repos/Directory.Build.targets(553,5): error : runtime.placeholder-rid.System.Text.Encoding.Extensions.4.0.11 [/tb/tarball_873964/repos/known-good.proj]
2020-11-03T11:44:15.9355028Z /tb/tarball_873964/repos/Directory.Build.targets(553,5): error : runtime.placeholder-rid.System.Threading.Tasks.4.0.11 [/tb/tarball_873964/repos/known-good.proj]
2020-11-03T11:44:15.9356265Z /tb/tarball_873964/repos/Directory.Build.targets(553,5): error : runtime.placeholder-rid.System.Diagnostics.Debug.4.0.11 [/tb/tarball_873964/repos/known-good.proj]
2020-11-03T11:44:15.9357879Z /tb/tarball_873964/repos/Directory.Build.targets(553,5): error : runtime.placeholder-rid.System.IO.FileSystem.4.0.1 [/tb/tarball_873964/repos/known-good.proj]
2020-11-03T11:44:15.9358939Z /tb/tarball_873964/repos/Directory.Build.targets(553,5): error : runtime.placeholder-rid.System.Runtime.Extensions.4.1.0 [/tb/tarball_873964/repos/known-good.proj]
2020-11-03T11:44:15.9359979Z /tb/tarball_873964/repos/Directory.Build.targets(553,5): error : runtime.placeholder-rid.System.Private.Uri.4.0.1 [/tb/tarball_873964/repos/known-good.proj]
2020-11-03T11:44:15.9361238Z /tb/tarball_873964/repos/Directory.Build.targets(553,5): error : System.Private.Uri.4.0.1 [/tb/tarball_873964/repos/known-good.proj]
2020-11-03T11:44:15.9362208Z /tb/tarball_873964/repos/Directory.Build.targets(553,5): error : System.Threading.Overlapped.4.0.1 [/tb/tarball_873964/repos/known-good.proj]

@tmds
Copy link
Member Author

tmds commented Nov 3, 2020

@dagood do you have some thoughts about the new packages?

What does the runtime.placeholder-rid. prefix mean?

Maybe testhost.deps.zip tells you something interesting? It's testhost.deps.json from src/vstest.0c0fafa64de5c73981e7c40a99d7363f57635f60/artifacts/Release/Microsoft.TestPlatform.TestHost/net5.0/testhost.deps.json. It has some of the packages and versions mentioned in the log:

@dagood
Copy link
Member

dagood commented Nov 3, 2020

That testhost.deps.json seems to be a subset of the info in src/testhost.x86/obj/project.assets.json. It looks like a ton of extra packages are brought in by the Newtonsoft.Json/9.0.1 package reference.

I think how it works is that dotnet restore brings in a bunch of stuff (especially if a RID is specified), then the MSBuild targets in build/publish filter it down later. Obviously not ideal for source-build. Not sure if there's a way to address the problem at restore time.

I'm able to repro the large amount of assets with this simple proj:

<Project Sdk="Microsoft.NET.Sdk">
  <PropertyGroup>
    <OutputType>Exe</OutputType>
    <TargetFramework>net5.0</TargetFramework>
    <RuntimeIdentifier>win7-x86</RuntimeIdentifier>
  </PropertyGroup>
  <ItemGroup>
    <PackageReference Include="Newtonsoft.Json" Version="9.0.1" />
  </ItemGroup>
</Project>

If we compiled Newtonsoft.Json for net5.0 rather than netstandard1.0 I think that would fix this. (Really, pretty much anything higher than netstandard1.0 would probably fix this.)

@dagood
Copy link
Member

dagood commented Nov 3, 2020

What does the runtime.placeholder-rid. prefix mean?

This is just what we replace runtime.*. with in the context of a baseline, so the baseline can work across platforms.

@dagood
Copy link
Member

dagood commented Nov 3, 2020

Ah, noticed that there was a patch previously that removed <RuntimeIdentifier>win7-x86</RuntimeIdentifier> and this PR removes that patch. That's the direct cause here. We have SBRP nupkgs for System.Diagnostics.Tools but not runtime.any.System.Diagnostics.Tools Not sure what the implications are either way.

@tmds
Copy link
Member Author

tmds commented Nov 5, 2020

CI is happy now. @dagood thank you for your help!

@dagood dagood requested review from crummel and dseefeld November 5, 2020 15:39
@@ -17,7 +17,6 @@
<PackagesOutput>$(ProjectDirectory)/artifacts/$(Configuration)/packages</PackagesOutput>
<RepoApiImplemented>false</RepoApiImplemented>
<DeterministicBuildOptOut>true</DeterministicBuildOptOut>
<GlobalJsonFile>$(ProjectDirectory)global.json</GlobalJsonFile>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure why I didn't notice this earlier... this seems strange, it should disable some of our override logic. Why is it removed?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll add it back. I was not sure if I could keep it since the global.json file specified a 3.1 sdk. What gets done to the file specified here?

Copy link
Member

@dagood dagood Nov 6, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It has a 5.0 SDK now: https://github.com/microsoft/vstest/blob/b195e2589980861425b331e73a859252c3f2b71a/global.json. But even with a 3.1 SDK, we need this--source-build overrides it with the 5.0 SDK that we can actually use at the time. (3.1 would be a prebuilt that the repo would try to download, or if we block the download, the repo may fail to build with the "mismatched" 5.0 SDK we actually have.) The override would be the prebuilt tooling version for prod builds, but also an N-1 for servicing builds, or an N version for bootstrapping.

@dagood dagood merged commit 6a459cc into dotnet:master Nov 6, 2020
@dagood
Copy link
Member

dagood commented Nov 6, 2020

Thanks! Merging this should remove the need to put together source-build-reference-packages versions of the 2.1 Microsoft.NETCore.*host* packages. They're already done with dotnet/source-build-reference-packages#171 and uptake is pending with #1864, but I think it's good to take the necessity away in case we want to trim SBRP in the future.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants