-
Notifications
You must be signed in to change notification settings - Fork 4.8k
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Unify dotnet/runtime on using Arcade -publish (#108990)
* Unify dotnet/runtime on using Arcade -publish Fixes dotnet/source-build#4239 Replaces #100004 Co-authored By: Viktor Hofer <[email protected]> * Use DotNetBuildOrchestrator to avoid executing in a repo source-build context. * Actually, we want this logic to run only when we're running in a non-VMR-like context, even repo build. * Restore, but only restore the toolset in the publish job * Fix Exists check and do the "restore toolset only" check one degree away from the Arcade SDK. * Set the "publish using pipelines" global property when publishing * Don't produce the product version files on repo source build.
- Loading branch information
1 parent
e11b867
commit 9e59acb
Showing
13 changed files
with
207 additions
and
461 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,39 +1,206 @@ | ||
| <Project> | ||
| <Project InitialTargets="ValidateDownloadedAssets"> | ||
|
|
||
| <PropertyGroup> | ||
| <ProducesDotNetReleaseShippingAssets>true</ProducesDotNetReleaseShippingAssets> | ||
| <!-- This avoids creating VS.*.symbols.nupkg packages that are identical to the original package. --> | ||
| <AutoGenerateSymbolPackages>false</AutoGenerateSymbolPackages> | ||
| <!-- Set PlatformName to TargetArchitecture to create unique build manifest files. --> | ||
| <PlatformName Condition="'$(TargetArchitecture)' != ''">$(TargetArchitecture)</PlatformName> | ||
| </PropertyGroup> | ||
|
|
||
| <!-- Include installer archives and packages which aren't globbed by default. | ||
| Don't include Symbols archive as it is already included in Arcade's Publish.proj, with correct blob path. --> | ||
| <Target Name="PublishRuntimeInstallers" | ||
| BeforeTargets="BeforePublish" | ||
| Condition="'$(DotNetBuildRepo)' == 'true'"> | ||
| <!-- Retrieve runtime's runtime pack product version. | ||
| <!-- | ||
| Look through the downloaded artifacts to figure out how to ship them. Creates item groups for | ||
| other types of publishing to use. | ||
| DownloadDirectory is expected to have some directory inside, which then contains a dir for each | ||
| build job name. For example, this nupkg would be found: | ||
| $(DownloadDirectory)IntermediateArtifacts\windows_x64\Shipping\a.nupkg | ||
| --> | ||
| <ItemGroup Condition="'$(DotNetBuildRepo)' != 'true'"> | ||
| <DownloadedArtifactFile Include="$(DownloadDirectory)**" /> | ||
| <DownloadedSymbolNupkgFile Include="$(DownloadDirectory)**\*.symbols.nupkg" /> | ||
| <DownloadedWixPdbFile Include="$(DownloadDirectory)**\*.wixpdb" /> | ||
| <DownloadedWixpackFile Include="$(DownloadDirectory)**\*.wixpack.zip" Condition="'$(PostBuildSign)' != 'true'" /> | ||
| <DownloadedWorkloadsVSInsertionFile Include="$(DownloadDirectory)*\workloads-vs\**\*" /> | ||
| <DownloadedNupkgFile Include="$(DownloadDirectory)**\*.nupkg" Exclude="@(DownloadedSymbolNupkgFile)" /> | ||
|
|
||
| <!-- Exclude symbol packages from have a NuGet signature. These are never pushed to NuGet.org or | ||
| other feeds (in fact, that have identical identity to their non-symbol variant) --> | ||
| <DownloadedSymbolPackagesWithoutPaths Include="@(DownloadedSymbolNupkgFile->'%(Filename)%(Extension)')" /> | ||
| <FileSignInfo Include="@(DownloadedSymbolPackagesWithoutPaths->Distinct())" CertificateName="None" /> | ||
|
|
||
| <!-- Add files that are not affected by filtering and create checksum for them. --> | ||
| <UploadToBlobStorageFile | ||
| Include="@(DownloadedArtifactFile)" | ||
| Exclude="@(DownloadedSymbolNupkgFile); | ||
| @(DownloadedNupkgFile); | ||
| @(DownloadedWixPdbFile); | ||
| @(DownloadedWorkloadsVSInsertionFile); | ||
| @(DownloadedWixpackFile)" /> | ||
|
|
||
| <!-- | ||
| Filter out the RID-specific (Runtime) nupkgs and RID-agnostic nupkgs. RID-specific packages | ||
| are published from every job. RID-agnostic nupkgs are built with the same ID/version by | ||
| every job, so one specific job's outputs must be picked to sign and publish. | ||
| --> | ||
|
|
||
| <!-- RID-specific framework packs. --> | ||
| <RuntimeNupkgFile | ||
| Include=" | ||
| $(DownloadDirectory)**\Microsoft.*.Runtime.*.nupkg; | ||
| $(DownloadDirectory)**\Microsoft.*.App.Host.*.nupkg; | ||
| $(DownloadDirectory)**\Microsoft.*.App.Crossgen2.*.nupkg" | ||
| Exclude="@(DownloadedSymbolNupkgFile)" /> | ||
|
|
||
| <!-- VS insertion packages, carrying RID-specific installers. --> | ||
| <RuntimeNupkgFile | ||
| Include="$(DownloadDirectory)**\VS.Redist.Common.*.nupkg" | ||
| Exclude="@(DownloadedSymbolNupkgFile)" /> | ||
|
|
||
| <!-- | ||
| Workloads VS insertion artifacts produced by src/workloads/workloads.csproj. Only grab | ||
| the zip artifacts as they're grouped by SDK feature band which correlates with specific VS versions. | ||
| --> | ||
| <WorkloadsVSInsertionFile Include="$(DownloadDirectory)*\workloads-vs\**\*.zip" /> | ||
|
|
||
| <!-- | ||
| Runtime packages associated with some identity packages. Need to exclude "runtime.native.*" | ||
| because Libraries produces some "runtime.native.Foo" packages with | ||
| "runtime.<rid>.runtime.native.Foo" identity packages. | ||
| --> | ||
| <RuntimeNupkgFile | ||
| Include="$(DownloadDirectory)**\runtime.*.nupkg" | ||
| Exclude=" | ||
| $(DownloadDirectory)**\runtime.native.*.nupkg; | ||
| @(DownloadedSymbolNupkgFile)" /> | ||
|
|
||
| <!-- | ||
| Packages that aren't matched above as RID-specific are considered RID-agnostic. Also include | ||
| the AllConfigurations packages from the Libraries build. | ||
| --> | ||
| <RidAgnosticNupkgToPublishFile | ||
| Include=" | ||
| $(DownloadDirectory)**\Microsoft.NET.Workload.Mono.Toolchain.*Manifest-*.nupkg; | ||
| $(DownloadDirectory)**\Microsoft.NET.Sdk.WebAssembly.Pack.*.nupkg; | ||
| $(DownloadDirectory)*\$(PublishRidAgnosticPackagesFromPlatform)\**\*.nupkg; | ||
| $(DownloadDirectory)*\*AllConfigurations\**\*.nupkg" | ||
| Exclude="@(RuntimeNupkgFile);@(DownloadedSymbolNupkgFile)" /> | ||
|
|
||
| <TransportPackagesToPublishFile | ||
| Include="$(DownloadDirectory)**\*Transport*.nupkg" | ||
| Exclude="@(RuntimeNupkgFile);@(RidAgnosticNupkgToPublishFile);@(DownloadedSymbolNupkgFile)" /> | ||
|
|
||
| <NupkgToPublishFile Include="@(RuntimeNupkgFile);@(RidAgnosticNupkgToPublishFile);@(TransportPackagesToPublishFile)" /> | ||
|
|
||
| <!-- | ||
| Assuming all symbol packages ship and can be found by turning .nupkg => .symbols.nupkg, find | ||
| them. Don't check for missing symbol packages here: some nupkgs don't have them for valid | ||
| reasons, such as the VS insertion packages that transport MSIs. Symbol package validation | ||
| will check for symbol completeness with file-by-file granularity rather than looking for | ||
| missing symbols.nupkg files: https://github.com/dotnet/arcade/issues/2499. | ||
| Handles several conventions: | ||
| * NonShipping packages have symbol nupkgs that are Shipping. | ||
| * Shipping packages have symbol packages in a "symbols" subdirectory. | ||
| --> | ||
| <PotentialSymbolNupkgToPublishFile | ||
| Include=" | ||
| @(NupkgToPublishFile->Replace('\NonShipping\', '\Shipping\')->Replace('.nupkg', '.symbols.nupkg')); | ||
| @(NupkgToPublishFile->Replace('\NonShipping\', '\NonShipping\symbols\')->Replace('.nupkg', '.symbols.nupkg')); | ||
| @(NupkgToPublishFile->Replace('\Shipping\', '\Shipping\symbols\')->Replace('.nupkg', '.symbols.nupkg'))" /> | ||
|
|
||
| <SymbolNupkgToPublishFile Include="@(PotentialSymbolNupkgToPublishFile->Distinct()->Exists())" /> | ||
|
|
||
| <!-- Packages --> | ||
| <Artifact Include="@(NupkgToPublishFile)" | ||
| IsShipping="$([System.String]::new('%(Identity)').Contains('\Shipping\'))" | ||
| PublishFlatContainer="false" /> | ||
| <Artifact Include="@(SymbolNupkgToPublishFile)" PublishFlatContainer="false" /> | ||
|
|
||
| <!-- Blob storage --> | ||
| <Artifact Include="@(UploadToBlobStorageFile)" | ||
| Exclude="@(NupkgToPublishFile);@(SymbolNupkgToPublishFile)" | ||
| IsShipping="$([System.String]::new('%(Identity)').Contains('\Shipping\'))" | ||
| ChecksumPath="%(FullPath).sha512" /> | ||
| <Artifact Include="@(WorkloadsVSInsertionFile)" | ||
| SubBlobFolder="workloads/" | ||
| IsShipping="$([System.String]::new('%(Identity)').Contains('\Shipping\'))" /> | ||
| </ItemGroup> | ||
|
|
||
| <Target Name="GetNonStableProductVersion"> | ||
| <!-- Retrieve the non-stable runtime pack product version. | ||
| Don't stabilize the package version in order to retrieve the VersionSuffix. --> | ||
| <MSBuild Projects="$(RepoRoot)src/installer/pkg/sfx/Microsoft.NETCore.App/Microsoft.NETCore.App.Runtime.sfxproj" | ||
| Targets="ReturnProductVersion" | ||
| Properties="IsShipping=false; | ||
| Crossgen2SdkOverridePropsPath=; | ||
| Crossgen2SdkOverrideTargetsPath="> | ||
| <Output TaskParameter="TargetOutputs" PropertyName="RuntimeRuntimePackProductVersion" /> | ||
| <Output TaskParameter="TargetOutputs" PropertyName="NonStableProductVersion" /> | ||
| </MSBuild> | ||
| </Target> | ||
|
|
||
| <!-- | ||
| Only generate the productVersion.txt and runtime-productVersion.txt files when we're building | ||
| either the full VMR repo or not building in the VMR infrastructure. | ||
| This ensures that we don't produce these files in the "Repo source build" builds, | ||
| but we do produce them in both the VMR and the runtime official build. | ||
| --> | ||
| <PropertyGroup> | ||
| <_ShouldGenerateProductVersionFiles Condition="'$(DotNetBuildRepo)' == 'true' and '$(DotNetBuildOrchestrator)' == 'true'">true</_ShouldGenerateProductVersionFiles> | ||
| <_ShouldGenerateProductVersionFiles Condition="'$(DotNetBuildRepo)' != 'true' and '$(DotNetBuildOrchestrator)' != 'true'">true</_ShouldGenerateProductVersionFiles> | ||
| </PropertyGroup> | ||
|
|
||
| <Target Name="GenerateProductVersionFiles" | ||
| DependsOnTargets="GetNonStableProductVersion" | ||
| BeforeTargets="PublishToAzureDevOpsArtifacts" | ||
| Condition="'$(_ShouldGenerateProductVersionFiles)' == 'true'"> | ||
| <!-- Retrieve the runtime pack product version. --> | ||
| <MSBuild Projects="$(RepoRoot)src/installer/pkg/sfx/Microsoft.NETCore.App/Microsoft.NETCore.App.Runtime.sfxproj" | ||
| Targets="ReturnProductVersion" | ||
| Properties="Crossgen2SdkOverridePropsPath=; | ||
| Crossgen2SdkOverrideTargetsPath="> | ||
| <Output TaskParameter="TargetOutputs" PropertyName="ProductVersionForVersionsFile" /> | ||
| </MSBuild> | ||
|
|
||
| <ItemGroup> | ||
| <InstallerToPublish Include="$(ArtifactsPackagesDir)**\*.tar.gz; | ||
| $(ArtifactsPackagesDir)**\*.zip; | ||
| $(ArtifactsPackagesDir)**\*.deb; | ||
| $(ArtifactsPackagesDir)**\*.rpm; | ||
| $(ArtifactsPackagesDir)**\*.pkg; | ||
| $(ArtifactsPackagesDir)**\*.exe; | ||
| $(ArtifactsPackagesDir)**\*.msi" | ||
| Exclude="$(ArtifactsPackagesDir)**\Symbols.runtime.tar.gz" /> | ||
|
|
||
| <ItemsToPushToBlobFeed Include="@(InstallerToPublish)" | ||
| IsShipping="$([System.String]::Copy('%(RecursiveDir)').StartsWith('Shipping'))" | ||
| PublishFlatContainer="true" | ||
| RelativeBlobPath="Runtime/$(RuntimeRuntimePackProductVersion)/%(Filename)%(Extension)" /> | ||
| <ProductVersionFile Include="$(ArtifactsShippingPackagesDir)productVersion.txt" /> | ||
| <ProductVersionFile Include="$(ArtifactsShippingPackagesDir)runtime-productVersion.txt" /> | ||
| </ItemGroup> | ||
|
|
||
| <!-- Generate productVersion.txt file containing the product version. --> | ||
| <WriteLinesToFile File="%(ProductVersionFile.Identity)" | ||
| Lines="$(ProductVersionForVersionsFile)" | ||
| Overwrite="true" | ||
| Encoding="ASCII" /> | ||
|
|
||
| <ItemGroup> | ||
| <Artifact Include="@(ProductVersionFile)" | ||
| RelativeBlobPath="Runtime/$(NonStableProductVersion)/%(Filename)%(Extension)" /> | ||
| </ItemGroup> | ||
| </Target> | ||
|
|
||
| <Target Name="AddRelativeBlobPathToInstallerArtifacts" | ||
| DependsOnTargets="GetNonStableProductVersion" | ||
| BeforeTargets="PublishToAzureDevOpsArtifacts" | ||
| AfterTargets="GenerateChecksumsFromArtifacts"> | ||
| <ItemGroup> | ||
| <Artifact Condition="'%(Artifact.PublishFlatContainer)' == 'true' and '%(Artifact.RelativeBlobPath)' == ''" | ||
| RelativeBlobPath="Runtime/$(NonStableProductVersion)/%(Artifact.SubBlobFolder)%(Filename)%(Extension)" /> | ||
| </ItemGroup> | ||
| </Target> | ||
|
|
||
| <Target Name="ValidateDownloadedAssets" Condition="'$(DotNetBuildRepo)' != 'true'"> | ||
| <Error Text="DownloadDirectory is not set." Condition="'$(DownloadDirectory)' == ''" /> | ||
| <Error Condition="'@(SymbolNupkgToPublishFile)' == ''" Text="No symbol packages found." /> | ||
|
|
||
| <!-- | ||
| Duplicate RuntimeNupkgFile items mean artifact upload will fail, but only after another hour | ||
| of signing. Detect this early. It's possible to automatically "fix" this with Distinct(), | ||
| however the patterns should be fairly specific: this is likely a build infra mistake that | ||
| should be corrected. | ||
| --> | ||
| <Error Text="Duplicate RuntimeNupkgFile entries for: %(RuntimeNupkgFile.Identity)" Condition="@(RuntimeNupkgFile->Count()) > 1" /> | ||
| </Target> | ||
|
|
||
| </Project> | ||
| </Project> |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.