[ArPow] NuGet.Client should publish source-build intermediate packages

[crummel]

NuGet Product Used

dotnet.exe

Product Version

NuGet client build process

Worked before?

No response

Impact

It's more difficult to complete my work

Repro Steps & Context

NuGet.Client now produces source-build intermediate packages (after NuGet/NuGet.Client#4105) but these aren't included in the manifest of binaries or published to BAR. To complete the product graph, we need to publish these with the non-source-built packages.

Related: dotnet/source-build#2310.

Verbose Logs

No response

[crummel]

@nkolev92 or @zivkan, is this something you'd be able to help us out with for Preview7?

[nkolev92]

cc @aortiz-msft

[zivkan]

@crummel how do other repos deal with package name + version collisions? I don't see how it's possible to publish both "source build" and "non source build" versions of the same package.

[crummel]

We don't need the individual packages (NuGet.Versioning etc) from the source-build leg published, just the Microsoft.SourceBuild.Intermediate.nuget-client nupkg - that contains the source-built individual packages and arcade-powered source-build knows to look for and unpack that when it needs NuGet packages.

[zivkan]

notes to implementer:

1. I noticed the source build package is version 1.0.0, so I'm assuming we have to fix that and get the real product version, otherwise only the first BAR publish will succeed, and afterwards there will be version conflicts since the package id + version is already on the relevant feed.

2. I think BAR publishing needs to be moved to a new job that happens after both source-build and Build_And_Unit_Test_NonRTM finish. source-build job needs to publish the nupkg as a pipeline/build artifact, and the new job should download all the nupkgs, then publish to BAR.

   • We could consider moving BAR publishing to the pipeline that copies our binaries to the fileshare. I also know now how we can make that pipeline source controled & yaml, if we're interested in making that effort. Frankly, I would like to see this, only do the builds in the build pipeline, and do all "publishing" (to bar and file share and the nuget-build feed)

[crummel]

Using Arcade to version and publish would also help with issues like dotnet/source-build#3249 where we see mismatches between the Microsoft-built version and the source-built version.

[crummel]

If we wanted to fix this without completely moving NuGet to Arcade, the build number is set in https://github.com/NuGet/NuGet.Client/blob/fdc39cc7f7541056c638ea982dc179f554849b7d/eng/pipelines/templates/Initialize_Build.yml and doesn't seem to be meaningful for them; it's just a different way of calculating it from the AzDo official build number than Arcade uses.

• We may be able to switch them to the Arcade calulation.
• Either way, the Initialize.yml should have run by the time source-build is choosing version numbers so we should be able to piggy-back on that rather than using the bad 1.0.0 version number.