[SBRP] Move back to project-to-project references for arcade-powered source-build

[dagood]

In 3.1, SBRP does something similar to project-to-project references via PackageReferences, nupkgs, and individual dotnet restores per project with very specific package feeds.

In 5.0 (dev branch as of writing), SBRP instead uses prebuilts for PackageReferences. This isn't actually a problem for building from source, because the bootstrap process rebuilds these against source-built artifacts. (It seems reasonable to leave it like this for 5.0 GA if we can't find time to address this.)

We can have the best of both worlds. The projects should build using each other's output DLLs as references, and then each project should package up its outputs. This removes the prebuilts but doesn't require an individual dotnet restore for every single project. ProjectReference is designed to do this kind of thing. (An early attempt to use ProjectReference didn't work out, leading to the 3.1 approach.)

[MichaelSimons]

@dagood - do you recall any specifics regarding An early attempt to use ProjectReference didn't work out, leading to the 3.1 approach.)?

The current approach has a number of UX issues specifically when adding new packages and ensuring the build order, addressing CG alerts requires multiple commits (the second commit is needed to update the SBRP intermediate after a build is available with the fix) and lastly this is the only reason we have to carry the SBRPs in the source-build artifacts which is a significant impact on the size.

[dagood]

IIRC @dseefeld told me he'd tried that earlier, and that's what the note is about. I don't think the issues were ever investigated in depth, we both just went with low-risk approaches that fairly obviously end up working instead of digging in and making ProjectReference work for us (potentially conflicting with SDK/MSBuild assumptions, etc).

I do think it's solvable with the right people and effort.

[ViktorHofer]

I agree with @dagood that SBRP should use ProjectReferences instead of PackageReferences which will solve a number of issues:

• SBRP ref pack generation tooling should add the project to eng/build.props #3038
• Make source-build-reference-packages use a local package cache dir #1605
• Make it reasonable to add packages to SBRP, as a downstream repo source-build champ who needs more #1969

Targeting packs like NETStandard.Library, Microsoft.NETFramework.ReferenceAssemblies and the *.Ref packs make this a bit challenging but I'm sure we could come up with a solution.

IMO we should start with changing the reference package source generator to emit ProjectReferences instead of PackageReferences. Targeting packs don't come into play here, as we don't directly reference them anymore now that we use implicit framework references.

After that, it would be good to clean-up all the existing non needed reference packages and then re-generate everything with the new template layout. During that, we will likely identify a few roslyn based GenAPI issues.