Design for unified behavior controls

[mmitche]

Describe the Problem

Right now, there is a rough assumption that the DotNetBuildFromSource property means building from source on Linux. This property is used to trim away package references, sources, target frameworks, and behaviors that should not be present when building from source on Linux (in offline mode).

The eventual goal is that Windows and Mac will use the same approach as Linux source build for building (a single all-up build per platforms), however the behavior and requirements will likely differ per platform. Examples:

• Mac and Windows source builds may use pre-builts not allowable on Linux
• Mac and Windows source builds may not trim away certain sources or package references not allowable on Linux.

Because the current source build schema is focused on Linux, we will need to redesign it to support multiple platforms.

In addition to this, improvements to the way that .NET builds will allow more organizations to build the product, and more organizations to include code or behavior in the source base that is not active during other builds. For instance, Microsoft may use specific functionality to sign binaries for its distribution, while other organizations may use another, Today, this pivot is largely represented by DotNetBuildFromSource

Describe the Solution

We need a coherent design for source build on other platforms. This design should:

• Allow for an arbitrary set of platforms
• Allow for different pre-built sets.
• Allow for online vs. offline behavior.
• Allow for different organizational behavior.
• Allow for trimming of TFM targets (e.g. don't target net472 on Linux). This ideally does not involve conditionalization in a vast number of projects.
• Still allow for platform-agnostic source build behavior (maybe use the existing properties)
• Provide a rollout plan that is a smooth offramp from the current design and into the new one. It would be ideal if this was a scriptable, mechanical change.

Additional Context

Sub-issues

• Unified Build Controls #2975

[dotnet-issue-labeler]

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

[mmitche]

/cc @MichaelSimons @richlander @crummel

[jkotas]

Mac and Windows source builds may not trim away certain sources or package references not allowable on Linux.

Not allowable in Linux distros. These dependencies can be perfectly fine when building for Linux in less stringent environments. There are number of reasons why it should be ok to allow these dependencies in less stringent environments. For example, some tests have dependencies like that.

We should decouple full stack build and offline build. Full stack build means that we are building the whole thing from VMR. Offline build means that everything has to be built from sources, no binary-only external packages allowed.

[mmitche]

Mac and Windows source builds may not trim away certain sources or package references not allowable on Linux.

Not allowable in Linux distros. These dependencies can be perfectly fine when building for Linux in less stringent environments. There are number of reasons why it should be ok to allow these dependencies in less stringent environments. For example, some tests have dependencies like that.

We should decouple full stack build and offline build. Full stack build means that we are building the whole thing from VMR. Offline build means that everything has to be built from sources, no binary-only external packages allowed.

Good points. So there are a few pivots:

• Platform - Differentiates what platform we're building for
• Source-build enabled? (eventually most of these switches go away) - These would go away in most repos (some repos with dual insertions like roslyn may keep them) after the current official build methodology is removed.
• Offline vs. Online - Switches allowable pre-builts and build behavior. This would not be functional on certain platforms, like Mac and Windows.

Maybe Offline vs. Online isn't the right terminology for this. Perhaps it should be Strict vs. Non-strict, or Full vs Source-Only?

[MichaelSimons]

@mmitche - is this something that is still in scope for .NET 8?

[mmitche]

Unsure whether we'd implement, but we'll get to the design in the .NET 8 timeframe. This is what we put on the schedule yesterday.

[mmitche]

https://microsoft-my.sharepoint.com/:w:/p/mmitche/EdchH8A4okpOoKhkki1sJEYB0LyAaaCyX5_6A887gL744Q?e=hpF1PI

[mmitche]

T Shirt size (remaining): Small

[mmitche]

Unless there are additional comments here, I am going to md-ify this doc and put it in the UnifiedBuild docs and call it done.

[mmitche]

Completed when dotnet/arcade#14407 is merged