[heft] Merge Heft RC to main

[dmichon-msft]

Summary

Merging the RC branch for heft-next into main.

Details

Fixes #3181 - See for the main discussion of the design of this overhaul.

At a high level, refactors Heft to support more granular build phases than merely build and test.
Heft will now internally perform task-based scheduling in a manner analogous to Rush, with the addition of treating watch mode as a first-class scenario all the way through the pipeline.

Configurability

The core Heft package no longer contains API-Extractor, TypeScript, or ESLint/TSLint support. These are now all their own separate plugin packages, so consumers that wish to use Heft without some or all of the above can do so without paying the cost of loading those features.

The heft.json file now specifies one or more named phases, each of which is expected to contain zero or more named tasks, forming a directed acyclic task graph. Each task invokes either exactly one plugin or exactly one "event" (which are just shorthand for calling utility plugins).

Boot Performance

All heft plugin packages are now required to provide a heft-plugin.json manifest, which provides a list of one or more plugin entry points, the options schemas, and any command line parameter definitions that are applicable to the plugins. This allows the Heft runtime to load the manifest files and determine what tasks to execute without having to load the plugin code itself until it is needed.

This avoids situations where a given Heft plugin may contain very expensive initialization code that is executed even for Heft commands that don't invoke the plugin.

Watch Mode

Plugins configured for watch mode are expected to perform their own file watching, though may utilize a provided watchGlobAsync hook for simple watch scenarios, and invoke a requestRun callback when changes are detected.

When the requestRun callback is invoked (either directly or internally by watchGlobAsync), the Heft scheduler determines if the change impacts work that has already completed, in which case it will abort any current run and start a new one, or only future work, in which case it proceeds normally.

How it was tested

Local runs, use of the RC in other repositories.

Impacted documentation

All Heft documentation.

[D4N14L]

🫡