[rush] Provide an option to make the committed shrinkwrap capture the exact installation plan

[octogonz]

Although rush install always produces a /deterministic/ outcome for a given input, the installation plan may not exactly correspond to the shrinkwrap file (aka "lockfile"). In other words, rush install is a deterministic function of common/config/rush/pnpm-lock.yaml file plus the package.json files for all the projects, not the shrinkwrap file alone.

Benefits: This feature was introduced to minimize spurious churn in the shrinkwrap file, which frequently causes annoying merge conflicts in a busy monorepo.

Drawbacks: People have found this feature counterintuitive, because it changes the meaning of the shrinkwrap file. Also in some situations it's important for the exact installation plan to be tracked by Git (e.g. when applying a hotfix to an old release branch, where you want to avoid disturbing any other package versions).

Algorithm details

Here's a summary of the algorithm when using PNPM:

There are 3 files:
A. common/config/rush/pnpm-lock.yaml - the "committed" version that is tracked by Git
B. common/temp/pnpm-lock.yaml - the version that PNPM sees when invoked in common/temp
C. common/temp/pnpm-lock-preinstall.yaml - a backup copy of pnpm-lock.yaml, for diagnostic purposes to compare with B

rush install reads A, makes some fixes to it, and writes out B and C before invoking pnpm install

pnpm install reads B and might rewrite it with some changes, making it different from C. (But by design, rush install does not copy these changes back to A; the deviations are allowable because they are supposed to be fully deterministic. The reason is to minimize churn that causes Git merge conflicts.)

rush update is like rush install but it does copy B -> A, but only when Rush determines that the changes are interesting.

rush update --full deletes B, then runs pnpm install to create B, then copies B -> A

Proposed changes

1. Add a rush.json setting for disabling this optimization. Maybe we could call it minimizeShrinkwrapChurn, which would be true when the optimization is enabled.

2. We could also add a command-line parameter such as rush update --resync-shrinkwrap to force copying common/temp/pnpm-lock.yaml --> common/config/rush/pnpm-lock.yaml if there are any differences.

3. When the files are different, Rush install could print an informational message to the log indicating this situation. Something like: "The installation has differences from the shrinkwrap file because minimizeShrinkwrapChurn was enabled."

@iclanton

[octogonz]

For reference, the optimization happens here:

https://github.com/microsoft/web-build-tools/blob/3355b5e5fb24913665e6e4de6cb680d2c03020ed/apps/rush-lib/src/logic/InstallManager.ts#L953-L959

[HarryGifford]

@sachinjoseph Do you know if there's been any progress on this?

[octogonz]

@iclanton could you clarify why this was closed? Was it fixed? Was it punted?

[sachinjoseph]

@octogonz Turning on pnpm --frozen-lockfile in experiments also disables shrinkwrap churn optimization.

[sachinjoseph]

@octogonz Do you mean that this issue should remain open until a --frozen-lockfile flag's equivalent is supported by Rush install on all the three package managers we currently support?