module: eliminate performance cost of ESM syntax detection for CommonJS entry points

[GeoffreyBooth]

This PR eliminates the performance penalty incurred by --experimental-detect-module for the common use case of CommonJS entry points:

hyperfine --warmup 10 --runs 30 \
  './node ./test/fixtures/snapshot/typescript.js' \
  './node --experimental-detect-module ./test/fixtures/snapshot/typescript.js'

Benchmark 1: ./node ./test/fixtures/snapshot/typescript.js
  Time (mean ± σ):     135.0 ms ±   3.3 ms    [User: 117.5 ms, System: 13.9 ms]
  Range (min … max):   129.0 ms … 143.5 ms    30 runs

Benchmark 2: ./node --experimental-detect-module ./test/fixtures/snapshot/typescript.js
  Time (mean ± σ):     135.3 ms ±   2.8 ms    [User: 117.3 ms, System: 13.9 ms]
  Range (min … max):   130.0 ms … 143.1 ms    30 runs

Summary
  ./node ./test/fixtures/snapshot/typescript.js ran
    1.00 ± 0.03 times faster than ./node --experimental-detect-module ./test/fixtures/snapshot/typescript.js

It does this by changing the startup flow for “ambiguous” entry points (where there’s no explicit file extension or package.json "type" field). On current main, there’s an initial file load and CommonJS parse to determine whether or not the such ambiguous entry points should be handled by the CommonJS or the ESM loader; and then the chosen loader reloads and reparses the entry point. In this PR, ambiguous files are always sent into the CommonJS loader, and only if it throws an error do we check if we should retry the entry point as ESM via the ESM loader. Because nothing “extra” happens for the common case of CommonJS entry points that parse successfully as CommonJS, there is no performance penalty for incurred for CommonJS entry points by having --experimental-detect-module enabled by default.

I haven’t gone further to investigate if additional optimizations are possible within the ESM loader; there probably are. But this was significant enough that I thought it could probably land on its own.

Additional benchmarks

./node benchmark/compare.js --new ./out/Release/node --old ./node-main --filter startup misc | Rscript benchmark/compare.R
[00:22:32|% 100| 2/2 files | 60/60 runs | 4/4 configs]: Done
                                                                                           confidence improvement accuracy (*)   (**)  (***)
misc/startup-cli-version.js count=30 cli='deps/corepack/dist/corepack.js'                                 0.80 %       ±1.15% ±1.53% ±1.99%
misc/startup-cli-version.js count=30 cli='deps/npm/bin/npm-cli.js'                                        0.72 %       ±1.17% ±1.56% ±2.03%
misc/startup-cli-version.js count=30 cli='deps/npm/bin/npx-cli.js'                                        0.60 %       ±1.32% ±1.76% ±2.29%
misc/startup-cli-version.js count=30 cli='tools/node_modules/eslint/bin/eslint.js'                       -0.24 %       ±1.50% ±1.99% ±2.60%
misc/startup-core.js count=30 mode='process' script='benchmark/fixtures/require-builtins'                 0.40 %       ±1.15% ±1.53% ±2.00%
misc/startup-core.js count=30 mode='process' script='test/fixtures/semicolon'                             0.84 %       ±1.14% ±1.52% ±1.97%
misc/startup-core.js count=30 mode='worker' script='benchmark/fixtures/require-builtins'                  1.55 %       ±2.40% ±3.19% ±4.15%
misc/startup-core.js count=30 mode='worker' script='test/fixtures/semicolon'                             -0.23 %       ±2.56% ±3.40% ±4.43%

Be aware that when doing many comparisons the risk of a false-positive
result increases. In this case, there are 8 comparisons, you can thus
expect the following amount of false-positive results:
  0.40 false positives, when considering a   5% risk acceptance (*, **, ***),
  0.08 false positives, when considering a   1% risk acceptance (**, ***),
  0.01 false positives, when considering a 0.1% risk acceptance (***)

@nodejs/loaders @nodejs/performance

[nodejs-github-bot]

Review requested:

• @nodejs/loaders

[mcollina]

lgtm

Can we flip the flag in v22?

[joyeecheung]

Definitely prefer this falling back from CJS loader behavior than the “check syntax first then choose the loader” approach before 👍 I think this approach was what I suggested at that time, not sure why it ended up the way it was. Though it would still be nicer if we could handle it via V8 caching but that can be a follow up.

[joyeecheung]

Can you add the typescript.js bundle into the startup-core or startup-cli-version benchmark and run it in the benchmark CI?

[joyeecheung]

Can we flip the flag in v22?

I think we should only flip the flag if we print a warning when the fallback is used (at least when it happens in non-entrypoints, or when the entry point is small enough). Users should mark the ESM explicitly wherever possible and be aware of the performance cost of the fallback. You probably don’t want thousands of your implicit .js ESMs to all get parsed as CJS first before falling back to ESM silently and wonder why your module loading is so slow…(and that could’ve been avoided if you just add a type filed to your package.json)

[mcollina]

Agreed on the warning. It would be great to ship it given it only simplifies dx.

[GeoffreyBooth]

Regarding the warning, I think we definitely want to avoid warning for two scenarios:

• An extensionless file outside of any package.json scope, like /usr/local/bin/foo
• Any files within node_modules

The user can’t add explicitness for the former without breaking their desired UX, and the user can’t fix the latter without patching dependencies.

The case I’m unsure on is node entry.js where there’s no package.json file (but there is an extension). The user could change the .js to .mjs or .cjs, but I’m not sure we want to push them to do so; they may have great reasons for needing to use .js (their editor, other tools, etc.). But if there’s no package.json present, it’s probably a single-file script or other very small script/app, where the performance overhead of detection is unlikely to be noticeable.

How about we warn only on ambiguous entry points where there is a package.json present? That probably covers most usage of detection, and if the user already has a package.json there shouldn’t be a reason that they can’t add a type field to it (whether that field’s value is module or commonjs).

[joyeecheung]

the user can’t fix the latter without patching dependencies.

Isn't that already the case? Right now this would not work anyway (it errors), why should we allow it without a warning? It seems to be encouraging package authors to publish their .js code without a type field in package.json. Right now what we have requires them to put "type": "module" field in package.json, I see no reason why we should lure them into stopping that...

I agree we can do that for extension less files. Though I'd still prefer we have something better for indicating ESM for them (e.g. magical comments or directives - the use of magical comments isn't new anyway, e.g. for source maps or V8 compilation hints).

[GeoffreyBooth]

I discovered that the warning was being printed incorrectly, because I was calling the error flow even when we were retrying as ESM. I fixed it so that the error flow is only used when we’re actually going to throw the CommonJS error, because we’re not retrying as ESM. So now the warning behavior is unchanged from before, which I think is fine for this PR. If we want to discuss warnings further that can be a separate PR.

[nodejs-github-bot]

CI: https://ci.nodejs.org/job/node-test-pull-request/57775/

[GeoffreyBooth]

Can you add the typescript.js bundle into the startup-core or startup-cli-version benchmark and run it in the benchmark CI?

Done for startup-core: https://ci.nodejs.org/view/Node.js%20benchmark/job/benchmark-node-micro-benchmarks/1495/console

The fixture isn’t the full TypeScript CLI. I don’t think there’s a way to get it to print the TypeScript version, so I didn’t add it to the startup-cli-version one.

[nodejs-github-bot]

CI: https://ci.nodejs.org/job/node-test-pull-request/57783/

[GeoffreyBooth]

There were some failing tests until I moved the try/catch inside the check for --experimental-detect-module; so with that change, this PR should be ready to land, unless @joyeecheung you had any other notes (or I haven’t resolved any of your notes)?

The few tests that fail when the flag is enabled by default seem mostly to do with error output, because per this PR with the flag we’re catching and rethrowing the attempt to run the entry point. I think I might have a solution for the error output, though it’ll take some work to get working with source maps; or it might not be necessary at all if we unflag this at the same time as #51977.

[nodejs-github-bot]

CI: https://ci.nodejs.org/job/node-test-pull-request/57795/

[nodejs-github-bot]

CI: https://ci.nodejs.org/job/node-test-pull-request/57849/

[nodejs-github-bot]

CI: https://ci.nodejs.org/job/node-test-pull-request/57850/

[nodejs-github-bot]

CI: https://ci.nodejs.org/job/node-test-pull-request/57851/

[nodejs-github-bot]

CI: https://ci.nodejs.org/job/node-test-pull-request/57852/

[nodejs-github-bot]

CI: https://ci.nodejs.org/job/node-test-pull-request/57853/

[nodejs-github-bot]

CI: https://ci.nodejs.org/job/node-test-pull-request/57855/

[nodejs-github-bot]

Commit Queue failed

- Loading data for nodejs/node/pull/52093
✔  Done loading data for nodejs/node/pull/52093
----------------------------------- PR info ------------------------------------
Title      module: eliminate performance cost of ESM syntax detection for CommonJS entry points (#52093)
   ⚠  Could not retrieve the email or name of the PR author's from user's GitHub profile!
Branch     GeoffreyBooth:optimize-cjs-detection -> nodejs:main
Labels     c++, module, lib / src, performance, esm, author ready, needs-ci, commit-queue-squash
Commits    16
 - module: eliminate performance cost of detection for cjs entry
 - DRY the C++ functions
 - restore warning to be printed only on error
 - add TypeScript fixture to benchmark
 - put try/catch inside our flag, to postpone for now having to deal wit…
 - Apply suggestions from code review
 - Remove cjsRetryAsESMCache
 - ScriptCompiler::kNoCompileOptions
 - export error messages from c++ as constants
 - USE()
 - format
 - review notes
 - Use string approach
 - simplify array conversion
 - lint
 - nit
Committers 1
 - Geoffrey Booth 
PR-URL: https://github.com/nodejs/node/pull/52093
Reviewed-By: Matteo Collina 
Reviewed-By: Yagiz Nizipli 
Reviewed-By: Joyee Cheung 
Reviewed-By: Jacob Smith 
------------------------------ Generated metadata ------------------------------
PR-URL: https://github.com/nodejs/node/pull/52093
Reviewed-By: Matteo Collina 
Reviewed-By: Yagiz Nizipli 
Reviewed-By: Joyee Cheung 
Reviewed-By: Jacob Smith 
--------------------------------------------------------------------------------
   ⚠  Commits were pushed since the last approving review:
   ⚠  - lint
   ⚠  - nit
   ℹ  This PR was created on Fri, 15 Mar 2024 05:07:50 GMT
   ✔  Approvals: 4
   ✔  - Matteo Collina (@mcollina) (TSC): https://github.com/nodejs/node/pull/52093#pullrequestreview-1938208612
   ✔  - Yagiz Nizipli (@anonrig) (TSC): https://github.com/nodejs/node/pull/52093#pullrequestreview-1944916206
   ✔  - Joyee Cheung (@joyeecheung) (TSC): https://github.com/nodejs/node/pull/52093#pullrequestreview-1946897096
   ✔  - Jacob Smith (@JakobJingleheimer): https://github.com/nodejs/node/pull/52093#pullrequestreview-1946928149
   ✔  Last GitHub CI successful
   ℹ  Last Full PR CI on 2024-03-20T12:55:16Z: https://ci.nodejs.org/job/node-test-pull-request/57855/
   ℹ  Last Benchmark CI on 2024-03-19T22:34:34Z: https://ci.nodejs.org/view/Node.js%20benchmark/job/benchmark-node-micro-benchmarks/1495/console
- Querying data for job/node-test-pull-request/57855/
   ✔  Last Jenkins CI successful
--------------------------------------------------------------------------------
   ✔  Aborted `git node land` session in /home/runner/work/node/node/.ncu

https://github.com/nodejs/node/actions/runs/8361463879

[nodejs-github-bot]

Landed in 8bc7459

[marco-ippolito]

Can you create a manual backport for v20? It doesnt land cleanly