Design Meeting Notes, 8/31/2022

[DanielRosenwasser]

Module Resolution Proposals

#50152

• What's the history?
  • classic - node-like extensionless resolution implemented wrong
  • node - a version of Node 11 and earlier resolution
  • node16 - a version of resolutiont that captures all the new Node 16 features for ESM.
• What's the problem?
  • Bundlers support lots of useful things. TypeScript doesn't really capture their behavior.
  • Example scenario:
    1. Your project starts off using the node resolution strategy.
    2. You add a dependency which specifies an exports field - but the node strategy doesn't understand that.
       • So TypeScript errors, but your bundler (e.g. Webpack) is fine.
    3. You swap to the node16 or nodenext module resolution options to fix it.
    4. TypeScript starts erroring on another dependency because it has "type": "module" but your project doesn't.
       • So TypeScript errors, but your bundler (e.g. Webpack) is fine.
    5. You switch your file to a module - either by renaming it to .mts or adding "type": "module" to your package.json.
    6. TypeScript starts erroring on a relative import because you didn't specify the extension.
       • So TypeScript errors, but your bundler (e.g. Webpack) is fine.
    7. You specify the .js extension as TypeScript suggests.
    8. Your bundler can no longer resolve the relative import because it doesn't know that a file ending in .js is supposed to map to .ts.
       • So your bundler (e.g. Webpack) errors, but TypeScript is fine.
    9. You have to now configure your bundler further.
• Friction!
• Many behaviors of Node16 have been brought into bundlers, and bundlers can continue to provide many features that provide a nice UX (e.g. extensionless resolution).
• Maybe a new hybrid mode?
  • Name is bikesheddable.
  • A basis for bundlers and maybe more.
  • node_modules lookups
  • Index files (e.g. ./folder -> ./folder/index.ts)
  • Extensionless (e.g. ./foo/bar -> ./foo/bar.ts)
  • Resolution of the exports field of package.json
  • .mts, .cts, "type" don't matter. Everything is
• minimal
  • A basis for browsers and maybe more.
  • No node_modules etc.
  • Just relative URL paths with required extensions.
    • URLs?
      • Yes, %20 is a space.
      • So ./our%20assets/file.js is our assets/file.js.
  • Does import "bar.js" resolve to relative in browser contexts?
    • Have to check - believe WHATWG reserved this?
    • We probably don't want to resolve relative even if that's the case.
• Maybe node becomes node-legacy
  • Feedback from the TC39 tools group was node -> node-legacy might not be a good call. Maybe call it node11
  • Does it make sense to call this node-cjs?
    • But it's the old node-cjs.
• So it sounds like this new hybrid/conventional mode really thinks of everything as possibly synchronous ES modules.
  • How does this work with top-level await?
  • Some runtime shimming
  • Things are a little different...but mostly works.
• Name the hybrid mode --moduleResolution bundler?
  • Accurate now, but
    • ts-node
    • bun
    • etc.
  • Not just one runtime.
• The name should communicate the scenario.
  • Yes, but the scenarios keep changing.

Yarn Plug'n'Play

#28289
#35206

• Way to work around some of the shortcomings from package manager installation and package resolution.
• Instead extracting all packages fully into node_modules and having your code resolve from the disk at runtime, Yarn calculates all the information ahead of time for packages.
• Yarn creates a .pnp.cjs file that intercepts require (and sometimes file system APIs)
• What's the advantage?
  • It's faster because it doesn't have to shuffle directory trees around.
  • Just plop a cached zip
  • "just check in your node_modules"
• Gotcha?
  • Non-standard-ish
  • Nothing in Node that emulates it.
• Have discussed it a few times. Why revisiting?
  • Resolution steps have been published.
• Where do these zip files come from?
  • There's a cache, but it gets created from the npm registry.
  • You can navigate it with the right VS Code extension!
    • It causes crashes because we don't think files can exist in zip files.
• Importing a resolver JS file is no longer a hard requirement.
  • Statically resolvable and specified.
  • Need to be able to read from a .zip file
• PnP + "check in your dependencies" is one of the few things that mitigates supply chain attacks.
  • But npm ci etc?
  • Not really the reason for this. It is a nice bonus maybe. But the convenience is just fast workflows.
• Okay, it's not in an executable file. Where is it?
  • .pnp.data.json
• Great, we have the JSON, so how do we do the zip file support?
  • Have to roll our own or add a dependency.
  • Old .cjs file did all this for us. It was nice!
    • But did it? We still need to support path completion, virtualizing files, etc. Things work now, but the demo keeps crashing on the mounted files.
  • zip support is technically optional
    • But not really?
• What about the old competing crux? Other possible strategies here?
• Versioning concerns of specification?
• Wrong abstraction? Shouldn't this be in Node.js anyway?
  • That's what's Yarn is trying to do out of band anyway - provide a way to leverage the right hooks.
  • Doesn't matter, TypeScript and friends need to reimplement their own custom resolution rules anyway.
• Can we make it easier for Yarn to patch these?
  • Unclear - seems very hard.
  • Has to patch the System host, the ModuleResolutionHost, and more.
• How does this compose between all the upcoming module work?
  • Substantial cost - short-term and long-term.
• Last time the executable JS was the biggest concern. Are we changing our view?
  • Took time to address the original concerns.
  • Not seeing community convergence between different package managers and runtimes - though is this a blocker?
  • Hard to promise that we can keep up with updates.
• What are the concerns?
  • Can't add every custom resolution algorithm - adding this means it'll be there forever.
  • Not clear on what the spec will add over time.
• What can we do to alleviate here? Work with the existing process?
  • Is there a way to make patching easier?