Rewrite readme

README.md

@@ -2,75 +2,22 @@
 
 ## Purpose
 
-The Node.js Modules Team maintains and actively develops the ECMAScript Modules (ESM) implementation in Node.js Core.
+The Node.js Modules Team maintains and actively develops the ECMAScript Modules (ESM) implementation in Node.js core.
 
 Work includes:
 
-* ESM Loader implementation
-* ESM Module Specifier Resolution Algorithm
-* Loader Hooks
-* ESM and CommonJS Interoperability
-* Node.js and Browser interoperability
-* VM Modules implementation
+* ESM loader implementation
+* ESM specifier resolution
+* Loader hooks
+* ESM and CommonJS interoperability
+* Node.js and browser interoperability
+* VM modules implementation
 
-## Current Efforts
+## Archives
 
-> ### [Current Plan for ECMAScript Modules support in Node.js](./doc/plan-for-new-modules-implementation.md)
+- **[Features](./doc/archive/features.md)**: As part of creating the ECMAScript Modules implementation for Node.js 12.0.0, [use cases](./doc/use-cases.md) were brainstormed, which led to a list of features. This feature list, which previously existed as a section in this repo's root README, informed what became the [plan for the implementation](./doc/archive/plan-for-new-modules-implementation.md).
 
-A new ECMAScript modules implementation shipped behind the `--experimental-modules` flag in Node.js 12.0.0. The flag was dropped in Node.js 13.2.0, although the implementation remains experimental and is still subject to change. A road map of the implementation’s development is [here](./doc/plan-for-new-modules-implementation.md).
-
-
-## Features
-
-Based on [these use cases](https://docs.google.com/document/d/10BBsIqdAXB9JR2KUzQGYbCiVugYBnxE4REBakX29yyo/edit) ([#55](https://github.com/nodejs/modules/issues/55)), our implementation aims to support the following features (subject to change):
-
-### Baseline Modules Implementation Features:
-* Spec compliance ([#132](https://github.com/nodejs/modules/issues/132))
-* Browser equivalence ([#133](https://github.com/nodejs/modules/issues/133))
-* Don’t break CommonJS ([#112](https://github.com/nodejs/modules/issues/112))
-* No refactoring ([#87](https://github.com/nodejs/modules/issues/87))
-
-#### Browser Interop:
-* Browser and Node compatibility without building ([#107](https://github.com/nodejs/modules/issues/107))
-* Browser-compatible specifier resolution ([#109](https://github.com/nodejs/modules/issues/109))
-* Browser-compatible builds for ESM and CommonJS ([#108](https://github.com/nodejs/modules/issues/108))
-
-#### CommonJS Interop:
-* Named exports when importing CJS ([#81](https://github.com/nodejs/modules/issues/81))
-* Multi-mode packages ([#94](https://github.com/nodejs/modules/issues/94))
-* Transparent interoperability for ESM importing CommonJS ([#100](https://github.com/nodejs/modules/issues/100))
-* Consumer-agnostic imports ([#105](https://github.com/nodejs/modules/issues/105))
-* Mixed module types within app/module; gradual migration from CommonJS to ESM ([#99](https://github.com/nodejs/modules/issues/99))
-* ESM in .js files ([#151](https://github.com/nodejs/modules/issues/151))
-
-### Existing Node.js Utility Features:
-
-* Importing non-JavaScript files ([#115](https://github.com/nodejs/modules/issues/115))
-* NodeJS contextual pathing use cases ([#121](https://github.com/nodejs/modules/issues/121))
-* ESM in executable files ([#152](https://github.com/nodejs/modules/issues/152))
-* Callable resolver ([#157](https://github.com/nodejs/modules/issues/157))
-
-### Loader Features:
-
-* Code coverage/instrumentation ([#95](https://github.com/nodejs/modules/issues/95))
-* Pluggable Loaders to support multiple use cases ([#82](https://github.com/nodejs/modules/issues/82))
-* Runtime loaders, transpilation at import time ([#96](https://github.com/nodejs/modules/issues/96))
-* Arbitrary sources for module source text ([#97](https://github.com/nodejs/modules/issues/97))
-* Mock modules (injection) ([#98](https://github.com/nodejs/modules/issues/98))
-* Specifier resolution customization ([#110](https://github.com/nodejs/modules/issues/110))
-* Package encapsulation ([#111](https://github.com/nodejs/modules/issues/111))
-* Conditional imports ([#113](https://github.com/nodejs/modules/issues/113))
-
-### WASM Features:
-
-* WASM modules (#[106](https://github.com/nodejs/modules/issues/106))
-
-### Developer and Tooling Features:
-
-* File / path / URL resolving (#[103](https://github.com/nodejs/modules/issues/103))
-* Import CommonJS without needing asynchronous syntax (#[116](https://github.com/nodejs/modules/issues/116))
-* Tree shaking (#[102](https://github.com/nodejs/modules/issues/102))
-* Polyfillability (#[101](https://github.com/nodejs/modules/issues/101))
+- **[Plan for New Modules Implementation](./doc/archive/plan-for-new-modules-implementation.md)**: This document summarized the work that went into the new ECMAScript Modules implementation that shipped in Node.js 12.0.0.
 
 ## Members
 

doc/archive/features.md

@@ -0,0 +1,55 @@
+> _This document has been archived._
+>
+> As part of creating the ECMAScript Modules implementation for Node.js 12.0.0, use cases were brainstormed, which led to a list of features. This feature list, which previously existed as a section in this repo's root [README](../../README.md), informed what became the [plan for the implementation](./plan-for-new-modules-implementation.md).
+
+# Features
+
+Based on [these use cases](./doc/use-cases.md) ([#55](https://github.com/nodejs/modules/issues/55)), our implementation aims to support the following features (subject to change):
+
+### Baseline Modules Implementation Features:
+* Spec compliance ([#132](https://github.com/nodejs/modules/issues/132))
+* Browser equivalence ([#133](https://github.com/nodejs/modules/issues/133))
+* Don’t break CommonJS ([#112](https://github.com/nodejs/modules/issues/112))
+* No refactoring ([#87](https://github.com/nodejs/modules/issues/87))
+
+#### Browser Interop:
+* Browser and Node compatibility without building ([#107](https://github.com/nodejs/modules/issues/107))
+* Browser-compatible specifier resolution ([#109](https://github.com/nodejs/modules/issues/109))
+* Browser-compatible builds for ESM and CommonJS ([#108](https://github.com/nodejs/modules/issues/108))
+
+#### CommonJS Interop:
+* Named exports when importing CJS ([#81](https://github.com/nodejs/modules/issues/81))
+* Multi-mode packages ([#94](https://github.com/nodejs/modules/issues/94))
+* Transparent interoperability for ESM importing CommonJS ([#100](https://github.com/nodejs/modules/issues/100))
+* Consumer-agnostic imports ([#105](https://github.com/nodejs/modules/issues/105))
+* Mixed module types within app/module; gradual migration from CommonJS to ESM ([#99](https://github.com/nodejs/modules/issues/99))
+* ESM in .js files ([#151](https://github.com/nodejs/modules/issues/151))
+
+### Existing Node.js Utility Features:
+
+* Importing non-JavaScript files ([#115](https://github.com/nodejs/modules/issues/115))
+* NodeJS contextual pathing use cases ([#121](https://github.com/nodejs/modules/issues/121))
+* ESM in executable files ([#152](https://github.com/nodejs/modules/issues/152))
+* Callable resolver ([#157](https://github.com/nodejs/modules/issues/157))
+
+### Loader Features:
+
+* Code coverage/instrumentation ([#95](https://github.com/nodejs/modules/issues/95))
+* Pluggable Loaders to support multiple use cases ([#82](https://github.com/nodejs/modules/issues/82))
+* Runtime loaders, transpilation at import time ([#96](https://github.com/nodejs/modules/issues/96))
+* Arbitrary sources for module source text ([#97](https://github.com/nodejs/modules/issues/97))
+* Mock modules (injection) ([#98](https://github.com/nodejs/modules/issues/98))
+* Specifier resolution customization ([#110](https://github.com/nodejs/modules/issues/110))
+* Package encapsulation ([#111](https://github.com/nodejs/modules/issues/111))
+* Conditional imports ([#113](https://github.com/nodejs/modules/issues/113))
+
+### WASM Features:
+
+* WASM modules (#[106](https://github.com/nodejs/modules/issues/106))
+
+### Developer and Tooling Features:
+
+* File / path / URL resolving (#[103](https://github.com/nodejs/modules/issues/103))
+* Import CommonJS without needing asynchronous syntax (#[116](https://github.com/nodejs/modules/issues/116))
+* Tree shaking (#[102](https://github.com/nodejs/modules/issues/102))
+* Polyfillability (#[101](https://github.com/nodejs/modules/issues/101))

doc/archive/plan-for-new-modules-implementation.md

@@ -0,0 +1,134 @@
+> _This document has been archived._
+>
+> This document summarized the work that went into the new ECMAScript Modules implementation that shipped in Node.js 12.0.0.
+
+# Plan for New Modules Implementation
+
+This document outlines the plan for building a new implementation to support ECMAScript modules in Node.js. The effort is split up into phases:
+
+* **Phase 0** branches off of current Node but removes much of the Node 8.5.0+ `--experimental-modules` implementation so that a new implementation could be built in its place.
+
+* **Phase 1** adds the “minimal kernel,” features that the modules working group felt would likely appear in any potential new ES modules implementation.
+
+* **Phase 2** fleshes out the implementation with enough functionality that it should be useful to average users as a minimum viable product.
+
+  - At the completion of Phase 2, the old `--experimental-modules` implementation was [replaced](https://github.com/nodejs/node/pull/26745) with this new one (still behind the `--experimental-modules` flag). It was released as part of Node 12 on 2019-04-23.
+
+* **Phase 3** improves user experience and extends the MVP.
+
+  - At the completion of Phase 3, the new implementation’s experimental flag was [dropped](https://github.com/nodejs/node/pull/29866). It was released as part of Node 13.2.0 on 2019-11-21.
+
+* **Phase 4** are items that were under development in earlier phases but weren’t finished when the new implementation’s experimental flag was dropped; these items may continue development after unflagging and potentially ship in later versions of Node.js.
+
+The effort is currently in **[Phase 4](#phase-4-further-improvements-after-unflagging)**
+
+At every phase, the following standards must be maintained:
+
+* Spec compliance ([#132](https://github.com/nodejs/modules/issues/132)): We must always follow the ES spec.
+* Browser equivalence ([#133](https://github.com/nodejs/modules/issues/133)): There’s room for debate in specific cases, but in general if Node is doing something that browsers also do, Node should do it in the same way. Alternatively, code that executes in both environments should produce identical results.
+* Don’t break CommonJS ([#112](https://github.com/nodejs/modules/issues/112)): We cannot cause breaking changes with regards to CommonJS.
+
+See also the [features list in the README](https://github.com/nodejs/modules#features).
+
+## Phase 0: Start Fresh
+
+From current shipping Node, the following changes were made to strip out most of the Node 8.5.0+ `--experimental-modules` implementation so that a new implementation could be built in its place:
+
+* Remove support in the `import` statement of formats other than ESM:
+  - No CommonJS.
+  - No JSON.
+  - No native modules.
+
+* Remove dynamic path searching:
+  - No extension adding.
+  - No directory resolution, including no support for `index.js` or `index.mjs`.
+  - No support for `main` field for ESM.
+
+* Remove current VM implementation
+
+* Remove current Loader implementation
+
+These changes were implemented in https://github.com/nodejs/ecmascript-modules/pull/6.
+
+## Phase 1: The Minimal Kernel
+
+The “minimal kernel” consists of features that the @nodejs/modules group have agreed will be necessary for all potential iterations of our ESM implementation. Phase 1 does _not_ include features that preclude other potential features or implementation approaches; and Phase 1 also does not include some features that should naturally be built in a later phase of development, for example because those features depend on features planned for Phase 1.
+
+* `module.createRequireFromPath` ([nodejs/node#19360](https://github.com/nodejs/node/pull/19360)) is the only way to import CommonJS into an ES module, for now.
+  - `import.meta.require` fails at runtime as opposed to import time. This is not desireable to all committee members.
+  - Hold off on `import` statements for CommonJS until more progress is made on the dynamic modules spec.
+  - Landed in core in https://github.com/nodejs/node/commit/246f6332e5a5f395d1e39a3594ee5d6fe869d622
+
+* `import` statements will only support files with an `.mjs` extension, and will import only ES modules, for now.
+  - No JSON or native modules; `createRequireFromPath` can be used to get these.
+
+* `import.meta.url`.
+  - Already in the existing implementation.
+
+* Dynamic `import()`.
+  - Already in the existing implementation.
+
+* Support for built-in modules with named exports
+  - Already in the existing implementation.
+
+## Phase 2: Minimum Viable Product: Required to Upstream
+
+Phase 2 fleshes out the implementation with enough functionality that it should be useful to average users as a minimum viable product. At the completion of Phase 2, the old `--experimental-modules` implementation was replaced with this new one (still behind the `--experimental-modules` flag).
+
+* Define semantics for importing a package entry point, e.g. `import _ from 'lodash'`
+  - Proposal: [“File Specifier Resolution” proposal](https://github.com/GeoffreyBooth/node-import-file-specifier-resolution-proposal) covers bare module specifier resolution of CommonJS packages.
+  - Landed in https://github.com/nodejs/ecmascript-modules/pull/28.
+
+* Define semantics for determining when to load sources as CommonJS or ES module for both the top-level main (`node x.js`) and dependency loading.
+  - Proposal: [“File Specifier Resolution” proposal](https://github.com/GeoffreyBooth/node-import-file-specifier-resolution-proposal) covers `import` statements of ESM files; and CommonJS files, package entry point and package deep imports.
+  - Landed in https://github.com/nodejs/ecmascript-modules/pull/28.
+
+* Define semantics for enabling ESM treatment of source code loaded via `--eval`, STDIN, and extensionless files (both with and without shebang lines).
+  - Proposal: [“Entry Points Proposal”](https://github.com/geoffreybooth/node-esm-entry-points-proposal) covers non-file forms of input as well as adding `--type` flag for controlling file-based input.
+  - Landed in https://github.com/nodejs/ecmascript-modules/pull/32.
+  - Renamed to `--entry-type` as part of upstream [PR](https://github.com/nodejs/node/pull/26745) to Node.js core.
+  - Renamed to `--intry-type` and limited to `--eval`, `--print` and `STDIN` as part of follow-up [PR](https://github.com/nodejs/node/pull/27184) to Node.js core.
+
+* File extension and directory index searching in ESM, behind its own flag, `--es-module-specifier-resolution`.
+  - See https://github.com/nodejs/modules/issues/268.
+  - Landed in https://github.com/nodejs/ecmascript-modules/pull/48.
+
+The work through the end of Phase 2 landed in Node.js `master` as part of https://github.com/nodejs/node/pull/26745 and was released in Node 12.0.0.
+
+## Phase 3: Path to Stability: Removing `--experimental-modules` Flag
+
+Phase 3 improves user experience and extends the MVP. Phase 3 is malleable based on how things proceed while working on this phase. At the end of this phase, the `--experimental-modules` flag is dropped.
+
+* Better mechanism for creating `require` function: `createRequire`.
+  - Landed in https://github.com/nodejs/node/pull/27405 and shipped in 12.2.0.
+
+* `"exports"` field: for consumers of a package, map the paths of deep imports to provide encapsulation (an explicit public API); pretty specifiers (no file exensions or paths that include things like `dist/`) and flexibility for future package versions renaming or moving files without affecting the package’s public API. Applies to both ESM and CommonJS.
+  - Proposal: [Package Exports Proposal](https://github.com/jkrems/proposal-pkg-exports).
+  - Landed in https://github.com/nodejs/node/pull/28568 and shipped in 12.7.0 behind `--experimental-exports`. Further improvements are being made as additional PRs against core.
+  - The separate `--experimental-exports` flag was dropped in https://github.com/nodejs/node/pull/29867, merging the feature with overall `--experimental-modules`.
+
+* Define behavior for builtin globals between CommonJS and ESM. Does modifying a builtin in one module system carry over into the other? If it does, we may have major performance concerns.
+  - Issue raised in: https://github.com/nodejs/node/pull/29426.
+  - Solution was to not sync bindings automatically, but provide an API to manually sync them when desired: `module.syncBuiltinESMExports()`.
+  - Landed in https://github.com/nodejs/node/pull/29737 and shipped in 12.12.0.
+
+* Shortcut to resolve to package root.
+  - Proposal: [Package `"name"` Resolution Proposal](https://github.com/guybedford/package-name-resolution).
+  - Discussion: https://github.com/nodejs/modules/issues/306.
+  - Landed in https://github.com/nodejs/node/pull/29327 behind the flag `--experimental-resolve-self`.
+
+* Finalize behavior of `import` of CommonJS files and packages.
+  - Overview: https://github.com/nodejs/modules/issues/264.
+  - At time of unflagging: `import` only the CommonJS default export, so `import _ from 'cjs-pkg'` but not `import { shuffle } from 'cjs-pkg'`.
+  - [Conditional exports](https://github.com/nodejs/node/pull/29978) allows creating an ESM wrapper to provide named exports of an otherwise all-CommonJS package; see [“Approach #1: Use an ES Module Wrapper.”](https://github.com/nodejs/node/blob/master/doc/api/esm.md#approach-1-use-an-es-module-wrapper)
+  - No further improvements are expected.
+
+* Dual CommonJS/ESM packages: Support packages with both CommonJS and ESM sources that can be used in either environment.
+  - At time of unflagging: `"main"` (or `"exports": { ".": "file.js" }` overriding `"main"`) points to exactly one file, and full filenames are required (by default), so there is no possibility of an `import` specifier pointing to different files in ESM versus CommonJS; unless `--experimental-conditional-exports` is used (see next bullet). Without that flag, dual packages must provide secondary entry point via a path, e.g. `'pkg/module'` or `'pkg/commonjs'`.
+  - With `--experimental-conditional-exports`, paths within the `package.json` `"exports"` block can have separate entry points per environment.
+  - Landed in https://github.com/nodejs/node/pull/29978 and https://github.com/nodejs/node/pull/30051 behind `--experimental-conditional-exports` flag.
+  - Unflagged in https://github.com/nodejs/node/pull/31001 and shipped in Node 13.7.0.
+
+## Phase 4: Further Improvements After Unflagging
+
+Now that the implementation has shipped, further efforts are listed on the [README](../README.md).