diff --git a/doc/api/cli.md b/doc/api/cli.md index 1469515f67..77757a60eb 100644 --- a/doc/api/cli.md +++ b/doc/api/cli.md @@ -134,6 +134,27 @@ added: v6.0.0 Enable FIPS-compliant crypto at startup. (Requires Node.js to be built with `./configure --openssl-fips`.) +### `--es-module-specifier-resolution=mode` + + +To be used in conjunction with `--experimental modules`. Sets the resolution +algorithm for resolving specifiers. Valid options are `explicit` and `node`. + +The default is `explicit`, which requires you to provide the full path to a +module. The `node` mode will enable support for optional file extensions and +the ability to import a directory that has an index file. + +Please see [customizing esm specifier resolution][] for example usage. + +### `--experimental-json-modules` + + +Enable experimental JSON support for the ES Module loader. + ### `--experimental-modules` Node.js contains support for ES Modules based upon the -[Node.js EP for ES Modules][] and the [ESM Minimal Kernel][]. +[Node.js EP for ES Modules][] and the [ecmascript-modules implementation][]. -The minimal feature set is designed to be compatible with all potential -future implementations. Expect major changes in the implementation including -interoperability support, specifier resolution, and default behavior. +Expect major changes in the implementation including interoperability support, +specifier resolution, and default behavior. ## Enabling -The `--experimental-modules` flag can be used to enable features for loading -ESM modules. +The `--experimental-modules` flag can be used to enable support for +ECMAScript modules (ES modules). + +## Running Node.js with an ECMAScript Module + +There are a few ways to start Node.js with an ES module as its input. -Once this has been set, files ending with `.mjs` will be able to be loaded -as ES Modules. +### Initial entry point with an .mjs extension + +A file ending with `.mjs` passed to Node.js as an initial entry point will be +loaded as an ES module. ```sh node --experimental-modules my-app.mjs ``` -## Features +### --type=module / -m flag - +Files ending with `.js` or `.mjs`, or lacking any extension, +will be loaded as ES modules when the `--type=module` flag is set. +This flag also has a shorthand alias `-m`. + +```sh +node --experimental-modules --type=module my-app.js +# or +node --experimental-modules -m my-app.js +``` + +For completeness there is also `--type=commonjs`, for explicitly running a `.js` +file as CommonJS. This is the default behavior if `--type` or `-m` is +unspecified. + +The `--type=module` or `-m` flags can also be used to configure Node.js to treat +as an ES module input sent in via `--eval` or `--print` (or `-e` or `-p`) or +piped to Node.js via `STDIN`. + +```sh +node --experimental-modules --type=module --eval \ + "import { sep } from 'path'; console.log(sep);" + +echo "import { sep } from 'path'; console.log(sep);" | \ + node --experimental-modules --type=module +``` + +### package.json "type" field + +Files ending with `.js` or `.mjs`, or lacking any extension, +will be loaded as ES modules when the nearest parent `package.json` file +contains a top-level field `"type"` with a value of `"module"`. + +The nearest parent `package.json` is defined as the first `package.json` found +when searching in the current folder, that folder’s parent, and so on up +until the root of the volume is reached. + + +```js +// package.json +{ + "type": "module" +} +``` + +```sh +# In same folder as above package.json +node --experimental-modules my-app.js # Runs as ES module +``` + +If the nearest parent `package.json` lacks a `"type"` field, or contains +`"type": "commonjs"`, extensionless and `.js` files are treated as CommonJS. +If the volume root is reached and no `package.json` is found, +Node.js defers to the deafult, a `package.json` with no `"type"` +field. + +## Package Scope and File Extensions + +A folder containing a `package.json` file, and all subfolders below that +folder down until the next folder containing another `package.json`, is +considered a _package scope_. The `"type"` field defines how `.js` and +extensionless files should be treated within a particular `package.json` file’s +package scope. Every package in a project’s `node_modules` folder contains its +own `package.json` file, so each project’s dependencies have their own package +scopes. A `package.json` lacking a `"type"` field is treated as if it contained +`"type": "commonjs"`. + +The package scope applies not only to initial entry points (`node +--experimental-modules my-app.js`) but also to files referenced by `import` +statements and `import()` expressions. + +```js +// my-app.js, in an ES module package scope because there is a package.json +// file in the same folder with "type": "module" + +import './startup/init.js'; +// Loaded as ES module since ./startup contains no package.json file, +// and therefore inherits the ES module package scope from one level up + +import 'commonjs-package'; +// Loaded as CommonJS since ./node_modules/commonjs-package/package.json +// lacks a "type" field or contains "type": "commonjs" + +import './node_modules/commonjs-package/index.js'; +// Loaded as CommonJS since ./node_modules/commonjs-package/package.json +// lacks a "type" field or contains "type": "commonjs" +``` + +Files ending with `.mjs` are always loaded as ES modules regardless of package +scope. -### Supported +Files ending with `.cjs` are always loaded as CommonJS regardless of package +scope. -Only the CLI argument for the main entry point to the program can be an entry -point into an ESM graph. Dynamic import can also be used to create entry points -into ESM graphs at runtime. +```js +import './legacy-file.cjs'; +// Loaded as CommonJS since .cjs is always loaded as CommonJS + +import 'commonjs-package/src/index.mjs'; +// Loaded as ES module since .mjs is always loaded as ES module +``` + +You can use the `.mjs` and `.cjs` extensions to mix types within the same +package scope: + +- Within a `"type": "module"` package scope, Node.js can be instructed to + interpret a particular file as CommonJS by naming it with a `.cjs` extension + (since both `.js` and `.mjs` files are treated as ES modules within a + `"module"` package scope). + +- Within a `"type": "commonjs"` package scope, Node.js can be instructed to + interpret a particular file as an ES module by naming it with an `.mjs` + extension (since both `.js` and `.cjs` files are treated as CommonJS within a + `"commonjs"` package scope). + +## Package Entry Points + +The `package.json` `"main"` field defines the entry point for a package, +whether the package is included into CommonJS via `require` or into an ES +module via `import`. + + +```js +// ./node_modules/es-module-package/package.json +{ + "type": "module", + "main": "./src/index.js" +} +``` +```js +// ./my-app.mjs + +import { something } from 'es-module-package'; +// Loads from ./node_modules/es-module-package/src/index.js +``` + +An attempt to `require` the above `es-module-package` would attempt to load +`./node_modules/es-module-package/src/index.js` as CommonJS, which would throw +an error as Node.js would not be able to parse the `export` statement in +CommonJS. + +As with `import` statements, for ES module usage the value of `"main"` must be +a full path including extension: `"./index.mjs"`, not `"./index"`. + +If the `package.json` `"type"` field is omitted, a `.js` file in `"main"` will +be interpreted as CommonJS. + +> Currently a package can define _either_ a CommonJS entry point **or** an ES +> module entry point; there is no way to specify separate entry points for +> CommonJS and ES module usage. This means that a package entry point can be +> included via `require` or via `import` but not both. +> +> Such a limitation makes it difficult for packages to support both new versions +> of Node.js that understand ES modules and older versions of Node.js that +> understand only CommonJS. There is work ongoing to remove this limitation, and +> it will very likely entail changes to the behavior of `"main"` as defined +> here. -#### import.meta +## import Specifiers + +### Terminology + +The _specifier_ of an `import` statement is the string after the `from` keyword, +e.g. `'path'` in `import { sep } from 'path'`. Specifiers are also used in +`export from` statements, and as the argument to an `import()` expression. + +There are four types of specifiers: + +- _Bare specifiers_ like `'some-package'`. They refer to an entry point of a + package by the package name. + +- _Deep import specifiers_ like `'some-package/lib/shuffle.mjs'`. They refer to + a path within a package prefixed by the package name. + +- _Relative specifiers_ like `'./startup.js'` or `'../config.mjs'`. They refer + to a path relative to the location of the importing file. + +- _Absolute specifiers_ like `'file:///opt/nodejs/config.js'`. They refer + directly and explicitly to a full path. + +Bare specifiers, and the bare specifier portion of deep import specifiers, are +strings; but everything else in a specifier is a URL. + +Only `file://` URLs are supported. A specifier like +`'https://example.com/app.js'` may be supported by browsers but it is not +supported in Node.js. + +Specifiers may not begin with `/` or `//`. These are reserved for potential +future use. The root of the current volume may be referenced via `file:///`. + +## import.meta * {Object} @@ -47,37 +243,44 @@ property: * `url` {string} The absolute `file:` URL of the module. -### Unsupported - -| Feature | Reason | -| --- | --- | -| `require('./foo.mjs')` | ES Modules have differing resolution and timing, use dynamic import | - -## Notable differences between `import` and `require` +## Differences Between ES Modules and CommonJS ### Mandatory file extensions -You must provide a file extension when using the `import` keyword. +You must provide a file extension when using the `import` keyword. Directory +indexes (e.g. `'./startup/index.js'`) must also be fully specified. -### No NODE_PATH +This behavior matches how `import` behaves in browser environments, assuming a +typically configured server. + +### No NODE_PATH `NODE_PATH` is not part of resolving `import` specifiers. Please use symlinks if this behavior is desired. -### No `require.extensions` +### No require, exports, module.exports, \_\_filename, \_\_dirname + +These CommonJS variables are not available in ES modules. + +`require` can be imported into an ES module using +[`module.createRequireFromPath()`][]. + +An equivalent for `__filename` and `__dirname` is [`import.meta.url`][]. + +### No require.extensions `require.extensions` is not used by `import`. The expectation is that loader hooks can provide this workflow in the future. -### No `require.cache` +### No require.cache `require.cache` is not used by `import`. It has a separate cache. -### URL based paths +### URL-based paths -ESM are resolved and cached based upon [URL](https://url.spec.whatwg.org/) -semantics. This means that files containing special characters such as `#` and -`?` need to be escaped. +ES modules are resolved and cached based upon +[URL](https://url.spec.whatwg.org/) semantics. This means that files containing +special characters such as `#` and `?` need to be escaped. Modules will be loaded multiple times if the `import` specifier used to resolve them have a different query or fragment. @@ -89,6 +292,59 @@ import './foo.mjs?query=2'; // loads ./foo.mjs with query of "?query=2" For now, only modules using the `file:` protocol can be loaded. +## Interoperability with CommonJS + +### require + +`require` always treats the files it references as CommonJS. This applies +whether `require` is used the traditional way within a CommonJS environment, or +in an ES module environment using [`module.createRequireFromPath()`][]. + +To include an ES module into CommonJS, use [`import()`][]. + +### import statements + +An `import` statement can reference either ES module or CommonJS JavaScript. +Other file types such as JSON and Native modules are not supported. For those, +use [`module.createRequireFromPath()`][]. + +`import` statements are permitted only in ES modules. For similar functionality +in CommonJS, see [`import()`][]. + +The _specifier_ of an `import` statement (the string after the `from` keyword) +can either be an URL-style relative path like `'./file.mjs'` or a package name +like `'fs'`. + +Like in CommonJS, files within packages can be accessed by appending a path to +the package name. + +```js +import { sin, cos } from 'geometry/trigonometry-functions.mjs'; +``` + +> Currently only the “default export” is supported for CommonJS files or +> packages: +> +> +> ```js +> import packageMain from 'commonjs-package'; // Works +> +> import { method } from 'commonjs-package'; // Errors +> ``` +> +> There are ongoing efforts to make the latter code possible. + +### import() expressions + +Dynamic `import()` is supported in both CommonJS and ES modules. It can be used +to include ES module files from CommonJS code. + +```js +(async () => { + await import('./my-app.mjs'); +})(); +``` + ## CommonJS, JSON, and Native Modules CommonJS, JSON, and Native modules can be used with [`module.createRequireFromPath()`][]. @@ -99,9 +355,9 @@ module.exports = 'cjs'; // esm.mjs import { createRequireFromPath as createRequire } from 'module'; -import { fileURLToPath as fromPath } from 'url'; +import { fileURLToPath as fromURL } from 'url'; -const require = createRequire(fromPath(import.meta.url)); +const require = createRequire(fromURL(import.meta.url)); const cjs = require('./cjs'); cjs === 'cjs'; // true @@ -138,6 +394,161 @@ fs.readFileSync = () => Buffer.from('Hello, ESM'); fs.readFileSync === readFileSync; ``` +## Experimental JSON Modules + +**Note: This API is still being designed and is subject to change.**. + +Currently importing JSON modules are only supported in the `commonjs` mode +and are loaded using the CJS loader. [WHATWG JSON modules][] are currently +being standardized, and are experimentally supported by including the +additional flag `--experimental-json-modules` when running Node.js. + +When the `--experimental-json-modules` flag is included both the +`commonjs` and `module` mode will use the new experimental JSON +loader. The imported JSON only exposes a `default`, there is no +support for named exports. A cache entry is created in the CommonJS +cache, to avoid duplication. The same object will be returned in +CommonJS if the JSON module has already been imported from the +same path. + +Assuming an `index.js` with + + +```js +import packageConfig from './package.json'; +``` + +You will need to include the `--experimental-json-modules` flag for the module +to work. + + +```bash +node --experimental-modules --type=module index.js # fails +node --experimental-modules --type=module --experimental-json-modules index.js # works +``` + +## Experimental Loader hooks + +**Note: This API is currently being redesigned and will still change.**. + + + +To customize the default module resolution, loader hooks can optionally be +provided via a `--loader ./loader-name.mjs` argument to Node.js. + +When hooks are used they only apply to ES module loading and not to any +CommonJS modules loaded. + +### Resolve hook + +The resolve hook returns the resolved file URL and module format for a +given module specifier and parent file URL: + +```js +const baseURL = new URL('file://'); +baseURL.pathname = `${process.cwd()}/`; + +export async function resolve(specifier, + parentModuleURL = baseURL, + defaultResolver) { + return { + url: new URL(specifier, parentModuleURL).href, + format: 'esm' + }; +} +``` + +The `parentModuleURL` is provided as `undefined` when performing main Node.js +load itself. + +The default Node.js ES module resolution function is provided as a third +argument to the resolver for easy compatibility workflows. + +In addition to returning the resolved file URL value, the resolve hook also +returns a `format` property specifying the module format of the resolved +module. This can be one of the following: + +| `format` | Description | +| --- | --- | +| `'module'` | Load a standard JavaScript module | +| `'commonjs'` | Load a Node.js CommonJS module | +| `'builtin'` | Load a Node.js builtin module | +| `'json'` | Load a JSON file | +| `'dynamic'` | Use a [dynamic instantiate hook][] | + +For example, a dummy loader to load JavaScript restricted to browser resolution +rules with only JS file extension and Node.js builtin modules support could +be written: + +```js +import path from 'path'; +import process from 'process'; +import Module from 'module'; + +const builtins = Module.builtinModules; +const JS_EXTENSIONS = new Set(['.js', '.mjs']); + +const baseURL = new URL('file://'); +baseURL.pathname = `${process.cwd()}/`; + +export function resolve(specifier, parentModuleURL = baseURL, defaultResolve) { + if (builtins.includes(specifier)) { + return { + url: specifier, + format: 'builtin' + }; + } + if (/^\.{0,2}[/]/.test(specifier) !== true && !specifier.startsWith('file:')) { + // For node_modules support: + // return defaultResolve(specifier, parentModuleURL); + throw new Error( + `imports must begin with '/', './', or '../'; '${specifier}' does not`); + } + const resolved = new URL(specifier, parentModuleURL); + const ext = path.extname(resolved.pathname); + if (!JS_EXTENSIONS.has(ext)) { + throw new Error( + `Cannot load file with non-JavaScript file extension ${ext}.`); + } + return { + url: resolved.href, + format: 'esm' + }; +} +``` + +With this loader, running: + +```console +NODE_OPTIONS='--experimental-modules --loader ./custom-loader.mjs' node x.js +``` + +would load the module `x.js` as an ES module with relative resolution support +(with `node_modules` loading skipped in this example). + +### Dynamic instantiate hook + +To create a custom dynamic module that doesn't correspond to one of the +existing `format` interpretations, the `dynamicInstantiate` hook can be used. +This hook is called only for modules that return `format: 'dynamic'` from +the `resolve` hook. + +```js +export async function dynamicInstantiate(url) { + return { + exports: ['customExportName'], + execute: (exports) => { + // Get and set functions provided for pre-allocated export names + exports.customExportName.set('value'); + } + }; +} +``` + +With the list of module exports provided upfront, the `execute` function will +then be called at the exact point of module evaluation order for that module +in the import tree. + ## Resolution Algorithm ### Features @@ -286,94 +697,35 @@ READ_PACKAGE_JSON(_packageURL_) -The default Node.js ES module resolution function is provided as a third -argument to the resolver for easy compatibility workflows. - -In addition to returning the resolved file URL value, the resolve hook also -returns a `format` property specifying the module format of the resolved -module. This can be one of the following: - -| `format` | Description | -| --- | --- | -| `'module'` | Load a standard JavaScript module | -| `'commonjs'` | Load a Node.js CommonJS module | -| `'builtin'` | Load a Node.js builtin module | -| `'dynamic'` | Use a [dynamic instantiate hook][] | - -For example, a dummy loader to load JavaScript restricted to browser resolution -rules with only JS file extension and Node.js builtin modules support could -be written: - -```js -import path from 'path'; -import process from 'process'; -import Module from 'module'; - -const builtins = Module.builtinModules; -const JS_EXTENSIONS = new Set(['.js', '.mjs']); - -const baseURL = new URL('file://'); -baseURL.pathname = `${process.cwd()}/`; - -export function resolve(specifier, parentModuleURL = baseURL, defaultResolve) { - if (builtins.includes(specifier)) { - return { - url: specifier, - format: 'builtin' - }; - } - if (/^\.{0,2}[/]/.test(specifier) !== true && !specifier.startsWith('file:')) { - // For node_modules support: - // return defaultResolve(specifier, parentModuleURL); - throw new Error( - `imports must begin with '/', './', or '../'; '${specifier}' does not`); - } - const resolved = new URL(specifier, parentModuleURL); - const ext = path.extname(resolved.pathname); - if (!JS_EXTENSIONS.has(ext)) { - throw new Error( - `Cannot load file with non-JavaScript file extension ${ext}.`); - } - return { - url: resolved.href, - format: 'esm' - }; -} -``` - -With this loader, running: - -```console -NODE_OPTIONS='--experimental-modules --loader ./custom-loader.mjs' node x.js -``` - -would load the module `x.js` as an ES module with relative resolution support -(with `node_modules` loading skipped in this example). - -### Dynamic instantiate hook - -To create a custom dynamic module that doesn't correspond to one of the -existing `format` interpretations, the `dynamicInstantiate` hook can be used. -This hook is called only for modules that return `format: 'dynamic'` from -the `resolve` hook. - -```js -export async function dynamicInstantiate(url) { - return { - exports: ['customExportName'], - execute: (exports) => { - // Get and set functions provided for pre-allocated export names - exports.customExportName.set('value'); - } - }; -} +### Customizing ESM specifier resolution algorithm + +The current specifier resolution does not support all default behavior of +the CommonJS loader. One of the behavior differences is automatic resolution +of file extensions and the ability to import directories that have an index +file. + +By using the `--es-module-specifier-resolution=[mode]` flag you can customize +the extension resolution algorithm. The default mode is `explicit`, which +requires the full path to a module be provided to the loader. To enable the +automatic extension resolution and importing from directories that include an +index file use the `node` mode. + +```bash +$ node --experimental-modules index.mjs +success! +$ node --experimental-modules index #Failure! +Error: Cannot find module +$ node --experimental-modules --es-module-specifier-resolution=node index +success! ``` -With the list of module exports provided upfront, the `execute` function will -then be called at the exact point of module evaluation order for that module -in the import tree. - +[`import`]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Statements/import +[`export`]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Statements/export +[CommonJS]: modules.html [Node.js EP for ES Modules]: https://github.com/nodejs/node-eps/blob/master/002-es-modules.md [dynamic instantiate hook]: #esm_dynamic_instantiate_hook [`module.createRequireFromPath()`]: modules.html#modules_module_createrequirefrompath_filename -[ESM Minimal Kernel]: https://github.com/nodejs/modules/blob/master/doc/plan-for-new-modules-implementation.md +[`import.meta.url`]: #esm_import_meta +[`import()`]: #esm_import-expressions +[ecmascript-modules implementation]: https://github.com/nodejs/modules/blob/master/doc/plan-for-new-modules-implementation.md +[WHATWG JSON modules]: https://github.com/whatwg/html/issues/4315