Add a set of features to the npm cli that provide support to managing multiple packages from within a singular top-level, root package.
This feature has been requested by the community for a long time. The primary motivation behind this RFC is to fully realize a set of features/functionality to support managing multiple packages that may or may not be used together.
The name “workspaces” is already established in the community with both Yarn and Pnpm implementing similar features under that same name so we chose to reuse it for the sake of simplicity to the larger community involved.
After sourcing feedback from the community, there are 3 major implementations/changes required in the npm cli in order to provide the feature set that would enable a better management of nested packages.
- Make the npm cli workspace-aware.
- Install: In a npm workspaces setup users expect to be able to install all nested packages and perform the associated lifecycle scripts from the Top-level workspace, it should also be aware of workspaces that have a dependency on one another and symlink them appropriately.
- Run some npm subcommands in the context of each workspace.
The set of features identified in this document are the ones that are essential to an initial MVP of the npm workspaces support. The community should expect further development of this feature based on the feedback we collected and documented at the end of this RFC.
First and foremost there’s the alternative of leaving the problem set for userland to solve, there’s already the very popular project Lerna that provides some of these features.
Also available is the alternative of supporting only the install (or bootstrap as Lerna names it) aspect of this proposal, following a less feature-rich approach but one that would still enable the basic goal of improving the user experience of managing multiple child packages but from all the feedback collected during the research phase of this RFC, this alternative is much less desirable to the community of maintainers involved.
We're following the lead of Yarn in supporting the workspaces package.json property which defines a list of paths, each of these paths may point to the location of a workspace in the file system but it also support globs.
package.json example:
{
"name": "workspace-example",
"version": "1.0.0",
"workspaces": {
"packages": [
"packages/*"
]
}
}
package.json shorthand example:
{
"name": "workspace-example",
"version": "1.0.0",
"workspaces": [
"packages/*"
]
}
The npm cli will read from the paths and globs defined in this workspaces configuration and look for valid package.json files in order to create a list of packages that will be treated as workspaces.
Change npm install (arborist) behavior to make it properly install dependencies for every workspace defined in the workspaces configuration described above.
Arborist should also be aware of all workspaces in order to correctly link to another internal workspace should it match the required semver version of an expected dependency anywhere in the installing tree. e.g:
// Given this package.json structure:
├── package.json { "workspaces": ["dep-a", "dep-b"] }
├── dep-a
│ └── package.json { "dependencies": { "dep-b": "^1.0.0" } }
└── dep-b
└── package.json { "version": "1.3.1" }
$ npm install
// Results in this symlinking structure:
├── node_modules
│ ├── dep-a -> ./dep-a
│ └── dep-b -> ./dep-b
├── dep-a
└── dep-b
For the initial workspaces implementation, we're going to stick with arborist's default algorithm that privileges hoisting packages but will place packages at nested node_modules whenever necessary.
Make npm cli subcommands npm workspaces-aware, so that running a command tries to run it within all workspaces as long as a workspaces configuration field is properly defined in package.json.
Only a subset of commands are to be supported:
fundList funding info for all workspaceslsList all packages including workspacesoutdatedList outdated deps including workspaces and its depsrun-scriptRun arbitrary scripts in all workspacesrebuildRebuild all workspacesrestartstartstoptestRun tests in all workspacesupdateUpdates a dependency across the entire installation tree, including workspacesviewView registry info but also including workspaces
A new npm cli configuration value should be introduced in order to allow for filtering out a subset of the workspaces in which to run these commands. e.g: --filter
├── package.json { "name": "foo", "workspaces": ["dep-a", "dep-b"] }
├── dep-a
│ └── package.json { "version": "1.0.0" }
└── dep-b
└── package.json { "version": "1.3.1" }
$ npm test
> [email protected] test /Users/username/foo
> echo "Error: no test specified" && exit 1
Error: no test specified
npm ERR! Test failed. See above for more details.
> [email protected] test /Users/username/foo/dep-a
> done
> [email protected] test /Users/username/foo/dep-b
> done
A Top-level workspace package may not be published to the registry and for all publishing purposes having a valid "workspaces" entry in a package.json is going to be the equivalent of "private": true.
Given a npm workspaces setup with the following contents:
$ cat ./package.json
{
"name": "foo",
"version": "1.0.0",
"workspaces": [
"./core/*",
"./packages/*"
],
dependencies: {
"lodash": "^4.x.x",
"libnpmutil": "^1.0.0"
}
}
$ cat ./core/libnpmutil/package.json
{
"name": "libnpmutil",
"version": "1.0.0",
"dependencies": {
"lodash": "^4.x.x"
}
}
$ cat ./packages/workspace-a/package.json
{
"name": "workspace-a",
"version": "1.7.3",
"peerDependencies": {
"react": "^16.x.x"
},
"dependencies": {
"workspace-b": "^2.0.0"
}
}
$ cat ./packages/worskpace-b/package.json
{
"name": "workspace-b",
"version": "2.1.1",
"peerDependencies": {
"react": "^16.x.x"
}
}
$ cat ./packages/workspace-c/package.json
{
"name": "workspace-c",
"version": "1.0.0",
"peerDependencies": {
"react": "^16.x.x"
},
"dependencies": {
"workspace-b": "^1.0.0"
}
}
Will result in the following symlinking structure:
$ tree
.
├── package-lock.json
├── node_modules
│ ├── lodash
│ ├── libnpmutil -> ./core/libnpmutil
│ ├── workspace-a -> ./packages/workspace-a
│ ├── workspace-b -> ./packages/workspace-b
│ ├── workspace-c -> ./packages/workspace-c
│ └── react
├── core
│ └── libnpmutil
└── packages
├── workspace-a
├── workspace-b
└─ worspace-c
└── node_modules
└── [email protected]
And the following package-lock.json files:
NOTE: The following lockfile is for illustration purpose only and its final shape might differ.
$ cat ./package-lock.json
{
"name": "foo",
"version": "1.0.0",
"lockfileVersion": 2,
"requires": true,
"packages": {
"": {
"name": "foo",
"version": "1.0.0",
"dependencies": {
"lodash": "^4.17.15",
"libnpmutil": "^1.0.0"
}
},
"core/libnpmutil": {
"name": "libnpmutil",
"version": "1.0.0"
},
"node_modules/lodash": {
"name": "lodash",
"version": "4.17.15",
"resolved": "https://registry.npmjs.org/lodash/-/lodash-4.17.15.tgz",
"integrity": "sha512-8xOcRHvCjnocdS5cpwXQXVzmmh5e5+saE2QGoeQmbKmRS6J3VQppPOIt0MnmE+4xlZoumy0GPG0D0MVIQbNA1A=="
},
"node_modules/libnpmutil": {
"resolved": "core/libnpmutil",
"link": true
},
"node_modules/workspace-a": {
"resolved": "packages/workspace-a",
"link": true
},
"node_modules/workspace-b": {
"resolved": "packages/workspace-b",
"link": true
},
"node_modules/workspace-c": {
"resolved": "packages/workspace-c",
"link": true
},
"packages/workspace-a": {
"name": "workspace-a",
"version": "1.7.3"
},
"packages/workspace-b": {
"name": "workspace-b",
"version": "1.0.0"
},
"packages/workspace-c": {
"name": "workspace-c",
"version": "1.0.0"
}
},
"dependencies": {
"lodash": {
"version": "4.17.15",
"resolved": "https://registry.npmjs.org/lodash/-/lodash-4.17.15.tgz",
"integrity": "sha512-8xOcRHvCjnocdS5cpwXQXVzmmh5e5+saE2QGoeQmbKmRS6J3VQppPOIt0MnmE+4xlZoumy0GPG0D0MVIQbNA1A=="
},
"libnpmutil": {
"version": "file:core/libnpmutil"
},
"workspace-a": {
"version": "file:packages/workspace-a"
},
"workspace-b": {
"version": "file:packages/workspace-b"
},
"workspace-c": {
"version": "file:packages/workspace-c",
"dependencies": {
"workspace-b": {
"version": "1.0.0",
"resolved": "https://registry.npmjs.org/workspace-b/-/workspace-b-1.0.0.tgz",
"integrity": "sha512-8xOcRHvCjnocdS5cpwXQXVzmmh5e5+saE2QGoeQmbKmRS6J3VQppPOIt0MnmE+4xlZoumy0GPG0D0MVIQbNA1A=="
},
}
}
}
}
During the discussions around this RFC it was brought up to our attention that a lot of the vocabulary surrounding what the larger JavaScript community understands as "workspaces" can be confusing, for the sake of keeping the discussion as productive as possible we're taking the extra step of documenting what each of the terms used here means:
- npm cli: The npm cli 😉
- npm workspaces: The feature name, meaning the ability to the npm cli to support a better workflow for working with multiple packages.
- workspaces: A set of workspaces.
- workspace: A nested package within the Top-level workspace file system that is explicitly defined as such via workspaces configuration.
- Top-level workspace: The root level package that contains a workspaces configuration defining workspaces.
- workspaces configuration: The blob of json configuration defined within
package.jsonthat declares where to find workspaces for this Top-level workspace package. - dependency: A package that is depended upon by another given package.
- dependent: A package which depends on another given package.
- symlink: A symbolic link between files.
- globs: String patterns that specifies sets of filenames with special characters.
- Arborist: The npm@7 install library
- hoisting packages: Bringing packages up a level in the context of an installation tree.
- Support for non-nested file structure workspaces (ex: adjacent directories)
- Should we support an opt-out config? e.g:
workspacesEnabled