[Discussion] Upgrade node and npm to latest LTS versions

[fluiddot]

Intro

The purpose of this issue is to serve as the source of truth for the topic related to upgrading the node and npm versions, as well as be the central point for discussing how to move it forward.

See also a related PR in WordPress core: WordPress/wordpress-develop#4028.

Context

Currently, Gutenberg allows the following versions:

• node 14+ (>=14.0.0)
• npm 6 (>=6.9.0 <7)

The problem is that the recommended node version is in maintenance and the End-of-life is April, 30th, 2023, so it would be great to upgrade it. However, new node versions use npm versions above 6, which are currently blocked due to issues with dependencies (see below under the Blockers section).

Goal

Upgrade to the latest LTS version of node:

• node 18.14.2
• npm 9.5.0

Or alternatively, if we find blockers using npm 9, upgrade to the latest LTS version that supports npm 8:

• node 18.13.0
• npm 8.19.3

_{NOTE: End-of-life of node 18 is 2025-04-30.}

Blockers

The main blockers at this point are related to dependencies/peer dependencies, in the following sections are described:

1. @testing-library/react-native dependency ✅

Error logs

npm ERR! code ERESOLVE
npm ERR! ERESOLVE could not resolve
npm ERR!
npm ERR! While resolving: @testing-library/[email protected]
npm ERR! Found: [email protected]
npm ERR! node_modules/jest
npm ERR!   dev jest@"27.4.5" from the root project
npm ERR!   peer jest@"^27.0.0" from [email protected]
npm ERR!   node_modules/jest-watch-typeahead
npm ERR!     dev jest-watch-typeahead@"1.0.0" from the root project
npm ERR!   7 more (snapshot-diff, @wordpress/e2e-test-utils, ...)
npm ERR!
npm ERR! Could not resolve dependency:
npm ERR! peerOptional jest@">=28.0.0" from @testing-library/[email protected]
npm ERR! node_modules/@testing-library/react-native
npm ERR!   dev @testing-library/react-native@"11.3.0" from the root project
npm ERR!
npm ERR! Conflicting peer dependency: [email protected]
npm ERR! node_modules/jest
npm ERR!   peerOptional jest@">=28.0.0" from @testing-library/[email protected]
npm ERR!   node_modules/@testing-library/react-native
npm ERR!     dev @testing-library/react-native@"11.3.0" from the root project

• Requires jest@">=28.0.0"
• Gutenberg is currently using [email protected]

🔧 How to solve it: There's an ongoing effort to upgrade Jest version to 29 in #47388, so we can either downgrade the dependency @testing-library/react-native to a previous version (10.1.1) or wait to the PR to be merged.

UPDATE: ✅ This issue has been solved with #47388.

2. react-native dependency ❌

Error logs

npm ERR! code ERESOLVE
npm ERR! ERESOLVE could not resolve
npm ERR!
npm ERR! While resolving: [email protected]
npm ERR! Found: [email protected]
npm ERR! node_modules/react
npm ERR!   dev react@"18.2.0" from the root project
npm ERR!   peer react@">=16.8.0" from @emotion/[email protected]
npm ERR!   node_modules/@emotion/primitives-core
npm ERR!     @emotion/primitives-core@"^11.0.0" from @emotion/[email protected]
npm ERR!     node_modules/@emotion/native
npm ERR!       dev @emotion/native@"11.0.0" from the root project
npm ERR!   83 more (@emotion/react, @emotion/styled, ...)
npm ERR!
npm ERR! Could not resolve dependency:
npm ERR! peer react@"18.0.0" from [email protected]
npm ERR! node_modules/react-native
npm ERR!   dev react-native@"0.69.4" from the root project
npm ERR!   peer react-native@">=0.14.0 <1" from @emotion/[email protected]
npm ERR!   node_modules/@emotion/native
npm ERR!     dev @emotion/native@"11.0.0" from the root project
npm ERR!   19 more (@react-native-clipboard/clipboard, ...)
npm ERR!
npm ERR! Conflicting peer dependency: [email protected]
npm ERR! node_modules/react
npm ERR!   peer react@"18.0.0" from [email protected]
npm ERR!   node_modules/react-native
npm ERR!     dev react-native@"0.69.4" from the root project
npm ERR!     peer react-native@">=0.14.0 <1" from @emotion/[email protected]
npm ERR!     node_modules/@emotion/native
npm ERR!       dev @emotion/native@"11.0.0" from the root project
npm ERR!     19 more (@react-native-clipboard/clipboard, ...)

• Requires react@"18.0.0"
• Gutenberg is currently using [email protected]

🔧 How to solve it: The optimal approach for this would be to upgrade React Native to version 0.71.0. However, in the meantime, we could override the dependency as a workaround.

3. Other warnings related to peer dependencies ⚠️

Warning logs

npm WARN ERESOLVE overriding peer dependency
npm WARN While resolving: @mdx-js/[email protected]
npm WARN Found: [email protected]
npm WARN node_modules/react
npm WARN   dev react@"18.2.0" from the root project
npm WARN   86 more (@emotion/primitives-core, @emotion/react, ...)
npm WARN
npm WARN Could not resolve dependency:
npm WARN peer react@"^16.13.1 || ^17.0.0" from @mdx-js/[email protected]
npm WARN node_modules/@mdx-js/react
npm WARN   @mdx-js/react@"^1.6.22" from @storybook/[email protected]
npm WARN   node_modules/@storybook/addon-docs
npm WARN
npm WARN Conflicting peer dependency: [email protected]
npm WARN node_modules/react
npm WARN   peer react@"^16.13.1 || ^17.0.0" from @mdx-js/[email protected]
npm WARN   node_modules/@mdx-js/react
npm WARN     @mdx-js/react@"^1.6.22" from @storybook/[email protected]
npm WARN     node_modules/@storybook/addon-docs
npm WARN ERESOLVE overriding peer dependency
npm WARN While resolving: [email protected]
npm WARN Found: [email protected]
npm WARN node_modules/ajv
npm WARN   dev ajv@"8.7.1" from the root project
npm WARN   4 more (ajv-draft-04, ajv-errors, ajv-formats, table)
npm WARN
npm WARN Could not resolve dependency:
npm WARN peer ajv@"^6.9.1" from [email protected]
npm WARN node_modules/ajv-keywords
npm WARN   ajv-keywords@"^3.4.1" from [email protected]
npm WARN   node_modules/@storybook/core-common/node_modules/webpack
npm WARN   4 more (webpack, webpack, schema-utils, schema-utils)
npm WARN
npm WARN Conflicting peer dependency: [email protected]
npm WARN node_modules/ajv
npm WARN   peer ajv@"^6.9.1" from [email protected]
npm WARN   node_modules/ajv-keywords
npm WARN     ajv-keywords@"^3.4.1" from [email protected]
npm WARN     node_modules/@storybook/core-common/node_modules/webpack
npm WARN     4 more (webpack, webpack, schema-utils, schema-utils)
npm WARN ERESOLVE overriding peer dependency
npm WARN While resolving: [email protected]
npm WARN Found: [email protected]
npm WARN node_modules/react
npm WARN   dev react@"18.2.0" from the root project
npm WARN   86 more (@emotion/primitives-core, @emotion/react, ...)
npm WARN
npm WARN Could not resolve dependency:
npm WARN peer react@"^0.14.8 || ^15.0.1 || ^16.0.0 || ^17.0.1" from [email protected]
npm WARN node_modules/react-element-to-jsx-string
npm WARN   react-element-to-jsx-string@"^14.3.4" from @storybook/[email protected]
npm WARN   node_modules/@storybook/react
npm WARN
npm WARN Conflicting peer dependency: [email protected]
npm WARN node_modules/react
npm WARN   peer react@"^0.14.8 || ^15.0.1 || ^16.0.0 || ^17.0.1" from [email protected]
npm WARN   node_modules/react-element-to-jsx-string
npm WARN     react-element-to-jsx-string@"^14.3.4" from @storybook/[email protected]
npm WARN     node_modules/@storybook/react
npm WARN ERESOLVE overriding peer dependency
npm WARN While resolving: [email protected]
npm WARN Found: [email protected]
npm WARN node_modules/react-dom
npm WARN   dev react-dom@"18.2.0" from the root project
npm WARN   52 more (@floating-ui/react-dom, @react-spring/web, ...)
npm WARN
npm WARN Could not resolve dependency:
npm WARN peer react-dom@"^0.14.8 || ^15.0.1 || ^16.0.0 || ^17.0.1" from [email protected]
npm WARN node_modules/react-element-to-jsx-string
npm WARN   react-element-to-jsx-string@"^14.3.4" from @storybook/[email protected]
npm WARN   node_modules/@storybook/react
npm WARN
npm WARN Conflicting peer dependency: [email protected]
npm WARN node_modules/react-dom
npm WARN   peer react-dom@"^0.14.8 || ^15.0.1 || ^16.0.0 || ^17.0.1" from [email protected]
npm WARN   node_modules/react-element-to-jsx-string
npm WARN     react-element-to-jsx-string@"^14.3.4" from @storybook/[email protected]
npm WARN     node_modules/@storybook/react
npm WARN ERESOLVE overriding peer dependency
npm WARN While resolving: [email protected]
npm WARN Found: [email protected]
npm WARN node_modules/react
npm WARN   dev react@"18.2.0" from the root project
npm WARN   86 more (@emotion/primitives-core, @emotion/react, ...)
npm WARN
npm WARN Could not resolve dependency:
npm WARN peer react@"^16.8.4 || ^17.0.0" from [email protected]
npm WARN node_modules/react-inspector
npm WARN   react-inspector@"^5.1.0" from @storybook/[email protected]
npm WARN   node_modules/@storybook/addon-actions
npm WARN
npm WARN Conflicting peer dependency: [email protected]
npm WARN node_modules/react
npm WARN   peer react@"^16.8.4 || ^17.0.0" from [email protected]
npm WARN   node_modules/react-inspector
npm WARN     react-inspector@"^5.1.0" from @storybook/[email protected]
npm WARN     node_modules/@storybook/addon-actions
npm WARN ERESOLVE overriding peer dependency
npm WARN While resolving: [email protected]
npm WARN Found: [email protected]
npm WARN node_modules/webpack
npm WARN   dev webpack@"5.65.0" from the root project
npm WARN   35 more (@pmmmwh/react-refresh-webpack-plugin, ...)
npm WARN
npm WARN Could not resolve dependency:
npm WARN peer webpack@"^2.0.0 || ^3.0.0 || ^4.0.0" from [email protected]
npm WARN node_modules/webpack-filter-warnings-plugin
npm WARN   webpack-filter-warnings-plugin@"^1.2.1" from @storybook/[email protected]
npm WARN   node_modules/@storybook/builder-webpack4
npm WARN
npm WARN Conflicting peer dependency: [email protected]
npm WARN node_modules/webpack
npm WARN   peer webpack@"^2.0.0 || ^3.0.0 || ^4.0.0" from [email protected]
npm WARN   node_modules/webpack-filter-warnings-plugin
npm WARN     webpack-filter-warnings-plugin@"^1.2.1" from @storybook/[email protected]
npm WARN     node_modules/@storybook/builder-webpack4

🔧 How to solve it: It's likely that these warnings won't block using npm 8 but it would be interesting to address them in the future, especially for using npm 9.

Affected dependencies:

• @mdx-js/react
• ajv-keywords
• react-element-to-jsx-string
• react-inspector
• webpack-filter-warnings-plugin

Related opened issues/PRs

• Packages: Unify peer dependencies between React Native packages #34801
• Allow developing Gutenberg with NPM v7 and v8 #46400
• Dependency conflict when installing dependencies using npm 8 #46443
• Cannot npm install with npm 7 and higher #46652
• Packages: Issues with peer dependencies with npm7+ after React 18 upgrade #48009
• [RNMobile] Test npm v8 #40517
• Fix dependency conflicts to allow using NPM 7 and 8 #47712

_{NOTE: Please feel free to edit the above with issues/PRs related to this topic.}

[fluiddot]

In relation to the React Native dependencies (@testing-library/react-native and react-native), I proposed a solution in this draft PR. The idea is to use the setting overrides and override these dependencies to the required versions:

gutenberg/package.json

Lines 246 to 253 in 9827fcb

	"overrides": {
	"[email protected]": {
	"react": "18.2.0"
	},
	"@testing-library/[email protected]": {
	"jest": "27.4.5"
	}
	},

NOTE: This solution is a temporary workaround until we upgrade the React Native and Jest versions.

[tomdevisser]

@fluiddot Thanks for this issue and the work you put in already, I think it's very important to address this. I have no experience with upgrading Node in a project like this, but will follow this to learn about it and think along where I can.

As there's not much time left before the end of April, is there a way we can give this a higher priority? Should it be added to a particular milestone?

And if we're not able to figure this out and merge it before the EOL, what are the risks of being late? Will things break, are there security implications or will it "just" not receive support anymore?

[gziolo]

@fluiddot, thank you for opening this issue and collecting related efforts to synchronize work.

I updated the description and included a reference to a related PR in WordPress core: WordPress/wordpress-develop#4028.

After April 30th, once Gutenberg runs with NodeJS 18 and npm 9, we will need to follow up with upgrading the minimum required version to Node 16.0.0 and npm 7.10.0 that ships with it. @gvgvgvijayan made me realize that after opening #48368 last week. We will have to update engines field in package.json files, notes in README files, and list those changes in CHANGELOG files as a breaking to enforce a major version bump for updated packages. I found some prior art in #43141.

[fluiddot]

I'm thinking about a potential plan in order to move the upgrade forward:

1. Fix the current dependency conflicts

Following the approach mentioned in this comment, we could fix the current dependency conflicts related to React native (@testing-library/react-native and react-native). This will allow starting using npm 7 or 8, although the package-lock.json file will be modified every time we install the dependencies.

ℹ️ I can work on this and have a PR ready today/tomorrow.

2. Decide the target version of node to upgrade to

We have two options:

• Upgrade to node 18.14.2 / npm 9.5.0.
• Upgrade to node 18.13.0 / npm 8.19.3.

If we go with the former, we'd need to fix other dependency conflicts that only happen when using npm 9. If we go with the latter, I think we could successfully migrate although we'd have the warnings mentioned in 3. Other warnings related to peer dependencies ⚠️ when installing the dependencies.

3. Upgrade node version

As @gziolo mentioned in this comment, we'd need to update engines field in package.json files, notes in README files, and list those changes in CHANGELOG files as a breaking to enforce a major version bump for updated packages. We could use #43141 as a guide to know what needs to be updated when creating a PR for this.

---

How does this plan sound to you? Please let me know if you have any thoughts/concerns. Thanks 🙇 !

[gziolo]

An excellent action plan, @fluiddot. I don't have strong opinions about npm version. We enforce the maximum version of Node and npm so increasing that to node 18.13.0 / npm 8.19.3 is still better than waiting more time. If that helps splitting that into two steps sounds reasonable.

[desrosj]

Thanks for working on this all! I haven't taken a deep dive into the challenges on the Gutenberg side yet, but I have on the Core side in Trac-56658 (the updates in both locations will need to be coordinated to happen relatively at the same time). For the related Core changes, see WordPress/wordpress-develop#4028.

It's my strong preference to upgrade to the latest Node and npm versions in one change instead of a two step process if possible. Each time we change the version requirements for these tools, we have to communicate it to contributors so that everyone can make any needed adjustments to their workflows. I know nvm makes this really easy, but in reality, not everyone has this installed and configured.

Another thing to keep in mind is that 7.x of npm upgrade the lock file format. When we update to npm >= 7.0.0, we should also update the engine at the same time. The lock file versions are forwards and backwards compatible, but contributors on 6.x will see the lock file convert back to the v1 format making it impossible for someone using npm 6.x to make any changes to the lock file.

[gziolo]

For visibility, @kevin940726 opened PR #48950 that upgrades Node to v18 and npm to v9. It contains more changes the discussed above, like:

• npm workspaces
• using npx instead of npm bin
• the minimum version for Node is set to v18 for WordPress packages (v14 end of maintenance at the end of April, v16 - end of October)

[fluiddot]

Now that #47388 is merged, I updated the PR's description to reflect that the issue with @testing-library/react-native dependency dependency is solved 🎊 .

[fluiddot]

Regarding #48950 PR, I also opened a companion PR that aims to solve the issue with the React Native dependency: #48990.

[noahtallen]

This should be partially resolved now with #53426, putting Gutenberg on Node 16 and NPM 8. Legacy peer dep mode is still enabled until we can fix the remaining issues! More here: https://make.wordpress.org/core/2023/08/09/updating-wordpress-to-use-more-modern-versions-of-node-js-npm/

[gziolo]

We still need to pick more changes prepared by @kevin940726 ing #48950:

• All documentation changes where Node 14 and npm 6 was listed as the requirement for contribution and plugin/theme development.
• Initiate breaking changes in all WordPress packages to increase the minimum Node and npm (where applicable, we might be able to drop the npm part as discussed).
• Enable support for npm workspaces.

There might be more work ready to land in the branch 😄

Finally, we should remove the legacy-peer-deps flag override and start using the new npm handling of peer dependencies. This might require some audit of how we handle peer dependencies in WordPress packages and increase the usage of this good practice.

[fluiddot]

⚠️ I'd like to share that node 16 (the current version we use in Gutenberg) has already reached its End-Of-Life (2023-09-11).

Based on the package.json file, I understand that we could use node 18 but the main restriction comes from npm and using version 9. As I shared in the Goal section of issue's description, I wonder if we could apply the workaround of using node 18 by pointing to version 18.13.0, as it still uses npm 8. WDY?

In relation to this, upgrading to node 18 will be a blocker for future upgrades of React Native as starting in version 0.73 it's likely that the minimum supported node version is 18.

[gziolo]

@fluiddot, can you also comment here https://make.wordpress.org/systems/2023/02/09/upgrade-nodejs-npm-on-the-build-server/ about EOL for Node 16?

[swissspidy]

Node 18 is now supported.

Ticket and PR for core to go to Node 18:

• https://core.trac.wordpress.org/ticket/59663
• Bump the minimum required version of Node.js to 20.x wordpress-develop#5515

[noahtallen]

By the way, Node 20 is now the current active LTS. @fluiddot, should we reach out to systems, try to get on v20, and skip v18 in the meantime?

[swissspidy]

Worth reaching out, but I'm skeptical about the timeline for that.

GitHub Actions is also skipping Node 18 and going straight to Node 20, see https://github.blog/changelog/2023-09-22-github-actions-transitioning-from-node-16-to-node-20/

If we could do that as well, it would mean having to maintain fewer Node versions across all branches.

On the other hand, the JS ecosystem isn't usually fast to catch up, so not sure how easy the migration would be for Node 20.

[noahtallen]

I don't think Node 20 is supposed to have many breaking changes, though I suppose it's worth trying in a branch!

[noahtallen]

Draft here, just to see what failures come up: #55561

[noahtallen]

Looks like CI is passing, so I think on the Gutenberg side, this will be an easy update. I'm not familiar with the core WP side though

[fluiddot]

By the way, Node 20 is now the current active LTS.

Hey @noahtallen 👋. Good point, following this documentation, node 18 is now in Maintenance mode until 2025-04-30. Hence, it would be ideal to use node 20 now that it's the current LTS.

In this regard, I'd like to note that the issues we had in the past have been related to upgrading npm, specifically when upgrading from version 6 to 8. Currently, we exclude npm version above 8 (reference), so I'm wondering if we'd identify potential blockers when upgrading to npm 10 🤔.

In any case, I'd lean toward choosing the easiest and quicker approach, either using version 18 or 20, as we still use node 16 which is already deprecated.

@fluiddot, should we reach out to systems, try to get on v20, and skip v18 in the meantime?

Sure, we could reach out to systems to notify them about using node 20. But as shared here, I'm also skeptical about a potential ETA.

[desrosj]

Worth reaching out, but I'm skeptical about the timeline for that.

I agree. It's unlikely we will get this support added on the build server this calendar year. But we can still make a request. If there are specific features in 20.x that could be utilized, we could build a more compelling case for it to be made available.

GitHub Actions is also skipping Node 18 and going straight to Node 20, see https://github.blog/changelog/2023-09-22-github-actions-transitioning-from-node-16-to-node-20/

Just noting that this only really applies to actions such as actions/checkout, though. actions/setup-node still supports any version of Node/npm. Good to look at what others are doing (and skipping 18 is also my preference), but it's not technically a restriction for our decision here.

18.x will also be in LTS maintenance mode for 18 months. There is a longer term security being on 18.x. 16.x was a bit of a unicorn because the EOL date was moved up by 7 months, and we were also on 14.x for a really long time.

I don't think Node 20 is supposed to have many breaking changes, though I suppose it's worth trying in a branch!

I think the issue that we may encounter will be dependencies having an upper limit to the version they allow similar to how we do. I think it's worth exploring so we know what the challenges will be, but there may be a bit of a waiting game for packages to update.

I'd also like to see us get to a state where we can remove legacy-peer-deps = true from .npmrc. I think that should be explored before 20.x.

With the WordPress 6.4 release coming out on November 7th, let's hold off on this until after that when contributors will have more time. I plan to circle back to this the following week and we can coordinate updating Core SVN and Gutenberg at the same time.

[noahtallen]

In my PR above, I did update to npm 10 at the same time, and everything appears to be working correctly in CI.

I'd love to remove legacy peer does too, but I don't think it makes a big difference which Node/npm version we're on when we do that.

By the way, do we know why it takes so long to make another Node binary available on the build servers?

[noahtallen]

I opened a wordpress-develop PR to test Node 20 and npm 10 for core: WordPress/wordpress-develop#5571

[noahtallen]

I don't think I have access to create posts on https://make.wordpress.org/systems/. Could someone else create the request so that we could at get the process started?

[fluiddot]

I don't think I have access to create posts on https://make.wordpress.org/systems/. Could someone else create the request so that we could at get the process started?

I don't have permission either to create posts there. @desrosj since you are the author of the post Upgrade NodeJS/npm on the Build Server, I wonder if you could help us with this. Thanks 🙇 !

[desrosj]

I couldn't find an open PR that stages updating to Node.js 18.x, so I've gone and made one: #56331.

I also updated WordPress/wordpress-develop#5515. One thing that's different than the usual approach is that I removed the upper limit to the versions that can be used. In my testing, there are no problems when using 20.x. Even if we can't use 20.x as the version used to build the final software, I think it's reasonable to allow use of it and test against it as long as the results are the same.

For example, recently the format of the package-lock.json file completely changed. Running the install command on version 7 vs. 6 would cause the format to flip back and forth, which was not ideal because it would potentially create a ton of churn on lock files. If a situation like this is encountered, we would need to explore pinning a version maximum again.

Happy to post again on the systems blog as well. But I'd like to have a better reasoning than "please install the latest version" because that will put the request into their backlog instead of at the forefront.

[noahtallen]

But I'd like to have a better reasoning than "please install the latest version" because that will put the request into their backlog instead of at the forefront.

I think the main reason to update is because Node version 16 is now at end of life. I'm not sure exactly what it means, but I assume no more bug (and maybe even security) fixes

Node 20 is the current LTS, and I think it should be any project's priority to remain on the current, stable, supported versions of software. We also technically don't want the latest version (21), just the current LTS, which is Node 20. ;)

I didn't realize that Node 18 was already available on that build server; that makes things easier for us! :)

However, I'm not sure there will ever be a more urgent reason, until Node 18 reaches EoL in 2025. I feel like this shouldn't be an ad-hoc process -- we should just be in the habit of updating to the current LTS as it becomes active each year. It often becomes more work the longer you wait. So maybe it'd still be worth creating the request, because we don't want it to suddenly become urgent at the last minute a year down the road. I guess the other approach is just doing it at the last minute when it becomes a security issue (e.g. Node 14 and now Node 16).

The other thing on my mind is that if we could just skip Node 18 and go to 20 right away, that makes developer's lives slightly easier :)

Just my opinion about it!

[flootr]

I second what @noahtallen says and would argue staying on top of the current LTS version of Node.js is what we should aim for.

[desrosj]

Thanks for all the work folks! The changes have been merged to both Gutenberg and WordPress Core, and I've gone and published an announcement: https://make.wordpress.org/core/2023/12/20/updating-wordpress-to-use-more-modern-versions-of-node-js-npm-2/.

See you all in April for 22.x 😉