-
Notifications
You must be signed in to change notification settings - Fork 8.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Ops] Bump Node.js to version 18 #160289
Merged
+378
−108
Merged
[Ops] Bump Node.js to version 18 #160289
Changes from 10 commits
Commits
Show all changes
27 commits
Select commit
Hold shift + click to select a range
ef3056c
feat: bump node.js to version 18.16.1
delanni 9b1f6cd
Set --openssl-legacy-provider for canvas runtime compilation
delanni 2eba4d5
feat: fix tests and types
delanni 16c32b8
feat: remove abort-controller, it's not compliant with the new node t…
delanni 4e42210
feat: Fix more type issues
delanni d4a66db
feat: set --openssl-legacy-provider as a default in the env setup
delanni 20eb469
feat: fix a few more tests related to the node bump
delanni cbb722d
Revert "feat: set --openssl-legacy-provider as a default in the env s…
delanni 147f4ed
feat: Fix last weird test error
delanni 1705f9f
feat: fix build integration test
delanni 3780e7f
docs: Add mentions for the custom node.js builds in the docs, add NOD…
delanni a49a631
chore: set resolutions for @types/node to a bit more strict version
delanni 2508c7f
Merge branch 'main' into bump-node-18
delanni 76a3eb9
Merge branch 'main' into bump-node-18
kibanamachine f335573
Merge branch 'main' into bump-node-18
kibanamachine 6c5026e
Merge branch 'main' into bump-node-18
mistic fe9654b
docs: Apply suggestions for docs
delanni 8ffbc3d
chore: remove debug option from bazel build
delanni d744829
feat: bump node's types to their latest
delanni 71e61b0
docs: remove NODE_BINARIES
delanni be95475
Remove duplicate reference to custom build
delanni 9519e2f
Merge branch 'main' into bump-node-18
kibanamachine 9b3dbca
Merge branch 'main' into bump-node-18
delanni d399a8e
Merge branch 'main' into bump-node-18
delanni 4b18fc3
feat: bump node.js to 18.17.0 and adjust types/docs
delanni c8f6be2
Merge branch 'main' into bump-node-18
kibanamachine 04072ea
feat: update sha hashes to glibc217 built resource hashes
delanni File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1 +1 @@ | ||
16.20.1 | ||
18.16.1 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1 +1 @@ | ||
16.20.1 | ||
18.16.1 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The hashes we give here belong to the official node.js distros, which I assume is fine, because these are binaries related to development, and not the ones we bundle with, right?
I wonder if we should change this bit to also load these files from our mirror, and use the custom-built variants on linux (for less hidden differences) - also using the proxy-cache mirror from line:32 might do away with a few of the Bazel cache warming job failures.
@elastic/kibana-operations - What do you think?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think it's a good idea, it could help us catch issues earlier.
Do we know if native binaries on the node distribution we're building are compatible with official distributions? Will a developer with a global, official installation of node and a bazel sandbox build with our glibc compatible version of something like
node-sass
work?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't think we need to worry about this tbh. Bazel should be short lived and its for development only. In addition to that Bazel has local caches so after fetching once it won't download the binaries again from the nodejs official mirror. Having that said I'm not opposed to change those lines to use our proxy cache links but I think there was something in the version of the bazel js rules we are in that was preventing us from doing that (not completely sure about this at the moment) and, again, as Bazel will be short lived I think we can avoid touching this for now 🙂
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@jbudz had a different concern though, whether the binaries are glibc compatible or whether that compatibility issue can happen with these binaries.
For now, we're not using a custom build pipeline to also build those binaries. We've just pre-built them for the missing platforms (modules version 108 for
linux-arm64
) and getting those from our proxy.Would it be a good way of testing this, if we use the artifacts from a successful "Build Kibana Distribution and Plugins" step, and use that to set up a CentOS7 image? Maybe
node-sass
is not used at the runtime butnode-re2
should be.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
it seems it's a good catch, @jbudz - I got kibana startup issues, then I tried requiring manually:
When using a re2 binary that was built with glibc2.17 (from earlier versions of the custom build pipeline), it can be imported, kibana works.
I tried the other way around, it seems the glibc217 built artifacts are going to be usable by other linux platforms too.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If I'm reading this correctly, does it means we will have to build and push node-sass and re2 native modules as part of our custom node js build pipeline to make sure we are always building those using glibc2.17? Otherwise they will fail in the machines consuming our custom node images. Is that what you were trying to mention here @delanni ? If thats the case we will probably have to recover your previous work for node-sass and re2 in the custom nodejs pipeline or document a way to build node-sass and node-re2 using glibc2.17 as needed and push those into ci-proxy-cache
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, I think. This is my impression from the tests. We'll need the libraries compiled using the custom node builds.
However, they're not exactly bound to the specific node.js version we're compiling them with, but the ones we have now will probably work for the whole Node 18.x.x family - so long as the modules versions (and apparently other native lib versions) don't change. We could probably live with the few pre-compiled binaries we have produced earlier, but for the long game, it's probably best to build them periodically.