forked from cockroachdb/cockroach
-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
bazel: upgrade to rules_nodejs 5.4.2
Upgrading to Bazel's rules_nodejs 5.x exposed a flaw in our previous Bazel integration: because rules_nodejs explicitly doesn't support yarn's "workspaces" feature [1] (in which common dependencies are hoisted to the lowest common parent directory), any NPM dependencies with different major versions between db-console and cluster-ui would get flattened to a single version. This left one of those packages using an unsupported (and un-requested) version of a dependency. Removing the yarn workspace layout and using separate Bazel repositories for each JS project ensured each project received the correct dependencies, but revealed incompatibilities with the requested versions. Upgrade rules_nodejs to the latest released version, remove yarn workspaces from the pkg/ui/ tree, and fix all revealed compatibility issues. [1] bazel-contrib/rules_nodejs#266
- Loading branch information
Showing
30 changed files
with
37,238 additions
and
18,813 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,16 @@ | ||
| #!/usr/bin/env bash | ||
| set -eu | ||
|
|
||
| if [ "$#" -ne 1 ]; then | ||
| echo "Usage: $0 PATH" | ||
| echo "Example: check_yarn_vendor_submodule.sh /path/to/cockroachdb/cockroach/pkg/ui/yarn-vendor" | ||
| exit 1 | ||
| fi | ||
|
|
||
| yarnVendorDir=$1 | ||
| if [ -z "$(ls -A $yarnVendorDir)" ]; then | ||
| echo "No packages available from yarn-vendor submodule." >&2 | ||
| echo "You may need to run 'git submodule update --init'." >&2 | ||
| exit 2 | ||
| fi | ||
|
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,85 @@ | ||
| load("@build_bazel_rules_nodejs//:index.bzl", "node_repositories", "yarn_install") | ||
|
|
||
| _doc = """ | ||
| Ensures the current user's global yarn cache (typically in ~/.cache/yarn) | ||
| contains all vendored yarn dependencies, so that later yarn_install calls can | ||
| be --offline without having access to the yarn-vendor submodule. | ||
| build_bazel_rules_nodejs's yarn_install can only get access to the | ||
| pkg/ui/yarn-vendor submodule by explicitly listing each .tgz file in that | ||
| submodule as a dependency, which causes every file to be copied individually | ||
| (in serial) and makes the yarn_install task unusably slow (> 50 minutes on a | ||
| 24-core Xeon machine with 32GB of RAM). To ensure yarn_install only uses | ||
| packages from the yarn-vendor submodule this rule helps to leverage the fact | ||
| that yarn_install has access to the user's "global" yarn cache, which is | ||
| typically used to accelerate `yarn install` by not repeatedly fetching packages | ||
| from remote locations. | ||
| Specifically, this rule performs one `yarn install` invocation for each package | ||
| from within the Bazel workspace (not the sandbox!) where access to | ||
| pkg/ui/yarn-vendor is guaranteed, and stores the resulting modules in a location | ||
| that intentionally won't be read. It relies on the fact that performing those | ||
| installations has a side-effect of adding each installed dependency to the | ||
| global yarn cache. | ||
| """ | ||
|
|
||
| def _seed_yarn_cache_impl(rctx): | ||
| workspace_root = str( | ||
| rctx.path( | ||
| Label("//:WORKSPACE") | ||
| ).dirname | ||
| ) | ||
|
|
||
| rctx.report_progress("Checking for yarn-vendor submodule...") | ||
| res = rctx.execute( | ||
| [ | ||
| rctx.path(Label("//build/bazelutil:check_yarn_vendor_submodule.sh")), | ||
| workspace_root + "/pkg/ui" | ||
| ], | ||
| ) | ||
| if res.return_code != 0: | ||
| fail("Unable to seed yarn cache: " + res.stderr) | ||
|
|
||
| paths = { | ||
| "protos": "pkg/ui/workspaces/db-console/src/js", | ||
| "cluster_ui": "pkg/ui/workspaces/cluster-ui", | ||
| "db_console": "pkg/ui/workspaces/db-console", | ||
| } | ||
| yarn_dir = str(rctx.path(Label("@yarn//:bin/yarn")).dirname) | ||
| for key, path in paths.items(): | ||
| rctx.report_progress("Seeding yarn cache for {}...".format(path)) | ||
| # Execute a script that uses the bazel-installed yarn (via $PATH) to install | ||
| # dependencies while running at the root of the *non-sandboxed* node | ||
| # package (argv[1]), but putting those dependencies in a repo-specific | ||
| # location (argv[2]) to avoid corrupting the local node_modules | ||
| # directories. | ||
| res = rctx.execute( | ||
| [ | ||
| rctx.path(Label("//build/bazelutil:seed_yarn_cache.sh")), | ||
| workspace_root + "/" + path, | ||
| rctx.path("node_modules." + key), | ||
| ], | ||
| environment = { | ||
| "PATH": "{}:{}".format(yarn_dir, rctx.os.environ["PATH"]) | ||
| }, | ||
| ) | ||
|
|
||
| if res.return_code != 0: | ||
| fail("Unable to seed yarn cache: " + res.stderr) | ||
|
|
||
| rctx.file( | ||
| ".seed", | ||
| content = "", | ||
| executable = False, | ||
| ) | ||
| rctx.file( | ||
| "BUILD.bazel", | ||
| content = """exports_files([".seed"])""", | ||
| executable = False, | ||
| ) | ||
|
|
||
| seed_yarn_cache = repository_rule( | ||
| implementation = _seed_yarn_cache_impl, | ||
| doc = _doc, | ||
| local = True, | ||
| ) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,20 @@ | ||
| #!/usr/bin/env bash | ||
| set -euo pipefail | ||
|
|
||
| if [ "$#" -ne 2 ]; then | ||
| echo "Usage: $0 PKG_PATH MODULES_DIR" | ||
| echo "Example: seed_yarn_cache.sh /path/to/cockroachdb/cockroach/pkg/ui/workspaces/cluster-ui /path/to/repo/node_modules.cluster_ui" | ||
| exit 1 | ||
| fi | ||
|
|
||
| rootDir=$1 | ||
| moduleDir=$2 | ||
|
|
||
| yarn \ | ||
| --cwd $rootDir \ | ||
| --modules-folder $moduleDir \ | ||
| --offline \ | ||
| --ignore-optional \ | ||
| --no-progress \ | ||
| --mutex network \ | ||
| install |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.