Skip to content

Commit

Permalink
Bump Bazel and dependencies to prepare for Bzlmod
Browse files Browse the repository at this point in the history
This begins the Bzlmod compatibility migration by updating Bazel to
version 7.3.2 and adding initial `MODULE.bazel` and `WORKSPACE.bzlmod`
files.

Part of: bazelbuild#1482

Though Bzlmod remains disabled, updating to Bazel 7.3.2 requred updating
or adding the following packages to maintain `WORKSPACE` compatibility.

In `rules_scala_setup()`:

- bazel_skylib: 1.4.1 => 1.7.1
- rules_cc: 0.0.6 => 0.0.10
- rules_java: 5.4.1 => 7.9.0
- rules_proto: 5.3.0-21.7 => 6.0.2

Dev dependencies in `WORKSPACE`:

- com_google_protobuf: 28.2
- rules_pkg: 1.0.1
- rules_jvm_external: 6.4
- com_google_absl: abseil-cpp-20240722.0
- zlib: 1.3.1

Of all of the new, explicit dev dependencies, only `com_google_protobuf`
will be necessary to include in `MODULE.bazel`. The Bzlmod mechanism
will discover these other transitive dev dependencies automatically.

Also removed the `rules_java_extra` repo from `WORKSPACE`, which
appeared unused.

---

Though the current `rules_java` version is 7.12.1, and largely works
with this repo, it requires a few temporary workarounds. Rather than
commit the workarounds, upgrading only to 7.9.0 now seems less crufty.

What follows is a very detailed explanation of what happens with 7.12.1
with Bazel 7.3.2, just to have it on the record.

---

The workaround is to change a few toolchain and macro file targets from
`@bazel_tools//tools/jdk:` to `@rules_java//toolchains:`. This isn't a
terribly bad or invasive workaround, but `@bazel_tools//tools/jdk:` is
clearly the canonical path. Best to keep it that way, lest we build up
technical debt.

Without the workaround, these targets would fail:

- //test/src/main/resources/java_sources:CompiledWithJava11
- //test/src/main/resources/java_sources:CompiledWithJava8
- //test/toolchains:java21_toolchain
- //test:JunitRuntimePlatform
- //test:JunitRuntimePlatform_test_runner
- //test:scala_binary_jdk_11

with this error:

```txt
ERROR: .../external/rules_java_builtin/toolchains/BUILD:254:14:

While resolving toolchains for target
@@rules_java_builtin//toolchains:platformclasspath (096dcc8):

No matching toolchains found for types
@@bazel_tools//tools/jdk:bootstrap_runtime_toolchain_type.
```

This appears to be a consequence of both upgrading the Bazel version
from 6.3.0 to 7.3.2 and updating `rules_java` to 7.12.1. The
`rules_java_builtin` repo is part of the `WORKSPACE` prefix that adds
implicit dependencies:

- https://bazel.build/external/migration#builtin-default-deps

This repo was added to 7.0.0-pre.20231011.2 in the following change,
mapped to `@rules_java` within the scope of the `@bazel_tools` repo:

- bazelbuild/bazel: Add rules_java_builtin to the users of Java modules
  bazelbuild/bazel@ff1abb2

This change tried to ensure `rules_java` remained compatible with
earlier Bazel versions. However, it changed all instances of
`@bazel_tools//tools/jdk:bootstrap_runtime_toolchain_type` to
`//toolchains:bootstrap_runtime_toolchain_type`:

- bazelbuild/rules_java: Make rules_java backwards compatible with Bazel
  6.3.0
  bazelbuild/rules_java@30ecf3f

Bazel has bumped `rules_java` in its `workspace_deps.bzl` from 7.9.0 to
7.11.0, but it's only available as of 8.0.0-pre.20240911.1.

- bazelbuild/bazel: Update rules_java 7.11.1 / java_tools 13.8
  bazelbuild/bazel#23571
  bazelbuild/bazel@f92124a

---

What I believe is happening is, under Bazel 7.3.2 and `rules_java`
7.12.1:

- Bazel creates `rules_java` 7.9.0 as `@rules_java_builtin` in the
  `WORKSPACE` prefix.

- `@bazel_tools` has `@rules_java` mapped to `@rules_java_builtin` when
  initialized during the `WORKSPACE` prefix, during which
  `@bazel_tools//tools/jdk` registers `alias()` targets to
  `@rules_java` toolchain targets. These aliased toolchains specify
  `@bazel_tools//tools/jdk:bootstrap_runtime_toolchain_type` in their
  `toolchains` attribute.

- `WORKSPACE` loads `@rules_java` 7.12.1 and registers all its
  toolchains with type
  `@rules_java//toolchains:bootstrap_runtime_toolchain_type`.

- Some `@rules_java` rules explicitly specifying toolchains from
  `@bazel_tools//tools/jdk` can't find them, because the
  `@bazel_tools//tools/jdk` toolchain aliases expect toolchains of type
  `@bazel_tools//tools/jdk:bootstrap_runtime_toolchain_type`.

This has broken other projects in the same way:

- bazelbuild/bazel: [Bazel CI] Downstream project broken by rules_java
  upgrade #23619
  bazelbuild/bazel#23619

These problems don't appear under Bzlmod, and `@rules_java_builtin` was
never required. This is because `WORKSPACE` executes its statements
sequentially, while Bzlmod builds the module dependency graph _before_
instantiating repositories (within module extensions).

It seems a fix is on the way that removes `@rules_java_builtin` from the
`WORKSPACE` prefix, and adds `@rules_java` to the suffix. At this
moment, though, it's not even in a prerelase:

- bazelbuild/bazel: Remove rules_java_builtin in WORKSPACE prefix
  bazelbuild/bazel@7506690

---

Note that the error message matches that from the following resolved
issue, but that issue was for non-Bzlmod child repos when `WORKSPACE`
was disabled.

- bazelbuild/bazel: Undefined @@rules_java_builtin repository with
  --noenable_workspace option
  bazelbuild/bazel#22754
  • Loading branch information
mbland committed Oct 3, 2024
1 parent 3da60a8 commit abc941a
Show file tree
Hide file tree
Showing 8 changed files with 88 additions and 33 deletions.
5 changes: 5 additions & 0 deletions .bazelrc
Original file line number Diff line number Diff line change
Expand Up @@ -3,3 +3,8 @@ build --enable_platform_specific_config
#Windows needs --worker_quit_after_build due to workers not being shut down when the compiler tools need to be rebuilt (resulting in 'file in use' errors). See Bazel Issue#10498.

build:windows --worker_quit_after_build --enable_runfiles

# Remove these upon completing Bzlmod compatibility work.
# - https://github.com/bazelbuild/rules_scala/issues/1482
build --noenable_bzlmod
build --enable_workspace
2 changes: 1 addition & 1 deletion .bazelversion
Original file line number Diff line number Diff line change
@@ -1 +1 @@
6.3.0
7.3.2
6 changes: 5 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -10,4 +10,8 @@ hash2
.vscode
unformatted-*.backup.scala
.scala-build
test/semanticdb/tempsrc
test/semanticdb/tempsrc

# Consider removing after Bzlmod conversion is complete.
# - https://github.com/bazelbuild/rules_scala/issues/1482
MODULE.bazel.lock
8 changes: 8 additions & 0 deletions MODULE.bazel
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
"""Bazel module definition for rules_scala"""

# Bzlmod compatibility work is currently underway. See:
# - https://github.com/bazelbuild/rules_scala/issues/1482

module(
name = "rules_scala", version = "6.6.1", repo_name = "io_bazel_rules_scala"
)
73 changes: 55 additions & 18 deletions WORKSPACE
Original file line number Diff line number Diff line change
Expand Up @@ -34,13 +34,13 @@ rules_scala_setup()

rules_scala_toolchain_deps_repositories(fetch_sources = True)

load("@rules_proto//proto:repositories.bzl", "rules_proto_dependencies", "rules_proto_toolchains")

# Declares @com_google_protobuf//:protoc pointing to released binary
# This should stop building protoc during bazel build
# See https://github.com/bazelbuild/rules_proto/pull/36
load("@rules_proto//proto:repositories.bzl", "rules_proto_dependencies")
rules_proto_dependencies()

load("@rules_proto//proto:setup.bzl", "rules_proto_setup")
rules_proto_setup()

load("@rules_proto//proto:toolchains.bzl", "rules_proto_toolchains")
rules_proto_toolchains()

load("//scala:scala_cross_version.bzl", "default_maven_server_urls")
Expand Down Expand Up @@ -144,19 +144,6 @@ go_rules_dependencies()

go_register_toolchains(version = "1.19.5")

# Explicitly pull in a different (newer) version of rules_java for remote jdks
rules_java_extra_version = "5.1.0"

rules_java_extra_sha = "d974a2d6e1a534856d1b60ad6a15e57f3970d8596fbb0bb17b9ee26ca209332a"

rules_java_extra_url = "https://github.com/bazelbuild/rules_java/releases/download/{}/rules_java-{}.tar.gz".format(rules_java_extra_version, rules_java_extra_version)

http_archive(
name = "rules_java_extra",
sha256 = rules_java_extra_sha,
url = rules_java_extra_url,
)

load("@rules_java//java:repositories.bzl", "remote_jdk8_repos")

# We need to select based on platform when we use these
Expand Down Expand Up @@ -209,3 +196,53 @@ load("//test/toolchains:jdk.bzl", "remote_jdk21_repositories", "remote_jdk21_too
remote_jdk21_repositories()

remote_jdk21_toolchains()

http_archive(
name = "com_google_protobuf",
sha256 = "b2340aa47faf7ef10a0328190319d3f3bee1b24f426d4ce8f4253b6f27ce16db",
strip_prefix = "protobuf-28.2",
url = "https://github.com/protocolbuffers/protobuf/releases/download/v28.2/protobuf-28.2.tar.gz",
)

http_archive(
name = "rules_pkg",
sha256 = "d20c951960ed77cb7b341c2a59488534e494d5ad1d30c4818c736d57772a9fef",
url = "https://github.com/bazelbuild/rules_pkg/releases/download/1.0.1/rules_pkg-1.0.1.tar.gz",
)

load("@rules_pkg//:deps.bzl", "rules_pkg_dependencies")

rules_pkg_dependencies()

RULES_JVM_EXTERNAL_TAG = "6.4"
RULES_JVM_EXTERNAL_SHA = "85776be6d8fe64abf26f463a8e12cd4c15be927348397180a01693610da7ec90"

http_archive(
name = "rules_jvm_external",
strip_prefix = "rules_jvm_external-%s" % RULES_JVM_EXTERNAL_TAG,
sha256 = RULES_JVM_EXTERNAL_SHA,
url = "https://github.com/bazel-contrib/rules_jvm_external/releases/download/%s/rules_jvm_external-%s.tar.gz" % (RULES_JVM_EXTERNAL_TAG, RULES_JVM_EXTERNAL_TAG)
)

load("@rules_jvm_external//:repositories.bzl", "rules_jvm_external_deps")

rules_jvm_external_deps()

load("@rules_jvm_external//:setup.bzl", "rules_jvm_external_setup")

rules_jvm_external_setup()

http_archive(
name = "com_google_absl",
strip_prefix = "abseil-cpp-20240722.0",
sha256 = "f50e5ac311a81382da7fa75b97310e4b9006474f9560ac46f54a9967f07d4ae3",
url = "https://github.com/abseil/abseil-cpp/releases/download/20240722.0/abseil-cpp-20240722.0.tar.gz",
)

http_archive(
name = "zlib",
strip_prefix = "zlib-1.3.1",
sha256 = "9a93b2b7dfdac77ceba5a558a580e74667dd6fede4585b91eefb60f03b72df23",
url = "https://github.com/madler/zlib/releases/download/v1.3.1/zlib-1.3.1.tar.gz",
build_file = "@com_google_protobuf//third_party:zlib.BUILD",
)
2 changes: 2 additions & 0 deletions WORKSPACE.bzlmod
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
# Bzlmod compatibility work is currently underway. See:
# - https://github.com/bazelbuild/rules_scala/issues/1482
23 changes: 11 additions & 12 deletions scala/private/macros/scala_repositories.bzl
Original file line number Diff line number Diff line change
Expand Up @@ -85,38 +85,37 @@ def rules_scala_setup(scala_compiler_srcjar = None):
if not native.existing_rule("bazel_skylib"):
http_archive(
name = "bazel_skylib",
sha256 = "b8a1527901774180afc798aeb28c4634bdccf19c4d98e7bdd1ce79d1fe9aaad7",
sha256 = "bc283cdfcd526a52c3201279cda4bc298652efa898b10b4db0837dc51652756f",
urls = [
"https://mirror.bazel.build/github.com/bazelbuild/bazel-skylib/releases/download/1.4.1/bazel-skylib-1.4.1.tar.gz",
"https://github.com/bazelbuild/bazel-skylib/releases/download/1.4.1/bazel-skylib-1.4.1.tar.gz",
"https://mirror.bazel.build/github.com/bazelbuild/bazel-skylib/releases/download/1.7.1/bazel-skylib-1.4.1.tar.gz",
"https://github.com/bazelbuild/bazel-skylib/releases/download/1.7.1/bazel-skylib-1.7.1.tar.gz",
],
)

if not native.existing_rule("rules_cc"):
http_archive(
name = "rules_cc",
urls = ["https://github.com/bazelbuild/rules_cc/releases/download/0.0.6/rules_cc-0.0.6.tar.gz"],
sha256 = "3d9e271e2876ba42e114c9b9bc51454e379cbf0ec9ef9d40e2ae4cec61a31b40",
strip_prefix = "rules_cc-0.0.6",
urls = ["https://github.com/bazelbuild/rules_cc/releases/download/0.0.10/rules_cc-0.0.10.tar.gz"],
sha256 = "65b67b81c6da378f136cc7e7e14ee08d5b9375973427eceb8c773a4f69fa7e49",
strip_prefix = "rules_cc-0.0.10",
)

if not native.existing_rule("rules_java"):
http_archive(
name = "rules_java",
urls = [
"https://github.com/bazelbuild/rules_java/releases/download/5.4.1/rules_java-5.4.1.tar.gz",
"https://github.com/bazelbuild/rules_java/releases/download/7.9.0/rules_java-7.9.0.tar.gz",
],
sha256 = "a1f82b730b9c6395d3653032bd7e3a660f9d5ddb1099f427c1e1fe768f92e395",
sha256 = "41131de4417de70b9597e6ebd515168ed0ba843a325dc54a81b92d7af9a7b3ea",
)

if not native.existing_rule("rules_proto"):
http_archive(
name = "rules_proto",
sha256 = "dc3fb206a2cb3441b485eb1e423165b231235a1ea9b031b4433cf7bc1fa460dd",
strip_prefix = "rules_proto-5.3.0-21.7",
sha256 = "6fb6767d1bef535310547e03247f7518b03487740c11b6c6adb7952033fe1295",
strip_prefix = "rules_proto-6.0.2",
urls = [
"https://mirror.bazel.build/github.com/bazelbuild/rules_proto/archive/refs/tags/5.3.0-21.7.tar.gz",
"https://github.com/bazelbuild/rules_proto/archive/refs/tags/5.3.0-21.7.tar.gz",
"https://github.com/bazelbuild/rules_proto/releases/download/6.0.2/rules_proto-6.0.2.tar.gz",
],
)

Expand Down
2 changes: 1 addition & 1 deletion test/proto_cross_repo_boundary/repo.bzl
Original file line number Diff line number Diff line change
Expand Up @@ -2,5 +2,5 @@ def proto_cross_repo_boundary_repository():
native.new_local_repository(
name = "proto_cross_repo_boundary",
path = "test/proto_cross_repo_boundary/repo",
build_file = "test/proto_cross_repo_boundary/repo/BUILD.repo",
build_file = "//test/proto_cross_repo_boundary:repo/BUILD.repo",
)

0 comments on commit abc941a

Please sign in to comment.