Difficult to override remote JDKs when using Bzlmod

[swarren12]

A bit of background context: I'm trying to upgrade a project running in Bazel 6.5.0 to use more recent versions of, well, everything. Unfortunately, there's a lot of custom stuff in there that makes it less than trivial to just bump up version numbers. To try and simplify things, I'm looking to migrate the existing Bazel 6.5.0 WORKPSACE file to have as much stuff in MODULE.bazel as possible. Unfortunately, rules_java is proving tricky...

Expected behaviour: defining custom remote_java_repositories in MODULE.bazel should prevent the default remotejdk... repositories from being fetched
Actual behaviour: the remote repositories (at least one) are still fetched

More details:

Currently, the version of rules_java is 7.3.1; I'm hesitant to update the version as some later versions require Bazel >= 7.0.0 and
right now that version just doesn't work at all for me (although, then, some of the most recent drop the requirement back down to >= 6.2.0?). The original WORKSPACE file looked something like this (somewhat paraphrased for brevity):

http_archive(name = "rules_java", urls = ["https://internal.repo/mirrors/rules_java/7.3.2/rules_java-7.3.2.tar.gz"] ...)

load("@bazel_tools//tools/jdk:remote_java_repository.bzl", "remote_java_repository")

remote_java_repository(name = "jdk11", urls =  ["https://internal.repo/packages/jdk/11/jdk-11.0.22-linux.tar.gz"] ...)

register_toolchains("@jdk11//:all")

When I try and migrate this to bzlmod, I end up with something that looks like the following:

bazel_dep(name = "rules_java", version = "7.3.1",  dev_dependency = True)

custom_java_repositories = use_extension("//:custom_java_repositories.bzl", "custom_java_repositories")
use_repo(custom_java_repositories, "jdk11")

register_toolchains("@jdk11//:all")

where custom_java_repositories.bzl is a module_extension that looks a bit like this:

load("@rules_java//toolchains:remote_java_repository.bzl", "remote_java_repository")

def _impl(modctx):
    remote_java_repository(name = "jdk11", urls = ["https://internal.repo/packages/jdk/11/jdk-11.0.22-linux.tar.gz"] ...)

custom_java_repositories = module_extension(implementation = _impl)

However, rules_java is still very insistent in attempting to download remotejdk21_linux from https://mirror.bazel.build/, which doesn't work and causes the build to fail. I've seen some other issues talking about registering the custom toolchains earlier, but I don't think that's possible since @bazel_tools//tools/jdk:remote_java_repository doesn't seem to work from within a module_extension.

(Also, just in case it's relevant, the examples here specify jdk11, but I'm actually doing 11, 17, and 21.)

I've also tried setting --javabase=@jdk11//:jdk (and --java_toolchain and `--host_...) but they don't seen to have any effect.

This is probably a "pebcak" error but it's tricky to work out what's going on without some Bzlmod examples / documentation. I'm hoping I'm just "doing it wrong" and that maybe there's a really obvious answer?

[fmeum]

Your MODULE.bazel snippet lacks a register_toolchains call similar to what you had in WORKSPACE. Could you try to add it?

[swarren12]

Your MODULE.bazel snippet lacks a register_toolchains call similar to what you had in WORKSPACE. Could you try to add it?

Sorry, this was my mistake constructing the snippet. There was a register_toolchains() call, I just missed it out. I've updated the original snippet with it in now, just in case the format is wrong,

Edit: just in case it's a version thing, the actual file looks closer to:

use_repo(custom_java_repositories, "jdk11", "jdk17", "jdk21")

register_toolchains(
    "@jdk11//:all",
    "@jdk17//:all",
    "@jdk21//:all",
)

[fmeum]

Cc @hvadehra

[fmeum]

Could you post the output of running with --toolchain_resolution_debug=.*?

[swarren12]

Log from the main project is incredibly spammy (and has other problems) so I made a small minimal reproduction. For convenience, it just uses the downloader config to block access to cdn.azul.com.

Attached are the logs from running bazel test //... --toolchain_resolution_debug=.* with both Bzlmod disabled and enabled.

workspace.log
bzlmod.log

Hopefully this is helpful!

[swarren12]

Thanks, this appears to work in the minimal case. I'll port it over to the original project and report back 🤞

[swarren12]

Edit: fixed it, was entirely an error on my part! I'd typed register_toolchains("@jdk11//:all") instead of register_toolchains("//:all").

Quick update: although it's playing nicely with the example repository, it's not behaving in the main project. Trying to investigate why, because I can't see anything obviously different. I'm assuming one of the other rules that is being pulled in is causing some kind of conflict, so I'll do a bit more digging.

Another minor observation is that if I no disable bzlmod on the example repository, it fails to build. I'm not too bothered about this, but I find it interesting that the behaviour appears to differ depending on how the rules are being loaded?

Update: running with --toolchain_resolution_debug=".*" on gives (amongst all the other spam):

INFO: ToolchainResolution:   Type @bazel_tools//tools/jdk:runtime_toolchain_type: target platform @local_config_platform//:host: execution @local_config_platform//:host: Selected toolchain @jdk11//:jdk
...
INFO: ToolchainResolution: Target platform @local_config_platform//:host: Selected execution platform @local_config_platform//:host, type @bazel_tools//tools/jdk:toolchain_type -> toolchain @bazel_tools//tools/jdk:toolchain_java11
INFO: ToolchainResolution:   Type @bazel_tools//tools/jdk:runtime_toolchain_type: target platform @local_config_platform//:host: execution @local_config_platform//:host: Selected toolchain @jdk11//:jdk
...
INFO: ToolchainResolution:     Type @bazel_tools//tools/jdk:runtime_toolchain_type: target platform @local_config_platform//:host: execution platform @local_config_platform//:host: Skipping toolchain @jdk11//:jdk; execution platform already has selected toolchain
...
INFO: ToolchainResolution:   Type @bazel_tools//tools/jdk:runtime_toolchain_type: target platform @local_config_platform//:host: execution @local_config_platform//:host: Selected toolchain @jdk11//:jdk
...
Analyzing: 2133 targets (1527 packages loaded, 37915 targets configured)
...
INFO: ToolchainResolution:     Type @bazel_tools//tools/jdk:runtime_toolchain_type: target platform @local_config_platform//:host: Rejected toolchain @jdk11//:jdk; mismatching config settings: prefix_version_setting
...
INFO: ToolchainResolution:   Type @bazel_tools//tools/jdk:runtime_toolchain_type: target platform @local_config_platform//:host: execution @local_config_platform//:host: Selected toolchain @remotejdk11_linux//:jdk

Not sure why it's being accepted at first and then rejected later. Continuing to investigate!

[swarren12]

@hvadehra just to confirm, this appears to all be working now. I'll leave the issue open for now, and if I have some free time at the end of the week I'll look at raising a PR with some Bzlmod examples if you think that would be helpful.

Thanks!

[BrandonHerrada]

This was an incredibly helpful example for me. Thanks!

[ed-irl]

I stumbled on this after having some difficulty with registering a java toolchain.

Would someone mind posting a complete example? I've done something like...

load("@rules_java//toolchains:remote_java_repository.bzl", "remote_java_repository")

def _impl(ctx):
    remote_java_repository(
        name = "jdk22",
        # When updating, find the release here https://adoptium.net/temurin/releases/
        # You will need to click the download link, which will forward you to a download page and automatically initiate
        # a download. This page will also contain a direct link which can be copied, and a link to the sha256.
        urls = [
            "https://github.com/adoptium/temurin22-binaries/releases/download/jdk-22.0.2%2B9/OpenJDK22U-jdk_aarch64_mac_hotspot_22.0.2_9.tar.gz",
        ],
        target_compatible_with = [
            "@platforms//os:macos",
            "@platforms//cpu:aarch64",
        ],
        version = "22",
        strip_prefix = "jdk-22.0.2+9",  # maybe add /Contents/Home?
        sha256 = "6f7bd166c46094cf48018b018b161eae838a01dddce9d6fc5800eed353d37b84",
    )

java_repos = module_extension(implementation = _impl)

and then in MODULE.bazel:

bazel_dep(name = "rules_java", version = "7.4.0")

custom_java_repositories = use_extension("//tools/java_repos:java_repos.bzl", "java_repos")
use_repo(custom_java_repositories, "jdk22", "jdk22_toolchain_config_repo")

register_toolchains("@jdk22_toolchain_config_repo//:all")

This gets me a runtime java toolchain but no JDK toolchain. Adding , "@jdk22//:all" to register_toolchains as suggested by the comments in remote_java_toolchain produces an error:

ERROR: Traceback (most recent call last):
        File "/private/var/tmp/_bazel_ekohlwey/bb77f95dcfbd0c560e2d2a842a7d7b94/external/_main~java_repos~jdk22/BUILD.bazel", line 76, column 66, in <toplevel>
                java = glob(["bin/java.exe", "bin/java"], allow_empty = True)[0],
Error: index out of range (index is 0, but sequence has 0 elements)
ERROR: package contains errors: : Traceback (most recent call last):
        File "/private/var/tmp/_bazel_ekohlwey/bb77f95dcfbd0c560e2d2a842a7d7b94/external/_main~java_repos~jdk22/BUILD.bazel", line 76, column 66, in <toplevel>
                java = glob(["bin/java.exe", "bin/java"], allow_empty = True)[0],
Error: index out of range (index is 0, but sequence has 0 elements)
ERROR: .../BUILD:128:12: While resolving toolchains for target ... (1558acc): invalid registered toolchain '@jdk22//:all': Error evaluating '@jdk22//:all': error loading package '@@_main~java_repos~jdk22//': Package '' contains errors
ERROR: Analysis of target '...' failed; build aborted

When I inspect the archive, however, it seems to be structurally similar to other packages (eg. zulu) I can find.

Anyways, a full example would be much appreciated!

[swarren12]

Adding , "@jdk22//:all" to register_toolchains as suggested by the comments in remote_java_toolchain produces an error:

At a guess, I think this is roughly what you want (minus the error, of course). From the stack you posted, it seems that it's failing whilst attempting to register the toolchain, due to not finding bin/java. I had a quick look at the archive and I noticed that the structure of the OSX one is a bit different to the Linux one.

strip_prefix = "jdk-22.0.2+9", # maybe add /Contents/Home?

What happens if you change this as per the comment, i.e. strip_prefix = "jdk-22.0.2+9/Contents/Home"?

When I inspect the archive, however, it seems to be structurally similar to other packages (eg. zulu) I can find.

I just had a quick nosey at the OSX package for Zulu 21, and I think the key difference is that one has a set of symlinks at the defined content root, whilst the Hostpot archive that you're grabbing does not.