rules_docker target //docs:update broken on M1 mac with 5.1.0rc2

[fmeum]

Description of the problem / feature request:

On an M1 Mac, rules_dockers target //docs:update` builds with Bazel 5.1.0rc1, but fails with a toolchain error with Bazel 5.1.0rc2, Bazel 5.1.0rc3 as well as on head.
This was initially discovered by @linzhp in #15043 (comment).

Bugs: what's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

$ git clone https://github.com/bazelbuild/rules_docker
$ cd rules_docker
$ USE_BAZEL_VERSION=5.1.0rc1 bazelisk build //docs:update
...
INFO: Build completed successfully, 5 total actions
$ USE_BAZEL_VERSION=5.1.0rc2 bazelisk build //docs:update
DEBUG: /private/var/tmp/_bazel_zplin/d6323f9e3146540840037c805ccd73ff/external/bazel_toolchains/rules/rbe_repo/version_check.bzl:59:14: 
Current running Bazel is not a release version and one was not defined explicitly in rbe_autoconfig target. Falling back to '4.0.0'
ERROR: /private/var/tmp/_bazel_zplin/d6323f9e3146540840037c805ccd73ff/external/bazel_tools/src/tools/launcher/BUILD:9:14: While resolving toolchains for target @bazel_tools//src/tools/launcher:launcher: No matching toolchains found for types @bazel_tools//tools/cpp:toolchain_type. Maybe --incompatible_use_cc_configure_from_rules_cc has been flipped and there is no default C++ toolchain added in the WORKSPACE file? See https://github.com/bazelbuild/bazel/issues/10134 for details and migration instructions.
ERROR: Analysis of target '//docs:update' failed; build aborted: 
INFO: Elapsed time: 6.538s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (50 packages loaded, 309 targets configured)

What operating system are you running Bazel on?

macOS M1

What's the output of bazel info release?

release 5.1.0rc2

Have you found anything relevant by searching the web?

No

Any other information, logs, or outputs that you want to share?

No

[fmeum]

@Wyverald

[fmeum]

My bisect points to 785c7ec.

[fmeum]

@linzhp Locally, on my M1 mac, the contents of external/platforms/cpu/BUILD indicate that @platforms//cpu:aarch64 is not yet an alias for @platforms//cpu:arm64, which would explain the issue. Does rules_docker perhaps pull in an old version of @platforms that does not yet contain bazelbuild/platforms@e20c932? As it stands, Bazel 5.1.0rc2 will not work on M1 Macs with outdated versions of @platforms.

@brentleyjones

[brentleyjones]

@thii @keith

[meteorcloudy]

There is a similar error reported back to rules_docker a long time ago: bazelbuild/rules_docker#1988

rules_docker still use Bazel 4.0.0 on CI: https://github.com/bazelbuild/rules_docker/blob/094a012750a9b12f3c92634d34cb16f3314ea2c2/.bazelci/presubmit.yml#L6

/cc @uhthomas

[Wyverald]

huh, I cannot reproduce this on my M1 mac at all. It builds completely fine:

2022/03/22 16:45:15 Downloading https://releases.bazel.build/5.1.0/rc2/bazel-5.1.0rc2-darwin-x86_64...
Extracting Bazel installation...
Starting local Bazel server and connecting to it...
INFO: Invocation ID: 352f0975-f7d7-4fed-a32d-b21264db35a8
DEBUG: /private/var/tmp/_bazel_wyv/a5cd0e91b3fbbd08f2cb64fffd950b6a/external/bazel_toolchains/rules/rbe_repo/checked_in.bzl:106:14: buildkite_config not using checked in configs as bazel rc version was used 
INFO: Analyzed target //docs:update (69 packages loaded, 529 targets configured).
INFO: Found 1 target...
Target //docs:update up-to-date:
  bazel-bin/docs/gen_update.sh
  bazel-bin/docs/update
INFO: Elapsed time: 22.377s, Critical Path: 0.04s
INFO: 1 process: 1 internal.
INFO: Build completed successfully, 1 total action

What's weirder is that my $OUTPUT_BASE/external/platforms/cpu/BUILD is indeed the old one (i.e. aarch64 is NOT an alias).

[linzhp]

@Wyverald I am able to build //docs:update using the Bazel binary downloaded by Bazelisk, but I can't build //docs:update using the Bazel binary built from source:

➜  rules_docker git:(master) /Users/zplin/Github/bazel/bazel-bin/src/bazel-dev build //docs:update 
Extracting Bazel installation...
Starting local Bazel server and connecting to it...
DEBUG: /private/var/tmp/_bazel_zplin/d6323f9e3146540840037c805ccd73ff/external/bazel_toolchains/rules/rbe_repo/version_check.bzl:59:14: 
Current running Bazel is not a release version and one was not defined explicitly in rbe_autoconfig target. Falling back to '4.0.0'
ERROR: /private/var/tmp/_bazel_zplin/d6323f9e3146540840037c805ccd73ff/external/bazel_tools/src/tools/launcher/BUILD:9:14: While resolving toolchains for target @bazel_tools//src/tools/launcher:launcher: No matching toolchains found for types @bazel_tools//tools/cpp:toolchain_type. Maybe --incompatible_use_cc_configure_from_rules_cc has been flipped and there is no default C++ toolchain added in the WORKSPACE file? See https://github.com/bazelbuild/bazel/issues/10134 for details and migration instructions.
ERROR: Analysis of target '//docs:update' failed; build aborted: 
INFO: Elapsed time: 7.763s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (58 packages loaded, 341 targets configured)

[fmeum]

huh, I cannot reproduce this on my M1 mac at all. It builds completely fine:

2022/03/22 16:45:15 Downloading https://releases.bazel.build/5.1.0/rc2/bazel-5.1.0rc2-darwin-x86_64...
...

@Wyverald Your Bazel is living in the (x86_64) simulation.

[meteorcloudy]

I did some debugging with the --experimental_repository_resolved_file flag, it turns out indeed it's because rules_docker is still using a very old version (681f1ee032566aa2d443cf0335d012925d9c58d4) of @platforms which is introduced as follow:

          "original_rule_class": "@bazel_tools//tools/build_defs/repo:http.bzl%http_archive",
          "definition_information": "Repository platforms instantiated at:\n  /Users/pcloudy/workspace/rules_docker/WORKSPACE:38:15: in <toplevel>\n  /Users/pcloudy/workspace/rules_docker/repositories/deps.bzl:38:12: in deps\n  /Users/pcloudy/workspace/rules_docker/repositories/go_repositories.bzl:39:26: in go_deps\n  /private/var/tmp/_bazel_pcloudy/8ebef652ed379d3bd5bf69a98d6be676/external/io_bazel_rules_go/go/private/repositories.bzl:44:11: in go_rules_dependencies\n  /private/var/tmp/_bazel_pcloudy/8ebef652ed379d3bd5bf69a98d6be676/external/io_bazel_rules_go/go/private/repositories.bzl:279:18: in _maybe\nRepository rule http_archive defined at:\n  /private/var/tmp/_bazel_pcloudy/8ebef652ed379d3bd5bf69a98d6be676/external/bazel_tools/tools/build_defs/repo/http.bzl:353:31: in <toplevel>\n",
          "original_attributes": {
               "name": "platforms",
               "generator_name": "platforms",
               "generator_function": "deps",
               "generator_location": None,
               "urls": [
                    "https://mirror.bazel.build/github.com/bazelbuild/platforms/archive/681f1ee032566aa2d443cf0335d012925d9c58d4.zip",
                    "https://github.com/bazelbuild/platforms/archive/681f1ee032566aa2d443cf0335d012925d9c58d4.zip"
               ],
               "sha256": "ae95e4bfcd9f66e9dc73a92cee0107fede74163f788e3deefe00f3aaae75c431",
               "strip_prefix": "platforms-681f1ee032566aa2d443cf0335d012925d9c58d4"
          },

I think this should be fixed from rules_docker instead of being a release blocker.

[Wyverald]

Your Bazel is living in the (x86_64) simulation.

Ah, that's why. Maybe it's because I have my Bazelisk installed from go install? How'd you set up Bazelisk to get the arm64 version?

[linzhp]

The Python version of Bazelisk will automatically use the arm64 version if you run it on M1 mac

[fmeum]

Based on @meteorcloudy's analysis, I will close this issue as it's something that should be fixed in rules_docker. Sorry for the false alarm, I just didn't want to risk letting this go unnoticed in case it were on Bazel.

[keith]

This old platforms issue might be hit by more folks as well google/flatbuffers#7195