Readme instructions related to RBE will only work on Linux

[mickael-carl]

Currently the README states that one can use the RBE container image by using the rbe_autoconfig rule from bazel-toolchains. However, this will only work when running on Linux, otherwise failing the build:

ERROR: An error occurred during the fetch of repository 'rbe_default':
   Traceback (most recent call last):
	File "/private/var/tmp/_bazel_mcarl/b866f019b4047dbc85cbb95ed2dc14e0/external/bazel_toolchains/rules/rbe_repo.bzl", line 505
		validate_host(ctx)
	File "/private/var/tmp/_bazel_mcarl/b866f019b4047dbc85cbb95ed2dc14e0/external/bazel_toolchains/rules/rbe_repo/util.bzl", line 147, in validate_host
		fail(<1 more arguments>)
Not running on linux host, cannot run rbe_autoconfig.

[MikhailTymchukFT]

I have different problem, but it also related to non-Linux deployment.

First, I run the local docker compose on MacOS Catalina 10.15.5:

docker-compose git:(master) ./run.sh
+ worker=worker-ubuntu16-04
+ sudo rm -rf bb worker-ubuntu16-04
Password:
+ mkdir -m 0777 worker-ubuntu16-04 worker-ubuntu16-04/build
+ mkdir -m 0700 worker-ubuntu16-04/cache
+ exec docker-compose up
Creating network "docker-compose_default" with the default driver
Pulling frontend (buildbarn/bb-storage:20200520T103928Z-32c5ba0)...
20200520T103928Z-32c5ba0: Pulling from buildbarn/bb-storage
e8d8785a314f: Already exists
c4039fbebd78: Pull complete
Digest: sha256:0b3ef680f92c9d074a1bf831cd5f7931369cb69ba9bfc296f3f28b800272c7f8
Status: Downloaded newer image for buildbarn/bb-storage:20200520T103928Z-32c5ba0
Pulling scheduler (buildbarn/bb-scheduler:20200528T061211Z-4dfbe5b)...
20200528T061211Z-4dfbe5b: Pulling from buildbarn/bb-scheduler
e8d8785a314f: Already exists
39eed11e6584: Pull complete
Digest: sha256:7a13dc4269c5baabf8ca98f9f56fee767135773f43291766b69fcee4da7de626
Status: Downloaded newer image for buildbarn/bb-scheduler:20200528T061211Z-4dfbe5b
Pulling browser (buildbarn/bb-browser:20200514T172056Z-1c2932b)...
20200514T172056Z-1c2932b: Pulling from buildbarn/bb-browser
e8d8785a314f: Already exists
abb0808c462c: Pull complete
da8a650ac503: Pull complete
85cea451eec0: Pull complete
Digest: sha256:99e00ebb1f41235b6ded6078eade055c28bc311d4a340370da7dd945bcb4855f
Status: Downloaded newer image for buildbarn/bb-browser:20200514T172056Z-1c2932b
Pulling worker-ubuntu16-04 (buildbarn/bb-worker:20200528T061211Z-4dfbe5b)...
20200528T061211Z-4dfbe5b: Pulling from buildbarn/bb-worker
e8d8785a314f: Already exists
6493ba6fd943: Pull complete
Digest: sha256:18775c0eef7606183405dcb259b6992047d6dcd43141f0b8f17e669dc51d0763
Status: Downloaded newer image for buildbarn/bb-worker:20200528T061211Z-4dfbe5b
Pulling runner-installer (buildbarn/bb-runner-installer:20200528T061211Z-4dfbe5b)...
20200528T061211Z-4dfbe5b: Pulling from buildbarn/bb-runner-installer
e2334dd9fee4: Pull complete
d4427b3ffe3f: Pull complete
228e897726cb: Pull complete
875b116ae1d7: Pull complete
85cea451eec0: Pull complete
Digest: sha256:acd29de0b78e2768da2b1a903d9b4e305c309e79316627563a4ef1c846f57f5e
Status: Downloaded newer image for buildbarn/bb-runner-installer:20200528T061211Z-4dfbe5b
Pulling runner-ubuntu16-04 (l.gcr.io/google/rbe-ubuntu16-04@sha256:b516a2d69537cb40a7c6a7d92d0008abb29fba8725243772bdaf2c83f1be2272)...
sha256:b516a2d69537cb40a7c6a7d92d0008abb29fba8725243772bdaf2c83f1be2272: Pulling from google/rbe-ubuntu16-04
bd47987755ba: Pull complete
831c222b21d8: Pull complete
3c2cba919283: Pull complete
6426cbdb4893: Pull complete
e8193af77640: Pull complete
Digest: sha256:b516a2d69537cb40a7c6a7d92d0008abb29fba8725243772bdaf2c83f1be2272
Status: Downloaded newer image for l.gcr.io/google/rbe-ubuntu16-04@sha256:b516a2d69537cb40a7c6a7d92d0008abb29fba8725243772bdaf2c83f1be2272
Creating docker-compose_scheduler_1          ... done
Creating docker-compose_browser_1            ... done
Creating docker-compose_frontend_1           ... done
Creating docker-compose_storage-1_1          ... done
Creating docker-compose_storage-0_1          ... done
Creating docker-compose_runner-installer_1   ... done
Creating docker-compose_worker-ubuntu16-04_1 ... done
Creating docker-compose_runner-ubuntu16-04_1 ... done
Attaching to docker-compose_browser_1, docker-compose_scheduler_1, docker-compose_runner-installer_1, docker-compose_worker-ubuntu16-04_1, docker-compose_frontend_1, docker-compose_storage-1_1, docker-compose_storage-0_1, docker-compose_runner-ubuntu16-04_1
runner-installer_1    | /bb
worker-ubuntu16-04_1  | 2020/07/06 17:01:16 Runner is not ready yet: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial unix /worker/runner: connect: no such file or directory"
worker-ubuntu16-04_1  | 2020/07/06 17:01:19 Runner is not ready yet: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial unix /worker/runner: connect: no such file or directory"
docker-compose_runner-installer_1 exited with code 0

From the another terminal I run the build:

bb-deployments git:(master) ✗ bazel build --config=mycluster-ubuntu16-04 @abseil-hello//:hello_main
INFO: Invocation ID: 2f9b811a-7c42-4bcb-9b90-c57ad7b08202
DEBUG: /private/var/tmp/_bazel_mikhailtymchuk/b6a42162c9844064bacf5d15342d9b51/external/bazel_toolchains/rules/rbe_repo.bzl:477:5: Bazel 2.1.0 is used in rbe_default.
INFO: Analyzed target @abseil-hello//:hello_main (0 packages loaded, 0 targets configured).
INFO: Found 1 target...
ERROR: /private/var/tmp/_bazel_mikhailtymchuk/b6a42162c9844064bacf5d15342d9b51/external/com_google_absl/absl/base/BUILD.bazel:40:1: C++ compilation of rule '@com_google_absl//absl/base:log_severity' failed (Exit 34). Note: Remote connection/protocol failed with: execution failed FAILED_PRECONDITION: No workers exist for instance "remote-execution" platform {"properties":[{"name":"OSFamily","value":"Linux"},{"name":"container-image","value":"docker://marketplace.gcr.io/google/rbe-ubuntu16-04@sha256:06f8f8e9f97daa1c15466536dc2a7ae6641d16962d7b58a393af8060e460f571"}]}
Target @abseil-hello//:hello_main failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 0.610s, Critical Path: 0.11s
INFO: 0 processes.
FAILED: Build did NOT complete successfully

What to adjust to make it work?

[finn-ball]

@MikhailTymchukFT - First of all, can you redeploy and make sure you wait until this line has completed:

docker-compose_runner-installer_1 exited with code 0

Try again, it will probably still fail. If it does, you could try this. Remove the following lines from the config and redeploy:
https://github.com/buildbarn/bb-deployments/blob/master/docker-compose/config/worker-ubuntu16-04.jsonnet#L22-L25

And also comment out lines 4, 8 and and 11 from your .bazelrc:

build:rbe-ubuntu16-04 --extra_execution_platforms=@rbe_default//config:platform
build:rbe-ubuntu16-04 --host_platform=@rbe_default//config:platform
build:rbe-ubuntu16-04 --platforms=@rbe_default//config:platform

[EdSchouten]

Try again, it will probably still fail. If it does, you could try this. Remove the following lines from the config and redeploy:
https://github.com/buildbarn/bb-deployments/blob/master/docker-compose/config/worker-ubuntu16-04.jsonnet#L22-L25

Huh? Why would you remove those lines entirely?

@MikhailTymchukFT I would suggest that you edit both config/worker-ubuntu16-04.jsonnet and docker-compose.yml. In both those files, you will find a reference to a container image called l.gcr.io/google/rbe-ubuntu16-04@sha256:..... You are likely using a newer version of bazel-toolchains that depends on a more recent version of this container image.

Just copy-paste the sha256:... that you see in Bazel's error message and paste that into those configuration files. If you then restart Buildbarn, it will use this container image as well, thereby bringing the toolchain definitions used by Bazel and the container image used by Buildbarn in sync.

[finn-ball]

Huh? Why would you remove those lines entirely?

Just to remove the variable of the platform from the problem and debug from there. Though as you said, it's probably an old container.

[MikhailTymchukFT]

@EdSchouten Cool, it works!

However, I have another issue, not completely related to Buildbarn.
The first successful run was with Bazel 2.1.0 which uses bazel_toolchains 10.0.0 version of config according to CONTAINER_TO_CONFIG_SPEC_NAMES mapping and _BAZEL_TO_CONFIG_SPEC_NAMES mapping.

Then I upgraded to Bazel 2.1.1 and built again. I saw a new message:

DEBUG: /private/var/tmp/_bazel_mikhailtymchuk/b6a42162c9844064bacf5d15342d9b51/external/bazel_toolchains/rules/rbe_repo.bzl:477:5: Bazel 2.1.0 is used in rbe_default.

Seems like I continue to use bazel_toolchains 10.0.0 config version, but Bazel 2.1.1 can use both 10.0.0 and 11.0.0:

"2.1.1": ["10.0.0", "11.0.0"]

How can I switch to 11.0.0? More specifically, I want to get rid of this debug message, is it the right way to do?

[moroten]

Seems like the problem was solved. rbe_autoconfig is deprecated. Maybe documentation of rbe_configs_gen would make sense. Feel free to make a PR around that.