AssertionError: Cannot find .runfiles directory

[sgowroji]

https://buildkite.com/bazel/bazel-at-head-plus-downstream/builds/3319#018ab077-c764-4cc2-8b32-9fb08619ef90

rules_python: rules_python (example bzlmod build file running gazelle and pip integration test on :ubuntu: 20.04 LTS (OpenJDK 11, gcc 9.4.0))
Logs:

 File "/var/lib/buildkite-agent/.cache/bazel/_bazel_buildkite-agent/3b10eb146ec4242f66f1b569edf0069e/execroot/_main/bazel-out/k8-opt-exec-ST-e846b08c7501/bin/external/rules_python_gazelle_plugin~override/python/std_modules", line 172, in FindModuleSpace
    raise AssertionError('Cannot find .runfiles directory for %s' % sys.argv[0])
AssertionError: Cannot find .runfiles directory for /var/lib/buildkite-agent/.cache/bazel/_bazel_buildkite-agent/3b10eb146ec4242f66f1b569edf0069e/execroot/_main/bazel-out/k8-opt-exec-ST-e846b08c7501/bin/external/rules_python_gazelle_plugin~override/python/std_modules
gazelle: failed to wait for std_modules: exit status 1

Culprit Found: 2f9c7d60ea676f9f8e929b20a3ff74e786995c91

CC Greenteam @comius

[comius]

cc @rickeylev

[rickeylev]

cc @f0rmiga -- Something about Gazelle unable to find the Python runfiles? But I thought Gazelle was written in Go, not Python?

[f0rmiga]

Seems related to #1415.

[aignas]

@rickeylev, gazelle plugin is written in go and it spawns a Python process which the go code is communicating to via stdin/stdout and it is included in the gazelle plugin via https://github.com/aignas/rules_python/blob/main/gazelle/python/BUILD.bazel#L21