Bazel does not (visibly?) begin loading phase when Starlark flag is set

[fmeum]

Description of the problem / feature request:

When a Starlark flag is set on the command line, Bazel never (visibly?) enters the loading phase. When there are repository rules that take a while to load (e.g. because they download or extract large files), this causes Bazel to get stuck after emitting the Invocation ID without any indication of what it is doing.

Bugs: what's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

$ git clone https://github.com/fmeum/bazel-hangs-on-startup
$ cd bazel-hangs-on-startup
$ bazel build //...
Starting local Bazel server and connecting to it...
INFO: Invocation ID: 3f8b3df5-f802-4557-840b-e2d5199384d4
Loading: 0 packages loaded
    Fetching @hangs_on_fetch; Will sleep now... 

<interrupt with Ctrl+C after some time, the fetch will hang for 600 seconds>
$ bazel build //... --//:flag
INFO: Invocation ID: 34b47010-e809-4de3-ba9c-096d8e406c50

Observe that in the second invocation, no Bazel status information will be printed until the repository rule times out. If Ctrl+C is used to interrupt the build, ERROR: //:flag :: Error loading option //:flag: null is printed.

What operating system are you running Bazel on?

Linux

What's the output of bazel info release?

Affects:

• release 4.2.1
• release 6.0.0-pre.20211019.1

Have you found anything relevant by searching the web?

No

Any other information, logs, or outputs that you want to share?

The reproducer is available here.

[fmeum]

@aranguyen Friendly ping. We have reduced the amount of Starlark flags to the absolute minimum because not knowing what Bazel is currently loading makes for a pretty bad UX.

[sdtwigg]

So, this is a bit tricky; what kind of fix do you think would work for your case here?

Note that, we kind of 'have' to do the loading of the Starlark flags (in order to get their types and default values). What about?

1. An option to report that (pre-)loading has started to handle the Starlark flags. (Would definitely have to be an option if you want a list of the Starlark flags that triggered the preload)

2. Actually deferring the loading-for-Starlark-options until the regular loading phase. I cannot guarantee this is possible, especially within a reasonable amount of time. (To do this would essentially require holding on to a BuildOptions or 'new' pseudo-BuildOptions that has non-canonical form since Starlark options may be the wrong type or improperly explicitly set to their defaults. I'm very nervous about this causing in havoc in top-level Bazel machinery that expects it to be a proper BuildOptions).

[fmeum]

1. would already be very helpful, even without additional information about the particular flag. 2. would of course be ideal, but I can see this being pretty difficult (and potentially requiring some hacks).

[aranguyen]

For option 1, I think we could add that at the beginning of the parse method for StarlarkOptionsParser https://source.corp.google.com/piper///depot/google3/third_party/bazel/src/main/java/com/google/devtools/build/lib/runtime/BlazeOptionHandler.java;l=263;rcl=431203742 since we're already passing in the env and eventHandler. Since this is a known behavior with starlark flag parsing and it's not blocking, I'm going to mark it as lower priority.

[fmeum]

Fixed by 51e6c38, which is in 6.1.0.