Undetected cyclic dependency between module extensions

[aherrmann]

Description of the bug:

It's possible to construct a cyclic dependency between bzlmod module extensions that goes undetected by Bazel but instead causes tags defined in MODULE.bazel to be ignored.

What's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

Check out https://github.com/aherrmann/repro-bzlmod-module-extension-cycle (66f4ab2 at the time of writing) and follow the steps in the README.

In that example module_a defines two extensions extension_1 and extension_2 in separate files extension_1.bzl and extension_2.bzl. The file implementing extension_2 loads a file that is generated by a repository rule that extension_1 invokes. Things work well so long as each extension is loaded (use_extension) from its dedicated file.

However, module_a includes extensions.bzl which re-exports both extensions. Trying to use this file to load the extensions from module_a yields an error message about cyclic dependencies as expected. However, loading this file from module_b raises no such error and instead causes the tags of extension_1 defined in module_b to be ignored.

Which operating system are you running Bazel on?

Ubuntu 22.04

What is the output of bazel info release?

release 6.0.0

If bazel info release returns development version or (@non-git), tell us how you built Bazel.

No response

What's the output of git remote get-url origin; git rev-parse master; git rev-parse HEAD ?

No response

Have you found anything relevant by searching the web?

No response

Any other information, logs, or outputs that you want to share?

Related to the more general question of whether module extensions can depend on files generated by other module extensions.

[Wyverald]

This is currently WAI, due to the fact that re-exporting an extension makes it a distinct extension.

The dependency graph (if you do the "Faulty behavior" part of your README) looks like this: (if you s/extension/ext)

EXT[@module_a//:exts.bzl%ext_1]    EXT[@module_a//:exts.bzl%ext_2]
                      V             V
                  FILE[@module_a//:exts.bzl]
                   V                V
FILE[@module_a//:ext_1.bzl]        FILE[@module_a//:ext_2.bzl]
                 ^                  V
                 |               FILE[@[@ext_1 from @module_a]//:defs.bzl] (*)
                 |                  V
              EXT[@module_a//:ext_1.bzl%ext_1]

And there are no cycles in this graph.

The node marked with (*) is telling: @module_a//:ext_2.bzl has a load("@ext_1//:defs.bzl", ...), in which the @ext_1 uses the definition from module_a instead of module_b (since @module_a//:ext_2.bzl is hosted in module_a). So @ext_1 actually ends up referring to the one in @module_a//:ext_1.bzl (bottom node), not @module_a//:exts.bzl (top-left node).

The missing tags thing is also WAI, since EXT[@module_a//:exts.bzl%ext_1] and EXT[@module_a//:ext_1.bzl%ext_1] are distinct extensions: the former used by module_b, and the latter used by module_a.

---

The question is then -- why does re-exporting an extension make it a distinct extension? The answer is a bit tricky... Right now, extensions are identified solely through the .bzl file they're defined in, and the name of the variable they're assigned to. If we wanted to ID an extension based on reference equality, we'd need to load the .bzl files defining them, and evaluate those .bzl files.

We need to ID the extensions in order to figure out the repo mapping; that is, use_repo(ext, "repo") is translated to a repo mapping entry @repo for @@module~1.2 is @@ext_module~3.4~ext~repo. To do this we need to know the "canonical location" of the definition of ext, which means that we'd need to fetch [email protected] (if extension ID is reference equality), or at least finish module version resolution so we know to use version 3.4 for ext_module (if extension ID is location+name).

And we need the repo mapping to parse any labels (this means evaluating any BUILD files, parsing label-typed command-line flags, etc).

All this together means that:

• Extension ID is location+name (status quo): we can start parsing labels as soon as module version resolution finishes.
• Extension ID is reference equality: we can start parsing labels after fetching all modules that host any extension used by any module in the dependency graph, and evaluating some .bzl files in those modules.

---

Given the above, and the fact that "re-exporting an extension makes it a distinct extension" shouldn't really be harmful in practice, I'm going to close this as WAI.

[aherrmann]

@Wyverald Big thank you for the in depth answer!
I didn't realize that the ID was location + name, thanks for clarifying and motivating that fact.
UPDATE: I've opened a PR to extend the documentation in #17634.