Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add --inject_repository and fix crash on overridden non-existent repo #23791

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

fmeum
Copy link
Collaborator

@fmeum fmeum commented Sep 27, 2024

Suggest the new flag instead of crashing when --override_repository is applied to a non-existent repo with --noenable_workspace.

Fixes #22691

RELNOTES: The new --inject_repository flag can be used to add new repositories via the CLI with --enable_bzlmod. Such repositories behave as if they were declared by local_repository via use_repo_rule in the root module.

@fmeum fmeum marked this pull request as ready for review September 27, 2024 15:59
@fmeum fmeum requested review from katre and removed request for a team, lberki and katre September 27, 2024 15:59
@github-actions github-actions bot added team-Configurability platforms, toolchains, cquery, select(), config transitions team-ExternalDeps External dependency handling, remote repositiories, WORKSPACE file. awaiting-review PR is awaiting review from an assigned reviewer labels Sep 27, 2024
if (repositoryName.isMain()) {
repoMapping =
repoMapping.withAdditionalMappings(
repositoryOverrides.stream().collect(toMap(RepositoryName::getName, name -> name)));
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a bit unfortunate as the resulting repo will have a canonical repo name that is also a valid apparent name. We could prevent that by branching on whether the name provided on the command line is a valid apparent name or not:

  • if it isn't, but is a valid canonical name, assume that it overrides an existing repo
  • if it is, generate a synthetic canonical repo name such as _main+_overrides+<name> for it.

What do you think? Would the resulting consistency be worth the increased complexity?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(tests are failing because REPOSITORY_OVERRIDES would now need to be injected in more tests, I can do that once we have agreed on the functionality)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could we maybe call the flag --inject_repository, similar to inject_repo vs override_repo? There's a nice bit of symmetry there, at least superficially... Then we could also say --inject_repository=X=foo always creates the repo with the canonical repo name of +_injected+X, or something.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I completely rewrote this PR. It now implements --inject_repository through a simulated use_repo_rule call for a local_repository, which guarantees behavior identical to that of a regular module extension repo.

@sgowroji sgowroji added awaiting-user-response Awaiting a response from the author and removed awaiting-review PR is awaiting review from an assigned reviewer labels Oct 11, 2024
RELNOTES: The new `--inject_repository` flag can be used to add new repositories via the CLI with `--noenable_workspace`. Such repositories behave as if they were declared by `local_repository` via `use_repo_rule` in the root module.
@fmeum fmeum changed the title Allow --override_repository to add repos with --noenable_workspace Add --inject_repository and fix crash on overridden non-existent repo Nov 6, 2024
@fmeum fmeum changed the title Add --inject_repository and fix crash on overridden non-existent repo Add --inject_repository and fix crash on overridden non-existent repo Nov 6, 2024
@fmeum
Copy link
Collaborator Author

fmeum commented Nov 6, 2024

There are a number of test failures referencing files under bazel-genfiles and I don't really know how those have been passing so far. I don't see --noincompatible_merge_genfiles_directory in any of the tests.

@Wyverald Wyverald added awaiting-review PR is awaiting review from an assigned reviewer and removed awaiting-user-response Awaiting a response from the author labels Nov 6, 2024
Copy link
Member

@Wyverald Wyverald left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like this approach!

@@ -95,6 +97,8 @@ public class ModuleFileFunction implements SkyFunction {

public static final Precomputed<Map<String, ModuleOverride>> MODULE_OVERRIDES =
new Precomputed<>("module_overrides");
public static final Precomputed<Map<String, PathFragment>> INJECTED_REPOSITORIES =
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let's use ImmutableMap everywhere (unless the values can be null, which doesn't seem to be the case)

new ModuleFileGlobals.ModuleExtensionProxy(
usageBuilder,
ModuleExtensionUsage.Proxy.builder()
.setDevDependency(true)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

does this matter at all? just checking. If not, I have a slight preference for not setting it for least surprise, but not a big deal.

+ " corresponding `local_repository` to the root module's MODULE.bazel file via"
+ " `use_repo_rule`. If the given path is an absolute path, it will be used as it is."
+ " If the given path is a relative path, it is relative to the current working"
+ " directory. If the given path starts with '%workspace%, it is relative to the"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
+ " directory. If the given path starts with '%workspace%, it is relative to the"
+ " directory. If the given path starts with '%workspace%', it is relative to the"

+ " If the given path is a relative path, it is relative to the current working"
+ " directory. If the given path starts with '%workspace%, it is relative to the"
+ " workspace root, which is the output of `bazel info workspace`. If the given path"
+ " is empty, then remove any previous overrides.")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
+ " is empty, then remove any previous overrides.")
+ " is empty, then remove any previous injections.")


public abstract String path();
}
public record RepositoryInjection(String apparentName, String path) {}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please add javadoc

}
if (repositoryMappingValue == RepositoryMappingValue.NOT_FOUND_VALUE
&& REPOSITORY_OVERRIDES.get(env).containsKey(key.repoName())) {
throw new RepositoryMappingFunctionException(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
awaiting-review PR is awaiting review from an assigned reviewer team-Configurability platforms, toolchains, cquery, select(), config transitions team-ExternalDeps External dependency handling, remote repositiories, WORKSPACE file.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Bazel crashes using external aspect + --noenable_workspace
3 participants