Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Emit labels in display form in Java rules #21180

Closed
wants to merge 1 commit into from

Conversation

fmeum
Copy link
Collaborator

@fmeum fmeum commented Feb 1, 2024

Buildozer fixes for Java strict deps violations referring to external repositories now contain labels in display form, which avoid canonical repository names that should never be added to BUILD files and aren't understood by buildozer.

The Starlark actions can use Label.to_display_form(). Java compilation actions, which are still implemented in Java, can't access the main repository mapping via BazelModuleContext and instead retrieve it from a new field on AnalysisEnvironment.

@fmeum fmeum requested review from katre and hvadehra and removed request for a team, lberki, Wyverald, meteorcloudy, ahumesky, ted-xie and katre February 1, 2024 21:11
@github-actions github-actions bot added team-Performance Issues for Performance teams team-Configurability platforms, toolchains, cquery, select(), config transitions team-ExternalDeps External dependency handling, remote repositiories, WORKSPACE file. team-Rules-Java Issues for Java rules awaiting-review PR is awaiting review from an assigned reviewer labels Feb 1, 2024
@fmeum
Copy link
Collaborator Author

fmeum commented Feb 1, 2024

Stacked on #21179, please ignore the first commit during review.

@fmeum fmeum removed team-Performance Issues for Performance teams team-Configurability platforms, toolchains, cquery, select(), config transitions team-ExternalDeps External dependency handling, remote repositiories, WORKSPACE file. labels Feb 1, 2024
The Starlark actions can use `Label.to_display_form()`. Java compilation
actions, which are still implemented in Java, can't access the main
repository mapping via `BazelModuleContext` and instead retrieve it from
a new field on `AnalysisEnvironment`.
@fmeum
Copy link
Collaborator Author

fmeum commented Feb 12, 2024

@hvadehra This PR is no longer stacked.

Copy link
Member

@hvadehra hvadehra left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM for java changes.

@meteorcloudy for tests now requiring network.

Copy link
Member

@meteorcloudy meteorcloudy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep, currently there is no way around it. Adding requires-network looks fine.

@meteorcloudy meteorcloudy added awaiting-PR-merge PR has been approved by a reviewer and is ready to be merge internally and removed awaiting-review PR is awaiting review from an assigned reviewer labels Feb 13, 2024
@fmeum
Copy link
Collaborator Author

fmeum commented Feb 13, 2024

@bazel-io fork 7.1.0

@github-actions github-actions bot removed the awaiting-PR-merge PR has been approved by a reviewer and is ready to be merge internally label Feb 16, 2024
@fmeum fmeum deleted the 20486-java branch February 16, 2024 22:35
iancha1992 pushed a commit to iancha1992/bazel that referenced this pull request Feb 16, 2024
Buildozer fixes for Java strict deps violations referring to external repositories now contain labels in display form, which avoid canonical repository names that should never be added to BUILD files and aren't understood by `buildozer`.

The Starlark actions can use `Label.to_display_form()`. Java compilation actions, which are still implemented in Java, can't access the main repository mapping via `BazelModuleContext` and instead retrieve it from a new field on `AnalysisEnvironment`.

Closes bazelbuild#21180.

PiperOrigin-RevId: 607803030
Change-Id: I0cdccabfde0217c9201cef9ca9d260b0c8ca27cd
github-merge-queue bot pushed a commit that referenced this pull request Feb 19, 2024
Buildozer fixes for Java strict deps violations referring to external
repositories now contain labels in display form, which avoid canonical
repository names that should never be added to BUILD files and aren't
understood by `buildozer`.

The Starlark actions can use `Label.to_display_form()`. Java compilation
actions, which are still implemented in Java, can't access the main
repository mapping via `BazelModuleContext` and instead retrieve it from
a new field on `AnalysisEnvironment`.

Closes #21180.

Commit
b11fa7a

PiperOrigin-RevId: 607803030
Change-Id: I0cdccabfde0217c9201cef9ca9d260b0c8ca27cd

---------

Co-authored-by: Fabian Meumertzheim <[email protected]>
Co-authored-by: hvadehra <[email protected]>
@meteorcloudy
Copy link
Member

Unfortunately, this change was rolled back at fd048d3 due to a memory regression.

@fmeum
Copy link
Collaborator Author

fmeum commented Feb 21, 2024

@hvadehra Do you know whether the Starlark part would have been fine by itself? If so, I could try to pass this argument into the native action from Starlark instead.

@hvadehra
Copy link
Member

@fmeum As noted in fd048d3, the issue is we're adding a new skyframe edge for every target/aspect. The exact cost of such a edge is a little hard to calculate in general due to skyframe optimizations. It can be particularly expensive for certain transitions in the number of edges (cc @justinhorvitz to keep me honest about this).

Additionally, we'd like to avoid the edge completely internally (i.e. no external repositories). Even for Bazel, it would be nice to only add the edge when required. Not sure if there's a good way to do that though. Solving the latter should also solve the former though.

@hvadehra
Copy link
Member

Do you know whether the Starlark part would have been fine by itself? If so, I could try to pass this argument into the native action from Starlark instead.

I tried out an experimental fix yesterday which dropped the edge in the target/aspect functions when external repos were not in play, and there was still some regression. So I suspect not. But I can try and measure more exactly.

@fmeum
Copy link
Collaborator Author

fmeum commented Feb 21, 2024

@hvadehra Thanks for the explanation. I will try to hide this edge in StarlarkBuiltinsValue instead. We won't need it anymore when all rules have been Starlarkified, so moving it there doesn't sound like too much of a hack. Would it be okay to have that single edge also internally or would you like me to make it conditional too?

@fmeum
Copy link
Collaborator Author

fmeum commented Feb 21, 2024

I tried out an experimental fix yesterday which dropped the edge in the target/aspect functions when external repos were not in play, and there was still some regression. So I suspect not. But I can try and measure more exactly.

I have already been a bit worried about the memory impact of an additional VectorArg + ImmutableList instance. I could get rid of the list, but also removing the VectorArg wouldn't really be feasible.

I will submit separate PRs for Starlark and native actions.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
team-Rules-Java Issues for Java rules
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants