Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix genrule autostamping #21407

Conversation

AlessandroPatti
Copy link
Contributor

Genrule does not support conditional stamping based on the value of the --stamp flag. This is because genrule treats the stamp attribute as boolean, while all other rules use the tristate type. After this change, the stamp attribute in genrule can be set to -1 to request conditional stamping.

@github-actions github-actions bot added the awaiting-review PR is awaiting review from an assigned reviewer label Feb 19, 2024
@AlessandroPatti
Copy link
Contributor Author

Also noticed that the documentation says:

Bazel never stamps binaries that are built for the exec configuration, regardless of this option or the stamp attribute

This is not true for genrule. Wonder if that is intentional since there's is a test verifying that.

@cdkrot
Copy link

cdkrot commented Feb 19, 2024

This is awesome, thank you @AlessandroPatti.

The documentation on stamp functionality is not exactly complete/consistent, I can confirm

@sgowroji sgowroji added the team-Configurability platforms, toolchains, cquery, select(), config transitions label Feb 20, 2024
@justinhorvitz
Copy link
Contributor

justinhorvitz commented Feb 20, 2024

Thanks for the fix. I'm going to patch this and modify it slightly to better converge with the google-internal genrule implementation.

Bazel never stamps binaries that are built for the exec configuration, regardless of this option or the stamp attribute

This is not true for genrule. Wonder if that is intentional since there's is a test verifying that.

Which test are you referring to? This is a bit of a concern for breaking users that rely on stamped genrules in the exec config, but that is not working as intended.

Edit: after further research, it does appear that this is WAI. Genrules are treated differently than binaries, see following comment.

@justinhorvitz
Copy link
Contributor

Also note that the linked documentation refers to binaries when talking about the exec configuration, not genrules. Internally, genrules are also currently allowed to be stamped (with stamp = 1) in the exec configuration. It will take some effort to see whether disallowing this would break anything (I guess that it will).

@AlessandroPatti
Copy link
Contributor Author

Which test are you referring to?

This is a bit of a concern for breaking users that rely on stamped genrules in the exec config, but that is not working as intended.

@justinhorvitz indeed this would be a breaking change and would likely require an incompatible flag, that is why I left it out of this pr, but wanted to call it out.

Also note that the linked documentation refers to binaries when talking about the exec configuration, not genrules

True, it mentions binaries only. Still, I find it odd that binaries and genrule behave differently and it seems desirable for tools to not be stamped to avoid massive cache invalidations. However, if this is the intended behaviour, maybe worth calling out explicitly in the documentation that this is not the case for genrules (what about starlark rules?)?

@justinhorvitz
Copy link
Contributor

Thanks, it took me a bit to realize that you left the exec configuration change out of this PR, but we're on the same page now. I have it out for review.

Most likely it was initially an oversight to allow stamping in the exec (back then host) configuration for genrules. In general, there are a lot of legacy issues with the stamp feature, and if we could redesign it from scratch it would look totally different (no per-rule attributes, only a per-build flag).

Starlark actions are also a mess with stamping. They allow unconditional access to the stamp files via ctx.info_file and ctx.version_file, no attribute necessary. It's probably a good thing this isn't documented, because it encourages stamp-based volatility regardless of the build configuration. I'm trying to get a cleanup effort prioritized to restrict access to the stamp files in starlark.

copybara-service bot pushed a commit that referenced this pull request Feb 21, 2024
Isolate two `@ForOverride` methods and make everything else `private` to make it clear what behavior may diverge between bazel and blaze.

PiperOrigin-RevId: 608999251
Change-Id: I16343d4a1b538d9c13544661ec421f13bf9b1c40
@github-actions github-actions bot removed the awaiting-review PR is awaiting review from an assigned reviewer label Feb 21, 2024
@brentleyjones
Copy link
Contributor

@bazel-io flag

@bazel-io bazel-io added the potential release blocker Flagged by community members using "@bazel-io flag". Should be added to a release blocker milestone label Feb 24, 2024
@keertk
Copy link
Member

keertk commented Feb 26, 2024

@bazel-io fork 7.1.0

@bazel-io bazel-io removed the potential release blocker Flagged by community members using "@bazel-io flag". Should be added to a release blocker milestone label Feb 26, 2024
iancha1992 pushed a commit to iancha1992/bazel that referenced this pull request Feb 27, 2024
Genrule does not support conditional stamping based on the value of the `--stamp` flag. This is because genrule treats the `stamp` attribute as `boolean`, while all other rules use the `tristate` type. After this change, the stamp attribute in genrule can be set to `-1` to request conditional stamping.

RELNOTES: `genrule` now supports setting `stamp = -1` to request conditional stamping (based on the value of the build's `--stamp` flag).

Closes bazelbuild#21407.

PiperOrigin-RevId: 609085052
Change-Id: I873941e9aaae3760a545c1900cd13cb60a9ada39
iancha1992 pushed a commit to iancha1992/bazel that referenced this pull request Feb 27, 2024
Isolate two `@ForOverride` methods and make everything else `private` to make it clear what behavior may diverge between bazel and blaze.

PiperOrigin-RevId: 608999251
Change-Id: I16343d4a1b538d9c13544661ec421f13bf9b1c40
iancha1992 pushed a commit to iancha1992/bazel that referenced this pull request Feb 27, 2024
Genrule does not support conditional stamping based on the value of the `--stamp` flag. This is because genrule treats the `stamp` attribute as `boolean`, while all other rules use the `tristate` type. After this change, the stamp attribute in genrule can be set to `-1` to request conditional stamping.

RELNOTES: `genrule` now supports setting `stamp = -1` to request conditional stamping (based on the value of the build's `--stamp` flag).

Closes bazelbuild#21407.

PiperOrigin-RevId: 609085052
Change-Id: I873941e9aaae3760a545c1900cd13cb60a9ada39
github-merge-queue bot pushed a commit that referenced this pull request Feb 28, 2024
Genrule does not support conditional stamping based on the value of the
`--stamp` flag. This is because genrule treats the `stamp` attribute as
`boolean`, while all other rules use the `tristate` type. After this
change, the stamp attribute in genrule can be set to `-1` to request
conditional stamping.

RELNOTES: `genrule` now supports setting `stamp = -1` to request
conditional stamping (based on the value of the build's `--stamp` flag).

Closes #21407.

Commit
cb901b7,
ee57d99

PiperOrigin-RevId: 609085052
Change-Id: I873941e9aaae3760a545c1900cd13cb60a9ada39

---------

Co-authored-by: Googler <[email protected]>
Co-authored-by: Alessandro Patti <[email protected]>
@iancha1992
Copy link
Member

The changes in this PR have been included in Bazel 7.1.0 RC2. Please test out the release candidate and report any issues as soon as possible.
If you're using Bazelisk, you can point to the latest RC by setting USE_BAZEL_VERSION=7.1.0rc2. Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
team-Configurability platforms, toolchains, cquery, select(), config transitions
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants