Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Cargo crates generated from git repositories are not reproducible #1927

Closed
cameron-martin opened this issue Apr 20, 2023 · 3 comments · Fixed by #1948
Closed

Cargo crates generated from git repositories are not reproducible #1927

cameron-martin opened this issue Apr 20, 2023 · 3 comments · Fixed by #1948

Comments

@cameron-martin
Copy link
Contributor

cameron-martin commented Apr 20, 2023 •
edited
Loading

If you have a crate loaded from git in your Cargo.toml, this gets loaded by bazel via git_repository. This leaves the .git directory either in the root or in .tmp_git_root. The .git directory contains files where the contents is not a function of the input attributes to git_repository, so technically this repository rule is not reproducible (see bazelbuild/bazel#18152). Rules_rust generates globs that depend on these files, making these rules unnecessarily re-run when the repository rule is re-run.

We should probably exclude .git from these globs.
@cameron-martin cameron-martin mentioned this issue Apr 20, 2023
Closed
@cameron-martin cameron-martin changed the title Cargo crates generated from git repositories are not reproducable Cargo crates generated from git repositories are not reproducible Apr 20, 2023
@cameron-martin
Copy link
Contributor Author

cameron-martin commented Apr 29, 2023 •
edited
Loading

I've submitted a PR to bazel that should fix this, although we should probably also exclude .tmp_git_root from these globs just to not include unnecessary files and to fix it for current bazel versions. I've submitted a PR for this.

cameron-martin added a commit to cameron-martin/rules_rust that referenced this issue Apr 29, 2023
@cameron-martin
Exclude .tmp_git_root from globs
8c3861d 
This directory contains the original git repo when the crate is from git. Including this directory currently makes these rules re-run whenever the repository rule re-runs, although this is fixed by bazelbuild/bazel#18271. Even after this fix, excluding this directory avoids depending on unnecessary files.

Fixes bazelbuild#1927.
@cameron-martin cameron-martin mentioned this issue Apr 29, 2023
Merged
cameron-martin added a commit to cameron-martin/rules_rust that referenced this issue May 1, 2023
@cameron-martin
Exclude .tmp_git_root from globs
435ba83 
This directory contains the original git repo when the crate is from git. Including this directory currently makes these rules re-run whenever the repository rule re-runs, although this is fixed by bazelbuild/bazel#18271. Even after this fix, excluding this directory avoids depending on unnecessary files.

Fixes bazelbuild#1927.
@snowp
Copy link

snowp commented May 26, 2023

We've been seeing poor caching related to the use of crate_universe git dependencies - should we expect that the Bazel fix should address this or would we also need the linked PR to change the globs?

@cameron-martin
Copy link
Contributor Author

The Bazel fix will fix it in new versions of Bazel (presumably 7 now), but the fix in rules_rust will fix it for existing versions.

cameron-martin added a commit to cameron-martin/rules_rust that referenced this issue Jun 22, 2023
@cameron-martin
Exclude .tmp_git_root from globs
6d0c004 
This directory contains the original git repo when the crate is from git. Including this directory currently makes these rules re-run whenever the repository rule re-runs, although this is fixed by bazelbuild/bazel#18271. Even after this fix, excluding this directory avoids depending on unnecessary files.

Fixes bazelbuild#1927.
UebelAndre pushed a commit that referenced this issue Jun 23, 2023
@cameron-martin
Exclude .tmp_git_root from globs (#1948)
36f8251 
* Exclude .tmp_git_root from globs

This directory contains the original git repo when the crate is from git. Including this directory currently makes these rules re-run whenever the repository rule re-runs, although this is fixed by bazelbuild/bazel#18271. Even after this fix, excluding this directory avoids depending on unnecessary files.

Fixes #1927.

* Regenerate vendored crates
freeformstu pushed a commit to freeformstu/rules_rust that referenced this issue Jun 24, 2023
@cameron-martin @freeformstu
Exclude .tmp_git_root from globs (bazelbuild#1948)
8d4e619 
* Exclude .tmp_git_root from globs

This directory contains the original git repo when the crate is from git. Including this directory currently makes these rules re-run whenever the repository rule re-runs, although this is fixed by bazelbuild/bazel#18271. Even after this fix, excluding this directory avoids depending on unnecessary files.

Fixes bazelbuild#1927.

* Regenerate vendored crates
Silcet pushed a commit to Silcet/rules_rust that referenced this issue Jul 10, 2023
@cameron-martin @mir-cmg
Exclude .tmp_git_root from globs (bazelbuild#1948)
93bfa63 
* Exclude .tmp_git_root from globs

This directory contains the original git repo when the crate is from git. Including this directory currently makes these rules re-run whenever the repository rule re-runs, although this is fixed by bazelbuild/bazel#18271. Even after this fix, excluding this directory avoids depending on unnecessary files.

Fixes bazelbuild#1927.

* Regenerate vendored crates
Silcet pushed a commit to Silcet/rules_rust that referenced this issue Jul 10, 2023
@cameron-martin @Silcet
Exclude .tmp_git_root from globs (bazelbuild#1948)
1d1a361 
* Exclude .tmp_git_root from globs

This directory contains the original git repo when the crate is from git. Including this directory currently makes these rules re-run whenever the repository rule re-runs, although this is fixed by bazelbuild/bazel#18271. Even after this fix, excluding this directory avoids depending on unnecessary files.

Fixes bazelbuild#1927.

* Regenerate vendored crates
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Exclude .tmp_git_root from globs cameron-martin/rules_rust
2 participants
@cameron-martin @snowp