test: deal with lockfile checksums dynamically #13739
Conversation
When generating local registries in the testsuite, the specific package checksums may vary from the original simply due to different `zlib` versions. For example, Fedora 40 is now using `zlib-ng-compat`. We can account for this dynamically by either formatting the checksum in the expected output string, or using `generate-lockfile` on projects and their expected output.
rustbot
added
A-testing-cargo-itself
| @@ -33,5 +35,7 @@ fn case() { | |||
| .stdout_matches(str![""]) | |||
| .stderr_matches(file!["stderr.term.svg"]); | |||
|
|
|||
| assert_ui().subset_matches(current_dir!().join("out"), &project_root); | |||
| let expected = Project::from_expected(current_dir!().join("out")); | |||
There was a problem hiding this comment.
The dynamic check seems fine to me, though I am a bit unsure since most UI tests in Cargo only asserts one command invocation.
@epage, do you have any input for this?
There was a problem hiding this comment.
Dynamically generating the snapshots like this runs counter to point of the snapshots
- We are bringing code under test into the process for verifying the code under tests
- This is brittle as a
generate-lockfilehappens to work right now for these specific tests but that likely won't be true always.
A good example of where this can fall flat in both cases is my on-going work with MSRV-aware resolver.
Something I've been considering is adding support for dynamic substitutions so a function could run and replace all checksums with a [CHECKSUM].
There was a problem hiding this comment.
Note that it's not just about expected output in the tests. In cargo_add::locked_unchanged, cargo itself complains that the input lockfile looks corrupt with its mismatched checksum.
|
Alternatively, I think setting |
|
Closing in favor of #13744. |
|
☔ The latest upstream changes (presumably #13744) made this pull request unmergeable. Please resolve the merge conflicts. |
When generating local registries in the testsuite, the specific package
checksums may vary from the original simply due to different
zlibversions. For example, Fedora 40 is now using
zlib-ng-compat. We canaccount for this dynamically by either formatting the checksum in the
expected output string, or using
generate-lockfileon projects andtheir expected output.