Remove `run_expect_error` to avoid tests incorrectly passing #14078

choznerol · 2024-06-15T15:36:33Z

What does this PR try to resolve?

Fixes #14076

How should we test and review this PR?

I pushed commits separately so the CI status can show the reproduction and the fix:

bc5d57c reproduce the bug: Its CI shouldn't have passed.
3c473b5 fixes the issue, so the CI for d9d11c4 is now correctly failing tests.
Turns out these test already have regression. 79a7bf4 fixes them.

Additional information

Thanks @weihanglo for suggesting the fix 3c473b5 in #14076 (comment).

rustbot · 2024-06-15T15:36:38Z

Thanks for the pull request, and welcome! The Rust team is excited to review your changes, and you should hear from @ehuss (or someone else) some time within the next two weeks.

Please see the contribution instructions for more information. Namely, in order to ensure the minimum review times lag, PR authors and assigned reviewers should ensure that the review label (S-waiting-on-review and S-waiting-on-author) stays updated, invoking these commands when appropriate:

@rustbot author: the review is finished, PR author should check the comments and take action accordingly
@rustbot review: the author is ready for a review, this PR will be queued again in the reviewer's queue

choznerol · 2024-06-15T16:39:54Z

tests/testsuite/freshness.rs

@@ -2934,7 +2934,7 @@ fn use_mtime_cache_in_cargo_home() {
    p.cargo("check -v")
        .env("CARGO_HOME", &cargo_home)
        .with_status(101)
-        .with_stderr(
+        .with_stderr_contains(


I know 79a7bf4 didn't fix these regressions the latest #14039 way. However, to avoid introducing unrelated diff (redaction, larger snapshot, etc.) to this PR, I thought minimum viable fix might be more suitable.

weihanglo · 2024-06-15T21:33:36Z

Thank you for your prompt fix!

@bors r+

bors · 2024-06-15T21:33:38Z

📌 Commit 79a7bf4 has been approved by weihanglo

It is now in the queue for this repository.

bors · 2024-06-15T21:56:15Z

🔒 Merge conflict

This pull request and the master branch diverged in a way that cannot be automatically merged. Please rebase on top of the latest master branch, and let the reviewer approve again.

How do I rebase?

Assuming self is your fork and upstream is this repository, you can resolve the conflict following these steps:

git checkout issue-14076-run_expect_error-test-incorrectly-passing (switch to your branch)
git fetch upstream master (retrieve the latest master)
git rebase upstream/master -p (rebase on top of it)
Follow the on-screen instruction to resolve conflicts (check git status if you got lost).
git push self issue-14076-run_expect_error-test-incorrectly-passing --force-with-lease (update this PR)

You may also read Git Rebasing to Resolve Conflicts by Drew Blessing for a short tutorial.

Please avoid the "Resolve conflicts" button on GitHub. It uses git merge instead of git rebase which makes the PR commit history more difficult to read.

Sometimes step 4 will complete without asking for resolution. This is usually due to difference between how Cargo.lock conflict is handled during merge and rebase. This is normal, and you should still perform step 5 to update this PR.

Error message

Auto-merging tests/testsuite/help.rs
CONFLICT (content): Merge conflict in tests/testsuite/help.rs
Automatic merge failed; fix conflicts and then commit the result.

bors · 2024-06-15T22:06:33Z

☔ The latest upstream changes (presumably #14079) made this pull request unmergeable. Please resolve the merge conflicts.

The `with_stderr_contains()` (as well as the new `with_stderr_data()` too, see rust-lang#14060) has no effect when using with `run_expect_error()`.

This reverts commit 18f4c3f.

weihanglo · 2024-06-16T01:21:55Z

@bors r+

bors · 2024-06-16T01:21:57Z

📌 Commit 6ae78e8 has been approved by weihanglo

It is now in the queue for this repository.

bors · 2024-06-16T01:23:04Z

⌛ Testing commit 6ae78e8 with merge 5586a47...

bors · 2024-06-16T01:50:47Z

☀️ Test successful - checks-actions
Approved by: weihanglo
Pushing 5586a47 to master...

Update cargo 13 commits in a1f47ec3f7cd076986f1bfcd7061f2e8cb1a726e..3ed207e416fb2f678a40cc79c02dcf4f936a21ce 2024-06-15 01:10:07 +0000 to 2024-06-18 19:18:22 +0000 - test: prefer raw string for regex reduction (rust-lang/cargo#14099) - test: migrate tree and tree_graph_features to snapbox (rust-lang/cargo#14094) - test: Migrate some files to snapbox (rust-lang/cargo#14069) - remove some legacy public dependency code from the resolver (rust-lang/cargo#14090) - fix(fix): Address problems with implicit -> explicit feature migration (rust-lang/cargo#14018) - refactor: 1.79 cleanup (rust-lang/cargo#14088) - test: migrate `git_(gc|shallow)` to snapbox (rust-lang/cargo#14087) - test: migrate timings_works to snapbox (rust-lang/cargo#14082) - test: migrate minimal_versions to snapbox (rust-lang/cargo#14080) - Remove `run_expect_error` to avoid tests incorrectly passing (rust-lang/cargo#14078) - test: migrate help to snapbox (rust-lang/cargo#14060) - test: Migrate tests/testsuite/co*.rs to snapbox (rust-lang/cargo#14079) - Use `std::fs::absolute` instead of reimplementing it (rust-lang/cargo#14075)

rustbot assigned ehuss Jun 15, 2024

rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. A-testing-cargo-itself Area: cargo's tests labels Jun 15, 2024

choznerol commented Jun 15, 2024

View reviewed changes

choznerol mentioned this pull request Jun 15, 2024

test: migrate help to snapbox #14060

Merged

choznerol marked this pull request as ready for review June 15, 2024 17:05

weihanglo approved these changes Jun 15, 2024

View reviewed changes

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Jun 15, 2024

bors added S-waiting-on-author Status: The marked PR is awaiting some action (such as code changes) from the PR author. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels Jun 15, 2024

choznerol added 4 commits June 16, 2024 09:13

Reproduce rust-lang#14076

afe9d17

The `with_stderr_contains()` (as well as the new `with_stderr_data()` too, see rust-lang#14060) has no effect when using with `run_expect_error()`.

Remove Execs::run_expect_error to avoid rust-lang#14076

fde1321

Revert "Reproduce rust-lang#14076"

b254699

This reverts commit 18f4c3f.

Fix regression since rust-lang#14076 is now resolved

6ae78e8

choznerol force-pushed the issue-14076-run_expect_error-test-incorrectly-passing branch from 79a7bf4 to 6ae78e8 Compare June 16, 2024 01:14

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-author Status: The marked PR is awaiting some action (such as code changes) from the PR author. labels Jun 16, 2024

bors merged commit 5586a47 into rust-lang:master Jun 16, 2024
22 checks passed

choznerol mentioned this pull request Jun 16, 2024

Rebase instruction include the deprecated --preserve-merges option rust-lang/homu#217

Open

weihanglo mentioned this pull request Jun 19, 2024

Update cargo rust-lang/rust#126657

Merged

rustbot added this to the 1.81.0 milestone Jun 19, 2024

choznerol deleted the issue-14076-run_expect_error-test-incorrectly-passing branch July 17, 2024 06:32

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Remove `run_expect_error` to avoid tests incorrectly passing #14078

Remove `run_expect_error` to avoid tests incorrectly passing #14078

choznerol commented Jun 15, 2024 •

edited

Loading

rustbot commented Jun 15, 2024

choznerol Jun 15, 2024 •

edited

Loading

weihanglo commented Jun 15, 2024

bors commented Jun 15, 2024

bors commented Jun 15, 2024

bors commented Jun 15, 2024

weihanglo commented Jun 16, 2024

bors commented Jun 16, 2024

bors commented Jun 16, 2024

bors commented Jun 16, 2024

Remove run_expect_error to avoid tests incorrectly passing #14078

Remove run_expect_error to avoid tests incorrectly passing #14078

Conversation

choznerol commented Jun 15, 2024 • edited Loading

What does this PR try to resolve?

How should we test and review this PR?

Additional information

rustbot commented Jun 15, 2024

choznerol Jun 15, 2024 • edited Loading

Choose a reason for hiding this comment

weihanglo commented Jun 15, 2024

bors commented Jun 15, 2024

bors commented Jun 15, 2024

bors commented Jun 15, 2024

weihanglo commented Jun 16, 2024

bors commented Jun 16, 2024

bors commented Jun 16, 2024

bors commented Jun 16, 2024

Remove `run_expect_error` to avoid tests incorrectly passing #14078

Remove `run_expect_error` to avoid tests incorrectly passing #14078

choznerol commented Jun 15, 2024 •

edited

Loading

choznerol Jun 15, 2024 •

edited

Loading