Revamp // run-fail wrt. --pass & support // build-fail & // check-fail

[Centril]

Revamp how // run-fail tests work internally by having a separate FailMode that does not interfere with PassMode. In particular, --pass check will now have no effect on // *-fail tests. Moreover, new test annotations // check-fail (the default) and // build-fail are added. The latter is useful to distinguish post-monomorphization failures from pre-monomorphization failures as seen throughout the PR. Finally, ensure that non-Ui tests do not listen to --pass check such that the flag can be used with e.g. ./x.py test --pass check directly.

Fixes #66929.
Fixes #67128.

r? @petrochenkov
cc @RalfJung @ninjasource

[ninjasource]

@Centril and @petrochenkov, there is a preexisting (and passing) run-fail unit test in ui called src/test/ui/test-panic-abort.rs which has the following header comments:

// compile-flags: --test -Cpanic=abort -Zpanic_abort_tests
// run-flags: --test-threads=1
// run-fail
// check-run-results
// exec-env:RUST_BACKTRACE=0

// ignore-wasm no panic or subprocess support
// ignore-emscripten no panic or subprocess support

It may be worthwhile adding the relevant flags above to the offending async run-fail tests. However, I'm not sure if this test gets any special treatment and I cannot reproduce the failures locally.

[Centril]

@petrochenkov I've rewritten the approach from scratch to something more useful and principled. :)

[petrochenkov]

I don't see how this is checked.
Looks like if the test starts failing at the "check" phase it will still pass with the // build-fail annotation.

Why are such annotations useful then?

[petrochenkov]

Also, how the // build-fail test set was collected?
This can be done by introducing some kind of analogue of --pass check, but for failing tests, but I don't see it in this PR.

[Centril]

Looks like if the test starts failing at the "check" phase it will still pass with the // build-fail annotation.

Ouch; Didn't think of that. To fix it, I think we'd need to compile the test twice, once as check-pass and once as build-fail.

Also, how the // build-fail test set was collected?

A set of tests started failing because they did not fail with --emit metadata.

This can be done by introducing some kind of analogue of --pass check, but for failing tests, but I don't see it in this PR.

Didn't feel necessary, but I can add --fail if you think it's useful.

[petrochenkov]

but I can add --fail if you think it's useful.

No, no, just running twice would probably be preferable, the number of such tests is not too large, but we need to measure.
This is material for another PR anyway.

[Centril]

I think I fixed it, but I had to refactor things to get rid of some implicit state to make it work. I think things should be less brittle & more understandable now. Let's see what the CI builder says. :)

[Centril]

PR builder is happy at last. 🎉

[petrochenkov]

@bors r+

[bors]

📌 Commit fdedf4c has been approved by petrochenkov

[Centril]

@bors p=199

[Centril]

Fixed the 32-bit-only tests wrt. build-fail.

@bors r=petrochenkov

[bors]

📌 Commit 73e6a21 has been approved by petrochenkov

[bors]

⌛ Testing commit 73e6a21 with merge 6e9172f...

[bors]

☀️ Test successful - checks-azure
Approved by: petrochenkov
Pushing 6e9172f to master...