Tidy check for test revisions that are mentioned but not declared

[Zalathar]

If a [revision] name appears in a test header directive or error annotation, but isn't declared in the //@ revisions: header, that is almost always a mistake.

In cases where a revision needs to be temporarily disabled, adding it to an //@ unused-revision-names: header will suppress these checks for that name.

Adding the wildcard name * to the unused list will suppress these checks for the entire file.

(None of the tests actually use *; it's just there because it was easy to add and could be handy as an escape hatch when dealing with other problems.)

---

Most of the existing problems discovered by this check were fairly straightforward to fix (or ignore); the trickiest cases are in borrowck tests.

[rustbot]

r? @estebank

rustbot has assigned @estebank.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

[rustbot]

Some changes occurred in src/tools/compiletest

cc @jieyouxu

Some changes occurred in tests/ui/check-cfg

cc @Urgau

[Zalathar]

It would be nice to have a variant of this for Filecheck directives as well, but that's probably trickier because they're more freeform and thus harder to identify reliably.

[bors]

☔ The latest upstream changes (presumably #124345) made this pull request unmergeable. Please resolve the merge conflicts.

[jieyouxu]

I feel like the long-term solution to revision-related funny business is to rework how compiletest directive collection, parsing and validation is done. This would include adding validation for revisions that are used but not declared (i.e. the problem this PR is trying to solve), duplicate, invalid (incl. invalid characters and such), etc.

I think having a tidy check in the meantime (before compiletest directive rework, which I plan to start work on towards the end of this month) is reasonable though. It just feels like it should be the job of compiletest rather than a tidy check.

[jieyouxu]

I'll roll a bootstrap reviewer as well because this mostly concerns tidy which falls under the purview of bootstrap. I think this PR is definitely rightfully addressing a "wtf moment" when writing and reading tests, I'm just not sure if it should be tidy's responsibility.

r? bootstrap

[Zalathar]

Yeah, my original implementation was integrated into compiletest, but it felt like I was adding more mess that would make future cleanups even trickier. So I decided to try a more self-contained tidy-based version instead.

So I’m fine with this being seen as a relatively temporary solution, to eventually be replaced with better checks inside compiletest itself.

[jieyouxu]

Also see more silliness for compiletest directive/revision handling:

• compiletest has some parsing footguns around revisions #123765
• compiletest silently fails on a name-value directive with a known name but missing colon but does not report an error #123760

[Mark-Simulacrum]

r? jieyouxu -- tidy seems like a reasonable place

[jieyouxu]

The implementation for this looks good to me. One minor nit about maybe leaving a FIXME for the tidy check to not forget to replace it with proper handling in compiletest in the long term.

[jieyouxu]

Should we leave a FIXME here (feel free to FIXME(jieyouxu)) to remind people (well, mostly me) that ideally we would have these checks as part of compiletest directive and revision handling?

[jieyouxu]

Feel free to r=me with or without the FIXME.

@bors delegate=Zalathar

[bors]

✌️ @Zalathar, you can now approve this pull request!

If @jieyouxu told you to "r=me" after making some further change, please make that change, then do @bors r=@jieyouxu

[Zalathar]

Rebased and added FIXME (diff).

@bors r=jieyouxu

[bors]

📌 Commit 14d56e8 has been approved by jieyouxu

It is now in the queue for this repository.

[bors]

⌛ Testing commit 14d56e8 with merge 8c7c151...

[bors]

☀️ Test successful - checks-actions
Approved by: jieyouxu
Pushing 8c7c151 to master...

[rust-timer]

Finished benchmarking commit (8c7c151): comparison URL.

Overall result: ❌✅ regressions and improvements - no action needed

@rustbot label: -perf-regression

Instruction count

This is a highly reliable metric that was used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	0.4%	[0.3%, 0.5%]	5
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-0.4%	[-0.4%, -0.4%]	1
All ❌✅ (primary)	-	-	0

Max RSS (memory usage)

This benchmark run did not return any relevant results for this metric.

Cycles

This benchmark run did not return any relevant results for this metric.

Binary size

This benchmark run did not return any relevant results for this metric.

Bootstrap: 673.446s -> 674.785s (0.20%)
Artifact size: 315.79 MiB -> 315.96 MiB (0.05%)