Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow coverage tests to ignore test modes, and to enable color in coverage reports #119034

Merged
merged 4 commits into from
Jan 5, 2024

Conversation

Zalathar
Copy link
Contributor

@Zalathar Zalathar commented Dec 17, 2023

This PR adds two new header directives to compiletest, intended for use by coverage tests (and by #119033 in particular).

The new headers are:

  • // ignore-mode-{mode} causes a test to not be run in a particular compiletest mode (e.g. ignore-mode-coverage-run).
    • This can theoretically be used by any test, but coverage tests are currently the only ones that automatically run in multiple modes, so it's not very useful for other kinds of test.
  • // llvm-cov-flags: --use-color makes coverage-run tests pass the flag --use-color when generating coverage reports.

@rustbot
Copy link
Collaborator

rustbot commented Dec 17, 2023

r? @compiler-errors

(rustbot has picked a reviewer for you, use r? to override)

@rustbot rustbot added A-testsuite Area: The testsuite used to check the correctness of rustc S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap) labels Dec 17, 2023
@Zalathar Zalathar changed the title Allow coverage tests to ignore test modes, and enable color in coverage reports Allow coverage tests to ignore test modes, and to enable color in coverage reports Dec 17, 2023
@compiler-errors
Copy link
Member

sorry, too busy to review this

r? compiler

@rustbot rustbot added the T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. label Dec 19, 2023
@rustbot rustbot assigned davidtwco and unassigned compiler-errors Dec 19, 2023
Copy link
Member

@davidtwco davidtwco left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

r=me unless you want to adopt my suggestion below

src/tools/compiletest/src/header.rs Outdated Show resolved Hide resolved
@davidtwco davidtwco added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Jan 3, 2024
@Zalathar Zalathar force-pushed the ignore-mode branch 2 times, most recently from 58a15e7 to 3342195 Compare January 4, 2024 00:13
@Zalathar
Copy link
Contributor Author

Zalathar commented Jan 4, 2024

Changed the new header from llvm-cov-use-color to llvm-cov-flags (diff).

Fairly straightforward; the only non-trivial part was having to extract a shared helper for splitting flags (for consistency with compile-flags).

@rustbot ready

@rustbot rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Jan 4, 2024
@Zalathar
Copy link
Contributor Author

Zalathar commented Jan 4, 2024

Having // llvm-cov-flags: could also be useful for #118305, since I can have individual tests enable branch coverage output in coverage reports, instead of unconditionally enabling it for all coverage tests.

Normally, each test in `tests/coverage` is automatically run in both
`coverage-map` mode and `coverage-run` mode.

This new family of directives allows an individual test to specify that it
should not be run in a particular mode.
@davidtwco
Copy link
Member

@bors r+

@bors
Copy link
Contributor

bors commented Jan 5, 2024

📌 Commit 731ba80 has been approved by davidtwco

It is now in the queue for this repository.

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Jan 5, 2024
bors added a commit to rust-lang-ci/rust that referenced this pull request Jan 5, 2024
…mpiler-errors

Rollup of 10 pull requests

Successful merges:

 - rust-lang#119034 (Allow coverage tests to ignore test modes, and to enable color in coverage reports)
 - rust-lang#119148 (Tweak suggestions for bare trait used as a type)
 - rust-lang#119538 (Cleanup error handlers: round 5)
 - rust-lang#119566 (Remove `-Zdump-mir-spanview`)
 - rust-lang#119567 (Remove `-Zreport-delayed-bugs`.)
 - rust-lang#119577 (Migrate memory overlap check from validator to lint)
 - rust-lang#119583 (Make `intrinsics::assume` const stable)
 - rust-lang#119586 ([rustdoc] Fix invalid handling for static method calls in jump to definition feature)
 - rust-lang#119588 (Move `i586-unknown-netbsd` from tier 2 to tier 3 platform support table)
 - rust-lang#119601 (`Emitter` cleanups)

r? `@ghost`
`@rustbot` modify labels: rollup
@bors bors merged commit d180e91 into rust-lang:master Jan 5, 2024
11 checks passed
@rustbot rustbot added this to the 1.77.0 milestone Jan 5, 2024
rust-timer added a commit to rust-lang-ci/rust that referenced this pull request Jan 5, 2024
Rollup merge of rust-lang#119034 - Zalathar:ignore-mode, r=davidtwco

Allow coverage tests to ignore test modes, and to enable color in coverage reports

This PR adds two new header directives to compiletest, intended for use by coverage tests (and by rust-lang#119033 in particular).

The new headers are:

- `// ignore-mode-{mode}` causes a test to not be run in a particular compiletest mode (e.g. `ignore-mode-coverage-run`).
  - This can theoretically be used by any test, but coverage tests are currently the only ones that automatically run in multiple modes, so it's not very useful for other kinds of test.
- `// llvm-cov-flags: --use-color` makes `coverage-run` tests pass the flag `--use-color` when generating coverage reports.
  - For most tests, non-coloured reports are easier to read and more portable across platforms. But for rust-lang#119033 specifically, we want to test that `llvm-cov` slices up source text correctly, which only happens when colour output is enabled.
@Zalathar Zalathar deleted the ignore-mode branch January 5, 2024 21:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-testsuite Area: The testsuite used to check the correctness of rustc S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap) T-compiler Relevant to the compiler team, which will review and decide on the PR/issue.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants