LLVM coverage regression in nightly-2022-01-15

[ecton]

Between nightly versions 2022-01-14 and 2022-01-15, grcov 0.8.2 lost track of 82% of the coverage of BonsaiDb when using the LLVM coverage option (-Zinstrument-coverage).

Other projects of mine continue to have code coverage correctly measured. I'm not where to start narrowing this down, as I'm not getting any errors or warnings printed.

Here are the steps to get good code coverage reports out of 2022-01-14:

git clone https://github.com/khonsulabs/bonsaidb.git
cd bonsaidb

rustup toolchain install nightly-2022-01-14 --component llvm-tools-preview
LLVM_PROFILE_FILE=%m.profraw RUSTFLAGS="-Zinstrument-coverage" cargo +nightly-2022-01-14 test --workspace --all-features --all-targets
RUST_TOOLCHAIN=nightly-2022-01-14 grcov . --binary-path ./target/debug/ -s . -t html --branch --ignore-not-existing --llvm -o coverage/ --ignore "target/*" --ignore "xtask/*"

If you look at the report in the coverage folder, the html report will show >80% coverage. Now, test with the next nightly:

# Clean up files
find . -name "*.profraw" -exec rm \{\} \;
rm -rf **/*.bonsaidb

rustup toolchain install nightly-2022-01-15 --component llvm-tools-preview
cargo clean
LLVM_PROFILE_FILE=%m.profraw RUSTFLAGS="-Zinstrument-coverage" cargo +nightly-2022-01-15 test --workspace --all-features --all-targets
RUST_TOOLCHAIN=nightly-2022-01-15 grcov . --binary-path ./target/debug/ -s . -t html --branch --ignore-not-existing --llvm -o coverage/ --ignore "target/*" --ignore "xtask/*"

If you look at the report again, the html report now shows < 1% coverage.

The quantity of profraw files are the same, and the sizes of the files are more or less equivalent. I'm not quite sure where to start narrowing this down since it's a unique issue to this project of mine -- but the only unique part I'm aware of is its size 😅.

For now I'll pin our CI to the last working nightly. Thank you in advance for any suggestions or help!

[fpoli]

We noticed the same when upgrading rustc in our project from nightly-2021-12-15 (December) to nightly-2022-01-17. Code coverage measured with -Zinstrument-coverage and grcov inexplicably dropped from 76% to 28%. Some crates that are clearly running don't show up at all in the reports.

tikv/tikv#11875 (comment) suggests that this is probably caused by #92142

[mati865]

^ taiki-e/cargo-llvm-cov#128 says nightly-2022-01-15 is also affected.

[Urgau]

cc @wesleywiser @richkadel

@rustbot label +requires-nightly +A-code-coverage

[wesleywiser]

Thanks for the report! I've tried reproducing locally and I'm able to see the same behavior. Running the LLVM tools directly, I see this when trying to process coverage data for the various binaries:

error: target/debug/deps/bonsaidb_keystorage_s3-438dd0d4a1f4a274: Failed to load coverage: Truncated coverage data
error: Could not load coverage information
error: target/debug/deps/bonsaidb_local-72f1e7d9d3d420d1: Failed to load coverage: Truncated coverage data
error: Could not load coverage information
error: target/debug/deps/bonsaidb_core-e06ccf75f46726e6: Failed to load coverage: Truncated coverage data
error: Could not load coverage information
error: target/debug/deps/bonsaidb_macros-7da7619a94109db7: Failed to load coverage: Truncated coverage data
error: Could not load coverage information
error: target/debug/deps/bonsaidb-9d5002ec7642a5fa: Failed to load coverage: Truncated coverage data
error: Could not load coverage information
error: target/debug/deps/bonsaidb-09fb12b0ad5856f2: Failed to load coverage: Truncated coverage data
error: Could not load coverage information
error: target/debug/deps/bonsaidb_server-41b846b92bb6128a: Failed to load coverage: Truncated coverage data
error: Could not load coverage information
error: target/debug/deps/bonsaidb-c4775d66b4dcbbc0: Failed to load coverage: Truncated coverage data
error: Could not load coverage information

which seems to be the same issue as was reported in a comment on #79645. I'm continuing to investigate ...

[wesleywiser]

The minimal repro seems to be

enum Never { }

impl Never {
    fn foo(self) {
        match self { }
    }
}

fn main() { }

I think I'll have a fix for this within the next few days.