cargo tarpaulin unable to produce coverage report on latest nightly toolchain

[mgaertne]

Describe the bug
When running cargo-tarpaulin on v0.27.3 with the latest nightly toolchain all tests pass, but fail to produce a coverage report, even on this repository. I get the following output:

...
test result: ok. 1209 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.78s

2024-03-13T18:24:21.340529Z  INFO cargo_tarpaulin::statemachine::instrumented: For binary: target\debug\deps\shinqlx-8d4e3f2d9130fbba.exe
2024-03-13T18:24:21.340746Z  INFO cargo_tarpaulin::statemachine::instrumented: Generated: target\tarpaulin\profraws\shinqlx-8d4e3f2d9130fbba.exe_4421533817759176802_0-24756.profraw
2024-03-13T18:24:21.340901Z  INFO cargo_tarpaulin::statemachine::instrumented: Merging coverage reports
2024-03-13T18:24:21.347839Z  INFO cargo_tarpaulin: No failing fast!
2024-03-13T18:24:21.578043Z  INFO cargo_tarpaulin::report: Coverage Results:
|| Tested/Total Lines:
||
NaN% coverage, 0/0 lines covered, NaN% change in coverage
Error: "Test failed during run"

Example can be found also on the latest gh actions run here when dependabot bumped some dependencies: https://github.com/xd009642/tarpaulin/actions/runs/8243334258

Build fails on nightly toolchain under linux, windows, and basically every system configured.

To Reproduce
Install rust nightly toolchain v0.78, and try to run cargo tarpaulin. Tried different engines, all failing.

Expected behavior
Not sure I need to write the obvious here.

[xd009642]

This should only be on the llvm engine with the new update to llvm 18. I have a WIP for this xd009642/llvm-profparser#36 but I am busy on preparing for rustnation so realistically I might not have time for it until the end of the month.

An alternative and something I was going to look to is if the user has https://github.com/rust-embedded/cargo-binutils installed using cargo profdata -- merge to merge into profdata files since they all seem to parse fine, and then parsing the merged profdata files as a fall back if my initial parsing fails. Which should be easier for someone else to PR as well (if anyone comes to this issue and wants to help out).

Looking at the logs it looks like all the tests pass on linux except 3, the LLVM sanity test and then 2 others which may or may not be related 🤔

[mgaertne]

So, I might cargo install cargo-tarpaulin from the llvm-profparser, and give it a try?

[xd009642]

no, for the profparser work it's kinda just porting from the LLVM code which I'm in the progress of. Tarpaulin uses it as a library so it wouldn't need tarpaulin to do the work.

For the bin-utils one it would be going into src/statemachine/instrumented.rs and where it loads the files and returns an error putting in a call to cargo profdata, so if we've generated say 2 profraw files doing: cargo profdata -- merge profile_1.profraw profile_2.profraw --output merged_$TIMESTAMP.profdata and then loading the profdata file

[soonum]

Hello there,
I've something similar using nightly toolchain (which was working fine before) but it fails at the parsing level:

test result: ok. 26 passed; 0 failed; 0 ignored; 0 measured; 240 filtered out; finished in 337.93s
2024-03-25T11:10:42.128864Z  INFO cargo_tarpaulin::statemachine::instrumented: For binary: target/release/deps/tfhe-b3a6e34d533d4af6
2024-03-25T11:10:42.128881Z  INFO cargo_tarpaulin::statemachine::instrumented: Generated: target/tarpaulin/profraws/tfhe-b3a6e34d533d4af6_12060749742692627106_0-14261.profraw
2024-03-25T11:10:42.128883Z  INFO cargo_tarpaulin::statemachine::instrumented: Merging coverage reports
2024-03-25T11:10:42.132122Z ERROR cargo_tarpaulin: Parsing failed
Error: "Parsing failed"
make: *** [Makefile:446: test_boolean_cov] Error 1
Error: Process completed with exit code 2.

Could this be related to the LLVM update too ?

[mgaertne]

Looks the same to me, though you probably have enabled failing fast, while I did not. That's the main difference in the output.

[xd009642]

There's a work in progress PR on this #1502 just got some nits to sort out if anyone wants to try an early version

[soonum]

There's a work in progress PR on this #1502 just got some nits to sort out if anyone wants to try an early version

It works just fine for me, thanks to this PR 🙇 . Any idea when this fix would be released ?

[xd009642]

So my profparsers PR that this is pulling via a git dependency breaks the stable tests in that repo somewhere so just in the cleanup phase of it. I'm aiming for this evening but if not hopefully Monday

[xd009642]

The fix for this is merged in, there's just one low-hanging issue I plan on tackling before doing a release under my sudden flurry of activity but you should see a release hopefully this weekend