Coverage Report Fails with Gradle 6.5

[jGleitz]

This plugin fails with Gradle version 6.5. It worked with the last version of Gradle (6.4.1).

More precisely, the output of jacocoTestReport now contain lines like these:

> Task :jacocoTestReport
[ant:jacocoReport] Classes in bundle 'broken-testkit-coverage' do no match with execution data. For report generation the same class files must be used as at runtime.
[ant:jacocoReport] Execution data for class de/joshuagleitze/gradle/CoverageTestPlugin$apply$1 does not match.
[ant:jacocoReport] Execution data for class de/joshuagleitze/gradle/CoverageTestPlugin does not match.
[ant:jacocoReport] Execution data for class de/joshuagleitze/gradle/CoverageTestPlugin$apply$1$1 does not match.

and jacoco reports 0% coverage on the plugin classes.

I have created a test repository that reproduces this bug and shows that it does not happen with Gradle 6.4.1: https://github.com/jGleitz/broken-testkit-coverage

[koral--]

Thanks, your project does not compile due to missing readAllBytes() function. However, after replacing with readBytes() issue does reproduce.
Moreover I've noticed that CoverageTestPlugin.kt file uses package de.joshuagleitze.gradle but it is not located in de/joshuagleitze/gradle directory. Is that intended?

[jGleitz]

does not compile due to missing readAllBytes() function

readAllBytes is @since Java 9. I guess you used Java ≤ 8 to compile the project? I have now added sourceCompatibility and targetCompatibilty to the build file.

Moreover I've noticed that CoverageTestPlugin.kt file uses package de.joshuagleitze.gradle but it is not located in de/joshuagleitze/gradle directory. Is that intended?

It is. Kotlin allows to omit the common prefix of packages and even recommends it.

[koral--]

OK, thanks for the clarification. Indeed I was using Java 8 (built in Android Studio).
I guess that maybe class id is changed for some reason https://www.eclemma.org/jacoco/trunk/doc/classids.html. I'll investigate that.

Regarding directories it seems that JaCoCo does not handle omitting them when embedding source code in reports. Source file "de/joshuagleitze/gradle/CoverageTestPlugin.kt" was not found during generation of report. is generated instead of source code snippet.

[koral--]

I haven't found anything in Gradle 6.5 changelog which may cause this issue.
I've also tried to specify classdumpdir parameter as https://www.eclemma.org/jacoco/trunk/doc/classids.html suggests as a workaround but without any luck.

I've created issue on Gradle bugtracker as a possible regression: gradle/gradle#13614

[jGleitz]

Gradle seems to have fixed this in 6.6-rc-4.

[CristianGM]

It's not really fixed, as Gradle engs said, Gradle now instruments the plugins so we should change how we use JaCoCo.
Right now it works because they reverted their instrumentation, but it will be broken again soon (probably 6.7)

[koral--]

From gradle/gradle#13614 (comment) it looks like Gradle added some workaround even in 6.6 RC1.
@CristianGM do you have any specific idea what exactly should we change in the plugin?

[jGleitz]

@koral-- gradle/gradle#2429 contains some workarounds for enabling offline instrumentation. Especially https://github.com/AureaMohammadAlavi/gradle-jacoo-offline-instrumentation seems to be useful. I have not tried it, though. I also don’t know how easy it would be to enable offline instrumentation transparently, as you need to change the test classpath.

[CristianGM]

I was going to tell you something similar, the official docs are https://www.jacoco.org/jacoco/trunk/doc/offline.html

But that plugin seems to be the best starting point.

In any case, I have no idea on how to implement it, I knew the concept but never played with it.

[joshuadavis]

I've just experienced the same issue. Downgrading to Gradle 6.3 worked, so I'll stick with that for now. Let me know if I can help with a fix. The basic error was 'classes do not match' as shown above.

Environment:

• Gradle 6.5.1 (6.3 works)
• pl.droidsonroids.jacoco.testkit version 1.0.7
• Java 11.0.2
• Jacoco 0.8.5