Error calculating coverage because of NPE in readClasspathManifest

[lalithsuresh]

I'm using the Gradle plugin version 1.5.2 to run pitest in a Junit5 project. I've configured pitest only in one sub-project for now (see the end of the issue for repro).

plugins {
    id 'info.solidsoft.pitest' version '1.5.2'
}

pitest {
    //adds dependency to org.pitest:pitest-junit5-plugin and sets "testPlugin" to "junit5"
    junit5PluginVersion = '0.12'
    // ...
}

It seems like the tests are running, but something messes up the code coverage calculation.

12:26:06 PM PIT >> INFO : MINION : 12:26:06 PM PIT >> INFO : Found  72 tests

12:26:06 PM PIT >> INFO : MINION : 12:26:06 PM PIT >> SEVERE : Error calculating coverage. Process will exit.
java.lang.NullPointerException
        at org.pitest.util.ManifestUtils.readClasspathManifest(ManifestUtils.java:78)
        at org.pitest.classpath.ClassPath.lambda$addEntriesFromClasspathManife
12:26:06 PM PIT >> INFO : MINION : st$2(ClassPath.java:143)
        at java.base/java.util.Optional.ifPresent(Optional.java:183)
        at org.pitest.classpath.ClassPath.addEntriesFromClasspathManifest(ClassPath.java:143)
        at org.pitest.classpath.ClassPath.getClassPathElementsAsFiles(ClassPath.java:128)
12:26:06 PM PIT >> INFO : MINION :
        at org.pitest.classpath.ClassPath.<init>(ClassPath.java:47)
        at org.pitest.classpath.ClassPathByteArraySource.<init>(ClassPathByteArraySource.java:31)
        at org.pitest.coverage.execute.CoverageMinion.getTestsFromParent(CoverageMinion.java:140)
        at org.pite
12:26:06 PM PIT >> INFO : MINION : st.coverage.execute.CoverageMinion.main(CoverageMinion.java:84)

12:26:06 PM PIT >> SEVERE : Coverage generator Minion exited abnormally due to UNKNOWN_ERROR
Exception in thread "main" org.pitest.util.PitError: Coverage generation minion exited abnormally!

Please copy and paste the information and the complete stacktrace below when reporting an issue
VM : OpenJDK 64-Bit Server VM
Vendor : Oracle Corporation
Version : 12.0.1+12
Uptime : 37527
Input ->
 1 : -Dfile.encoding=UTF-8
 2 : -Duser.country=US
 3 : -Duser.language=en
 4 : -Duser.variant
BootClassPathSupported : false


Please copy and paste the information and the complete stacktrace below when reporting an issue
VM : OpenJDK 64-Bit Server VM
Vendor : Oracle Corporation
Version : 12.0.1+12
Uptime : 37528
Input ->
 1 : -Dfile.encoding=UTF-8
 2 : -Duser.country=US
 3 : -Duser.language=en
 4 : -Duser.variant
BootClassPathSupported : false

        at org.pitest.util.Unchecked.translateCheckedException(Unchecked.java:20)
        at org.pitest.coverage.execute.DefaultCoverageGenerator.calculateCoverage(DefaultCoverageGenerator.java:105)
        at org.pitest.coverage.execute.DefaultCoverageGenerator.calculateCoverage(DefaultCoverageGenerator.java:51)
        at org.pitest.mutationtest.tooling.MutationCoverage.runReport(MutationCoverage.java:115)
        at org.pitest.mutationtest.tooling.EntryPoint.execute(EntryPoint.java:121)
        at org.pitest.mutationtest.tooling.EntryPoint.execute(EntryPoint.java:51)
        at org.pitest.mutationtest.commandline.MutationCoverageReport.runReport(MutationCoverageReport.java:87)
        at org.pitest.mutationtest.commandline.MutationCoverageReport.main(MutationCoverageReport.java:45)
Caused by: org.pitest.util.PitError: Coverage generation minion exited abnormally!

Please copy and paste the information and the complete stacktrace below when reporting an issue
VM : OpenJDK 64-Bit Server VM
Vendor : Oracle Corporation
Version : 12.0.1+12
Uptime : 37527
Input ->
 1 : -Dfile.encoding=UTF-8
 2 : -Duser.country=US
 3 : -Duser.language=en
 4 : -Duser.variant
BootClassPathSupported : false

        at org.pitest.coverage.execute.DefaultCoverageGenerator.gatherCoverageData(DefaultCoverageGenerator.java:148)
        at org.pitest.coverage.execute.DefaultCoverageGenerator.calculateCoverage(DefaultCoverageGenerator.java:89)
        ... 6 more

> Task :dcm:pitest FAILED

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':dcm:pitest'.
> Process 'command '/Library/Java/JavaVirtualMachines/openjdk-12.0.1.jdk/Contents/Home/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

If you'd like to reproduce this failure:

$: g clone -b pitest https://github.com/vmware/declarative-cluster-management/tree/pitest
$: cd declarative-cluster-management
$: ./gradlew pitest

You can see the most recent commit to see the changes I introduced.

diff --git a/dcm/build.gradle b/dcm/build.gradle
index 082e424..dbc8aea 100644
--- a/dcm/build.gradle
+++ b/dcm/build.gradle
@@ -6,8 +6,10 @@
 plugins {
     id 'maven-publish'
     id 'signing'
+    id 'info.solidsoft.pitest' version '1.5.2'
 }

+
 dependencies {
     implementation "org.freemarker:freemarker:${freemarkerVersion}"
     implementation "org.apache.commons:commons-csv:${commonsCsvVersion}"
@@ -32,6 +34,12 @@ java {
     withSourcesJar()
 }

+pitest {
+    //adds dependency to org.pitest:pitest-junit5-plugin and sets "testPlugin" to "junit5"
+    junit5PluginVersion = '0.12'
+    // ...
+}
+
 publishing {
     repositories {
         maven {

Thanks!

[szpak]

I investigated that topic. Initially, I was a little bit surprised as using a file to pass classpath to PIT (which seems to throw NPE) is not enabled by default in the Gradle plugin. However, it turned out that internally it is used by default anyway. The problem here is caused by the fact that there is:

/home/foobar/.gradle/caches/.../com.skaggsm/classpath-resource-extractor/0.2.0/.../classpath-resource-extractor-0.2.0.jar

on the classpath. It is caught by:

pitest/pitest/src/main/java/org/pitest/classpath/ClassPath.java

Lines 140 to 144 in fab8ddf

	private static void addEntriesFromClasspathManifest(final Set<File> elements) {
	Optional<File> maybeJar = elements.stream().filter( f -> f.getName().startsWith("classpath") && f.getName().endsWith(".jar"))
	.findFirst();
	maybeJar.ifPresent(file -> elements.addAll(ManifestUtils.readClasspathManifest(file)));
	}

It effectively fails down the line on reading information from that JAR:

5:39:18 PM PIT >> INFO : MINION : 5:39:18 PM PIT >> SEVERE : Error calculating coverage. Process will exit.
java.lang.NullPointerException: Cannot invoke "String.split(String)" because "cp" is null
        at org.pitest.util.ManifestUtils.readClasspathManifest(ManifestUtils.java:78)

While using a file to pass the classpath elements seems to be a good idea in general, but the file prefix should be more unique (unless the whole name/path is passed in some other way):

pitest/pitest/src/main/java/org/pitest/util/ManifestUtils.java

Line 62 in fab8ddf

File jarFile = File.createTempFile("classpath", ".jar");

I created #823, but maybe @hcoles will want to fix it in the more sophisticated way.

As a workaround, I propose to use a SNAPSHOT built from by branch. I don't know if there is any workaround available in the current version (excluding temporal removing that dependency or its local re-release it with different name).

================================================================================
- Statistics
================================================================================
>> Line Coverage: 1902/3331 (57%)
>> Generated 1451 mutations Killed 609 (42%)
>> Mutations with no coverage 655. Test strength 77%
>> Ran 1533 tests (1.06 tests per mutation)
:dcm:pitest (Thread[Execution worker for ':',5,main]) completed. Took 7 mins 20.996 secs.

[lalithsuresh]

Thanks @szpak , it works fine with a SNAPSHOT build from your branch!

[iadcode]

@hcoles, it seems that this issue was fixed for a time but in commit 7876f3c9723e6072e7968282f62027ab677e456f, file pitest/src/main/java/org/pitest/classpath/ClassPath.java, the fix was reverted to the previous version. Can I ask if this is intentional? I seems to have run into the same issue due to having a classpath-scanner jar on my classpath. I'm using pitest 1.6.3 but this issue exists on master too.

[hcoles]

@iadcode Thanks for digging into this. No, that was not intentional and is very concerning. That commit was meant to just be uninteresting static analysis fixes, I can't imagine how it reverted something.

I'll revert that change and go through the rest of it.