J9 (9.0.4) threw NullPointerException while other JVMs threw AssertionError.

[fuzzy000]

JVM Versions

We used the following versions of J9:

openjdk version "1.8.0_275"
OpenJDK Runtime Environment (build 1.8.0_275-b01)
Eclipse OpenJ9 VM (build openj9-0.23.0, JRE 1.8.0 Linux amd64-64-Bit Compressed References 20201110_845 (JIT enabled, AOT enabled)
OpenJ9   - 0394ef754
OMR      - 582366ae5
JCL      - b52d2ff7ee based on jdk8u275-b01)

openjdk version "9.0.4-adoptopenjdk"
OpenJDK Runtime Environment (build 9.0.4-adoptopenjdk+12)
Eclipse OpenJ9 VM (build openj9-0.9.0, JRE 9 Linux amd64-64-Bit Compressed References 20180814_248 (JIT enabled, AOT enabled)
OpenJ9   - 24e53631
OMR      - fad6bf6e
JCL      - feec4d2ae based on jdk-9.0.4+12)

openjdk version "11.0.9" 2020-10-20
OpenJDK Runtime Environment AdoptOpenJDK (build 11.0.9+11)
Eclipse OpenJ9 VM AdoptOpenJDK (build openj9-0.23.0, JRE 11 Linux amd64-64-Bit Compressed References 20201022_810 (JIT enabled, AOT enabled)
OpenJ9   - 0394ef754
OMR      - 582366ae5
JCL      - 3b09cfd7e9 based on jdk-11.0.9+11)

The used OpenJDK versions:

openjdk version "1.8.0_275"
OpenJDK Runtime Environment (AdoptOpenJDK)(build 1.8.0_275-b01)
OpenJDK 64-Bit Server VM (AdoptOpenJDK)(build 25.275-b01, mixed mode)

openjdk version "9.0.4"
OpenJDK Runtime Environment (build 9.0.4+11)
OpenJDK 64-Bit Server VM (build 9.0.4+11, mixed mode)

openjdk version "11.0.9" 2020-10-20
OpenJDK Runtime Environment AdoptOpenJDK (build 11.0.9+11)
OpenJDK 64-Bit Server VM AdoptOpenJDK (build 11.0.9+11, mixed mode)

Summary of the Problem

We made some changes to a class file in the JUnit project, and let several JVMs execute the corresponding JUnit test. We found that in a run, J9 (9.0.4) threw NullPointerException while other JVMs, including J9 (1.8.0_275), J9 (11.0.9) and OpenJDK (8u275, 9.0.4, 11.0.9), threw AssertionError.

For example,
J9 (9.0.4):

There were 2 failures:
1) neitherLogSuccessNorFailedForViolatedAssumption(org.junit.rules.TestWatchmanTest)
java.lang.NullPointerException
        at org.junit.rules.TestWatchmanTest.neitherLogSuccessNorFailedForViolatedAssumption(TestWatchmanTest.java:35)
2) logFailingTest(org.junit.rules.TestWatchmanTest)
java.lang.NullPointerException
        at org.junit.rules.TestWatchmanTest.logFailingTest(TestWatchmanTest.java:59)

FAILURES!!!
Tests run: 2,  Failures: 2

J9 (1.8.0_275) and J9 (11.0.9)

There were 2 failures:
1) neitherLogSuccessNorFailedForViolatedAssumption(org.junit.rules.TestWatchmanTest)
java.lang.AssertionError:
Expected: is "starting finished "
     but: was ""
        at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:20)
        at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:8)
        at org.junit.rules.TestWatchmanTest.neitherLogSuccessNorFailedForViolatedAssumption(TestWatchmanTest.java:35)
2) logFailingTest(org.junit.rules.TestWatchmanTest)
java.lang.AssertionError:
Expected: is "starting failed finished "
     but: was ""
        at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:20)
        at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:8)
        at org.junit.rules.TestWatchmanTest.logFailingTest(TestWatchmanTest.java:59)

FAILURES!!!
Tests run: 2,  Failures: 2

Diagnostic files

NPEIssue.zip

Steps to reproduce the behavior:

1. extract the NPEIssue.zip
2. In directory NPEIssue, run command java -cp sootOutput/junit-junit/:hamcrest-all-1.3.jar:junit-4.12.jar org.junit.runner.JUnitCore org.junit.rules.TestWatchmanTest

Execution environment

• OS and version: Ubuntu 16.04.6 LTS
• CPU model: Intel(R) Xeon(R) CPU E5-4610 v4 @ 1.80GHz
• Number of CPU cores: 4CPUs, each has 10 cores
• Size of physical memory: 16384 MB * 20

[pshipton]

Java 9 is way out of support.