J9 failed to detect that the class is not defined during the loading stage? #9323

fuzzy000 · 2020-04-22T18:33:08Z

Java -version output

The issue happened in following versions:

openjdk version "1.8.0_232"
OpenJDK Runtime Environment (build 1.8.0_232-b09)
Eclipse OpenJ9 VM (build openj9-0.17.0, JRE 1.8.0 Linux amd64-64-Bit Compressed References 20191017_442 (JIT enabled, AOT enabled)
OpenJ9   - 77c1cf708
OMR      - 20db4fbc
JCL      - 97b5ec8f383 based on jdk8u232-b09)

openjdk version "9.0.4-adoptopenjdk"
OpenJDK Runtime Environment (build 9.0.4-adoptopenjdk+12)
Eclipse OpenJ9 VM (build openj9-0.9.0, JRE 9 Linux amd64-64-Bit Compressed References 20180814_248 (JIT enabled, AOT enabled)
OpenJ9   - 24e53631
OMR      - fad6bf6e
JCL      - feec4d2ae based on jdk-9.0.4+12)

openjdk version "11.0.5" 2019-10-15
OpenJDK Runtime Environment AdoptOpenJDK (build 11.0.5+10)
Eclipse OpenJ9 VM AdoptOpenJDK (build openj9-0.17.0, JRE 11 Linux amd64-64-Bit Compressed References 20191016_358 (JIT enabled, AOT enabled)
OpenJ9   - 77c1cf708
OMR      - 20db4fbc
JCL      - 2a7af5674b based on jdk-11.0.5+10)

Summary of problem

We made some changes to a class file in the Apache Ant project (org/apache/tools/ant/Project.class), and let several JVMs run the corresponding test file (ProjectTest.class) to test that class. OpenJDK series report NoClassDefFoundError while J9 series report VerifyError.

Actually, although the file name is Project.class, the content in it is a class named ProjectHelperRepository. When JVM is processing a class file, it should follow this order: loading, linking, and initializing. NoClassDefFoundError is thrown at the loading stage, and VerifyError is thrown at the linking stage (verification happens during linking). So in this case, since J9 directly throws a VerifyError, maybe J9 failed to check that the class Project is not defined during the loading stage? Is that a defect?

Diagnostic files

Verify3.zip

Extract Verify3.zip
In directory Verify3, run command java -cp sootOutput/junit-ant/:hamcrest-core-1.3.jar:junit-4.12.jar org.junit.runner.JUnitCore org.apache.tools.ant.ProjectTest

Execution environment

OS and version: Ubuntu 16.04.6 LTS
CPU model: Intel(R) Xeon(R) CPU E5-4610 v4 @ 1.80GHz
Number of CPU cores: 4CPUs, each has 10 cores
Size of physical memory: 16384 MB * 20

The text was updated successfully, but these errors were encountered:

DanHeidinga · 2020-04-22T18:44:43Z

I've assigned this to the 0.21.0 milestone which tracks issues for our next release. It may take a few days before someone can dig further into this.

gacholio · 2020-04-23T19:01:46Z

We are throwing the VerifyError from the loading phase, not linking:

#12 0x00007fb13933975f in setCurrentExceptionUTF (vmThread=0x19cce00, exceptionNumber=58, detailUTF=0x7fb1343d3a10 "JVMCFRE072 entries not sorted in lookupswitch bytecode; class=org/apache/tools/ant/Project, method=<clinit>()V, pc=29\nException Details:\n  Location:\n    org/apache/tools/ant/Project.<clinit>()V @29: J"...) at exceptionsupport.c:65
#13 0x00007fb1393db787 in createROMClassFromClassFile (currentThread=0x19cce00, loadData=0x7fb13a2fc8a0, localBuffer=0x7fb13a2fca10) at defineclass.c:926
#14 0x00007fb1393d8ae1 in internalDefineClass (vmThread=0x19cce00, className=0x7fb13a2fca30, classNameLength=28, classData=0xfffaf660 "\312\376\272\276", classDataLength=60190, classDataObject=0x0, classLoader=0x7fb13406a5b8, protectionDomain=0xfff04218, options=4097, existingROMClass=0x0, hostClass=0x0, localBuffer=0x7fb13a2fca10) at defineclass.c:141

gacholio · 2020-04-23T19:14:55Z

Removing the lookupswitch ordering verification code results in the expected NoClassDefFoundError, so this is just an ordering issue with the verification tests (if this is really an issue at all).

sharon-wang · 2020-05-11T15:23:50Z

I'm doing some JVM spec reading to confirm the appropriate timing of these errors.

sharon-wang · 2020-05-12T19:55:01Z

From the Creation and Loading section of the JVM spec, it states:

If an error occurs during class loading, then an instance of a subclass of LinkageError must be thrown at a point in the program that (directly or indirectly) uses the class or interface being loaded.

Both VerifyError and NoClassDefFoundError subclass LinkageError, so no issues there.

Zooming in on:

...thrown at a point in the program that (directly or indirectly) uses the class or interface being loaded

My understanding here is that the error can be reported at the Linking phase, even if the error itself occurred during Loading.

The spec also states that:

If the Java Virtual Machine ever attempts to load a class C during verification (§5.4.1) or resolution (§5.4.3) (but not initialization (§5.5)), and the class loader that is used to initiate loading of C throws an instance of ClassNotFoundException, then the Java Virtual Machine must throw an instance of NoClassDefFoundError whose cause is the instance of ClassNotFoundException.

I don't think this is the same situation as the NoClassDefFoundError we are looking into. This situation would involve Project.class not being accessible in the classpath, i.e. if it were renamed to something else, deleted, or moved to an inaccessible location.

Overall, if I'm understanding correctly, it seems like the spec is permissive of both the Hotspot and OpenJ9 forms of handling the class error. However, NoClassDefFoundError would make it more clear that the class name mismatch is the cause of the failure.

Here's the output of java -cp sootOutput/junit-ant/:hamcrest-core-1.3.jar:junit-4.12.jar org.junit.runner.JUnitCore org.apache.tools.ant.ProjectTest for reference:

OpenJ9

openjdk version "14.0.1" 2020-04-14
OpenJDK Runtime Environment AdoptOpenJDK (build 14.0.1+7)
Eclipse OpenJ9 VM AdoptOpenJDK (build openj9-0.20.0, JRE 14 Linux amd64-64-Bit Compressed References 20200416_44 (JIT enabled, AOT enabled)
OpenJ9   - 05fa2d361
OMR      - d4365f371
JCL      - 5757187cae based on jdk-14.0.1+7)

1) initializationError(org.apache.tools.ant.ProjectTest)
java.lang.VerifyError: JVMCFRE072 entries not sorted in lookupswitch bytecode; class=org/apache/tools/ant/Project, method=<clinit>()V, pc=29
Exception Details:
  Location:
    org/apache/tools/ant/Project.<clinit>()V @29: JBlookupswitch
  Reason:
    Error exists in the bytecode.
  Exception Handler Table:
    bci [155, 201] => handler: 204
	at java.base/java.lang.ClassLoader.defineClassImpl(Native Method)
	at java.base/java.lang.ClassLoader.defineClassInternal(ClassLoader.java:479)
	at java.base/java.lang.ClassLoader.defineClass(ClassLoader.java:440)
	at java.base/java.security.SecureClassLoader.defineClass(SecureClassLoader.java:151)
	at java.base/jdk.internal.loader.BuiltinClassLoader.defineClass(BuiltinClassLoader.java:1133)
	at java.base/jdk.internal.loader.BuiltinClassLoader.findClassOnClassPathOrNull(BuiltinClassLoader.java:920)
	at java.base/jdk.internal.loader.BuiltinClassLoader.loadClassOrNull(BuiltinClassLoader.java:828)
	at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:786)
	at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)
	at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:1081)
	at java.base/java.lang.Class.getDeclaredMethodsImpl(Native Method)
	at java.base/java.lang.Class.getDeclaredMethods(Class.java:1134)
	at org.junit.internal.MethodSorter.getDeclaredMethods(MethodSorter.java:54)
	at org.junit.runners.model.TestClass.scanAnnotatedMembers(TestClass.java:65)
	at org.junit.runners.model.TestClass.<init>(TestClass.java:57)
	at org.junit.runners.ParentRunner.createTestClass(ParentRunner.java:88)
	at org.junit.runners.ParentRunner.<init>(ParentRunner.java:83)
	at org.junit.runners.BlockJUnit4ClassRunner.<init>(BlockJUnit4ClassRunner.java:65)
	at org.junit.internal.builders.JUnit4Builder.runnerForClass(JUnit4Builder.java:10)
	at org.junit.runners.model.RunnerBuilder.safeRunnerForClass(RunnerBuilder.java:59)
	at org.junit.internal.builders.AllDefaultPossibilitiesBuilder.runnerForClass(AllDefaultPossibilitiesBuilder.java:26)
	at org.junit.runner.Computer.getRunner(Computer.java:40)
	at org.junit.runner.Computer$1.runnerForClass(Computer.java:31)
	at org.junit.runners.model.RunnerBuilder.safeRunnerForClass(RunnerBuilder.java:59)
	at org.junit.runners.model.RunnerBuilder.runners(RunnerBuilder.java:101)
	at org.junit.runners.model.RunnerBuilder.runners(RunnerBuilder.java:87)
	at org.junit.runners.Suite.<init>(Suite.java:81)
	at org.junit.runner.Computer.getSuite(Computer.java:28)
	at org.junit.runner.Request.classes(Request.java:75)
	at org.junit.runner.JUnitCommandLineParseResult.createRequest(JUnitCommandLineParseResult.java:118)
	at org.junit.runner.JUnitCore.runMain(JUnitCore.java:77)
	at org.junit.runner.JUnitCore.main(JUnitCore.java:36)

Hotspot

openjdk version "14.0.1" 2020-04-14
OpenJDK Runtime Environment AdoptOpenJDK (build 14.0.1+7-202005072031)
OpenJDK 64-Bit Server VM AdoptOpenJDK (build 14.0.1+7-202005072031, mixed mode, sharing

1) initializationError(org.apache.tools.ant.ProjectTest)
java.lang.NoClassDefFoundError: org/apache/tools/ant/ProjectHelperRepository (wrong name: org/apache/tools/ant/Project)
	at java.base/java.lang.ClassLoader.defineClass1(Native Method)
	at java.base/java.lang.ClassLoader.defineClass(ClassLoader.java:1017)
	at java.base/java.security.SecureClassLoader.defineClass(SecureClassLoader.java:151)
	at java.base/jdk.internal.loader.BuiltinClassLoader.defineClass(BuiltinClassLoader.java:821)
	at java.base/jdk.internal.loader.BuiltinClassLoader.findClassOnClassPathOrNull(BuiltinClassLoader.java:719)
	at java.base/jdk.internal.loader.BuiltinClassLoader.loadClassOrNull(BuiltinClassLoader.java:642)
	at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:600)
	at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)
	at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522)
	at java.base/java.lang.Class.getDeclaredMethods0(Native Method)
	at java.base/java.lang.Class.privateGetDeclaredMethods(Class.java:3244)
	at java.base/java.lang.Class.getDeclaredMethods(Class.java:2387)
	at org.junit.internal.MethodSorter.getDeclaredMethods(MethodSorter.java:54)
	at org.junit.runners.model.TestClass.scanAnnotatedMembers(TestClass.java:65)
	at org.junit.runners.model.TestClass.<init>(TestClass.java:57)
	at org.junit.runners.ParentRunner.createTestClass(ParentRunner.java:88)
	at org.junit.runners.ParentRunner.<init>(ParentRunner.java:83)
	at org.junit.runners.BlockJUnit4ClassRunner.<init>(BlockJUnit4ClassRunner.java:65)
	at org.junit.internal.builders.JUnit4Builder.runnerForClass(JUnit4Builder.java:10)
	at org.junit.runners.model.RunnerBuilder.safeRunnerForClass(RunnerBuilder.java:59)
	at org.junit.internal.builders.AllDefaultPossibilitiesBuilder.runnerForClass(AllDefaultPossibilitiesBuilder.java:26)
	at org.junit.runner.Computer.getRunner(Computer.java:40)
	at org.junit.runner.Computer$1.runnerForClass(Computer.java:31)
	at org.junit.runners.model.RunnerBuilder.safeRunnerForClass(RunnerBuilder.java:59)
	at org.junit.runners.model.RunnerBuilder.runners(RunnerBuilder.java:101)
	at org.junit.runners.model.RunnerBuilder.runners(RunnerBuilder.java:87)
	at org.junit.runners.Suite.<init>(Suite.java:81)
	at org.junit.runner.Computer.getSuite(Computer.java:28)
	at org.junit.runner.Request.classes(Request.java:75)
	at org.junit.runner.JUnitCommandLineParseResult.createRequest(JUnitCommandLineParseResult.java:118)
	at org.junit.runner.JUnitCore.runMain(JUnitCore.java:77)
	at org.junit.runner.JUnitCore.main(JUnitCore.java:36)

sharon-wang · 2020-05-27T20:37:39Z

Hi @fuzzy000!

The fix for this issue, as well as #9336 and #9357 has been merged. If you'd like to test out the changes, grab a nightly build with date 27 May 2020 or later at https://adoptopenjdk.net/nightly.html?jvmVariant=openj9.

fuzzy000 added the userRaised label Apr 22, 2020

DanHeidinga added the comp:vm label Apr 22, 2020

DanHeidinga added this to the Release 0.21 (Java 8, 11, 14) July refresh milestone Apr 22, 2020

fuzzy000 mentioned this issue Apr 24, 2020

J9 failed to execute all JUnit tests in a test file (caused by initializationError) #9357

Closed

sharon-wang mentioned this issue May 22, 2020

Move lookupswitch match order verification to second pass #9660

Merged

gacholio closed this as completed May 25, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

J9 failed to detect that the class is not defined during the loading stage? #9323

J9 failed to detect that the class is not defined during the loading stage? #9323

fuzzy000 commented Apr 22, 2020

DanHeidinga commented Apr 22, 2020

gacholio commented Apr 23, 2020

gacholio commented Apr 23, 2020

sharon-wang commented May 11, 2020

sharon-wang commented May 12, 2020

sharon-wang commented May 27, 2020

J9 failed to detect that the class is not defined during the loading stage? #9323

J9 failed to detect that the class is not defined during the loading stage? #9323

Comments

fuzzy000 commented Apr 22, 2020

Java -version output

Summary of problem

Diagnostic files

Execution environment

DanHeidinga commented Apr 22, 2020

gacholio commented Apr 23, 2020

gacholio commented Apr 23, 2020

sharon-wang commented May 11, 2020

sharon-wang commented May 12, 2020

sharon-wang commented May 27, 2020