Kotlin code fails to compile due to java.lang.IncompatibleClassChangeError: MyClass and MyClass$special$$inlined$sortedByDescending$1 disagree on InnerClasses attribute

[zakkak]

Describe the issue
Since the merge of ce3d101 several Quarkus tests including kotlin code have started failing.

The issue seems to be originating from the fact that the kotlin compiler doesn't properly(?) generate the InnerClasses attribute for outer classes of generated inner classes. Inspecting the generated class files (with javap -v we see that the generated (by the kotlin compiler) inner class has its InnerClasses attribute set while the original outer class does not. This results in getDeclaringClass0 to fail with java.lang.IncompatibleClassChangeError: MyClass and MyClass$special$$inlined$sortedByDescending$1 disagree on InnerClasses attribute during native image generation.

Steps to reproduce the issue
@jerboaa successfully created standalone reproducers that do not rely on Quarkus.

Case 1

git clone [email protected]:jerboaa/kotlin-icce.git --branch sortedByIssue
cd kotlin-icce
./run.sh path/to/graalvm-22.3-dev-build

Case 2

git clone [email protected]:jerboaa/kotlin-icce.git --branch vetoableRepro
cd kotlin-icce
./run.sh path/to/graalvm-22.3-dev-build

Describe GraalVM and your environment:

• GraalVM version: fb7d0d1
• JDK major version: 17
• OS: Fedora 36
• Architecture: AMD64

More details

The trace for vetoableRepro looks like this:

========================================================================================================================
GraalVM Native Image: Generating 'kr' (executable)...
========================================================================================================================
[1/7] Initializing...                                                                                    (3.9s @ 0.21GB)
 Version info: 'GraalVM 22.3.0-dev Java 17 CE'
 Java version info: '17.0.4+7-jvmci-22.3-b02'
 C compiler: gcc (redhat, x86_64, 12.2.1)
 Garbage collector: Serial GC
[2/7] Performing analysis...  []                                                                         (3.4s @ 0.51GB)
   1,339 (64.75%) of  2,068 classes reachable
   1,085 (39.43%) of  2,752 fields reachable
   4,381 (56.98%) of  7,689 methods reachable
      24 classes,     0 fields, and     0 methods registered for reflection

Fatal error: com.oracle.graal.pointsto.util.AnalysisError$ParsingError: Error encountered while parsing hello.TestKt.getHelloStringFromJava() 
Parsing context:
   at hello.TestKt.getHelloStringFromJava(Test.kt:20)
   at hello.JavaHello.main(JavaHello.java:12)
   at com.oracle.svm.core.JavaMainWrapper.runCore0(JavaMainWrapper.java:169)
   at com.oracle.svm.core.JavaMainWrapper.runCore(JavaMainWrapper.java:129)

	at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.util.AnalysisError.parsingError(AnalysisError.java:153)
	at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.flow.MethodTypeFlow.createFlowsGraph(MethodTypeFlow.java:104)
	at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.flow.MethodTypeFlow.ensureFlowsGraphCreated(MethodTypeFlow.java:83)
	at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.flow.MethodTypeFlow.getOrCreateMethodFlowsGraph(MethodTypeFlow.java:65)
	at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.typestate.DefaultStaticInvokeTypeFlow.update(DefaultStaticInvokeTypeFlow.java:64)
	at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.PointsToAnalysis$1.run(PointsToAnalysis.java:488)
	at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.util.CompletionExecutor.executeCommand(CompletionExecutor.java:193)
	at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.util.CompletionExecutor.lambda$executeService$0(CompletionExecutor.java:177)
	at java.base/java.util.concurrent.ForkJoinTask$RunnableExecuteAction.exec(ForkJoinTask.java:1395)
	at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373)
	at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182)
	at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655)
	at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622)
	at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165)
Caused by: org.graalvm.compiler.java.BytecodeParser$BytecodeParserError: com.oracle.graal.pointsto.constraints.UnsupportedFeatureException: Discovered a type for which getDeclaringClass0 cannot be called: hello.Test$special$$inlined$vetoable$1. This error is reported at image build time because class hello.Test$special$$inlined$vetoable$1 is registered for linking at image build time by command line
	at parsing hello.Test.<init>(Test.kt:27)
	at jdk.internal.vm.compiler/org.graalvm.compiler.java.BytecodeParser.throwParserError(BytecodeParser.java:2518)
	at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.phases.SharedGraphBuilderPhase$SharedBytecodeParser.throwParserError(SharedGraphBuilderPhase.java:110)
	at jdk.internal.vm.compiler/org.graalvm.compiler.java.BytecodeParser.iterateBytecodesForBlock(BytecodeParser.java:3379)
	at jdk.internal.vm.compiler/org.graalvm.compiler.java.BytecodeParser.handleBytecodeBlock(BytecodeParser.java:3331)
	at jdk.internal.vm.compiler/org.graalvm.compiler.java.BytecodeParser.processBlock(BytecodeParser.java:3176)
	at jdk.internal.vm.compiler/org.graalvm.compiler.java.BytecodeParser.build(BytecodeParser.java:1138)
	at jdk.internal.vm.compiler/org.graalvm.compiler.java.BytecodeParser.buildRootMethod(BytecodeParser.java:1030)
	at jdk.internal.vm.compiler/org.graalvm.compiler.java.GraphBuilderPhase$Instance.run(GraphBuilderPhase.java:97)
	at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.phases.SharedGraphBuilderPhase.run(SharedGraphBuilderPhase.java:84)
	at jdk.internal.vm.compiler/org.graalvm.compiler.phases.Phase.run(Phase.java:49)
	at jdk.internal.vm.compiler/org.graalvm.compiler.phases.BasePhase.apply(BasePhase.java:446)
	at jdk.internal.vm.compiler/org.graalvm.compiler.phases.Phase.apply(Phase.java:42)
	at jdk.internal.vm.compiler/org.graalvm.compiler.phases.Phase.apply(Phase.java:38)
	at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.flow.AnalysisParsedGraph.parseBytecode(AnalysisParsedGraph.java:135)
	at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.meta.AnalysisMethod.ensureGraphParsed(AnalysisMethod.java:685)
	at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.phases.InlineBeforeAnalysisGraphDecoder.lookupEncodedGraph(InlineBeforeAnalysis.java:180)
	at jdk.internal.vm.compiler/org.graalvm.compiler.replacements.PEGraphDecoder.doInline(PEGraphDecoder.java:1162)
	at jdk.internal.vm.compiler/org.graalvm.compiler.replacements.PEGraphDecoder.tryInline(PEGraphDecoder.java:1145)
	at jdk.internal.vm.compiler/org.graalvm.compiler.replacements.PEGraphDecoder.trySimplifyInvoke(PEGraphDecoder.java:1003)
	at jdk.internal.vm.compiler/org.graalvm.compiler.replacements.PEGraphDecoder.handleInvoke(PEGraphDecoder.java:957)
	at jdk.internal.vm.compiler/org.graalvm.compiler.nodes.GraphDecoder.processNextNode(GraphDecoder.java:817)
	at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.phases.InlineBeforeAnalysisGraphDecoder.processNextNode(InlineBeforeAnalysis.java:240)
	at jdk.internal.vm.compiler/org.graalvm.compiler.nodes.GraphDecoder.decode(GraphDecoder.java:548)
	at jdk.internal.vm.compiler/org.graalvm.compiler.replacements.PEGraphDecoder.decode(PEGraphDecoder.java:833)
	at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.phases.InlineBeforeAnalysis.decodeGraph(InlineBeforeAnalysis.java:98)
	at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.flow.MethodTypeFlowBuilder.parse(MethodTypeFlowBuilder.java:177)
	at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.flow.MethodTypeFlowBuilder.apply(MethodTypeFlowBuilder.java:344)
	at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.flow.MethodTypeFlow.createFlowsGraph(MethodTypeFlow.java:93)
	... 12 more
Caused by: com.oracle.graal.pointsto.constraints.UnsupportedFeatureException: Discovered a type for which getDeclaringClass0 cannot be called: hello.Test$special$$inlined$vetoable$1. This error is reported at image build time because class hello.Test$special$$inlined$vetoable$1 is registered for linking at image build time by command line
	at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.SVMHost.handleLinkageError(SVMHost.java:426)
	at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.SVMHost.getDeclaringClass(SVMHost.java:412)
	at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.SVMHost.createHub(SVMHost.java:400)
	at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.SVMHost.registerType(SVMHost.java:246)
	at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.meta.AnalysisUniverse.createType(AnalysisUniverse.java:303)
	at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.meta.AnalysisUniverse.lookupAllowUnresolved(AnalysisUniverse.java:213)
	at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.infrastructure.WrappedConstantPool.lookupType(WrappedConstantPool.java:255)
	at jdk.internal.vm.compiler/org.graalvm.compiler.java.BytecodeParser.lookupType(BytecodeParser.java:4178)
	at jdk.internal.vm.compiler/org.graalvm.compiler.java.BytecodeParser.genNewInstance(BytecodeParser.java:4464)
	at jdk.internal.vm.compiler/org.graalvm.compiler.java.BytecodeParser.processBytecode(BytecodeParser.java:5246)
	at jdk.internal.vm.compiler/org.graalvm.compiler.java.BytecodeParser.iterateBytecodesForBlock(BytecodeParser.java:3371)
	... 37 more
Caused by: java.lang.IncompatibleClassChangeError: hello.Test and hello.Test$special$$inlined$vetoable$1 disagree on InnerClasses attribute
	at java.base/java.lang.Class.getDeclaringClass0(Native Method)
	at java.base/jdk.internal.reflect.GeneratedMethodAccessor10.invoke(Unknown Source)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
	at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.SVMHost.getDeclaringClass(SVMHost.java:409)
	... 46 more
------------------------------------------------------------------------------------------------------------------------
                        0.2s (2.9% of total time) in 10 GCs | Peak RSS: 0.94GB | CPU load: 5.52
========================================================================================================================
Failed generating 'kr' after 7.5s.
Error: Image build request failed with exit status 1

[zakkak]

cc @pejovica

[jerboaa]

What's curious is that it's unclear which linkage error Graal is trying to detect here with --link-at-build-time. There is no linkage error at runtime when using plain Java.

$ java -cp ./target/mixed-code-hello-world-1.0-SNAPSHOT-jar-with-dependencies.jar hello.JavaHello
Hello from Kotlin!
class java.util.Collections$SingletonList (Kotlin reflection is not available)
Hello from Java!
$ java -Xlog:class+load=info -cp ./target/mixed-code-hello-world-1.0-SNAPSHOT-jar-with-dependencies.jar hello.JavaHello | grep sort
[0.069s][info][class,load] hello.Test$special$$inlined$sortedByDescending$1 source: file:/home/sgehwolf/Documents/mandrel/bugs/upstream-graal/kotlin_getdeclaredclass0_repro/kotlin-icce/target/mixed-code-hello-world-1.0-SNAPSHOT-jar-with-dependencies.jar

[zakkak]

When using Java the inner class is not being accessed through getDeclaringClass0 which is the one testing for the agreement in the InnerClasses attribute. When using native-image, getDeclaringClass0 is explicitly invoked to get the declaring class of the inner class, resulting in the observed issue.

The way I see it there are two possible issues:

1. Kotlin generates invalid bytecode and the JVM fails to catch it. If that's true, there are two issues to be fixed, one at the kotlin compiler and another in the JVM itself as I would expect the JVM to detect and report the issue at class load.
2. GraalVM wrongfully invokes getDeclaringClass0 in this case, resulting in the observed issue.

[jerboaa]

When using Java the inner class is not being accessed through getDeclaringClass0 which is the one testing for the agreement in the InnerClasses attribute. When using native-image, getDeclaringClass0 is explicitly invoked to get the declaring class of the inner class, resulting in the observed issue.

Yes, for it to surface with plain java getDeclaringClass() call is necessary. Repro of https://github.com/jerboaa/kotlin-icce main branch.

The way I see it there are two possible issues:

1. Kotlin generates invalid bytecode and the JVM fails to catch it. If that's true, there are two issues to be fixed, one at the kotlin compiler and another in the JVM itself as I would expect the JVM to detect and report the issue at class load.

Yes. Proving the latter is something I'm failing at the moment.

2. GraalVM wrongfully invokes `getDeclaringClass0` in this case, resulting in the observed issue.

That seems a possibility too. Removal of --link-at-build-time makes the image generation work and it behaves the same way as the java program at runtime (no link error). So this seems to suggest this check is overzealous?

[zakkak]

Thank you for the fix @pejovica !