MethodTooLargeException when too many classes to register for reflection

[essobedo]

Describe the bug

For the project camel-quarkus, I'm working on an extension allowing to marshall and unmarshall SWIFT messages by leveraging the library https://github.com/prowide/prowide-iso20022.

The problem with this library is the fact that it contains more than 3 thousand classes that must be registered for reflection. With the current code, we end up with an error of type:

[ERROR] Failed to execute goal io.quarkus:quarkus-maven-plugin:2.15.0.CR1:build (quarkus-build) on project camel-quarkus-integration-test-swift: Failed to build quarkus application: io.quarkus.builder.BuildException: Build failure: Build failed due to errors
[ERROR] 	[error]: Build step io.quarkus.deployment.steps.NativeImageFeatureStep#generateFeature threw an exception: org.objectweb.asm.MethodTooLargeException: Method too large: io/quarkus/runner/Feature.registerForReflection (Lorg/graalvm/nativeimage/hosted/Feature$BeforeAnalysisAccess;)V
[ERROR] 	at org.objectweb.asm.MethodWriter.computeMethodInfoSize(MethodWriter.java:2088)
[ERROR] 	at org.objectweb.asm.ClassWriter.toByteArray(ClassWriter.java:511)
[ERROR] 	at io.quarkus.gizmo.ClassCreator.writeTo(ClassCreator.java:219)
[ERROR] 	at io.quarkus.gizmo.ClassCreator.close(ClassCreator.java:230)
[ERROR] 	at io.quarkus.deployment.steps.NativeImageFeatureStep.generateFeature(NativeImageFeatureStep.java:727)
[ERROR] 	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[ERROR] 	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[ERROR] 	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[ERROR] 	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
[ERROR] 	at io.quarkus.deployment.ExtensionLoader$3.execute(ExtensionLoader.java:909)
[ERROR] 	at io.quarkus.builder.BuildContext.run(BuildContext.java:281)
[ERROR] 	at org.jboss.threads.ContextHandler$1.runWith(ContextHandler.java:18)
[ERROR] 	at org.jboss.threads.EnhancedQueueExecutor$Task.run(EnhancedQueueExecutor.java:2449)
[ERROR] 	at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1478)
[ERROR] 	at java.base/java.lang.Thread.run(Thread.java:829)
[ERROR] 	at org.jboss.threads.JBossThread.run(JBossThread.java:501)

Expected behavior

No failure even with several thousands of classes to register for reflection

Actual behavior

We get a MethodTooLargeException when trying to get a native image.

How to Reproduce?

1 . Checkout the branch 29693/fix-MethodTooLargeException-when-too-many-classes-to-register of https://github.com/essobedo/beefy-scenarios
2. Go to 024-quarkus-extension-with-many-classes-to-register
4. Execute the command mvn clean install -Pnative

The pending PR is quarkus-qe/beefy-scenarios#324

Output of uname -a or ver

Darwin LT-C02CP37VMD6R 22.1.0 Darwin Kernel Version 22.1.0: Sun Oct 9 20:14:54 PDT 2022; root:xnu-8792.41.9~2/RELEASE_X86_64 x86_64

Output of java -version

openjdk version "11.0.17" 2022-10-18 OpenJDK Runtime Environment GraalVM CE 22.3.0 (build 11.0.17+8-jvmci-22.3-b08) OpenJDK 64-Bit Server VM GraalVM CE 22.3.0 (build 11.0.17+8-jvmci-22.3-b08, mixed mode, sharing)

GraalVM version (if different from Java)

No response

Quarkus version or git rev

2.15.0.CR1

Build tool (ie. output of mvnw --version or gradlew --version)

No response

Additional information

No response

[essobedo]

The relevant build steps:

    @BuildStep
    void indexDependencies(BuildProducer<IndexDependencyBuildItem> indexedDependency) {
        // The dependency for MX message types
        indexedDependency.produce(new IndexDependencyBuildItem("com.prowidesoftware", "pw-iso20022"));
        // The dependency for MT message types
        indexedDependency.produce(new IndexDependencyBuildItem("com.prowidesoftware", "pw-swift-core"));
    }

    @BuildStep
    void registerMessageTypesForReflection(BuildProducer<ReflectiveClassBuildItem> reflectiveClass,
            CombinedIndexBuildItem combinedIndex) {

        IndexView index = combinedIndex.getIndex();

        String[] messageTypeClasses = index.getAllKnownSubclasses(DotName.createSimple(AbstractMessage.class.getName()))
                .stream()
                .map(classInfo -> classInfo.name().toString())
                .toArray(String[]::new);

        // Register all message types for refection
        reflectiveClass.produce(new ReflectiveClassBuildItem(false, true, messageTypeClasses));
    }

[essobedo]

Should be fixed by #29694

[quarkus-bot]

/cc @Sanne, @aloubyansky, @gsmet, @radcortez, @stuartwdouglas

[mkouba]

How to Reproduce?

1 . Checkout the branch 4220/camel-swift-extension of https://github.com/apache/camel-quarkus 2. Go to integration-tests/swift 3. Execute the command mvn clean install -Pnative

In fact, you need to build the project first and I get [ERROR] Failed to execute goal org.codehaus.gmaven:groovy-maven-plugin:2.1.1:execute (validate-github-workflows) on project camel-quarkus: Execution validate-github-workflows of goal org.codehaus.gmaven:groovy-maven-plugin:2.1.1:execute failed: Integration tests not executed by the CI: swift when I try to run mvn clean install -DskipTests in the project root first. Is there a way to skip this check?

[essobedo]

@mkouba That's why I would like to write an integration to reproduce, it would be simpler. Any idea where I could add it?

[essobedo]

How to Reproduce?

1 . Checkout the branch 4220/camel-swift-extension of https://github.com/apache/camel-quarkus 2. Go to integration-tests/swift 3. Execute the command mvn clean install -Pnative

In fact, you need to build the project first and I get [ERROR] Failed to execute goal org.codehaus.gmaven:groovy-maven-plugin:2.1.1:execute (validate-github-workflows) on project camel-quarkus: Execution validate-github-workflows of goal org.codehaus.gmaven:groovy-maven-plugin:2.1.1:execute failed: Integration tests not executed by the CI: swift when I try to run mvn clean install -DskipTests in the project root first. Is there a way to skip this check?

Try with mvn clean install -Dquickly from the root directory

[mkouba]

Try with mvn clean install -Dquickly from the root directory

Still requires camel 3.20.0-SNAPSHOT: Could not find artifact org.apache.camel:camel-catalog:jar:3.20.0-SNAPSHOT.

[essobedo]

It won't work anyway my bad because the migration to Camel 3.20 has not been done yet so you will get a build failure. That's why I need to write an IT

[essobedo]

@zakkak Great news, many thanks for your huge effort on this problem 🙏

[zakkak]

You are welcome. Thank you for the great help!