Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Native Image failure building Kafka #8914

Closed
a74nh opened this issue Apr 28, 2020 · 6 comments · Fixed by #12653
Closed

Native Image failure building Kafka #8914

a74nh opened this issue Apr 28, 2020 · 6 comments · Fixed by #12653
Labels
kind/bug Something isn't working
Milestone

Comments

@a74nh
Copy link

a74nh commented Apr 28, 2020

As mentioned here: oracle/graal#2379 (comment)

Building Quarkus using NI, where graal has been built with JDK8, Kaftka fails to build as native image

Machine: Ubuntu 16.04.3 LTS (GNU/Linux 4.4.0-142-generic x86_64)
Quarkus: a527712
Graal: b47aaf4
OpenJDK: openjdk1.8.0_242-jvmci-20.0-b02

$ mvn install -Dnative -Dquarkus.native.additional-build-args=-H:GenerateDebugInfo=1 -rf :quarkus-integration-test-kafka

[quarkus-integration-test-kafka-999-SNAPSHOT-runner:35543]   (features):   1,337.32 ms,  4.20 GB
[quarkus-integration-test-kafka-999-SNAPSHOT-runner:35543]     analysis:  50,361.47 ms,  4.20 GB
Error: Detected a started Thread in the image heap. Threads running in the image generator are no longer running at image run time.  To see how this object got instantiated use -H:+TraceClassInitialization. The object was probably created by a class initializer and is reachable from a static field. You can request class initialization at image run time by using the option --initialize-at-run-time=<class-name>. Or you can write your own initialization methods and call them explicitly from your main entry point.
Detailed message:
Trace: Object was reached by
        reading field org.apache.avro.generic.GenericDatumReader.creator of
                constant org.apache.avro.specific.SpecificDatumReader@299e79d3 reached by
        reading field io.quarkus.it.kafka.avro.Pet.READER$

com.oracle.svm.core.util.UserError$UserException: Detected a started Thread in the image heap. Threads running in the image generator are no longer running at image run time.  To see how this object got instantiated use -H:+TraceClassInitialization. The object was probably created by a class initializer and is reachable from a static field. You can request class initialization at image run time by using the option --initialize-at-run-time=<class-name>. Or you can write your own initialization methods and call them explicitly from your main entry point.
Detailed message:
Trace: Object was reached by
        reading field org.apache.avro.generic.GenericDatumReader.creator of
                constant org.apache.avro.specific.SpecificDatumReader@299e79d3 reached by
        reading field io.quarkus.it.kafka.avro.Pet.READER$

        at com.oracle.svm.core.util.UserError.abort(UserError.java:79)
        at com.oracle.svm.hosted.FallbackFeature.reportAsFallback(FallbackFeature.java:217)
        at com.oracle.svm.hosted.NativeImageGenerator.runPointsToAnalysis(NativeImageGenerator.java:753)
        at com.oracle.svm.hosted.NativeImageGenerator.doRun(NativeImageGenerator.java:538)

Full log files attached.
kafka.log
kafka.out.txt

@rmarting
Copy link

IMHO this issue could be wrong described because it says that Kafka is the cause of the native image failure, when the root cause is use Avro classes, right?

This issue is still running with the following dependencies:

  • Quarkus 1.8.1.Final
  • Avro Schemas generated with org.apache.avro:avro-maven-plugin:1.10.0
  • quarkus-avro extension
  • JDK 11
  • GraalVM 20.2.0-java11 Image

The Native Image failure in this scenario is:

[INFO] [io.quarkus.deployment.pkg.steps.NativeImageBuildStep] Running Quarkus native-image plugin on GraalVM Version 20.2.0 (Java Version 11.0.8)
[INFO] [io.quarkus.deployment.pkg.steps.NativeImageBuildStep] podman run -v /home/rmarting/Workspaces/ws-streams/kafka-clients-quarkus-sample/target/kafka-clients-quarkus-sample-1.0.0-SNAPSHOT-native-image-source-jar:/project:z --env LANG=C --user 1000:1000 --userns=keep-id --rm quay.io/quarkus/ubi-quarkus-native-image:20.2.0-java11 -J-Dsun.nio.ch.maxUpdateArraySize=100 -J-Djava.util.logging.manager=org.jboss.logmanager.LogManager -J-Davro.disable.unsafe=true -J-Dvertx.logger-delegate-factory-class-name=io.quarkus.vertx.core.runtime.VertxLogDelegateFactory -J-Dvertx.disableDnsResolver=true -J-Dio.netty.leakDetection.level=DISABLED -J-Dio.netty.allocator.maxOrder=1 -J-Duser.language=en -J-Dfile.encoding=UTF-8 --initialize-at-build-time= -H:InitialCollectionPolicy=com.oracle.svm.core.genscavenge.CollectionPolicy\$BySpaceAndTime -H:+JNI -jar kafka-clients-quarkus-sample-1.0.0-SNAPSHOT-runner.jar -H:FallbackThreshold=0 -H:+ReportExceptionStackTraces -H:-AddAllCharsets -H:EnableURLProtocols=http --no-server -H:-UseServiceLoaderFeature -H:+StackTrace kafka-clients-quarkus-sample-1.0.0-SNAPSHOT-runner
[kafka-clients-quarkus-sample-1.0.0-SNAPSHOT-runner:20]    classlist:   7,501.59 ms,  1.18 GB
[kafka-clients-quarkus-sample-1.0.0-SNAPSHOT-runner:20]        (cap):     692.55 ms,  1.18 GB
[kafka-clients-quarkus-sample-1.0.0-SNAPSHOT-runner:20]        setup:   2,130.07 ms,  1.18 GB
[kafka-clients-quarkus-sample-1.0.0-SNAPSHOT-runner:20]     analysis:  11,945.89 ms,  1.95 GB
Error: Classes that should be initialized at run time got initialized during image building:
 com.rmarting.kafka.schema.avro.Message the class was requested to be initialized at run time (from feature io.quarkus.runner.AutoFeature.beforeAnalysis). To see why com.rmarting.kafka.schema.avro.Message got initialized use -H:+TraceClassInitialization
org.apache.avro.generic.GenericDatumReader the class was requested to be initialized at run time (from feature io.quarkus.runner.AutoFeature.beforeAnalysis). To see why org.apache.avro.generic.GenericDatumReader got initialized use -H:+TraceClassInitialization

com.oracle.svm.core.util.UserError$UserException: Classes that should be initialized at run time got initialized during image building:
 com.rmarting.kafka.schema.avro.Message the class was requested to be initialized at run time (from feature io.quarkus.runner.AutoFeature.beforeAnalysis). To see why com.rmarting.kafka.schema.avro.Message got initialized use -H:+TraceClassInitialization
org.apache.avro.generic.GenericDatumReader the class was requested to be initialized at run time (from feature io.quarkus.runner.AutoFeature.beforeAnalysis). To see why org.apache.avro.generic.GenericDatumReader got initialized use -H:+TraceClassInitialization

	at com.oracle.svm.core.util.UserError.abort(UserError.java:68)
	at com.oracle.svm.hosted.classinitialization.ConfigurableClassInitialization.checkDelayedInitialization(ConfigurableClassInitialization.java:526)
	at com.oracle.svm.hosted.classinitialization.ClassInitializationFeature.duringAnalysis(ClassInitializationFeature.java:227)
	at com.oracle.svm.hosted.NativeImageGenerator.lambda$runPointsToAnalysis$8(NativeImageGenerator.java:732)
	at com.oracle.svm.hosted.FeatureHandler.forEachFeature(FeatureHandler.java:70)
	at com.oracle.svm.hosted.NativeImageGenerator.runPointsToAnalysis(NativeImageGenerator.java:732)
	at com.oracle.svm.hosted.NativeImageGenerator.doRun(NativeImageGenerator.java:555)
	at com.oracle.svm.hosted.NativeImageGenerator.lambda$run$0(NativeImageGenerator.java:468)
	at java.base/java.util.concurrent.ForkJoinTask$AdaptedRunnableAction.exec(ForkJoinTask.java:1407)
	at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290)
	at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1020)
	at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1656)
	at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1594)
	at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:183)
Error: Image build request failed with exit status 1

Quarkus Avro extension is seems experimental, where I can find information about this extension? I only found this blog

@xabe
Copy link

xabe commented Sep 27, 2020

Hello, I have the same problem, I have a project that has 3 modules:

  • Avro
  • Consumer
  • Producer

When I run the module to produce the integration tests I get the same error as @rmarting, while consumer module works correctly in mode native.

[INFO] [io.quarkus.deployment.pkg.steps.NativeImageBuildStep] /Users/chabir/.sdkman/candidates/java/20.2.0.r11-grl/bin/native-image -J-Djava.util.logging.manager=org.jboss.logmanager.LogManager -J-Dsun.nio.ch.maxUpdateArraySize=100 -J-Davro.disable.unsafe=true -J-Dvertx.logger-delegate-factory-class-name=io.quarkus.vertx.core.runtime.VertxLogDelegateFactory -J-Dvertx.disableDnsResolver=true -J-Dio.netty.leakDetection.level=DISABLED -J-Dio.netty.allocator.maxOrder=1 -J-Duser.language=es -J-Dfile.encoding=UTF-8 -H:-ParseRuntimeOptions -H:+TraceClassInitialization -H:GenerateDebugInfo=1 --initialize-at-build-time= -H:InitialCollectionPolicy=com.oracle.svm.core.genscavenge.CollectionPolicy$BySpaceAndTime -H:+JNI -jar quarkus-producer-1.0.0-SNAPSHOT-runner.jar -H:FallbackThreshold=0 -H:+ReportExceptionStackTraces -H:-AddAllCharsets -H:EnableURLProtocols=http --no-server -H:-UseServiceLoaderFeature -H:+StackTrace quarkus-producer-1.0.0-SNAPSHOT-runner

[quarkus-producer-1.0.0-SNAPSHOT-runner:50981] classlist: 12,652.06 ms, 1.68 GB
[quarkus-producer-1.0.0-SNAPSHOT-runner:50981] (cap): 4,775.82 ms, 1.68 GB
[quarkus-producer-1.0.0-SNAPSHOT-runner:50981] setup: 12,287.91 ms, 2.68 GB
10:11:06,715 INFO [org.hib.val.int.uti.Version] HV000001: Hibernate Validator 6.1.5.Final
[quarkus-producer-1.0.0-SNAPSHOT-runner:50981] analysis: 23,009.33 ms, 4.67 GB
Error: Classes that should be initialized at run time got initialized during image building:
com.xabe.avro.v1.MessageEnvelope the class was requested to be initialized at run time (from feature io.quarkus.runner.AutoFeature.beforeAnalysis). com.xabe.avro.v1.MessageEnvelope has been initialized without the native-image initialization instrumentation and the stack trace can't be tracked. Try avoiding to initialize the class that caused initialization of com.xabe.avro.v1.MessageEnvelope
org.apache.avro.generic.GenericDatumReader the class was requested to be initialized at run time (from feature io.quarkus.runner.AutoFeature.beforeAnalysis). org.apache.avro.generic.GenericDatumReader has been initialized without the native-image initialization instrumentation and the stack trace can't be tracked. Try avoiding to initialize the class that caused initialization of org.apache.avro.generic.GenericDatumReader

com.oracle.svm.core.util.UserError$UserException: Classes that should be initialized at run time got initialized during image building:
com.xabe.avro.v1.MessageEnvelope the class was requested to be initialized at run time (from feature io.quarkus.runner.AutoFeature.beforeAnalysis). com.xabe.avro.v1.MessageEnvelope has been initialized without the native-image initialization instrumentation and the stack trace can't be tracked. Try avoiding to initialize the class that caused initialization of com.xabe.avro.v1.MessageEnvelope
org.apache.avro.generic.GenericDatumReader the class was requested to be initialized at run time (from feature io.quarkus.runner.AutoFeature.beforeAnalysis). org.apache.avro.generic.GenericDatumReader has been initialized without the native-image initialization instrumentation and the stack trace can't be tracked. Try avoiding to initialize the class that caused initialization of org.apache.avro.generic.GenericDatumReader

at com.oracle.svm.core.util.UserError.abort(UserError.java:68)
at com.oracle.svm.hosted.classinitialization.ConfigurableClassInitialization.checkDelayedInitialization(ConfigurableClassInitialization.java:526)
at com.oracle.svm.hosted.classinitialization.ClassInitializationFeature.duringAnalysis(ClassInitializationFeature.java:227)
at com.oracle.svm.hosted.NativeImageGenerator.lambda$runPointsToAnalysis$8(NativeImageGenerator.java:732)
at com.oracle.svm.hosted.FeatureHandler.forEachFeature(FeatureHandler.java:70)
at com.oracle.svm.hosted.NativeImageGenerator.runPointsToAnalysis(NativeImageGenerator.java:732)
at com.oracle.svm.hosted.NativeImageGenerator.doRun(NativeImageGenerator.java:555)
at com.oracle.svm.hosted.NativeImageGenerator.lambda$run$0(NativeImageGenerator.java:468)
at java.base/java.util.concurrent.ForkJoinTask$AdaptedRunnableAction.exec(ForkJoinTask.java:1407)
at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290)
at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1020)
at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1656)
at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1594)
at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:183)

example

@missourian55
Copy link

I am still seeing this error in Quarkus 1.12.0.Final. Any recommendations?

Stacktrace

[kafka-quickstart-1.0-SNAPSHOT-runner:27652]     analysis:  60,748.99 ms,  4.43 GB
Error: Detected a started Thread in the image heap. Threads running in the image generator are no longer running at image runtime.  To see how this object got instantiated use --trace-object-instantiation=com.oracle.svm.hosted.NativeImageGenerator$1. The object was probably created by a class initializer and is reachable from a static field. You can request class initialization at image runtime by using the option --initialize-at-run-time=<class-name>. Or you can write your own initialization methods and call them explicitly from your main entry point.
Detailed message:
Trace: Object was reached by
	reading field org.apache.avro.generic.GenericDatumReader.creator of
		constant org.apache.avro.specific.SpecificDatumReader@39f7a6db reached by
	reading field org.acme.kafka.Movie.READER$

com.oracle.svm.core.util.UserError$UserException: Detected a started Thread in the image heap. Threads running in the image generator are no longer running at image runtime.  To see how this object got instantiated use --trace-object-instantiation=com.oracle.svm.hosted.NativeImageGenerator$1. The object was probably created by a class initializer and is reachable from a static field. You can request class initialization at image runtime by using the option --initialize-at-run-time=<class-name>. Or you can write your own initialization methods and call them explicitly from your main entry point.
Detailed message:
Trace: Object was reached by
	reading field org.apache.avro.generic.GenericDatumReader.creator of
		constant org.apache.avro.specific.SpecificDatumReader@39f7a6db reached by
	reading field org.acme.kafka.Movie.READER$

	at com.oracle.svm.core.util.UserError.abort(UserError.java:82)

Configuration

mp.messaging.outgoing.movies.connector=smallrye-kafka
mp.messaging.outgoing.movies.value.serializer=io.confluent.kafka.serializers.KafkaAvroSerializer
mp.messaging.outgoing.movies.schema.registry.url=${kafka.schema.registry.url}
mp.messaging.outgoing.movies.specific.avro.reader=true

mp.messaging.incoming.movies-from-kafka.connector=smallrye-kafka
mp.messaging.incoming.movies-from-kafka.topic=movies
mp.messaging.incoming.movies-from-kafka.value.deserializer=io.confluent.kafka.serializers.KafkaAvroDeserializer
mp.messaging.incoming.movies-from-kafka.auto.offset.reset=earliest
mp.messaging.incoming.movies-from-kafka.enable.auto.commit=false
mp.messaging.incoming.movies-from-kafka.group.id=movies
mp.messaging.incoming.movies-from-kafka.schema.registry.url=${kafka.schema.registry.url}
mp.messaging.incoming.movies-from-kafka.specific.avro.reader=true

@cescoffier
Copy link
Member

Are you sing the Quarkus Avro extension?

If so, please provide a reproducer attached to a new issue.

@missourian55
Copy link

Are you sing the Quarkus Avro extension?

If so, please provide a reproducer attached to a new issue.

My bad, I was missing that dependency. Thanks for pointing me to it. Curious, how in JVM mode its working without that dependency?

@cescoffier
Copy link
Member

A Quarkus extension is more than a dependency. It collects metadata on your application and compute the right configuration to provide to the native compiler.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

6 participants