[native-image] [Buildtime] Getting a java.lang.NoClassDefFoundError: groovy/lang/GroovyObject when running a native-image #1731

neomatrix369 · 2019-10-09T17:37:35Z

On the back of issue #1711, although we have been able to overcome the reported error with work-around, we now get the below error when building the native-image:

[MLPMnist-1.0.0-bin:36759]    classlist: 122,300.22 ms
[MLPMnist-1.0.0-bin:36759]        (cap):   4,958.27 ms
[MLPMnist-1.0.0-bin:36759]        setup:   7,281.72 ms
[MLPMnist-1.0.0-bin:36759]   (typeflow):  60,394.09 ms
[MLPMnist-1.0.0-bin:36759]    (objects):  34,212.76 ms
[MLPMnist-1.0.0-bin:36759]   (features):  10,437.41 ms
[MLPMnist-1.0.0-bin:36759]     analysis: 106,791.53 ms
[MLPMnist-1.0.0-bin:36759]     (clinit):   3,764.72 ms
[MLPMnist-1.0.0-bin:36759]     universe:  11,561.49 ms
Fatal error: java.lang.NoClassDefFoundError
	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
	at java.util.concurrent.ForkJoinTask.getThrowableException(ForkJoinTask.java:598)
	at java.util.concurrent.ForkJoinTask.get(ForkJoinTask.java:1005)
	at com.oracle.svm.hosted.NativeImageGenerator.run(NativeImageGenerator.java:461)
	at com.oracle.svm.hosted.NativeImageGeneratorRunner.buildImage(NativeImageGeneratorRunner.java:310)
	at com.oracle.svm.hosted.NativeImageGeneratorRunner.build(NativeImageGeneratorRunner.java:448)
	at com.oracle.svm.hosted.server.NativeImageBuildServer.executeCompilation(NativeImageBuildServer.java:397)
	at com.oracle.svm.hosted.server.NativeImageBuildServer.lambda$processCommand$8(NativeImageBuildServer.java:334)
	at com.oracle.svm.hosted.server.NativeImageBuildServer.withJVMContext(NativeImageBuildServer.java:415)
	at com.oracle.svm.hosted.server.NativeImageBuildServer.processCommand(NativeImageBuildServer.java:331)
	at com.oracle.svm.hosted.server.NativeImageBuildServer.processRequest(NativeImageBuildServer.java:275)
	at com.oracle.svm.hosted.server.NativeImageBuildServer.lambda$serve$7(NativeImageBuildServer.java:235)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.NoClassDefFoundError: groovy/lang/GroovyObject
	at java.lang.ClassLoader.defineClass1(Native Method)
	at java.lang.ClassLoader.defineClass(ClassLoader.java:763)
	at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
	at java.net.URLClassLoader.defineClass(URLClassLoader.java:468)
	at java.net.URLClassLoader.access$100(URLClassLoader.java:74)
	at java.net.URLClassLoader$1.run(URLClassLoader.java:369)
	at java.net.URLClassLoader$1.run(URLClassLoader.java:363)
	at java.security.AccessController.doPrivileged(Native Method)
	at java.net.URLClassLoader.findClass(URLClassLoader.java:362)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
	at java.lang.Class.getDeclaredMethods0(Native Method)
	at java.lang.Class.privateGetDeclaredMethods(Class.java:2701)
	at java.lang.Class.getDeclaredMethods(Class.java:1975)
	at com.oracle.svm.jni.access.JNIAccessibleMethod.lambda$finishBeforeCompilation$0(JNIAccessibleMethod.java:134)
	at com.oracle.svm.jni.access.JNIAccessibleMember.findHidingSubclasses(JNIAccessibleMember.java:88)
	at com.oracle.svm.jni.access.JNIAccessibleMember.setHidingSubclasses(JNIAccessibleMember.java:80)
	at com.oracle.svm.jni.access.JNIAccessibleMethod.finishBeforeCompilation(JNIAccessibleMethod.java:134)
	at com.oracle.svm.jni.access.JNIAccessFeature.beforeCompilation(JNIAccessFeature.java:338)
	at com.oracle.svm.hosted.NativeImageGenerator.lambda$doRun$2(NativeImageGenerator.java:575)
	at com.oracle.svm.hosted.FeatureHandler.forEachFeature(FeatureHandler.java:63)
	at com.oracle.svm.hosted.NativeImageGenerator.doRun(NativeImageGenerator.java:575)
	at com.oracle.svm.hosted.NativeImageGenerator.lambda$run$0(NativeImageGenerator.java:444)
	at java.util.concurrent.ForkJoinTask$AdaptedRunnableAction.exec(ForkJoinTask.java:1386)
	at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
	at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)
	at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)
	at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:157)
Caused by: java.lang.ClassNotFoundException: groovy.lang.GroovyObject
	at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
	... 28 more
Error: Image build request failed with exit status 1

I did have a look at #953 and #1306 but it does not appear to be these - are they related?

neomatrix369 · 2019-10-09T17:38:40Z

Jar is available at #1711 (comment) in order to reproduce the issue

neomatrix369 · 2019-10-10T14:26:20Z

I'll correct myself from earlier, this happens while trying to build the native-image, not when running it.

neomatrix369 · 2019-10-13T17:23:44Z

@peter-hofer did the Jar work for you? Do you have any hints for me? Pls do let me know otherwise.

neomatrix369 · 2019-10-19T16:07:12Z

@peter-hofer any update on this?

nestis · 2019-10-20T12:15:25Z

Hi,
I'm facing the same issue just after start using H:JNIConfigurationFile param.

As workaround, I tried to add Groovy as dep in my application's pom.xml, but didnt work. The error went away, but new ones arose. Those errors were solved by adding new deps in my pom, but this ended up in an endless process without a final solution.

Code is on my company private repo, but will try to share if needed. Its just a simple SpringBoot(2.2)+SpringCloudConfig application.

thomaswue · 2019-10-20T13:47:50Z

CC @peter-hofer @cstancu

peter-hofer · 2019-10-21T20:29:07Z

@peter-hofer did the Jar work for you? Do you have any hints for me? Pls do let me know otherwise.

I get this error with the JAR from your download:

[MLPMnist-1.0.0-bin:25236]    classlist:  32,069.66 ms
[MLPMnist-1.0.0-bin:25236]        setup:     176.13 ms
Error: Error parsing JNI configuration in jar:file:/home/ph/github/1731/target/MLPMnist-1.0.0-bin.jar!/META-INF/native-image/org.deeplearning4j.feedforward.mnist/MLPMnist/jni-config.json:
line 25 column 0 Trailing comma is not allowed in JSON
...

The jni-config.json from src doesn't seem to have that trailing comma, so I'm guessing it's a stale build. Rebuilding the JAR doesn't work for lack of a POM however. Also, how did you solve the initializer error from #1711 (comment) , is the solution already incorporated into that source bundle?

Code is on my company private repo, but will try to share if needed. Its just a simple SpringBoot(2.2)+SpringCloudConfig application.

Thanks for commenting @nestis, another example would be very welcome.

neomatrix369 · 2019-10-21T21:14:39Z

#1711 (comment)

Thanks for looking at this @peter-hofer, I'll revisit the jar and update the link and let you know.

I solved the issue in #1711 by passing in some parameters at runtime, which were specific to the app, which isn't GraalVM specific, but I have added that to the scripts I provided (AFAIK) - I'll recheck when I take a look again.

You shouldn't get those errors you mentioned, as I didn't get them during my build.

@nestis thanks for your input.

neomatrix369 · 2019-10-21T21:17:11Z

This is my version of jni-config.json:

[
{
  "name":"org.bytedeco.javacpp.Pointer",
  "allDeclaredFields":true,
  "allPublicFields":true,
  "allDeclaredMethods":true,
  "allPublicMethods":true,
  "allDeclaredConstructors":true,
  "allPublicConstructors":true,
  "methods":[
    {"name":"<init>","parameterTypes":[] }, 
    {"name":"<init>","parameterTypes":["org.bytedeco.javacpp.Pointer"] }
  ]
},
{
  "name":"org.nd4j.nativeblas.NativeOps",
  "allDeclaredFields":true,
  "allPublicFields":true,
  "allDeclaredMethods":true,
  "allPublicMethods":true,
  "allDeclaredConstructors":true,
  "allPublicConstructors":true,
  "methods":[{"name":"<init>","parameterTypes":["org.bytedeco.javacpp.Pointer"] }]
}
]

neomatrix369 · 2019-10-21T21:35:26Z

I tried to build the native-image using the same jar I provided earlier and got the same error:

Build on Server(pid: 8615, port: 63593)*
[MLPMnist-1.0.0-bin:8615]    classlist:  46,093.79 ms
[MLPMnist-1.0.0-bin:8615]        (cap):  11,330.59 ms
[MLPMnist-1.0.0-bin:8615]        setup:  15,278.62 ms
[MLPMnist-1.0.0-bin:8615]   (typeflow):  43,014.60 ms
[MLPMnist-1.0.0-bin:8615]    (objects):  31,322.72 ms
[MLPMnist-1.0.0-bin:8615]   (features):  11,874.58 ms
[MLPMnist-1.0.0-bin:8615]     analysis:  88,666.84 ms
Printing call tree to /Users/swami/git-repos/hands-on-workshops/deeplearning/valohai/MLPMnist-dl4j-example-graalvm/native-image/reports/call_tree_MLPMnist-1.0.0-bin_20191021_223059.txt
Printing list of used classes to /Users/swami/git-repos/hands-on-workshops/deeplearning/valohai/MLPMnist-dl4j-example-graalvm/native-image/reports/used_classes_MLPMnist-1.0.0-bin_20191021_223106.txt
Printing list of used packages to /Users/swami/git-repos/hands-on-workshops/deeplearning/valohai/MLPMnist-dl4j-example-graalvm/native-image/reports/used_packages_MLPMnist-1.0.0-bin_20191021_223106.txt
[MLPMnist-1.0.0-bin:8615]     (clinit):   1,364.88 ms
[MLPMnist-1.0.0-bin:8615]     universe:   4,499.57 ms
Fatal error: java.lang.NoClassDefFoundError
	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
	at java.util.concurrent.ForkJoinTask.getThrowableException(ForkJoinTask.java:598)
	at java.util.concurrent.ForkJoinTask.get(ForkJoinTask.java:1005)
	at com.oracle.svm.hosted.NativeImageGenerator.run(NativeImageGenerator.java:461)
	at com.oracle.svm.hosted.NativeImageGeneratorRunner.buildImage(NativeImageGeneratorRunner.java:310)
	at com.oracle.svm.hosted.NativeImageGeneratorRunner.build(NativeImageGeneratorRunner.java:448)
	at com.oracle.svm.hosted.server.NativeImageBuildServer.executeCompilation(NativeImageBuildServer.java:397)
	at com.oracle.svm.hosted.server.NativeImageBuildServer.lambda$processCommand$8(NativeImageBuildServer.java:334)
	at com.oracle.svm.hosted.server.NativeImageBuildServer.withJVMContext(NativeImageBuildServer.java:415)
	at com.oracle.svm.hosted.server.NativeImageBuildServer.processCommand(NativeImageBuildServer.java:331)
	at com.oracle.svm.hosted.server.NativeImageBuildServer.processRequest(NativeImageBuildServer.java:275)
	at com.oracle.svm.hosted.server.NativeImageBuildServer.lambda$serve$7(NativeImageBuildServer.java:235)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.NoClassDefFoundError: groovy/lang/GroovyObject
	at java.lang.ClassLoader.defineClass1(Native Method)
	at java.lang.ClassLoader.defineClass(ClassLoader.java:756)
	at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
	at java.net.URLClassLoader.defineClass(URLClassLoader.java:468)
	at java.net.URLClassLoader.access$100(URLClassLoader.java:74)
	at java.net.URLClassLoader$1.run(URLClassLoader.java:369)
	at java.net.URLClassLoader$1.run(URLClassLoader.java:363)
	at java.security.AccessController.doPrivileged(Native Method)
	at java.net.URLClassLoader.findClass(URLClassLoader.java:362)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
	at java.lang.Class.getDeclaredMethods0(Native Method)
	at java.lang.Class.privateGetDeclaredMethods(Class.java:2701)
	at java.lang.Class.getDeclaredMethods(Class.java:1975)
	at com.oracle.svm.jni.access.JNIAccessibleMethod.lambda$finishBeforeCompilation$0(JNIAccessibleMethod.java:134)
	at com.oracle.svm.jni.access.JNIAccessibleMember.findHidingSubclasses(JNIAccessibleMember.java:88)
	at com.oracle.svm.jni.access.JNIAccessibleMember.setHidingSubclasses(JNIAccessibleMember.java:80)
	at com.oracle.svm.jni.access.JNIAccessibleMethod.finishBeforeCompilation(JNIAccessibleMethod.java:134)
	at com.oracle.svm.jni.access.JNIAccessFeature.beforeCompilation(JNIAccessFeature.java:338)
	at com.oracle.svm.hosted.NativeImageGenerator.lambda$doRun$2(NativeImageGenerator.java:575)
	at com.oracle.svm.hosted.FeatureHandler.forEachFeature(FeatureHandler.java:63)
	at com.oracle.svm.hosted.NativeImageGenerator.doRun(NativeImageGenerator.java:575)
	at com.oracle.svm.hosted.NativeImageGenerator.lambda$run$0(NativeImageGenerator.java:444)
	at java.util.concurrent.ForkJoinTask$AdaptedRunnableAction.exec(ForkJoinTask.java:1386)
	at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
	at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)
	at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)
	at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:157)
Caused by: java.lang.ClassNotFoundException: groovy.lang.GroovyObject
	at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
	... 28 more
Error: Image build request failed with exit status 1

I'll still upload a new zip with everything you need in it. Instructions to use are the same as in the #1711 (comment) comment. Avoid using native-image-builder.sh --fullBuild as you need the pom for it.

neomatrix369 · 2019-10-21T21:54:55Z

I have recompiled it with GraalVM 19.2.1 now and uploaded it to https://www.dropbox.com/s/9tng5gzix7whof5/native-image-gitIssue1711.zip?dl=0, please check and let me know if you have any issues reproducing it.

Please do use the native-image-builder.sh script provided as it has all the params needed to get it to the point where it fails for me.

neomatrix369 · 2019-10-21T21:59:15Z

I have one more question about the below:

Error: Error parsing JNI configuration in jar:file:/home/ph/github/1731/target/MLPMnist-1.0.0-bin.jar!/META-INF/native-image/org.deeplearning4j.feedforward.mnist/MLPMnist/jni-config.json:

But only applies if you are using my native-image-builder.sh script provided,
the above shows that the native-image compiler is using the config files embedded inside the jar even though we are passing them via the CLI:

<---snipped--->
    OPTIONS="${OPTIONS} -H:ReflectionConfigurationFiles=${NATIVE_IMAGE_META_INF_FOLDER}/reflect-config.json"
    OPTIONS="${OPTIONS} -H:DynamicProxyConfigurationFiles=${NATIVE_IMAGE_META_INF_FOLDER}/proxy-config.json"
    OPTIONS="${OPTIONS} -H:ResourceConfigurationFiles=${NATIVE_IMAGE_META_INF_FOLDER}/resource-config.json"
    OPTIONS="${OPTIONS} -H:JNIConfigurationFiles=${NATIVE_IMAGE_META_INF_FOLDER}/jni-config.json"
    OPTIONS="${OPTIONS} -H:IncludeResources=${NATIVE_IMAGE_META_INF_FOLDER}/services/*"
<---snipped--->

I could be wrong but would be good to know if this is expected behaviour?

I'm doing the below:

createNativeImageMetaInfFolder() {
    CURRENT_DIR=$(pwd)
    ## Current folder:	NATIVE_IMAGE_META_INF_FOLDER="${CURRENT_DIR}/META-INF/native-image/org.deeplearning4j.feedforward.mnist/MLPMnist/"
    NATIVE_IMAGE_META_INF_FOLDER="${CURRENT_DIR}/../src/main/resources/META-INF/native-image/org.deeplearning4j.feedforward.mnist/MLPMnist/"
    mkdir -p ${NATIVE_IMAGE_META_INF_FOLDER}
}

to assign NATIVE_IMAGE_META_INF_FOLDER the location of the config files, i'm supposing this is the right way to do it and the native-image compiler is picking it up, in case it does not , would be great to get a warning when running it, that your supplied paths were incorrect falling back to <---- whatever method it chooses --->.

peter-hofer · 2019-10-22T08:06:00Z

Thanks for providing the new JAR. I'm using a current build of GraalVM and your configuration scripts. I'm now getting this, which seems to be caused by a potential reflective access to org.bytedeco.javacpp.Pointer.deallocatorThread that makes this field reachable, which already contains a started thread during the image build. Could this still be an issue with the javacpp JAR? It might also be triggered by a change between 19.2.1 and now.

Build on Server(pid: 9984, port: 45125)*
[MLPMnist-1.0.0-bin:9984]    classlist:  34,410.83 ms
[MLPMnist-1.0.0-bin:9984]        (cap):   1,010.77 ms
[MLPMnist-1.0.0-bin:9984]        setup:   2,826.97 ms
java[MLPMnist-1.0.0-bin:9984]     analysis:  25,285.28 ms
Printing call tree to /home/ph/github/1731/native-image/reports/call_tree_MLPMnist-1.0.0-bin_20191022_092515.txt
Printing list of used methods to /home/ph/github/1731/native-image/reports/used_methods_MLPMnist-1.0.0-bin_20191022_092517.txt
Printing list of used classes to /home/ph/github/1731/native-image/reports/used_classes_MLPMnist-1.0.0-bin_20191022_092517.txt
Printing list of used packages to /home/ph/github/1731/native-image/reports/used_packages_MLPMnist-1.0.0-bin_20191022_092517.txt
Fatal error: com.oracle.graal.pointsto.util.AnalysisError$ParsingError: Error encountered while parsing com.oracle.svm.reflect.Pointer_deallocatorThread_330a2bfd42a35375c44762314d9730587206b5b5.get(java.lang.Object)
Parsing context:
        parsing java.lang.reflect.Field.get(Field.java:418)
        parsing org.deeplearning4j.nn.conf.layers.Layer.clone(Layer.java:128)
        parsing org.deeplearning4j.nn.conf.layers.BaseLayer.clone(BaseLayer.java:98)
        parsing org.deeplearning4j.nn.conf.layers.BaseLayer.clone(BaseLayer.java:43)
        parsing org.deeplearning4j.nn.conf.NeuralNetConfiguration$Builder.clone(NeuralNetConfiguration.java:690)
        parsing org.deeplearning4j.nn.conf.NeuralNetConfiguration$ListBuilder.layer(NeuralNetConfiguration.java:178)
        parsing org.deeplearning4j.nn.conf.NeuralNetConfiguration$ListBuilder.layer(NeuralNetConfiguration.java:189)
        parsing org.deeplearning4j.feedforward.mnist.MLPMnistSingleLayerTrain.execute(MLPMnistSingleLayerTrain.java:69)
        parsing org.deeplearning4j.feedforward.mnist.MLPMnistSingleLayerRunner.execute(MLPMnistSingleLayerRunner.java:101)
        parsing org.deeplearning4j.feedforward.mnist.MLPMnistSingleLayerRunner.main(MLPMnistSingleLayerRunner.java:72)
        parsing com.oracle.svm.core.JavaMainWrapper.runCore(JavaMainWrapper.java:151)
        parsing com.oracle.svm.core.JavaMainWrapper.run(JavaMainWrapper.java:186)
        parsing com.oracle.svm.core.code.IsolateEnterStub.JavaMainWrapper_run_5087f5482cc9a6abc971913ece43acb471d2631b(generated:0)

        at com.oracle.graal.pointsto.util.AnalysisError.parsingError(AnalysisError.java:138)
        at com.oracle.graal.pointsto.flow.MethodTypeFlow.doParse(MethodTypeFlow.java:327)
        at com.oracle.graal.pointsto.flow.MethodTypeFlow.ensureParsed(MethodTypeFlow.java:300)
        at com.oracle.graal.pointsto.flow.MethodTypeFlow.addContext(MethodTypeFlow.java:107)
        at com.oracle.graal.pointsto.DefaultAnalysisPolicy$DefaultVirtualInvokeTypeFlow.onObservedUpdate(DefaultAnalysisPolicy.java:191)
        at com.oracle.graal.pointsto.flow.TypeFlow.notifyObservers(TypeFlow.java:343)
        at com.oracle.graal.pointsto.flow.TypeFlow.update(TypeFlow.java:385)
        at com.oracle.graal.pointsto.BigBang$2.run(BigBang.java:511)
        at com.oracle.graal.pointsto.util.CompletionExecutor.lambda$execute$0(CompletionExecutor.java:171)
        at java.base/java.util.concurrent.ForkJoinTask$RunnableExecuteAction.exec(ForkJoinTask.java:1426)
        at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290)
        at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1020)
        at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1656)
        at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1594)
        at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:177)
Caused by: org.graalvm.compiler.graph.GraalGraphError: com.oracle.graal.pointsto.constraints.UnsupportedFeatureException: Detected a started Thread in the image heap. Threads running in the image generator are no longer running at image run time.  To see how this object got instantiated use -H:+TraceClassInitialization. The object was probably created by a class initializer and is reachable from a static field. You can request class initialization at image run time by using the option --initialize-at-build-time=<class-name>. Or you can write your own initialization methods and call them explicitly from your main entry point.
        at node: 4|LoadField#deallocatorThread
        at jdk.internal.vm.compiler/org.graalvm.compiler.phases.common.CanonicalizerPhase$Instance.tryCanonicalize(CanonicalizerPhase.java:397)
        at jdk.internal.vm.compiler/org.graalvm.compiler.phases.common.CanonicalizerPhase$Instance.processNode(CanonicalizerPhase.java:325)
        at jdk.internal.vm.compiler/org.graalvm.compiler.phases.common.CanonicalizerPhase$Instance.processWorkSet(CanonicalizerPhase.java:302)
        at jdk.internal.vm.compiler/org.graalvm.compiler.phases.common.CanonicalizerPhase$Instance.run(CanonicalizerPhase.java:264)
        at jdk.internal.vm.compiler/org.graalvm.compiler.phases.common.CanonicalizerPhase.run(CanonicalizerPhase.java:177)
        at jdk.internal.vm.compiler/org.graalvm.compiler.phases.common.CanonicalizerPhase.run(CanonicalizerPhase.java:73)
        at jdk.internal.vm.compiler/org.graalvm.compiler.phases.BasePhase.apply(BasePhase.java:197)
        at jdk.internal.vm.compiler/org.graalvm.compiler.phases.BasePhase.apply(BasePhase.java:139)
        at com.oracle.graal.pointsto.flow.MethodTypeFlowBuilder.parse(MethodTypeFlowBuilder.java:231)
        at com.oracle.graal.pointsto.flow.MethodTypeFlowBuilder.apply(MethodTypeFlowBuilder.java:340)
        at com.oracle.graal.pointsto.flow.MethodTypeFlow.doParse(MethodTypeFlow.java:310)
        ... 13 more
Caused by: com.oracle.graal.pointsto.constraints.UnsupportedFeatureException: Detected a started Thread in the image heap. Threads running in the image generator are no longer running at image run time.  To see how this object got instantiated use -H:+TraceClassInitialization. The object was probably created by a class initializer and is reachable from a static field. You can request class initialization at image run time by using the option --initialize-at-build-time=<class-name>. Or you can write your own initialization methods and call them explicitly from your main entry point.
        at com.oracle.svm.hosted.image.DisallowedImageHeapObjectFeature.error(DisallowedImageHeapObjectFeature.java:129)
        at com.oracle.svm.hosted.image.DisallowedImageHeapObjectFeature.replacer(DisallowedImageHeapObjectFeature.java:75)
        at com.oracle.graal.pointsto.meta.AnalysisUniverse.replaceObject(AnalysisUniverse.java:534)
        at com.oracle.svm.hosted.ameta.AnalysisConstantReflectionProvider.replaceObject(AnalysisConstantReflectionProvider.java:201)
        at com.oracle.svm.hosted.ameta.AnalysisConstantReflectionProvider.interceptValue(AnalysisConstantReflectionProvider.java:172)
        at com.oracle.svm.hosted.ameta.AnalysisConstantReflectionProvider.readValue(AnalysisConstantReflectionProvider.java:101)
        at com.oracle.svm.hosted.ameta.AnalysisConstantReflectionProvider.readFieldValue(AnalysisConstantReflectionProvider.java:77)
        at jdk.internal.vm.compiler/org.graalvm.compiler.nodes.util.ConstantFoldUtil$1.readValue(ConstantFoldUtil.java:51)
        at jdk.internal.vm.compiler/org.graalvm.compiler.core.common.spi.JavaConstantFieldProvider.readConstantField(JavaConstantFieldProvider.java:84)
        at com.oracle.svm.hosted.ameta.AnalysisConstantFieldProvider.readConstantField(AnalysisConstantFieldProvider.java:72)
        at jdk.internal.vm.compiler/org.graalvm.compiler.nodes.util.ConstantFoldUtil.tryConstantFold(ConstantFoldUtil.java:47)
        at jdk.internal.vm.compiler/org.graalvm.compiler.nodes.java.LoadFieldNode.asConstant(LoadFieldNode.java:164)
        at jdk.internal.vm.compiler/org.graalvm.compiler.nodes.java.LoadFieldNode.canonical(LoadFieldNode.java:132)
        at jdk.internal.vm.compiler/org.graalvm.compiler.nodes.java.LoadFieldNode.canonical(LoadFieldNode.java:123)
        at jdk.internal.vm.compiler/org.graalvm.compiler.nodes.java.LoadFieldNode.canonical(LoadFieldNode.java:68)
        at jdk.internal.vm.compiler/org.graalvm.compiler.graph.spi.Canonicalizable$Unary.canonical(Canonicalizable.java:99)
        at jdk.internal.vm.compiler/org.graalvm.compiler.phases.common.CanonicalizerPhase$Instance.tryCanonicalize(CanonicalizerPhase.java:391)
        ... 23 more
Error: Image build request failed with exit status 1

... the above shows that the native-image compiler is using the config files embedded inside the jar even though we are passing them via the CLI:

That is to be expected: our design explicitly takes all the configuration files that it can find as input, rather than having them override each other in some way. The idea is that you can use a Java library in your project that brings its own native-image configuration files in its JAR instead of you having to provide those configuration files yourself.

peter-hofer · 2019-10-22T10:32:16Z

The build script doesn't seem to have picked up the javacpp.jar from my GraalVM build, so after changing that, the above problems are gone. What I get now is:

Build on Server(pid: 18343, port: 44839)*
[MLPMnist-1.0.0-bin:18343]    classlist:  33,344.15 ms
[MLPMnist-1.0.0-bin:18343]        (cap):     781.64 ms
[MLPMnist-1.0.0-bin:18343]        setup:   2,645.82 ms
[MLPMnist-1.0.0-bin:18343]   (typeflow):  21,700.13 ms
[MLPMnist-1.0.0-bin:18343]    (objects):  23,980.31 ms
[MLPMnist-1.0.0-bin:18343]   (features):   2,984.23 ms
[MLPMnist-1.0.0-bin:18343]     analysis:  50,211.21 ms
Printing call tree to /home/ph/github/1731/native-image/reports/call_tree_MLPMnist-1.0.0-bin_20191022_114733.txt
Printing list of used methods to /home/ph/github/1731/native-image/reports/used_methods_MLPMnist-1.0.0-bin_20191022_114736.txt
Printing list of used classes to /home/ph/github/1731/native-image/reports/used_classes_MLPMnist-1.0.0-bin_20191022_114737.txt
Printing list of used packages to /home/ph/github/1731/native-image/reports/used_packages_MLPMnist-1.0.0-bin_20191022_114737.txt
[MLPMnist-1.0.0-bin:18343]     (clinit):     622.13 ms
[MLPMnist-1.0.0-bin:18343]     universe:   1,582.10 ms
Error: Incompatible change of initialization policy for java.io.RandomAccessFile: trying to change RERUN for JDK native code support via JNI to BUILD_TIME super type of org.deeplearning4j.datasets.mnist.MnistDbFile
com.oracle.svm.core.util.UserError$UserException: Incompatible change of initialization policy for java.io.RandomAccessFile: trying to change RERUN for JDK native code support via JNI to BUILD_TIME super type of org.deeplearning4j.datasets.mnist.MnistDbFile
        at com.oracle.svm.core.util.UserError.abort(UserError.java:65)
        at com.oracle.svm.hosted.classinitialization.ClassInitializationConfiguration.insertRec(ClassInitializationConfiguration.java:96)
        at com.oracle.svm.hosted.classinitialization.ClassInitializationConfiguration.insertRec(ClassInitializationConfiguration.java:107)
        at com.oracle.svm.hosted.classinitialization.ClassInitializationConfiguration.insertRec(ClassInitializationConfiguration.java:107)
        at com.oracle.svm.hosted.classinitialization.ClassInitializationConfiguration.insertRec(ClassInitializationConfiguration.java:107)
        at com.oracle.svm.hosted.classinitialization.ClassInitializationConfiguration.insert(ClassInitializationConfiguration.java:63)
        at com.oracle.svm.hosted.classinitialization.ConfigurableClassInitialization.forceInitializeHosted(ConfigurableClassInitialization.java:456)
        at com.oracle.svm.hosted.classinitialization.ConfigurableClassInitialization.forceInitializeHosted(ConfigurableClassInitialization.java:460)
        at com.oracle.svm.hosted.classinitialization.ClassInitializationFeature.lambda$initializeSafeDelayedClasses$11(ClassInitializationFeature.java:285)
        at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183)
        at java.base/java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:177)
        at java.base/java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:177)
        at java.base/java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:177)
        at java.base/java.util.HashMap$KeySpliterator.forEachRemaining(HashMap.java:1603)
        at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484)
        at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474)
        at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150)
        at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173)
        at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
        at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:497)
        at com.oracle.svm.hosted.classinitialization.ClassInitializationFeature.initializeSafeDelayedClasses(ClassInitializationFeature.java:282)
        at com.oracle.svm.hosted.classinitialization.ClassInitializationFeature.beforeCompilation(ClassInitializationFeature.java:222)
        at com.oracle.svm.hosted.NativeImageGenerator.lambda$doRun$2(NativeImageGenerator.java:571)
        at com.oracle.svm.hosted.FeatureHandler.forEachFeature(FeatureHandler.java:63)
        at com.oracle.svm.hosted.NativeImageGenerator.doRun(NativeImageGenerator.java:571)
        at com.oracle.svm.hosted.NativeImageGenerator.lambda$run$0(NativeImageGenerator.java:446)
        at java.base/java.util.concurrent.ForkJoinTask$AdaptedRunnableAction.exec(ForkJoinTask.java:1407)
        at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290)
        at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1020)
        at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1656)
        at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1594)
        at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:177)
Error: Image build request failed with exit status 1

This is most likely caused by an instance of MnistDbFile making its way into the image heap, typically via a static field (directly or indirectly). Because that class derives from RandomAccessFile which might be open during the image build, this is state that can't be serialized into an image. Can you look into that?

neomatrix369 · 2019-10-24T13:29:07Z

Thanks for providing the new JAR. I'm using a current build of GraalVM and your configuration scripts. I'm now getting this, which seems to be caused by a potential reflective access to org.bytedeco.javacpp.Pointer.deallocatorThread that makes this field reachable, which already contains a started thread during the image build. Could this still be an issue with the javacpp JAR? It might also be triggered by a change between 19.2.1 and now.

Build on Server(pid: 9984, port: 45125)*
[MLPMnist-1.0.0-bin:9984]    classlist:  34,410.83 ms
[MLPMnist-1.0.0-bin:9984]        (cap):   1,010.77 ms
[MLPMnist-1.0.0-bin:9984]        setup:   2,826.97 ms
java[MLPMnist-1.0.0-bin:9984]     analysis:  25,285.28 ms
Printing call tree to /home/ph/github/1731/native-image/reports/call_tree_MLPMnist-1.0.0-bin_20191022_092515.txt
Printing list of used methods to /home/ph/github/1731/native-image/reports/used_methods_MLPMnist-1.0.0-bin_20191022_092517.txt
Printing list of used classes to /home/ph/github/1731/native-image/reports/used_classes_MLPMnist-1.0.0-bin_20191022_092517.txt
Printing list of used packages to /home/ph/github/1731/native-image/reports/used_packages_MLPMnist-1.0.0-bin_20191022_092517.txt
Fatal error: com.oracle.graal.pointsto.util.AnalysisError$ParsingError: Error encountered while parsing com.oracle.svm.reflect.Pointer_deallocatorThread_330a2bfd42a35375c44762314d9730587206b5b5.get(java.lang.Object)
Parsing context:
        parsing java.lang.reflect.Field.get(Field.java:418)
        parsing org.deeplearning4j.nn.conf.layers.Layer.clone(Layer.java:128)
        parsing org.deeplearning4j.nn.conf.layers.BaseLayer.clone(BaseLayer.java:98)
        parsing org.deeplearning4j.nn.conf.layers.BaseLayer.clone(BaseLayer.java:43)
        parsing org.deeplearning4j.nn.conf.NeuralNetConfiguration$Builder.clone(NeuralNetConfiguration.java:690)
        parsing org.deeplearning4j.nn.conf.NeuralNetConfiguration$ListBuilder.layer(NeuralNetConfiguration.java:178)
        parsing org.deeplearning4j.nn.conf.NeuralNetConfiguration$ListBuilder.layer(NeuralNetConfiguration.java:189)
        parsing org.deeplearning4j.feedforward.mnist.MLPMnistSingleLayerTrain.execute(MLPMnistSingleLayerTrain.java:69)
        parsing org.deeplearning4j.feedforward.mnist.MLPMnistSingleLayerRunner.execute(MLPMnistSingleLayerRunner.java:101)
        parsing org.deeplearning4j.feedforward.mnist.MLPMnistSingleLayerRunner.main(MLPMnistSingleLayerRunner.java:72)
        parsing com.oracle.svm.core.JavaMainWrapper.runCore(JavaMainWrapper.java:151)
        parsing com.oracle.svm.core.JavaMainWrapper.run(JavaMainWrapper.java:186)
        parsing com.oracle.svm.core.code.IsolateEnterStub.JavaMainWrapper_run_5087f5482cc9a6abc971913ece43acb471d2631b(generated:0)

        at com.oracle.graal.pointsto.util.AnalysisError.parsingError(AnalysisError.java:138)
        at com.oracle.graal.pointsto.flow.MethodTypeFlow.doParse(MethodTypeFlow.java:327)
        at com.oracle.graal.pointsto.flow.MethodTypeFlow.ensureParsed(MethodTypeFlow.java:300)
        at com.oracle.graal.pointsto.flow.MethodTypeFlow.addContext(MethodTypeFlow.java:107)
        at com.oracle.graal.pointsto.DefaultAnalysisPolicy$DefaultVirtualInvokeTypeFlow.onObservedUpdate(DefaultAnalysisPolicy.java:191)
        at com.oracle.graal.pointsto.flow.TypeFlow.notifyObservers(TypeFlow.java:343)
        at com.oracle.graal.pointsto.flow.TypeFlow.update(TypeFlow.java:385)
        at com.oracle.graal.pointsto.BigBang$2.run(BigBang.java:511)
        at com.oracle.graal.pointsto.util.CompletionExecutor.lambda$execute$0(CompletionExecutor.java:171)
        at java.base/java.util.concurrent.ForkJoinTask$RunnableExecuteAction.exec(ForkJoinTask.java:1426)
        at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290)
        at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1020)
        at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1656)
        at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1594)
        at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:177)
Caused by: org.graalvm.compiler.graph.GraalGraphError: com.oracle.graal.pointsto.constraints.UnsupportedFeatureException: Detected a started Thread in the image heap. Threads running in the image generator are no longer running at image run time.  To see how this object got instantiated use -H:+TraceClassInitialization. The object was probably created by a class initializer and is reachable from a static field. You can request class initialization at image run time by using the option --initialize-at-build-time=<class-name>. Or you can write your own initialization methods and call them explicitly from your main entry point.
        at node: 4|LoadField#deallocatorThread
        at jdk.internal.vm.compiler/org.graalvm.compiler.phases.common.CanonicalizerPhase$Instance.tryCanonicalize(CanonicalizerPhase.java:397)
        at jdk.internal.vm.compiler/org.graalvm.compiler.phases.common.CanonicalizerPhase$Instance.processNode(CanonicalizerPhase.java:325)
        at jdk.internal.vm.compiler/org.graalvm.compiler.phases.common.CanonicalizerPhase$Instance.processWorkSet(CanonicalizerPhase.java:302)
        at jdk.internal.vm.compiler/org.graalvm.compiler.phases.common.CanonicalizerPhase$Instance.run(CanonicalizerPhase.java:264)
        at jdk.internal.vm.compiler/org.graalvm.compiler.phases.common.CanonicalizerPhase.run(CanonicalizerPhase.java:177)
        at jdk.internal.vm.compiler/org.graalvm.compiler.phases.common.CanonicalizerPhase.run(CanonicalizerPhase.java:73)
        at jdk.internal.vm.compiler/org.graalvm.compiler.phases.BasePhase.apply(BasePhase.java:197)
        at jdk.internal.vm.compiler/org.graalvm.compiler.phases.BasePhase.apply(BasePhase.java:139)
        at com.oracle.graal.pointsto.flow.MethodTypeFlowBuilder.parse(MethodTypeFlowBuilder.java:231)
        at com.oracle.graal.pointsto.flow.MethodTypeFlowBuilder.apply(MethodTypeFlowBuilder.java:340)
        at com.oracle.graal.pointsto.flow.MethodTypeFlow.doParse(MethodTypeFlow.java:310)
        ... 13 more
Caused by: com.oracle.graal.pointsto.constraints.UnsupportedFeatureException: Detected a started Thread in the image heap. Threads running in the image generator are no longer running at image run time.  To see how this object got instantiated use -H:+TraceClassInitialization. The object was probably created by a class initializer and is reachable from a static field. You can request class initialization at image run time by using the option --initialize-at-build-time=<class-name>. Or you can write your own initialization methods and call them explicitly from your main entry point.
        at com.oracle.svm.hosted.image.DisallowedImageHeapObjectFeature.error(DisallowedImageHeapObjectFeature.java:129)
        at com.oracle.svm.hosted.image.DisallowedImageHeapObjectFeature.replacer(DisallowedImageHeapObjectFeature.java:75)
        at com.oracle.graal.pointsto.meta.AnalysisUniverse.replaceObject(AnalysisUniverse.java:534)
        at com.oracle.svm.hosted.ameta.AnalysisConstantReflectionProvider.replaceObject(AnalysisConstantReflectionProvider.java:201)
        at com.oracle.svm.hosted.ameta.AnalysisConstantReflectionProvider.interceptValue(AnalysisConstantReflectionProvider.java:172)
        at com.oracle.svm.hosted.ameta.AnalysisConstantReflectionProvider.readValue(AnalysisConstantReflectionProvider.java:101)
        at com.oracle.svm.hosted.ameta.AnalysisConstantReflectionProvider.readFieldValue(AnalysisConstantReflectionProvider.java:77)
        at jdk.internal.vm.compiler/org.graalvm.compiler.nodes.util.ConstantFoldUtil$1.readValue(ConstantFoldUtil.java:51)
        at jdk.internal.vm.compiler/org.graalvm.compiler.core.common.spi.JavaConstantFieldProvider.readConstantField(JavaConstantFieldProvider.java:84)
        at com.oracle.svm.hosted.ameta.AnalysisConstantFieldProvider.readConstantField(AnalysisConstantFieldProvider.java:72)
        at jdk.internal.vm.compiler/org.graalvm.compiler.nodes.util.ConstantFoldUtil.tryConstantFold(ConstantFoldUtil.java:47)
        at jdk.internal.vm.compiler/org.graalvm.compiler.nodes.java.LoadFieldNode.asConstant(LoadFieldNode.java:164)
        at jdk.internal.vm.compiler/org.graalvm.compiler.nodes.java.LoadFieldNode.canonical(LoadFieldNode.java:132)
        at jdk.internal.vm.compiler/org.graalvm.compiler.nodes.java.LoadFieldNode.canonical(LoadFieldNode.java:123)
        at jdk.internal.vm.compiler/org.graalvm.compiler.nodes.java.LoadFieldNode.canonical(LoadFieldNode.java:68)
        at jdk.internal.vm.compiler/org.graalvm.compiler.graph.spi.Canonicalizable$Unary.canonical(Canonicalizable.java:99)
        at jdk.internal.vm.compiler/org.graalvm.compiler.phases.common.CanonicalizerPhase$Instance.tryCanonicalize(CanonicalizerPhase.java:391)
        ... 23 more
Error: Image build request failed with exit status 1

... the above shows that the native-image compiler is using the config files embedded inside the jar even though we are passing them via the CLI:

That is to be expected: our design explicitly takes all the configuration files that it can find as input, rather than having them override each other in some way. The idea is that you can use a Java library in your project that brings its own native-image configuration files in its JAR instead of you having to provide those configuration files yourself.

So the javacpp.jar library has been omitted with intention as it was clashing with the one with GraalVM (both my jar and GraalVM collided with this) - this issue might have been fixed by now by at that time I raised #1711 and was advised in the conversation #1711 (comment) to drop the library from GraalVM SDK when building the native-image.

neomatrix369 · 2019-10-24T13:33:38Z

The build script doesn't seem to have picked up the javacpp.jar from my GraalVM build, so after changing that, the above problems are gone. What I get now is:

Build on Server(pid: 18343, port: 44839)*
[MLPMnist-1.0.0-bin:18343]    classlist:  33,344.15 ms
[MLPMnist-1.0.0-bin:18343]        (cap):     781.64 ms
[MLPMnist-1.0.0-bin:18343]        setup:   2,645.82 ms
[MLPMnist-1.0.0-bin:18343]   (typeflow):  21,700.13 ms
[MLPMnist-1.0.0-bin:18343]    (objects):  23,980.31 ms
[MLPMnist-1.0.0-bin:18343]   (features):   2,984.23 ms
[MLPMnist-1.0.0-bin:18343]     analysis:  50,211.21 ms
Printing call tree to /home/ph/github/1731/native-image/reports/call_tree_MLPMnist-1.0.0-bin_20191022_114733.txt
Printing list of used methods to /home/ph/github/1731/native-image/reports/used_methods_MLPMnist-1.0.0-bin_20191022_114736.txt
Printing list of used classes to /home/ph/github/1731/native-image/reports/used_classes_MLPMnist-1.0.0-bin_20191022_114737.txt
Printing list of used packages to /home/ph/github/1731/native-image/reports/used_packages_MLPMnist-1.0.0-bin_20191022_114737.txt
[MLPMnist-1.0.0-bin:18343]     (clinit):     622.13 ms
[MLPMnist-1.0.0-bin:18343]     universe:   1,582.10 ms
Error: Incompatible change of initialization policy for java.io.RandomAccessFile: trying to change RERUN for JDK native code support via JNI to BUILD_TIME super type of org.deeplearning4j.datasets.mnist.MnistDbFile
com.oracle.svm.core.util.UserError$UserException: Incompatible change of initialization policy for java.io.RandomAccessFile: trying to change RERUN for JDK native code support via JNI to BUILD_TIME super type of org.deeplearning4j.datasets.mnist.MnistDbFile
        at com.oracle.svm.core.util.UserError.abort(UserError.java:65)
        at com.oracle.svm.hosted.classinitialization.ClassInitializationConfiguration.insertRec(ClassInitializationConfiguration.java:96)
        at com.oracle.svm.hosted.classinitialization.ClassInitializationConfiguration.insertRec(ClassInitializationConfiguration.java:107)
        at com.oracle.svm.hosted.classinitialization.ClassInitializationConfiguration.insertRec(ClassInitializationConfiguration.java:107)
        at com.oracle.svm.hosted.classinitialization.ClassInitializationConfiguration.insertRec(ClassInitializationConfiguration.java:107)
        at com.oracle.svm.hosted.classinitialization.ClassInitializationConfiguration.insert(ClassInitializationConfiguration.java:63)
        at com.oracle.svm.hosted.classinitialization.ConfigurableClassInitialization.forceInitializeHosted(ConfigurableClassInitialization.java:456)
        at com.oracle.svm.hosted.classinitialization.ConfigurableClassInitialization.forceInitializeHosted(ConfigurableClassInitialization.java:460)
        at com.oracle.svm.hosted.classinitialization.ClassInitializationFeature.lambda$initializeSafeDelayedClasses$11(ClassInitializationFeature.java:285)
        at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183)
        at java.base/java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:177)
        at java.base/java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:177)
        at java.base/java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:177)
        at java.base/java.util.HashMap$KeySpliterator.forEachRemaining(HashMap.java:1603)
        at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484)
        at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474)
        at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150)
        at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173)
        at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
        at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:497)
        at com.oracle.svm.hosted.classinitialization.ClassInitializationFeature.initializeSafeDelayedClasses(ClassInitializationFeature.java:282)
        at com.oracle.svm.hosted.classinitialization.ClassInitializationFeature.beforeCompilation(ClassInitializationFeature.java:222)
        at com.oracle.svm.hosted.NativeImageGenerator.lambda$doRun$2(NativeImageGenerator.java:571)
        at com.oracle.svm.hosted.FeatureHandler.forEachFeature(FeatureHandler.java:63)
        at com.oracle.svm.hosted.NativeImageGenerator.doRun(NativeImageGenerator.java:571)
        at com.oracle.svm.hosted.NativeImageGenerator.lambda$run$0(NativeImageGenerator.java:446)
        at java.base/java.util.concurrent.ForkJoinTask$AdaptedRunnableAction.exec(ForkJoinTask.java:1407)
        at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290)
        at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1020)
        at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1656)
        at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1594)
        at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:177)
Error: Image build request failed with exit status 1

This is most likely caused by an instance of MnistDbFile making its way into the image heap, typically via a static field (directly or indirectly). Because that class derives from RandomAccessFile which might be open during the image build, this is state that can't be serialized into an image. Can you look into that?

I actually don't get these errors I'll share what I get (for both Linux and MacOS) - perhaps one of our environments is interfering with building of the native-image, see https://gist.github.com/neomatrix369/9720802c2c3f6b5e6927b62c30a47187.

I'll try to reproduce this in a clean isolated environment to see if the issue persists.

peter-hofer · 2019-10-29T09:35:44Z

I got rid of the errors from the log you posted (Pointer_deallocate, started thread,etc) by using javacpp from GraalVM. Are you saying that GraalVM javacpp should be replaced with your version instead?

peter-hofer · 2019-10-31T16:45:13Z

Regardless, this issue seems to be caused by an incomplete classpath that doesn't include GroovyObject: we walk the subclasses of a class to discover any methods that shadow a method registered for JNI calls and then come across a class of a parameter or return type which cannot be found, which might not be an issue otherwise. This is not trivial to fix correctly and we might need to use JVMCI here so we can deal with unresolved types properly.

neomatrix369 · 2019-10-31T22:50:34Z

I got rid of the errors from the log you posted (Pointer_deallocate, started thread,etc) by using javacpp from GraalVM. Are you saying that GraalVM javacpp should be replaced with your version instead?

Okay I see, no actually I didn't use javacpp at all to attempt to build the native-image, although when I did use javacpp, starting 19.2.1 I didn't get any errors as I did previously. My observation so far has been that whether we leave javacpp in the path or not we don't get the previous error since 19.2.1 - not sure what has changed.

neomatrix369 · 2019-10-31T22:52:02Z

Regardless, this issue seems to be caused by an incomplete classpath that doesn't include GroovyObject: we walk the subclasses of a class to discover any methods that shadow a method registered for JNI calls and then come across a class of a parameter or return type which cannot be found, which might not be an issue otherwise. This is not trivial to fix correctly and we might need to use JVMCI here so we can deal with unresolved types properly.

Thanks again - great finding, I'll leave it to your side to work on it further.

Do you need any more details from me or anything else to help with any further investigation or fixing of the issue?

peter-hofer · 2019-11-14T10:15:54Z

Should be fixed in 0fef136.

dougxc assigned cstancu Oct 9, 2019

neomatrix369 changed the title ~~[native-image] [Runtime] Getting a java.lang.NoClassDefFoundError: groovy/lang/GroovyObject when running a native-image~~ [native-image] [Buildtime] Getting a java.lang.NoClassDefFoundError: groovy/lang/GroovyObject when running a native-image Oct 10, 2019

cstancu assigned peter-hofer Oct 10, 2019

cstancu added the native-image label Oct 10, 2019

cstancu removed their assignment Oct 10, 2019

peter-hofer closed this as completed Nov 14, 2019

LightCampy mentioned this issue Aug 25, 2020

Error: Incompatible change of initialization policy for java.io.RandomAccessFile: trying to change RERUN for JDK native code support via JNI to BUILD_TIME super type of org.apache.fontbox.ttf.BufferedRandomAccessFile #2723

Closed

altro3 mentioned this issue May 14, 2022

Upgrade libs. Fix versions for test suites micronaut-projects/micronaut-grpc#483

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[native-image] [Buildtime] Getting a java.lang.NoClassDefFoundError: groovy/lang/GroovyObject when running a native-image #1731

[native-image] [Buildtime] Getting a java.lang.NoClassDefFoundError: groovy/lang/GroovyObject when running a native-image #1731

neomatrix369 commented Oct 9, 2019 •

edited

Loading

neomatrix369 commented Oct 9, 2019 •

edited

Loading

neomatrix369 commented Oct 10, 2019

neomatrix369 commented Oct 13, 2019

neomatrix369 commented Oct 19, 2019

nestis commented Oct 20, 2019 •

edited

Loading

thomaswue commented Oct 20, 2019

peter-hofer commented Oct 21, 2019 •

edited

Loading

neomatrix369 commented Oct 21, 2019 •

edited

Loading

neomatrix369 commented Oct 21, 2019

neomatrix369 commented Oct 21, 2019 •

edited

Loading

neomatrix369 commented Oct 21, 2019 •

edited

Loading

neomatrix369 commented Oct 21, 2019 •

edited

Loading

peter-hofer commented Oct 22, 2019

peter-hofer commented Oct 22, 2019

neomatrix369 commented Oct 24, 2019 •

edited

Loading

neomatrix369 commented Oct 24, 2019

peter-hofer commented Oct 29, 2019

peter-hofer commented Oct 31, 2019 •

edited

Loading

neomatrix369 commented Oct 31, 2019

neomatrix369 commented Oct 31, 2019

peter-hofer commented Nov 14, 2019

[native-image] [Buildtime] Getting a java.lang.NoClassDefFoundError: groovy/lang/GroovyObject when running a native-image #1731

[native-image] [Buildtime] Getting a java.lang.NoClassDefFoundError: groovy/lang/GroovyObject when running a native-image #1731

Comments

neomatrix369 commented Oct 9, 2019 • edited Loading

neomatrix369 commented Oct 9, 2019 • edited Loading

neomatrix369 commented Oct 10, 2019

neomatrix369 commented Oct 13, 2019

neomatrix369 commented Oct 19, 2019

nestis commented Oct 20, 2019 • edited Loading

thomaswue commented Oct 20, 2019

peter-hofer commented Oct 21, 2019 • edited Loading

neomatrix369 commented Oct 21, 2019 • edited Loading

neomatrix369 commented Oct 21, 2019

neomatrix369 commented Oct 21, 2019 • edited Loading

neomatrix369 commented Oct 21, 2019 • edited Loading

neomatrix369 commented Oct 21, 2019 • edited Loading

peter-hofer commented Oct 22, 2019

peter-hofer commented Oct 22, 2019

neomatrix369 commented Oct 24, 2019 • edited Loading

neomatrix369 commented Oct 24, 2019

peter-hofer commented Oct 29, 2019

peter-hofer commented Oct 31, 2019 • edited Loading

neomatrix369 commented Oct 31, 2019

neomatrix369 commented Oct 31, 2019

peter-hofer commented Nov 14, 2019

neomatrix369 commented Oct 9, 2019 •

edited

Loading

neomatrix369 commented Oct 9, 2019 •

edited

Loading

nestis commented Oct 20, 2019 •

edited

Loading

peter-hofer commented Oct 21, 2019 •

edited

Loading

neomatrix369 commented Oct 21, 2019 •

edited

Loading

neomatrix369 commented Oct 21, 2019 •

edited

Loading

neomatrix369 commented Oct 21, 2019 •

edited

Loading

neomatrix369 commented Oct 21, 2019 •

edited

Loading

neomatrix369 commented Oct 24, 2019 •

edited

Loading

peter-hofer commented Oct 31, 2019 •

edited

Loading