GraalVM-19 fails to build native-images for Clojure-1.10.0

[BrunoBonacci]

GraalVM version:

openjdk version "1.8.0_212"
OpenJDK Runtime Environment (build 1.8.0_212-20190420112649.buildslave.jdk8u-src-tar--b03)
OpenJDK GraalVM CE 19.0.0 (build 25.212-b03-jvmci-19-b01, mixed mode)

Fails to build a native image for a simple main function with Clojure-1.10 with the following error:

~/graalvm-ce-19.0.0/Contents/Home/bin/native-image --no-fallback --report-unsupported-elements-at-runtime -jar ./out/simple-clojure-1.10.0-uberjar.jar -H:Name=./out/simple-clojure-1.10.0-graal-19.0.0
Build on Server(pid: 66429, port: 56533)
[./out/simple-clojure-1.10.0-graal-19.0.0:66429]    classlist:     893.66 ms
[./out/simple-clojure-1.10.0-graal-19.0.0:66429]        (cap):     784.93 ms
[./out/simple-clojure-1.10.0-graal-19.0.0:66429]        setup:   1,032.85 ms
[./out/simple-clojure-1.10.0-graal-19.0.0:66429]     analysis:  22,805.33 ms
Error: Unsupported features in 5 methods
Detailed message:
Error: com.oracle.graal.pointsto.constraints.UnsupportedFeatureException: Invoke with MethodHandle argument could not be reduced to at most a single call: java.lang.invoke.MethodHandle.bindTo(Object)
Trace:
	at parsing java.lang.invoke.MethodHandleImpl.makePairwiseConvertByEditor(MethodHandleImpl.java:221)
Call path from entry point to java.lang.invoke.MethodHandleImpl.makePairwiseConvertByEditor(MethodHandle, MethodType, boolean, boolean):
	at java.lang.invoke.MethodHandleImpl.makePairwiseConvertByEditor(MethodHandleImpl.java:207)
	at java.lang.invoke.MethodHandleImpl.makePairwiseConvert(MethodHandleImpl.java:194)
	at java.lang.invoke.MethodHandleImpl.makePairwiseConvert(MethodHandleImpl.java:380)
	at java.lang.invoke.MethodHandle.asTypeUncached(MethodHandle.java:776)
	at java.lang.invoke.MethodHandle.asType(MethodHandle.java:761)
	at java.lang.invoke.Invokers.checkGenericType(Invokers.java:321)
	at java.lang.invoke.LambdaForm$MH/1760776497.invoke_MT(LambdaForm$MH)
	at clojure.lang.Reflector.canAccess(Reflector.java:49)
	at clojure.lang.Reflector.invokeMatchingMethod(Reflector.java:156)
	at clojure.lang.Reflector.invokeStaticMethod(Reflector.java:332)
	at clojure.lang.LispReader$CtorReader.readRecord(LispReader.java:1497)
	at clojure.lang.LispReader$CtorReader.invoke(LispReader.java:1427)
	at clojure.lang.AFn.applyToHelper(AFn.java:165)
	at clojure.lang.AFn.applyTo(AFn.java:144)
	at simple.main.main(Unknown Source)
	at com.oracle.svm.core.JavaMainWrapper.run(JavaMainWrapper.java:153)
	at com.oracle.svm.core.code.IsolateEnterStub.JavaMainWrapper_run_5087f5482cc9a6abc971913ece43acb471d2631b(generated:0)
Error: com.oracle.graal.pointsto.constraints.UnsupportedFeatureException: Invoke with MethodHandle argument could not be reduced to at most a single call: java.lang.invoke.MethodHandleImpl$BindCaller.prepareForInvoker(MethodHandle)
Trace:
	at parsing java.lang.invoke.MethodHandleImpl$BindCaller.makeInjectedInvoker(MethodHandleImpl.java:1166)
Call path from entry point to java.lang.invoke.MethodHandleImpl$BindCaller.makeInjectedInvoker(Class):
	at java.lang.invoke.MethodHandleImpl$BindCaller.makeInjectedInvoker(MethodHandleImpl.java:1141)
	at java.lang.invoke.MethodHandleImpl$BindCaller.access$300(MethodHandleImpl.java:1122)
	at java.lang.invoke.MethodHandleImpl$BindCaller$1.computeValue(MethodHandleImpl.java:1175)
	at java.lang.invoke.MethodHandleImpl$BindCaller$1.computeValue(MethodHandleImpl.java:1173)
	at com.oracle.svm.core.jdk.Target_java_lang_ClassValue.get(JavaLangSubstitutions.java:512)
	at java.lang.invoke.MethodHandleImpl$BindCaller.bindCaller(MethodHandleImpl.java:1136)
	at java.lang.invoke.MethodHandleImpl.bindCaller(MethodHandleImpl.java:1117)
	at java.lang.invoke.MethodHandles$Lookup.maybeBindCaller(MethodHandles.java:1681)
	at java.lang.invoke.MethodHandles$Lookup.getDirectMethodCommon(MethodHandles.java:1669)
	at java.lang.invoke.MethodHandles$Lookup.getDirectMethod(MethodHandles.java:1605)
	at java.lang.invoke.MethodHandles$Lookup.findStatic(MethodHandles.java:781)
	at java.lang.invoke.BoundMethodHandle$Factory.makeCbmhCtor(BoundMethodHandle.java:818)
	at java.lang.invoke.BoundMethodHandle$Factory.makeCtors(BoundMethodHandle.java:763)
	at java.lang.invoke.BoundMethodHandle$SpeciesData.<init>(BoundMethodHandle.java:349)
	at java.lang.invoke.BoundMethodHandle$SpeciesData$1.apply(BoundMethodHandle.java:389)
	at java.lang.invoke.BoundMethodHandle$SpeciesData$1.apply(BoundMethodHandle.java:383)
	at java.util.concurrent.ConcurrentHashMap.computeIfAbsent(ConcurrentHashMap.java:1660)
	at java.lang.invoke.BoundMethodHandle$SpeciesData.get(BoundMethodHandle.java:383)
	at java.lang.invoke.BoundMethodHandle$SpeciesData.extendWith(BoundMethodHandle.java:378)
	at java.lang.invoke.BoundMethodHandle$Species_L.copyWithExtendL(BoundMethodHandle.java:233)
	at java.lang.invoke.MethodHandle.asCollector(MethodHandle.java:1005)
	at java.lang.invoke.MethodHandleImpl$AsVarargsCollector.asTypeUncached(MethodHandleImpl.java:502)
	at java.lang.invoke.MethodHandle.asType(MethodHandle.java:761)
	at java.lang.invoke.Invokers.checkGenericType(Invokers.java:321)
	at java.lang.invoke.LambdaForm$MH/1760776497.invoke_MT(LambdaForm$MH)
	at clojure.lang.Reflector.canAccess(Reflector.java:49)
	at clojure.lang.Reflector.invokeMatchingMethod(Reflector.java:156)
	at clojure.lang.Reflector.invokeStaticMethod(Reflector.java:332)
	at clojure.lang.LispReader$CtorReader.readRecord(LispReader.java:1497)
	at clojure.lang.LispReader$CtorReader.invoke(LispReader.java:1427)
	at clojure.lang.AFn.applyToHelper(AFn.java:165)
	at clojure.lang.AFn.applyTo(AFn.java:144)
	at simple.main.main(Unknown Source)
	at com.oracle.svm.core.JavaMainWrapper.run(JavaMainWrapper.java:153)
	at com.oracle.svm.core.code.IsolateEnterStub.JavaMainWrapper_run_5087f5482cc9a6abc971913ece43acb471d2631b(generated:0)
Error: com.oracle.graal.pointsto.constraints.UnsupportedFeatureException: Invoke with MethodHandle argument could not be reduced to at most a single call: java.lang.invoke.MethodHandleImpl.buildVarargsArray(MethodHandle, MethodHandle, int)
Trace:
	at parsing java.lang.invoke.MethodHandleImpl.varargsArray(MethodHandleImpl.java:1634)
Call path from entry point to java.lang.invoke.MethodHandleImpl.varargsArray(Class, int):
	at java.lang.invoke.MethodHandleImpl.varargsArray(MethodHandleImpl.java:1611)
	at java.lang.invoke.MethodHandle.asCollector(MethodHandle.java:999)
	at java.lang.invoke.MethodHandleImpl$AsVarargsCollector.asTypeUncached(MethodHandleImpl.java:502)
	at java.lang.invoke.MethodHandle.asType(MethodHandle.java:761)
	at java.lang.invoke.Invokers.checkGenericType(Invokers.java:321)
	at java.lang.invoke.LambdaForm$MH/1760776497.invoke_MT(LambdaForm$MH)
	at clojure.lang.Reflector.canAccess(Reflector.java:49)
	at clojure.lang.Reflector.invokeMatchingMethod(Reflector.java:156)
	at clojure.lang.Reflector.invokeStaticMethod(Reflector.java:332)
	at clojure.lang.LispReader$CtorReader.readRecord(LispReader.java:1497)
	at clojure.lang.LispReader$CtorReader.invoke(LispReader.java:1427)
	at clojure.lang.AFn.applyToHelper(AFn.java:165)
	at clojure.lang.AFn.applyTo(AFn.java:144)
	at simple.main.main(Unknown Source)
	at com.oracle.svm.core.JavaMainWrapper.run(JavaMainWrapper.java:153)
	at com.oracle.svm.core.code.IsolateEnterStub.JavaMainWrapper_run_5087f5482cc9a6abc971913ece43acb471d2631b(generated:0)
Error: type is not available in this platform: org.graalvm.nativeimage.hosted.Feature$BeforeAnalysisAccess
Trace: 	object java.lang.Class[]
	object java.lang.invoke.MethodType
	object java.lang.invoke.MethodType$ConcurrentWeakInternSet$WeakEntry
	object java.util.concurrent.ConcurrentHashMap$Node
	object java.util.concurrent.ConcurrentHashMap$Node[]
	object java.util.concurrent.ConcurrentHashMap
	object java.lang.invoke.MethodType$ConcurrentWeakInternSet
	method java.lang.invoke.MethodType.makeImpl(Class, Class[], boolean)
Call path from entry point to java.lang.invoke.MethodType.makeImpl(Class, Class[], boolean):
	at java.lang.invoke.MethodType.makeImpl(MethodType.java:301)
	at java.lang.invoke.MethodType.methodType(MethodType.java:221)
	at java.lang.invoke.MethodType.asCollectorType(MethodType.java:529)
	at java.lang.invoke.MethodHandle.asCollector(MethodHandle.java:998)
	at java.lang.invoke.MethodHandleImpl$AsVarargsCollector.asTypeUncached(MethodHandleImpl.java:502)
	at java.lang.invoke.MethodHandle.asType(MethodHandle.java:761)
	at java.lang.invoke.Invokers.checkGenericType(Invokers.java:321)
	at java.lang.invoke.LambdaForm$MH/1760776497.invoke_MT(LambdaForm$MH)
	at clojure.lang.Reflector.canAccess(Reflector.java:49)
	at clojure.lang.Reflector.invokeMatchingMethod(Reflector.java:156)
	at clojure.lang.Reflector.invokeStaticMethod(Reflector.java:332)
	at clojure.lang.LispReader$CtorReader.readRecord(LispReader.java:1497)
	at clojure.lang.LispReader$CtorReader.invoke(LispReader.java:1427)
	at clojure.lang.AFn.applyToHelper(AFn.java:165)
	at clojure.lang.AFn.applyTo(AFn.java:144)
	at simple.main.main(Unknown Source)
	at com.oracle.svm.core.JavaMainWrapper.run(JavaMainWrapper.java:153)
	at com.oracle.svm.core.code.IsolateEnterStub.JavaMainWrapper_run_5087f5482cc9a6abc971913ece43acb471d2631b(generated:0)
Error: type is not available in this platform: org.graalvm.nativeimage.impl.ImageSingletonsSupport

Error: Use -H:+ReportExceptionStackTraces to print stacktrace of underlying exception
Error: Image build request failed with exit status 1

This works fine with version: OpenJDK GraalVM CE 1.0.0-rc14 (build 25.202-b08-jvmci-0.56, mixed mode)

To reproduce the jars here all the steps:
https://github.com/BrunoBonacci/graalvm-clojure/tree/master/clojure

In attachment the standalone JAR:
simple-clojure-1.10.0-uberjar.tar.gz

[BrunoBonacci]

After additional tests, it seems that the regression was introduced after GraalVM-CE-1.0.0-rc16 as it is working with this version.

[vjovanov]

Here as well running initializers at run time has revealed other issues. So the quick way out is to pass --initialize-at-build-time.

I am investigating what is the correct set of packages that need to be initialized at build time.

[vjovanov]

It should be --initialize-at-build-time=clojure but then I get:

[./simple-clojure-1.10.0-graal-19.0.0:68215]        setup:   2,158.30 ms
[./simple-clojure-1.10.0-graal-19.0.0:68215]     analysis:  15,500.99 ms
Error: unbalanced monitors: mismatch at monitorexit, 96|LoadField#lockee__5436__auto__ != 3|LoadField#lockee__5436__auto__
Detailed message:
Call path from entry point to clojure.spec.gen.alpha$dynaload$fn__2628.invoke(): 
	at clojure.spec.gen.alpha$dynaload$fn__2628.invoke(alpha.clj:21)
	at clojure.lang.AFn.run(AFn.java:22)
	at com.oracle.svm.core.jdk.RuntimeSupport.executeHooks(RuntimeSupport.java:144)
	at com.oracle.svm.core.jdk.RuntimeSupport.executeStartupHooks(RuntimeSupport.java:89)
	at com.oracle.svm.core.JavaMainWrapper.run(JavaMainWrapper.java:147)
	at com.oracle.svm.core.code.IsolateEnterStub.JavaMainWrapper_run_5087f5482cc9a6abc971913ece43acb471d2631b(generated:0)

We will have to investigate why this happens and make proper config for clojure. Until then --initialize-at-build-time will work as before.

[BrunoBonacci]

Hi,
clojure-1.10.0.jar depends on two additional jars:

 [org.clojure/clojure "1.10.0"]
   [org.clojure/core.specs.alpha "0.2.44"]
   [org.clojure/spec.alpha "0.2.176"]

Here is the cumulative list of packages:

clojure
clojure/asm
clojure/asm/commons
clojure/core
clojure/core/protocols
clojure/core/proxy$clojure/lang
clojure/core/proxy$java/io
clojure/data
clojure/inspector/proxy$java/lang
clojure/inspector/proxy$javax/swing/table
clojure/instant/proxy$java/lang
clojure/java
clojure/java/api
clojure/java/browse_ui/proxy$java/lang
clojure/java/io
clojure/lang
clojure/pprint
clojure/pprint/proxy$java/io
clojure/reflect
clojure/reflect/proxy$clojure/asm
clojure/repl/proxy$java/io
clojure/repl/proxy$java/lang
clojure/spec
clojure/spec/alpha
clojure/spec/gen
clojure/spec/test
clojure/test
clojure/xml/proxy$java/lang

[BrunoBonacci]

I understand this issue, although it manifests itself with a different error it can be resolved with the same considerations of #1265. So I think we can close this one and follow up the investigations for the minimal set of --initialize-at-build-time flag with the related ticket.

[GradySimon]

For people finding this from Google (like me) for whom --initialize-at-build-time doesn't fix it, in my case it was my use of pprint/pprint that was triggering this error:

Error: unbalanced monitors: mismatch at monitorexit, 96|LoadField#lockee__5436__auto__ != 3|LoadField#lockee__5436__auto__
Call path from entry point to clojure.spec.gen.alpha$dynaload$fn__2628.invoke(): 
	at clojure.spec.gen.alpha$dynaload$fn__2628.invoke(alpha.clj:21)
	at clojure.lang.AFn.run(AFn.java:22)
	at java.lang.Thread.run(Thread.java:748)
	at com.oracle.svm.core.thread.JavaThreads.threadStartRoutine(JavaThreads.java:460)
	at com.oracle.svm.core.posix.thread.PosixJavaThreads.pthreadStartRoutine(PosixJavaThreads.java:193)
	at com.oracle.svm.core.code.IsolateEnterStub.PosixJavaThreads_pthreadStartRoutine_e1f4a8c0039f8337338252cd8734f63a79b5e3df(generated:0)

In addition to using --initialize-at-build-time, I needed to remove the require and use of pprint to make it possible to build native-images again. There are probably other libraries that somehow cause native-image building to fail, possibly through dependency on unsupported parts of spec.

For the record, my code does use spec directly, but not spec.gen, which seems indicated as the problem in the error.

[vjovanov]

@GradySimon I think you should open a separate issue for this. It is a compilation bug that needs to be fixed. Also, please provide a reproducer.