quarkus-hibernate-orm extension is breaking native-image generation #2482
Comments
|
It's weird because AFAICS the error looks more related to Undertow than to Hibernate ORM. Any chance you could share a minimal reproducer? |
|
Attached a basic hello world that reproduces the issue. just run: |
|
Thanks for the reproducer! That kind of exception usually means we have a substitution for some component which is also being substituted by graalvm. We'll need to debug the compiler to figure out the details, but I suspect it's not a specific issue of the |
|
I hit this issue when playing with https://github.com/AdamBien/quarkee |
|
I did some tests as Iw as curious since it goes into my usual demo path. If you add a So I suppose there is a state here when Hibernate is not started that leads to a call path that is problematic. It is not necessary the blank ORM, it might be just agreal or just narayana but I could not reproduce with just narayana or agroal as direct dependency. Actualy with just narayana as a dep it works, with agroal as a dependency I have a different GraalVM possible stack trace error. |
|
CC @gsmet to pick your curiosity ;) |
|
Yeah, my curiosity was picked a few weeks ago. I tried to get a viable error from GraalVM and had to tweak the code a bit to avoid some false positives. In the end I ended up with a suspicion that maybe it could be related to Narayana. But it's really hard to say what's going on. I was sidetracked (a lot) but this one is still on my list. |
|
Blame @nmcl ! :) |
|
I've been in touch with @mmusgrov , he's going to send some improvements to the Narayana extension. In particular we should be able to get rid of the substitutions, maybe wait for that, it might be easier to figure out what's going on here? |
|
What's your expected timeframe @mmusgrov ? 1d 1w 1m 6m 1y never? |
|
FWIW It's still an issue with current master as of today. |
|
Right. I don’t think we should wait for the transaction improvements before we investigation. |
|
Any update / plan for this issue ? |
|
So debugging this a bit, I have been able to narrow this down and solve one part of this issue. I can reproduce the issue consistently if I use a simple application with a dependency on the (even against latest upstream master). With that command it throws a bunch of substrate exceptions which are hard to understand. So I introduced the in my pom.xml for the native image build and that showed up some interesting exceptions. For example of them is: Debugging this showed that, this exception happens because the Hibernate ORM extension has this (I'm not an exepert, but) the way Substrate seems to deal with deletes is that it loads the target class ( But of course, that doesn't mean we won't run into exceptions at runtime on first access. So I decided to remove that param and rebuild and this time I see lesser number of errors - 3 of them to be precise. All of those 3 are: So it appears that the [1] https://github.com/hibernate/hibernate-orm/blob/master/hibernate-core/hibernate-core.gradle#L52 |
|
The big question is: why does it work when you really use Hibernate ORM :).
…On Tue, Oct 8, 2019 at 4:08 PM Jaikiran ***@***.***> wrote:
So debugging this a bit, I have been able to narrow this down and solve
one part of this issue. I can reproduce the issue consistently if I use a
simple application with a dependency on the quarkus-hibernate-orm
extension (and no other extension) and then run:
mvn clean verify -Pnative
(even against latest upstream master). With that command it throws a bunch
of substrate exceptions which are hard to understand. So I introduced the
<additionalBuildArgs>--report-unsupported-elements-at-runtime</additionalBuildArgs>
in my pom.xml for the native image build and that showed up some
interesting exceptions. For example of them is:
Fatal error: java.lang.NoClassDefFoundError
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at java.util.concurrent.ForkJoinTask.getThrowableException(ForkJoinTask.java:598)
at java.util.concurrent.ForkJoinTask.get(ForkJoinTask.java:1005)
at com.oracle.svm.hosted.NativeImageGenerator.run(NativeImageGenerator.java:461)
at com.oracle.svm.hosted.NativeImageGeneratorRunner.buildImage(NativeImageGeneratorRunner.java:310)
at com.oracle.svm.hosted.NativeImageGeneratorRunner.build(NativeImageGeneratorRunner.java:448)
at com.oracle.svm.hosted.NativeImageGeneratorRunner.main(NativeImageGeneratorRunner.java:113)
Caused by: java.lang.NoClassDefFoundError: javax/security/jacc/PolicyContextException
at java.lang.Class.getDeclaredMethods0(Native Method)
at java.lang.Class.privateGetDeclaredMethods(Class.java:2701)
at java.lang.Class.getDeclaredMethods(Class.java:1975)
at com.oracle.svm.hosted.substitute.AnnotationSubstitutionProcessor.handleDeletedClass(AnnotationSubstitutionProcessor.java:437)
at com.oracle.svm.hosted.substitute.AnnotationSubstitutionProcessor.handleClass(AnnotationSubstitutionProcessor.java:270)
at com.oracle.svm.hosted.substitute.AnnotationSubstitutionProcessor.init(AnnotationSubstitutionProcessor.java:230)
at com.oracle.svm.hosted.NativeImageGenerator.createDeclarativeSubstitutionProcessor(NativeImageGenerator.java:875)
at com.oracle.svm.hosted.NativeImageGenerator.setupNativeImage(NativeImageGenerator.java:824)
at com.oracle.svm.hosted.NativeImageGenerator.doRun(NativeImageGenerator.java:524)
at com.oracle.svm.hosted.NativeImageGenerator.lambda$run$0(NativeImageGenerator.java:444)
at java.util.concurrent.ForkJoinTask$AdaptedRunnableAction.exec(ForkJoinTask.java:1386)
at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)
at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)
at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:157)
Caused by: java.lang.ClassNotFoundException: javax.security.jacc.PolicyContextException
at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 15 more
Debugging this showed that, this exception happens because the Hibernate
ORM extension has this @delete substitution:
@TargetClass(className = "org.hibernate.secure.internal.StandardJaccServiceImpl")
@delete
public final class Delete_StandardJaccServiceImpl {
}
(I'm not an exepert, but) the way Substrate seems to deal with deletes is
that it loads the target class (
org.hibernate.secure.internal.StandardJaccServiceImpl) first and then
does a getDeclaredMethods and getDeclaredFields on it, which obviously
means that it needs the necessary dependency classes to be present (or else
you will run into the exception above). StandardJaccServiceImpl has
depedency on jacc classes. Looking at the quarkus-hiberate-orm extension
it doesn't list the dependency on the artifact which brings in these
javax.security.jacc.* classes. Looking at the upstream hibernate-orm
project, that dependency is actually a provided scope dependency[1], so
the inclusion of hibernate-core artifact doesn't bring this in. So I
updated our quarkus-hibernate-orm extension to bring in that dependency
explicitly (pull request at [2]) just to allow this substitution to work
correctly and then rebuilt the quickstart project with this newer
quarkus-hibernate-orm. This time there were no more errors with:
<additionalBuildArgs>--report-unsupported-elements-at-runtime</additionalBuildArgs>
But of course, that doesn't mean we won't run into exceptions at runtime
on first access. So I decided to remove that param and rebuild and this
time I see lesser number of errors - 3 of them to be precise. All of those
3 are:
Caused by: com.oracle.svm.hosted.substitute.DeletedElementException: Unsupported method java.lang.Class.getExecutableTypeAnnotationBytes(Executable) is reachable: The declaring class of this element has been substituted, but this element is not present in the substitution class
Caused by: com.oracle.svm.hosted.substitute.DeletedElementException: Unsupported method java.lang.Class.getRawTypeAnnotations() is reachable: The declaring class of this element has been substituted, but this element is not present in the substitution class
Caused by: com.oracle.svm.hosted.substitute.DeletedElementException: Unsupported method java.lang.Class.getConstantPool() is reachable: The declaring class of this element has been substituted, but this element is not present in the substitution class
So it appears that the java.lang.Class is actually substituted (right in
the substrate VM here[3]) and that substituted version doesn't support
these getExecutableTypeAnnotationBytes getRawTypeAnnotations and
getConstantPool methods. But some piece of code within this quarkus
application seems to be ending up calling it. I have the analysis tree
generated out of this run but I'm yet to fully understand how to parse
these substrate VM logs.
[1]
https://github.com/hibernate/hibernate-orm/blob/master/hibernate-core/hibernate-core.gradle#L52
[2] #4440 <#4440>
[3]
https://github.com/oracle/graal/blob/master/substratevm/src/com.oracle.svm.core/src/com/oracle/svm/core/hub/DynamicHub.java#L97
—
You are receiving this because you were assigned.
Reply to this email directly, view it on GitHub
<#2482?email_source=notifications&email_token=AAJYOBL2J3C43QKXCRK66Y3QNSH4LA5CNFSM4HNS5AQKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEAUJOKA#issuecomment-539531048>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AAJYOBMNF4MNI47PACJR2V3QNSH4LANCNFSM4HNS5AQA>
.
|
Good question - I was curious too :) But turns out it doesn't work even with that "use an |
I was thinking of asking about this at the graal project github repo, but it looks like @Sanne already ran into this previously and has reported it here oracle/graal#819 |
|
@Sanne says in the issue that he could get rid of the entire class to solve the issue for him so I'm not sure it applies to the case where we delete the entire class. |
OK but we have projects working? Typically, this very simple project work OK: https://github.com/gsmet/hibernate-search-demo . So there's something that makes it work somehow. |
Actually, even that project fails if I add the following in the pom.xml and run It fails with: Quarkus version : latest upstream master Removing that None of this looks right, so let me take a step back and see what part of this is Graal specific and what part is a genuine error that we need to sort out in Quarkus extension. |
|
I've had numerous problems with "report-unsupported-elements-at-runtime";
if you're using it the compiler follows quite different logic in the flow
analysis.
I don't think we should support that mode, we certainly don't test for it
so please don't use it :)
…On Tue, 8 Oct 2019 at 16:08, Jaikiran ***@***.***> wrote:
OK but we have projects working? Typically, this very simple project work
OK: https://github.com/gsmet/hibernate-search-demo .
So there's something that makes it work somehow.
Actually, even that project fails *if I add the following in the pom.xml*
and run mvn clean verify -Pnative:
@@ -106,6 +106,8 @@
</goals>
<configuration>
<enableHttpUrlHandler>true</enableHttpUrlHandler>
+ <additionalBuildArgs>--report-unsupported-elements-at-runtime</additionalBuildArgs>
+
</configuration>
It fails with:
[INFO] [io.quarkus.creator.phase.nativeimage.NativeImagePhase] graalvm/graalvm-ce-19.2.0.1/Contents/Home/bin/native-image -J-Djava.util.logging.manager=org.jboss.logmanager.LogManager -J-Dio.netty.leakDetection.level=DISABLED -J-Dvertx.logger-delegate-factory-class-name=io.quarkus.vertx.core.runtime.VertxLogDelegateFactory -J-Dsun.nio.ch.maxUpdateArraySize=100 -J-Dio.netty.allocator.maxOrder=1 -J-Dvertx.disableDnsResolver=true --report-unsupported-elements-at-runtime --initialize-at-build-time= -H:InitialCollectionPolicy=com.oracle.svm.core.genscavenge.CollectionPolicy$BySpaceAndTime -jar hibernate-search-demo-1.0-SNAPSHOT-runner.jar -J-Djava.util.concurrent.ForkJoinPool.common.parallelism=1 -H:FallbackThreshold=0 -H:+ReportExceptionStackTraces -H:+PrintAnalysisCallTree -H:Log=registerResource: -H:-AddAllCharsets -H:EnableURLProtocols=http -H:-JNI --no-server -H:-UseServiceLoaderFeature -H:+StackTrace
[hibernate-search-demo-1.0-SNAPSHOT-runner:94789] classlist: 10,352.37 ms
[hibernate-search-demo-1.0-SNAPSHOT-runner:94789] setup: 400.41 ms
Fatal error: java.lang.NoClassDefFoundError
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at java.util.concurrent.ForkJoinTask.getThrowableException(ForkJoinTask.java:598)
at java.util.concurrent.ForkJoinTask.get(ForkJoinTask.java:1005)
at com.oracle.svm.hosted.NativeImageGenerator.run(NativeImageGenerator.java:461)
at com.oracle.svm.hosted.NativeImageGeneratorRunner.buildImage(NativeImageGeneratorRunner.java:310)
at com.oracle.svm.hosted.NativeImageGeneratorRunner.build(NativeImageGeneratorRunner.java:448)
at com.oracle.svm.hosted.NativeImageGeneratorRunner.main(NativeImageGeneratorRunner.java:113)
Caused by: java.lang.NoClassDefFoundError: javax/security/jacc/PolicyContextException
at java.lang.Class.getDeclaredMethods0(Native Method)
at java.lang.Class.privateGetDeclaredMethods(Class.java:2701)
at java.lang.Class.getDeclaredMethods(Class.java:1975)
at com.oracle.svm.hosted.substitute.AnnotationSubstitutionProcessor.handleDeletedClass(AnnotationSubstitutionProcessor.java:437)
at com.oracle.svm.hosted.substitute.AnnotationSubstitutionProcessor.handleClass(AnnotationSubstitutionProcessor.java:270)
at com.oracle.svm.hosted.substitute.AnnotationSubstitutionProcessor.init(AnnotationSubstitutionProcessor.java:230)
at com.oracle.svm.hosted.NativeImageGenerator.createDeclarativeSubstitutionProcessor(NativeImageGenerator.java:875)
at com.oracle.svm.hosted.NativeImageGenerator.setupNativeImage(NativeImageGenerator.java:824)
at com.oracle.svm.hosted.NativeImageGenerator.doRun(NativeImageGenerator.java:524)
at com.oracle.svm.hosted.NativeImageGenerator.lambda$run$0(NativeImageGenerator.java:444)
at java.util.concurrent.ForkJoinTask$AdaptedRunnableAction.exec(ForkJoinTask.java:1386)
at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)
at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)
at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:157)
Caused by: java.lang.ClassNotFoundException: javax.security.jacc.PolicyContextException
at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 15 more
Quarkus version : latest upstream master
Graal VM version: 19.2.0.1
Removing that --report-unsupported-elements-at-runtime gets it to pass
fine.
None of this looks right, so let me take a step back and see what part of
this is Graal specific and what part is a genuine error that we need to
sort out in Quarkus extension.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#2482?email_source=notifications&email_token=AAAKMTKOT6ON3FCFIGRWNLTQNSO5HA5CNFSM4HNS5AQKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEAUQHJY#issuecomment-539558823>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AAAKMTJRXLQBAASF6VKSYNLQNSO5HANCNFSM4HNS5AQA>
.
|
Same here. It does look like a bug the way it's implemented. So I've raised it at graal oracle/graal#1725 |
|
I might have a solution soon for this, was working on #5262 and it turns out it might be a duplicate. |
|
sent a PR :) |
Java version: OpenJDK GraalVM CE 1.0.0-rc16
OS: MacOS (but also breaks the docker image generation)
Quarkus Version : <quarkus.version>0.15.0</quarkus.version>
mvn package -Pnativeis throwing the following errors:
if I comment the dependency in pom.xml , everything starts working as expected
The text was updated successfully, but these errors were encountered: