[WIP] Support both GraalVM 19.2.1 and 19.3.0.2 - 19.2.1 SDK & 19.2.1 GraalVM image #6503

gwenneg · 2020-01-10T20:54:36Z

This PR tests one of the Graal SDK and image combinations decribed here.

svm version (SDK): 19.2.1
GraalVM image version : 19.2.1

geoand · 2020-01-11T10:01:01Z

I dunno what is wrong with CI and it's not testing this... Can you try one more force push please @gwenneg ?

dmlloyd · 2020-01-11T10:06:55Z

I got it @geoand. I think it's because it has "WIP" in the title.

dmlloyd · 2020-01-11T10:08:06Z

Oh! Never mind. It immediately cancels when I try to trigger it manually. I have no idea why...

geoand · 2020-01-11T10:09:16Z

That could very well be the case @dmlloyd but I am pretty sure it didn't used to be a problem.
@gsmet, @ia3andy did you perhaps change some setting for the repo that could be causing this now?

dmlloyd · 2020-01-11T10:11:12Z

I thought maybe the WIP check was preventing the rest of the checks from proceeding. Still tinkering a little...

dmlloyd · 2020-01-11T10:14:00Z

Going to try one more thing.... bear with me...

dmlloyd · 2020-01-11T10:14:33Z

Nope, I give up. I have no idea why this won't test.

geoand · 2020-01-11T10:16:36Z

Thanks for trying!

This one is super weird indeed...

gwenneg · 2020-01-11T12:46:55Z

I'm promoting the PR to see if it helps, but it's not meant to be reviewed or merged for now.

gwenneg · 2020-01-11T12:53:43Z

Well, now I failed a rebase... this PR is cursed! Let me clean that up :)

gwenneg · 2020-01-11T13:00:35Z

Annnnd CI finally accepted to build it ! :) There was a conflict in build-parent/pom.xml, CI started building the PR as soon as I rebased this on master and fixed the conflict.

Thanks for your help @geoand and @dmlloyd.

gwenneg · 2020-01-11T15:03:21Z

This PR shares with #6487 a quarkus-integration-test-jsch failure (which I haven't tried to fix yet):

WARNING: The sunec native library, required by the SunEC provider, could not be loaded. This library is usually shipped as part of the JDK and can be found under <JAVA_HOME>/jre/lib/<platform>/libsunec.so. It is loaded at run time via System.loadLibrary("sunec"), the first time services from SunEC are accessed. To use this provider's services the java.library.path system property needs to be set accordingly to point to a location that contains libsunec.so. Note that if java.library.path is not set it defaults to the current working directory.
Jan 11, 2020 2:31:45 PM org.apache.sshd.server.keyprovider.AbstractGeneratorHostKeyProvider resolveKeyPairs
WARN: resolveKeyPair(/tmp/host2828919426301705818key) Failed (EOFException) to load: null
2020-01-11 14:31:45,418 ERROR [io.qua.ver.htt.run.QuarkusErrorHandler] (executor-thread-1) HTTP Request to /jsch?host=fv-az53&port=34429 failed, error id: 1f9550dc-b3b7-4b00-8f20-bbaabbb125bc-1: org.jboss.resteasy.spi.UnhandledException: java.lang.UnsatisfiedLinkError: sun.security.ec.ECKeyPairGenerator.isCurveSupported([B)Z [symbol: Java_sun_security_ec_ECKeyPairGenerator_isCurveSupported or Java_sun_security_ec_ECKeyPairGenerator_isCurveSupported___3B]
	at org.jboss.resteasy.core.ExceptionHandler.handleApplicationException(ExceptionHandler.java:106)
	at org.jboss.resteasy.core.ExceptionHandler.handleException(ExceptionHandler.java:372)
	at org.jboss.resteasy.core.SynchronousDispatcher.writeException(SynchronousDispatcher.java:209)
	at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:496)
	at org.jboss.resteasy.core.SynchronousDispatcher.lambda$invoke$4(SynchronousDispatcher.java:252)
	at org.jboss.resteasy.core.SynchronousDispatcher.lambda$preprocess$0(SynchronousDispatcher.java:153)
	at org.jboss.resteasy.core.interception.jaxrs.PreMatchContainerRequestContext.filter(PreMatchContainerRequestContext.java:363)
	at org.jboss.resteasy.core.SynchronousDispatcher.preprocess(SynchronousDispatcher.java:156)
	at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:238)
	at io.quarkus.resteasy.runtime.standalone.RequestDispatcher.service(RequestDispatcher.java:73)
	at io.quarkus.resteasy.runtime.standalone.VertxRequestHandler.dispatch(VertxRequestHandler.java:120)
	at io.quarkus.resteasy.runtime.standalone.VertxRequestHandler.access$000(VertxRequestHandler.java:36)
	at io.quarkus.resteasy.runtime.standalone.VertxRequestHandler$1.run(VertxRequestHandler.java:85)
	at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35)
	at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:2011)
	at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1535)
	at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1395)
	at org.jboss.threads.DelegatingRunnable.run(DelegatingRunnable.java:29)
	at org.jboss.threads.ThreadLocalResettingRunnable.run(ThreadLocalResettingRunnable.java:29)
	at java.lang.Thread.run(Thread.java:748)
	at org.jboss.threads.JBossThread.run(JBossThread.java:479)
	at com.oracle.svm.core.thread.JavaThreads.threadStartRoutine(JavaThreads.java:460)
	at com.oracle.svm.core.posix.thread.PosixJavaThreads.pthreadStartRoutine(PosixJavaThreads.java:193)
Caused by: java.lang.UnsatisfiedLinkError: sun.security.ec.ECKeyPairGenerator.isCurveSupported([B)Z [symbol: Java_sun_security_ec_ECKeyPairGenerator_isCurveSupported or Java_sun_security_ec_ECKeyPairGenerator_isCurveSupported___3B]
	at com.oracle.svm.jni.access.JNINativeLinkage.getOrFindEntryPoint(JNINativeLinkage.java:145)
	at com.oracle.svm.jni.JNIGeneratedMethodSupport.nativeCallAddress(JNIGeneratedMethodSupport.java:57)
	at sun.security.ec.ECKeyPairGenerator.isCurveSupported(ECKeyPairGenerator.java)
	at sun.security.ec.ECKeyPairGenerator.ensureCurveIsSupported(ECKeyPairGenerator.java:135)
	at sun.security.ec.ECKeyPairGenerator.initialize(ECKeyPairGenerator.java:114)
	at java.security.KeyPairGenerator$Delegate.initialize(KeyPairGenerator.java:674)
	at java.security.KeyPairGenerator.initialize(KeyPairGenerator.java:411)
	at com.jcraft.jsch.jce.KeyPairGenECDSA.init(KeyPairGenECDSA.java:54)
	at com.jcraft.jsch.jce.ECDHN.init(ECDHN.java:46)
	at com.jcraft.jsch.DHECN.init(DHECN.java:81)
	at com.jcraft.jsch.Session.checkKex(Session.java:2542)
	at com.jcraft.jsch.Session.checkKexes(Session.java:2519)
	at com.jcraft.jsch.Session.send_kexinit(Session.java:634)
	at com.jcraft.jsch.Session.connect(Session.java:307)
	at com.jcraft.jsch.Session.connect(Session.java:183)
	at io.quarkus.it.jsch.JSchResource.connect(JSchResource.java:19)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.jboss.resteasy.core.MethodInjectorImpl.invoke(MethodInjectorImpl.java:151)
	at org.jboss.resteasy.core.MethodInjectorImpl.lambda$invoke$3(MethodInjectorImpl.java:122)
	at java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:616)
	at java.util.concurrent.CompletableFuture.uniApplyStage(CompletableFuture.java:628)
	at java.util.concurrent.CompletableFuture.thenApply(CompletableFuture.java:1996)
	at java.util.concurrent.CompletableFuture.thenApply(CompletableFuture.java:110)
	at org.jboss.resteasy.core.MethodInjectorImpl.invoke(MethodInjectorImpl.java:122)
	at org.jboss.resteasy.core.ResourceMethodInvoker.internalInvokeOnTarget(ResourceMethodInvoker.java:594)
	at org.jboss.resteasy.core.ResourceMethodInvoker.invokeOnTargetAfterFilter(ResourceMethodInvoker.java:468)
	at org.jboss.resteasy.core.ResourceMethodInvoker.lambda$invokeOnTarget$2(ResourceMethodInvoker.java:421)
	at org.jboss.resteasy.core.interception.jaxrs.PreMatchContainerRequestContext.filter(PreMatchContainerRequestContext.java:363)
	at org.jboss.resteasy.core.ResourceMethodInvoker.invokeOnTarget(ResourceMethodInvoker.java:423)
	at org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:391)
	at org.jboss.resteasy.core.ResourceMethodInvoker.lambda$invoke$1(ResourceMethodInvoker.java:365)
	at java.util.concurrent.CompletableFuture.uniComposeStage(CompletableFuture.java:995)
	at java.util.concurrent.CompletableFuture.thenCompose(CompletableFuture.java:2137)
	at java.util.concurrent.CompletableFuture.thenCompose(CompletableFuture.java:110)
	at org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:365)
	at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:477)
	... 19 more

gsmet · 2020-01-11T18:18:16Z

@gwenneg you should try to enable JNI for the JSCH extension and see how it goes.

gwenneg · 2020-01-11T21:55:45Z

@gsmet JNI is actually already enabled during the image generation used for the failing test. Here's the Docker command:
[INFO] [io.quarkus.deployment.pkg.steps.NativeImageBuildStep] docker run -v /home/vsts/work/1/s/integration-tests/jsch/target/quarkus-integration-test-jsch-999-SNAPSHOT-native-image-source-jar:/project:z --user 1001:117 --rm quay.io/quarkus/ubi-quarkus-native-image:19.2.1 -J-Dsun.nio.ch.maxUpdateArraySize=100 -J-Djava.util.logging.manager=org.jboss.logmanager.LogManager -J-Dvertx.logger-delegate-factory-class-name=io.quarkus.vertx.core.runtime.VertxLogDelegateFactory -J-Dvertx.disableDnsResolver=true -J-Dio.netty.leakDetection.level=DISABLED -J-Dio.netty.allocator.maxOrder=1 --initialize-at-build-time= -H:InitialCollectionPolicy=com.oracle.svm.core.genscavenge.CollectionPolicy$BySpaceAndTime -jar quarkus-integration-test-jsch-999-SNAPSHOT-runner.jar -H:FallbackThreshold=0 -H:+ReportExceptionStackTraces -J-Xmx6g -H:+AddAllCharsets -H:EnableURLProtocols=http --enable-all-security-services -H:+JNI --no-server -H:-UseServiceLoaderFeature -H:+StackTrace quarkus-integration-test-jsch-999-SNAPSHOT-runner

I am now focusing on fixing the test error since that's the only issue so far while using the 19.2.1 SDK with JDK 8 (JDK 11 still needs to be tested).

gwenneg · 2020-01-12T13:17:08Z

From what I can see locally, quarkus-integration-test-jsch is failing because the sunec lib is not loaded, which itself is caused by a missing GRAALVM_HOME environment variable.

This looks like an easy issue to fix, but I don't know why this module is the only one affected. Is the environment variable set in the ubi-quarkus-native-image:19.2.1?

dmlloyd · 2020-01-12T14:08:46Z

Hm I wonder why it relies on GRAALVM_HOME for that. I should be loading it from the JDK's home, shouldn't it?

gwenneg · 2020-01-12T14:15:37Z

Well, I had a look at what's going on during NativeImageConfigBuildStep.java#L83-L122 execution, but maybe there's another way to load the sunec lib. This code wasn't even involved at first, until I added f13d93a.

gwenneg · 2020-01-12T14:18:09Z

By the way, I think NativeImageConfigBuildStep.java#L83-L122 will be partly or maybe completely useless (I'm not sure about the cacerts part) once we're done with the GraalVM 19.3.x upgrade since GraalVM 19.3 [...] eliminates the need for shipping JDK libraries such as libsunec.so along with native images that use Java crypto services (source).

dmlloyd · 2020-01-12T14:20:07Z

In that case maybe the right fix is to have a version check?

gsmet · 2020-01-12T14:57:54Z

Jsch was extracted from the jgit extension and everything was working before the split. So I would have a look at how the jgit extension was doing this in 1.1.

gwenneg · 2020-01-12T22:53:58Z

@gsmet The test that is failing now didn't exist back then in 1.1 so I'm not sure this test has ever been run using GraalVM 19.2.1. Maybe @gastaldi could confirm this.

gastaldi · 2020-01-12T23:46:30Z

extensions/jsch/deployment/src/main/java/io/quarkus/jsch/deployment/JSchProcessor.java

+
+    @BuildStep
+    FeatureBuildItem feature() {
+        return new FeatureBuildItem(FeatureBuildItem.JSCH);


This extension was purposely designed to not produce a FeatureBuildItem since we didn't want to appear in the initialization logs

That's good to know, thanks @gastaldi. This was part of a test that didn't produce the expected effect, so it would probably have been removed anyway.

geoand · 2020-01-13T21:16:59Z

@gwenneg I assume that this need a rebase now?

gwenneg · 2020-01-14T09:30:01Z

@geoand I'll rebase tonight, but it's not really important. A small part of this PR has been pushed on master, but the code is the same in the end.

geoand · 2020-01-14T09:31:40Z

OK, it's just that github shows conflicts :)

gwenneg · 2020-01-16T01:23:22Z

Superseded by #6574.

This was referenced Jan 10, 2020

[WIP] Support both GraalVM 19.2.1 and 19.3.0.2 - 19.2.1 SDK edition #6502

Closed

[WIP] Support both GraalVM 19.2.1 and 19.3.0.2 - 19.3.0.2 SDK & 19.2.1 GraalVM image #6487

Closed

dmlloyd changed the title ~~[WIP] Support both GraalVM 19.2.1 and 19.3.0.2 - 19.2.1 SDK edition~~ Support both GraalVM 19.2.1 and 19.3.0.2 - 19.2.1 SDK edition Jan 11, 2020

dmlloyd closed this Jan 11, 2020

dmlloyd reopened this Jan 11, 2020

gwenneg force-pushed the support-graalvm-19.2.1-and-19.3.0.2 branch from 9de1dd4 to 10c4083 Compare January 11, 2020 12:45

gwenneg marked this pull request as ready for review January 11, 2020 12:46

gwenneg force-pushed the support-graalvm-19.2.1-and-19.3.0.2 branch 2 times, most recently from ee628d9 to 5c6c26c Compare January 11, 2020 12:51

gwenneg force-pushed the support-graalvm-19.2.1-and-19.3.0.2 branch from 5c6c26c to fa59b6c Compare January 11, 2020 12:57

gwenneg added 2 commits January 11, 2020 13:58

Support both GraalVM 19.2.1 and 19.3.0.2

41aeb61

Use 19.2.1 SDK instead of 19.3.0.2 SDK

9b8585a

gwenneg force-pushed the support-graalvm-19.2.1-and-19.3.0.2 branch from fa59b6c to 9b8585a Compare January 11, 2020 12:59

gwenneg changed the title ~~Support both GraalVM 19.2.1 and 19.3.0.2 - 19.2.1 SDK edition~~ [WIP] Support both GraalVM 19.2.1 and 19.3.0.2 - 19.2.1 SDK edition Jan 11, 2020

gwenneg changed the title ~~[WIP] Support both GraalVM 19.2.1 and 19.3.0.2 - 19.2.1 SDK edition~~ [WIP] Support both GraalVM 19.2.1 and 19.3.0.2 - 19.2.1 SDK & 19.2.1 GraalVM image Jan 11, 2020

Try fixing quarkus-integration-test-jsch failure

f13d93a

Force java.library.path (this is only a test)

827a9d7

gastaldi reviewed Jan 12, 2020

View reviewed changes

Revert non-working changes

39a3b25

gwenneg closed this Jan 16, 2020

gwenneg added the triage/invalid This doesn't seem right label Jan 16, 2020

sberyozkin mentioned this pull request Jan 17, 2020

OIDC adapter stabilization #5874

Closed

10 tasks

gwenneg deleted the support-graalvm-19.2.1-and-19.3.0.2 branch January 17, 2020 21:48

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[WIP] Support both GraalVM 19.2.1 and 19.3.0.2 - 19.2.1 SDK & 19.2.1 GraalVM image #6503

[WIP] Support both GraalVM 19.2.1 and 19.3.0.2 - 19.2.1 SDK & 19.2.1 GraalVM image #6503

gwenneg commented Jan 10, 2020 •

edited

Loading

geoand commented Jan 11, 2020

dmlloyd commented Jan 11, 2020

dmlloyd commented Jan 11, 2020

geoand commented Jan 11, 2020

dmlloyd commented Jan 11, 2020

dmlloyd commented Jan 11, 2020

dmlloyd commented Jan 11, 2020

geoand commented Jan 11, 2020

gwenneg commented Jan 11, 2020

gwenneg commented Jan 11, 2020 •

edited

Loading

gwenneg commented Jan 11, 2020 •

edited

Loading

gwenneg commented Jan 11, 2020 •

edited

Loading

gsmet commented Jan 11, 2020

gwenneg commented Jan 11, 2020 •

edited

Loading

gwenneg commented Jan 12, 2020 •

edited

Loading

dmlloyd commented Jan 12, 2020

gwenneg commented Jan 12, 2020

gwenneg commented Jan 12, 2020 •

edited

Loading

dmlloyd commented Jan 12, 2020

gsmet commented Jan 12, 2020

gwenneg commented Jan 12, 2020 •

edited

Loading

gastaldi Jan 12, 2020

gwenneg Jan 13, 2020

geoand commented Jan 13, 2020

gwenneg commented Jan 14, 2020

geoand commented Jan 14, 2020

gwenneg commented Jan 16, 2020

[WIP] Support both GraalVM 19.2.1 and 19.3.0.2 - 19.2.1 SDK & 19.2.1 GraalVM image #6503

[WIP] Support both GraalVM 19.2.1 and 19.3.0.2 - 19.2.1 SDK & 19.2.1 GraalVM image #6503

Conversation

gwenneg commented Jan 10, 2020 • edited Loading

geoand commented Jan 11, 2020

dmlloyd commented Jan 11, 2020

dmlloyd commented Jan 11, 2020

geoand commented Jan 11, 2020

dmlloyd commented Jan 11, 2020

dmlloyd commented Jan 11, 2020

dmlloyd commented Jan 11, 2020

geoand commented Jan 11, 2020

gwenneg commented Jan 11, 2020

gwenneg commented Jan 11, 2020 • edited Loading

gwenneg commented Jan 11, 2020 • edited Loading

gwenneg commented Jan 11, 2020 • edited Loading

gsmet commented Jan 11, 2020

gwenneg commented Jan 11, 2020 • edited Loading

gwenneg commented Jan 12, 2020 • edited Loading

dmlloyd commented Jan 12, 2020

gwenneg commented Jan 12, 2020

gwenneg commented Jan 12, 2020 • edited Loading

dmlloyd commented Jan 12, 2020

gsmet commented Jan 12, 2020

gwenneg commented Jan 12, 2020 • edited Loading

gastaldi Jan 12, 2020

Choose a reason for hiding this comment

gwenneg Jan 13, 2020

Choose a reason for hiding this comment

geoand commented Jan 13, 2020

gwenneg commented Jan 14, 2020

geoand commented Jan 14, 2020

gwenneg commented Jan 16, 2020

gwenneg commented Jan 10, 2020 •

edited

Loading

gwenneg commented Jan 11, 2020 •

edited

Loading

gwenneg commented Jan 11, 2020 •

edited

Loading

gwenneg commented Jan 11, 2020 •

edited

Loading

gwenneg commented Jan 11, 2020 •

edited

Loading

gwenneg commented Jan 12, 2020 •

edited

Loading

gwenneg commented Jan 12, 2020 •

edited

Loading

gwenneg commented Jan 12, 2020 •

edited

Loading