Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Upgrades ojdbc to the latest version of 23 as of this writing #9441

Merged
merged 9 commits into from
Nov 20, 2024

Conversation

ljnelson
Copy link
Member

@ljnelson ljnelson commented Oct 28, 2024

See also: #9351.

This PR updates Helidon's usage of the various ojdbc and ucp* artifacts to the latest version available from the 23 line.

Due to internal changes in the Universal Connection Pool, formerly Helidon-provided special-case support is being removed for the following properties:

  • serviceName: Per the UCP team's instructions, the customer should use UCPConnectionBuilder::serviceName instead while acquiring Connections
  • pdbRoles: Per the UCP team's instructions, the customer should use UCPConnectionBuilder::pdbRoles instead while acquiring Connections
  • tnsServiceName: There was special support for ignoring this property in prior versions of Helidon; that support is removed

It is unclear whether Helidon's special-case support for these properties was actually being used. For serviceName and pdbRoles, setting these properties would only have effect when the Universal Connection Pool's associated "connection factory" (the "real" DataSource, usually supplied by a database vendor) also has an equivalently-named property. Moreover, the semantics of serviceName appear to be valid only when the connection factory is supplied by Oracle. All other former usages of this property would have essentially no effect.

@ljnelson ljnelson added dependencies Pull requests that update a dependency file 4.x Version 4.x labels Oct 28, 2024
@ljnelson ljnelson self-assigned this Oct 28, 2024
@oracle-contributor-agreement oracle-contributor-agreement bot added the OCA Verified All contributors have signed the Oracle Contributor Agreement. label Oct 28, 2024
@ljnelson
Copy link
Member Author

Looks like the UCP changed in some non-backwards compatible way between 21.x and 23.x; investigating

@ljnelson
Copy link
Member Author

ljnelson commented Nov 4, 2024

A test involving previously successful injection of a PoolDataSource with fred as a serviceName is now failing with, ultimately:

Caused by: java.lang.AssertionError: service mismatch: this.service=fred, smServiceName=
	at oracle.ucp.common.CoreConnectionImpl.<init>(CoreConnectionImpl.java:124)

Not sure what smServiceName is; need more research on the actual UCP classes' bytecode.

{time passes}

After disassembling the UCP I can see that this version has added a new check that ensures that some set of properties (it is unclear whether it is a set that applies to (a) the connection pool, (b) the PoolDataSource and/or (c) the driver it ultimately wraps) contains a service name entry somewhere that is the same as the serviceName found on the PoolDataSource (which Helidon has set programmatically for a long time). This seems to be related to setting up something called a ServiceMember, over which Helidon can have no control.

Digging further, it looks like the Properties object that is interrogated is returned by the UniversalPooledConnectionImpl class, via its getDatabaseConnectionProperties method. This method always returns a new, empty Properties object for some reason. I'm not sure whether that's a bug or not, but it does mean that the Service constructor, which accepts such a Properties object, will always get an empty one, and all of its get-property-or-use-defaults operations will always return the defaults (i.e. an empty properties object here is useless).

That means that the ServiceMember will never report anything other than "" as its service name, and that service name will never match a PoolDataSourceImpl with a non-empty service name.

@ljnelson
Copy link
Member Author

ljnelson commented Nov 9, 2024

The latest commit on this PR deliberately removes all special-cased property handling we were doing before, letting the UCP do the validation work instead. This is both more elegant (the special-casing was never going to scale, and this very PR got burned by it in a prior commit) and required (the upgrade of UCP from 21.x to 23.x introduced assertions in the UCP codebase that were not there previously and which were preventing the upgrade). Open to all comments on this situation.

Copy link
Member

@barchetta barchetta left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall this looks good. Are you aware of any customer impact of this in the area of compatibility? If so, can you summarize in the PR Description?

@ljnelson
Copy link
Member Author

Description updated. I'll be working on the native image failure next.

@ljnelson
Copy link
Member Author

ljnelson commented Nov 13, 2024

From some debugging, it looks like the native image test is not failing due to any change made by this PR, but by the omission of a --enable-url-protocols=http native image option. I'm not sure why this PR in particular uncovered this omission or why this test has not failed before. Moreover, the native test in question (mp-2) builds and runs just fine (I hesitate to say it) on my machine.

EDIT: I can reproduce the error but only when I run the test via target/helidon-tests-integration-packaging-mp-2 -Djavax.sql.DataSource.test.dataSource.url=jdbc:oracle:thin:@localhost:32769/XE -Djavax.sql.DataSource.test.dataSource.password=oracle123, i.e. not via mvn verify -Pnative-image. Maybe mvn verify here does not in fact run any tests?

The following error message does not show up in the test run but does appear in the downloadable test archive, which includes standard output:

jakarta.ws.rs.ProcessingException: java.net.MalformedURLException: Accessing a URL protocol that was not enabled. The URL protocol http is supported but not enabled by default. It must be enabled by adding the --enable-url-protocols=http option to the native-image command.
	at org.glassfish.jersey.client.internal.HttpUrlConnector.apply(HttpUrlConnector.java:288)
	at org.glassfish.jersey.client.ClientRuntime.invoke(ClientRuntime.java:300)
	at org.glassfish.jersey.client.JerseyInvocation.lambda$invoke$0(JerseyInvocation.java:674)
	at org.glassfish.jersey.client.JerseyInvocation.call(JerseyInvocation.java:709)
	at org.glassfish.jersey.client.JerseyInvocation.lambda$runInScope$3(JerseyInvocation.java:703)
	at org.glassfish.jersey.internal.Errors.process(Errors.java:292)
	at org.glassfish.jersey.internal.Errors.process(Errors.java:274)
	at org.glassfish.jersey.internal.Errors.process(Errors.java:205)
	at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:391)
	at org.glassfish.jersey.client.JerseyInvocation.runInScope(JerseyInvocation.java:703)
	at org.glassfish.jersey.client.JerseyInvocation.invoke(JerseyInvocation.java:673)
	at org.glassfish.jersey.client.JerseyInvocation$Builder.method(JerseyInvocation.java:413)
	at org.glassfish.jersey.client.JerseyInvocation$Builder.get(JerseyInvocation.java:313)
	at io.helidon.tests.integration.packaging.mp2.Mp2Main.validateJaxrs(Mp2Main.java:81)
	at io.helidon.tests.integration.packaging.mp2.Mp2Main.test(Mp2Main.java:72)
	at io.helidon.tests.integration.packaging.mp2.Mp2Main.main(Mp2Main.java:50)
	at [email protected]/java.lang.invoke.LambdaForm$DMH/sa346b79c.invokeStaticInit(LambdaForm$DMH)
Caused by: java.net.MalformedURLException: Accessing a URL protocol that was not enabled. The URL protocol http is supported but not enabled by default. It must be enabled by adding the --enable-url-protocols=http option to the native-image command.
	at org.graalvm.nativeimage.builder/com.oracle.svm.core.jdk.JavaNetSubstitutions.unsupported(JavaNetSubstitutions.java:253)
	at org.graalvm.nativeimage.builder/com.oracle.svm.core.jdk.JavaNetSubstitutions.getURLStreamHandler(JavaNetSubstitutions.java:227)
	at [email protected]/java.net.URL.getURLStreamHandler(URL.java:79)
	at [email protected]/java.net.URL.<init>(URL.java:778)
	at [email protected]/java.net.URL.of(URL.java:913)
	at [email protected]/java.net.URI.toURL(URI.java:1172)
	at org.glassfish.jersey.client.internal.HttpUrlConnector._apply(HttpUrlConnector.java:391)
	at org.glassfish.jersey.client.internal.HttpUrlConnector.apply(HttpUrlConnector.java:286)
	... 16 more

@ljnelson
Copy link
Member Author

Adding --enable-url-protocols=http to tests/integration/packaging/mp-2/src/main/resources/META-INF/native-image/io.helidon.tests.integration.packaging/helidon-tests-integration-packaging-mp2/native-image.properties and re-running the test (via main again) now yields:

Caused by: org.graalvm.nativeimage.MissingReflectionRegistrationError: The program tried to reflectively invoke method public java.lang.String(java.lang.String) without it being registered for runtime reflection. Add public java.lang.String(java.lang.String) to the reflection metadata to solve this problem. See https://www.graalvm.org/latest/reference-manual/native-image/metadata/#reflection for help.
	at org.graalvm.nativeimage.builder/com.oracle.svm.core.reflect.MissingReflectionRegistrationUtils.forQueriedOnlyExecutable(MissingReflectionRegistrationUtils.java:72)
	at [email protected]/java.lang.reflect.Constructor.acquireConstructorAccessor(Constructor.java:74)
	at [email protected]/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:499)
	at [email protected]/java.lang.reflect.Constructor.newInstance(Constructor.java:486)
	at org.glassfish.jersey.internal.inject.ParamConverters$StringConstructor$1._fromString(ParamConverters.java:143)
	at org.glassfish.jersey.internal.inject.ParamConverters$AbstractStringReader.fromString(ParamConverters.java:92)
	at org.glassfish.jersey.server.internal.inject.AbstractParamValueExtractor.convert(AbstractParamValueExtractor.java:116)
	at org.glassfish.jersey.server.internal.inject.AbstractParamValueExtractor.fromString(AbstractParamValueExtractor.java:107)
	at org.glassfish.jersey.server.internal.inject.SingleValueExtractor.extract(SingleValueExtractor.java:61)
	at org.glassfish.jersey.server.internal.inject.PathParamValueParamProvider$PathParamValueProvider.apply(PathParamValueParamProvider.java:92)
	at org.glassfish.jersey.server.internal.inject.PathParamValueParamProvider$PathParamValueProvider.apply(PathParamValueParamProvider.java:79)
	at org.glassfish.jersey.server.spi.internal.ParamValueFactoryWithSource.apply(ParamValueFactoryWithSource.java:50)
	at org.glassfish.jersey.server.spi.internal.ParameterValueHelper.getParameterValues(ParameterValueHelper.java:64)
	at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$AbstractMethodParamInvoker.getParamValues(JavaResourceMethodDispatcherProvider.java:109)
	at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176)
	at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:93)
	at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:478)
	at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:400)
	at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81)

@ljnelson
Copy link
Member Author

I've pushed dbf4198 following the instructions in the mp-2 test failure. This change is technically out of scope for this PR but I'd like other Graal-related errors to start showing up.

@ljnelson
Copy link
Member Author

Good (I guess); with the Graal-suggested command line option of --enable-url-protocols=http now the pipeline (not just my machine) is reporting:

org.graalvm.nativeimage.MissingReflectionRegistrationError: The program tried to reflectively invoke method public java.lang.String(java.lang.String) without it being registered for runtime reflection. Add public java.lang.String(java.lang.String) to the reflection metadata to solve this problem. See https://www.graalvm.org/latest/reference-manual/native-image/metadata/#reflection for help.
	at org.graalvm.nativeimage.builder/com.oracle.svm.core.reflect.MissingReflectionRegistrationUtils.forQueriedOnlyExecutable(MissingReflectionRegistrationUtils.java:72)
	at [email protected]/java.lang.reflect.Constructor.acquireConstructorAccessor(Constructor.java:74)
	at [email protected]/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:499)
	at [email protected]/java.lang.reflect.Constructor.newInstance(Constructor.java:486)
	at org.glassfish.jersey.internal.inject.ParamConverters$StringConstructor$1._fromString(ParamConverters.java:143)
	at org.glassfish.jersey.internal.inject.ParamConverters$AbstractStringReader.fromString(ParamConverters.java:92)
	at org.glassfish.jersey.server.internal.inject.AbstractParamValueExtractor.convert(AbstractParamValueExtractor.java:116)
	at org.glassfish.jersey.server.internal.inject.AbstractParamValueExtractor.fromString(AbstractParamValueExtractor.java:107)
	at org.glassfish.jersey.server.internal.inject.SingleValueExtractor.extract(SingleValueExtractor.java:61)
	at org.glassfish.jersey.server.internal.inject.PathParamValueParamProvider$PathParamValueProvider.apply(PathParamValueParamProvider.java:92)
	at org.glassfish.jersey.server.internal.inject.PathParamValueParamProvider$PathParamValueProvider.apply(PathParamValueParamProvider.java:79)
	at org.glassfish.jersey.server.spi.internal.ParamValueFactoryWithSource.apply(ParamValueFactoryWithSource.java:50)
	at org.glassfish.jersey.server.spi.internal.ParameterValueHelper.getParameterValues(ParameterValueHelper.java:64)
	at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$AbstractMethodParamInvoker.getParamValues(JavaResourceMethodDispatcherProvider.java:109)
	at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$TypeOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:219)
	at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:93)
	at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:478)
	at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:400)
	at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81)
	at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:274)
	at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248)
	at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244)
	at org.glassfish.jersey.internal.Errors.process(Errors.java:292)
	at org.glassfish.jersey.internal.Errors.process(Errors.java:274)
	at org.glassfish.jersey.internal.Errors.process(Errors.java:244)
	at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:266)
	at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:253)
	at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:696)
	at io.helidon.microprofile.server.JaxRsService.doHandle(JaxRsService.java:228)
	at io.helidon.microprofile.server.JaxRsService.lambda$handle$2(JaxRsService.java:185)
	at io.helidon.common.context.Contexts.runInContext(Contexts.java:117)
	at io.helidon.microprofile.server.JaxRsService.handle(JaxRsService.java:185)
	at io.helidon.webserver.http.HttpRoutingImpl$RoutingExecutor.doRoute(HttpRoutingImpl.java:165)
	at io.helidon.webserver.http.HttpRoutingImpl$RoutingExecutor.call(HttpRoutingImpl.java:124)
	at io.helidon.webserver.http.HttpRoutingImpl$RoutingExecutor.call(HttpRoutingImpl.java:102)
	at io.helidon.webserver.http.ErrorHandlers.runWithErrorHandling(ErrorHandlers.java:76)
	at io.helidon.webserver.http.Filters$FilterChainImpl.proceed(Filters.java:121)
	at io.helidon.common.context.Contexts.runInContext(Contexts.java:117)
	at io.helidon.webserver.context.ContextRoutingFeature.filter(ContextRoutingFeature.java:50)
	at io.helidon.webserver.http.Filters$FilterChainImpl.proceed(Filters.java:119)
	at io.helidon.webserver.http.Filters.executeFilters(Filters.java:87)
	at io.helidon.webserver.http.Filters.lambda$filter$0(Filters.java:83)
	at io.helidon.webserver.http.ErrorHandlers.runWithErrorHandling(ErrorHandlers.java:76)
	at io.helidon.webserver.http.Filters.filter(Filters.java:83)
	at io.helidon.webserver.http.HttpRoutingImpl.route(HttpRoutingImpl.java:73)
	at io.helidon.webserver.http1.Http1Connection.route(Http1Connection.java:544)
	at io.helidon.webserver.http1.Http1Connection.handle(Http1Connection.java:223)
	at io.helidon.webserver.ConnectionHandler.run(ConnectionHandler.java:169)
	at io.helidon.common.task.InterruptableTask.call(InterruptableTask.java:47)
	at io.helidon.webserver.ThreadPerTaskExecutor$ThreadBoundFuture.run(ThreadPerTaskExecutor.java:239)
	at [email protected]/java.lang.Thread.runWith(Thread.java:1596)
	at [email protected]/java.lang.VirtualThread.run(VirtualThread.java:309)
	at [email protected]/java.lang.VirtualThread$VThreadContinuation$1.run(VirtualThread.java:190)

@ljnelson
Copy link
Member Author

Following the instructions output by GraalVM and with @tvallin's verification and help I've added reflect-config.json to this native image test. The PR itself does not require it, nor is the error related to the PR at all. I am still not sure how the mp-2 test has ever passed if this reflection configuration information is suddenly necessary now, but wasn't before.

@ljnelson
Copy link
Member Author

Partial mystery solved: the Oracle JDBC ojdbc11 artifact, at version 21.15.0.0, has the following in its native-image.properties file it ships with:

  -H:EnableURLProtocols=http \
  -H:EnableURLProtocols=https

These lines are not present in the native-image.properties file shipped with ojdbc11 version 23.6.0.24.10 (nor, I would think, should they be). So that explains that issue.

@ljnelson
Copy link
Member Author

Following #9510, I have removed reflect-config.json from the mp-2 test and have instead introduced a provisional reflect-config.json in jersey/server.

barchetta
barchetta previously approved these changes Nov 20, 2024
…d lets the UCP take care of all validation

Signed-off-by: Laird Nelson <[email protected]>
…-image.properties in an attempt to get the test to pass, or at least to expose other errors

Signed-off-by: Laird Nelson <[email protected]>
…hange not needed by this PR but is necessary for the otherwise unmodified test to pass

Signed-off-by: Laird Nelson <[email protected]>
…ain parameter conversion use cases; adds useful properties to some integration tests to permit native image tracing more easily

Signed-off-by: Laird Nelson <[email protected]>
@ljnelson
Copy link
Member Author

Rebased to accommodate Netty and OCI upgrades in dependencies/pom.xml.

barchetta
barchetta previously approved these changes Nov 20, 2024
…-longer-needed ucp dependency in messaging/connectors/aq.

Signed-off-by: Laird Nelson <[email protected]>
integrations/db/ojdbc/pom.xml Show resolved Hide resolved
@ljnelson ljnelson merged commit 3af58fe into helidon-io:main Nov 20, 2024
46 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
4.x Version 4.x dependencies Pull requests that update a dependency file OCA Verified All contributors have signed the Oracle Contributor Agreement.
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

Jersey does not supply "commonly used" native image reflection metadata
2 participants