Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Native build error on Quarkus 2.2.1.Final when using Opentelemetry #19877

Closed
wabrit opened this issue Sep 2, 2021 · 3 comments · Fixed by #19897
Closed

Native build error on Quarkus 2.2.1.Final when using Opentelemetry #19877

wabrit opened this issue Sep 2, 2021 · 3 comments · Fixed by #19897
Assignees
@geoand
Labels
area/container-image area/kubernetes area/tracing kind/bug Something isn't working
Milestone
2.2.2.Final

Comments

@wabrit
Copy link

wabrit commented Sep 2, 2021 •
edited
Loading

Describe the bug

In migrating a Quarkus app from 2.1.2.Final to 2.2.1.Final an NPE occurs in the io.quarkus.opentelemetry.runtime.QuarkusContextStorage.getVertxContext() method during native build, causing the build to fail.

Expected behavior

The native build should complete successfully.

Actual behavior

The following error occurs:

[INFO] [io.quarkus.deployment.pkg.steps.NativeImageBuildStep] Running Quarkus native-image plugin on GraalVM 21.2.0 Java 11 CE (Java Version 11.0.12+6-jvmci-21.2-b08)
[INFO] [io.quarkus.deployment.pkg.steps.NativeImageBuildRunner] docker run --env LANG=C --rm -v //c/Dev/git_repos/microservices/foobar/target/foobar-0.0.2-SNAPSHOT-native-image-source-jar:/project:z --name build-native-OWTtz quay.io/quarkus/ubi-quarkus-native-image:21.2-java11 -J-Dsun.nio.ch.maxUpdateArraySize=100 -J-Djava.util.logging.manager=org.jboss.logmanager.LogManager -J-Dvertx.logger-delegate-factory-class-name=io.quarkus.vertx.core.runtime.VertxLogDelegateFactory -J-Dvertx.disableDnsResolver=true -J-Dio.netty.leakDetection.level=DISABLED -J-Dio.netty.allocator.maxOrder=3 -J-Duser.language=en -J-Duser.country=GB -J-Dfile.encoding=UTF-8 -H:InitialCollectionPolicy=com.oracle.svm.core.genscavenge.CollectionPolicy\$BySpaceAndTime -H:+JNI -H:+AllowFoldMethods -jar foobar-0.0.2-SNAPSHOT-runner.jar -H:FallbackThreshold=0 -H:+ReportExceptionStackTraces -H:-AddAllCharsets -H:EnableURLProtocols=http,https -H:-UseServiceLoaderFeature -H:+StackTrace -H:-ParseOnce foobar-0.0.2-SNAPSHOT-runner
[foobar-0.0.2-SNAPSHOT-runner:25]    classlist:  17,997.62 ms,  1.18 GB
[foobar-0.0.2-SNAPSHOT-runner:25]        (cap):   1,337.82 ms,  1.18 GB
[foobar-0.0.2-SNAPSHOT-runner:25]        setup:   4,936.58 ms,  1.19 GB
The bundle named: messages, has not been found. If the bundle is part of a module, verify the bundle name is a fully qualified class name. Otherwise verify the bundle path is accessible in the classpath.
17:47:19,065 SEVERE [io.ope.api.GlobalOpenTelemetry] Error automatically configuring OpenTelemetry SDK. OpenTelemetry will not be enabled.: java.lang.NullPointerException
        at io.quarkus.opentelemetry.runtime.QuarkusContextStorage.getVertxContext(QuarkusContextStorage.java:62)
        at io.quarkus.opentelemetry.runtime.QuarkusContextStorage.current(QuarkusContextStorage.java:54)
        at io.opentelemetry.context.Context.current(Context.java:86)
        at io.opentelemetry.sdk.metrics.internal.state.SynchronousMetricStorage.bind(SynchronousMetricStorage.java:66)
        at io.opentelemetry.sdk.metrics.LongCounterSdk.bind(LongCounterSdk.java:52)
        at io.opentelemetry.exporter.otlp.trace.OtlpGrpcSpanExporter.<init>(OtlpGrpcSpanExporter.java:73)
        at io.opentelemetry.exporter.otlp.trace.OtlpGrpcSpanExporterBuilder.build(OtlpGrpcSpanExporterBuilder.java:152)
        at io.opentelemetry.sdk.autoconfigure.SpanExporterConfiguration.configureOtlpSpans(SpanExporterConfiguration.java:113)
        at io.opentelemetry.sdk.autoconfigure.SpanExporterConfiguration.configureExporter(SpanExporterConfiguration.java:43)
        at io.opentelemetry.sdk.autoconfigure.TracerProviderConfiguration.configureTracerProvider(TracerProviderConfiguration.java:51)
        at io.opentelemetry.sdk.autoconfigure.OpenTelemetrySdkAutoConfiguration.initialize(OpenTelemetrySdkAutoConfiguration.java:64)
        at io.opentelemetry.sdk.autoconfigure.OpenTelemetrySdkAutoConfiguration.initialize(OpenTelemetrySdkAutoConfiguration.java:44)
        at io.opentelemetry.sdk.autoconfigure.OpenTelemetrySdkAutoConfiguration.initialize(OpenTelemetrySdkAutoConfiguration.java:33)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
        at io.opentelemetry.api.GlobalOpenTelemetry.maybeAutoConfigure(GlobalOpenTelemetry.java:171)
        at io.opentelemetry.api.GlobalOpenTelemetry.get(GlobalOpenTelemetry.java:57)
        at io.opentelemetry.api.GlobalOpenTelemetry.getPropagators(GlobalOpenTelemetry.java:156)
        at io.quarkus.opentelemetry.runtime.tracing.restclient.ClientTracingFilter.<clinit>(ClientTracingFilter.java:29)
        at java.base/jdk.internal.misc.Unsafe.ensureClassInitialized0(Native Method)
        at java.base/jdk.internal.misc.Unsafe.ensureClassInitialized(Unsafe.java:1042)
        at jdk.unsupported/sun.misc.Unsafe.ensureClassInitialized(Unsafe.java:698)
        at jdk.internal.vm.compiler/org.graalvm.compiler.serviceprovider.GraalUnsafeAccess.ensureClassInitialized(GraalUnsafeAccess.java:77)
        at com.oracle.svm.hosted.classinitialization.ConfigurableClassInitialization.ensureClassInitialized(ConfigurableClassInitialization.java:178)
        at com.oracle.svm.hosted.classinitialization.ConfigurableClassInitialization.computeInitKindAndMaybeInitializeClass(ConfigurableClassInitialization.java:648)
        at com.oracle.svm.hosted.classinitialization.ConfigurableClassInitialization.computeInitKindAndMaybeInitializeClass(ConfigurableClassInitialization.java:131)
        at com.oracle.svm.hosted.classinitialization.ConfigurableClassInitialization.shouldInitializeAtRuntime(ConfigurableClassInitialization.java:159)
        at com.oracle.svm.hosted.SVMHost.isInitialized(SVMHost.java:307)
        at com.oracle.graal.pointsto.meta.AnalysisType.isInitialized(AnalysisType.java:819)
        at jdk.internal.vm.compiler/org.graalvm.compiler.java.BytecodeParser.maybeEagerlyInitialize(BytecodeParser.java:4389)
        at jdk.internal.vm.compiler/org.graalvm.compiler.java.BytecodeParser.genNewInstance(BytecodeParser.java:4587)
        at jdk.internal.vm.compiler/org.graalvm.compiler.java.BytecodeParser.genNewInstance(BytecodeParser.java:4576)
        at jdk.internal.vm.compiler/org.graalvm.compiler.java.BytecodeParser.genNewInstance(BytecodeParser.java:4571)
        at jdk.internal.vm.compiler/org.graalvm.compiler.java.BytecodeParser.processBytecode(BytecodeParser.java:5376)
        at jdk.internal.vm.compiler/org.graalvm.compiler.java.BytecodeParser.iterateBytecodesForBlock(BytecodeParser.java:3431)
        at jdk.internal.vm.compiler/org.graalvm.compiler.java.BytecodeParser.handleBytecodeBlock(BytecodeParser.java:3391)
        at jdk.internal.vm.compiler/org.graalvm.compiler.java.BytecodeParser.processBlock(BytecodeParser.java:3236)
        at jdk.internal.vm.compiler/org.graalvm.compiler.java.BytecodeParser.build(BytecodeParser.java:1122)
        at jdk.internal.vm.compiler/org.graalvm.compiler.java.BytecodeParser.buildRootMethod(BytecodeParser.java:1007)
        at jdk.internal.vm.compiler/org.graalvm.compiler.java.GraphBuilderPhase$Instance.run(GraphBuilderPhase.java:84)
        at com.oracle.svm.hosted.phases.SharedGraphBuilderPhase.run(SharedGraphBuilderPhase.java:81)
        at jdk.internal.vm.compiler/org.graalvm.compiler.phases.Phase.run(Phase.java:49)
        at jdk.internal.vm.compiler/org.graalvm.compiler.phases.BasePhase.apply(BasePhase.java:212)
        at jdk.internal.vm.compiler/org.graalvm.compiler.phases.Phase.apply(Phase.java:42)
        at jdk.internal.vm.compiler/org.graalvm.compiler.phases.Phase.apply(Phase.java:38)
        at com.oracle.graal.pointsto.flow.AnalysisParsedGraph.parseBytecode(AnalysisParsedGraph.java:131)
        at com.oracle.svm.hosted.SVMHost.parseBytecode(SVMHost.java:709)
        at com.oracle.graal.pointsto.meta.AnalysisMethod.ensureGraphParsed(AnalysisMethod.java:605)
        at com.oracle.graal.pointsto.flow.MethodTypeFlowBuilder.parse(MethodTypeFlowBuilder.java:163)
        at com.oracle.graal.pointsto.flow.MethodTypeFlowBuilder.apply(MethodTypeFlowBuilder.java:321)
        at com.oracle.graal.pointsto.flow.MethodTypeFlow.createTypeFlow(MethodTypeFlow.java:293)
        at com.oracle.graal.pointsto.flow.MethodTypeFlow.ensureTypeFlowCreated(MethodTypeFlow.java:282)
        at com.oracle.graal.pointsto.flow.MethodTypeFlow.addContext(MethodTypeFlow.java:103)
        at com.oracle.graal.pointsto.DefaultAnalysisPolicy$DefaultVirtualInvokeTypeFlow.onObservedUpdate(DefaultAnalysisPolicy.java:222)
        at com.oracle.graal.pointsto.flow.TypeFlow$1.run(TypeFlow.java:434)
        at com.oracle.graal.pointsto.util.CompletionExecutor.executeCommand(CompletionExecutor.java:188)
        at com.oracle.graal.pointsto.util.CompletionExecutor.lambda$executeService$0(CompletionExecutor.java:172)
        at java.base/java.util.concurrent.ForkJoinTask$RunnableExecuteAction.exec(ForkJoinTask.java:1426)
        at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290)
        at java.base/java.util.concurrent.ForkJoinPool.awaitQuiescence(ForkJoinPool.java:2984)
        at com.oracle.graal.pointsto.util.CompletionExecutor.complete(CompletionExecutor.java:238)
        at com.oracle.graal.pointsto.BigBang.doTypeflow(BigBang.java:602)
        at com.oracle.graal.pointsto.BigBang.finish(BigBang.java:577)
        at com.oracle.svm.hosted.NativeImageGenerator.runPointsToAnalysis(NativeImageGenerator.java:707)
        at com.oracle.svm.hosted.NativeImageGenerator.doRun(NativeImageGenerator.java:532)
        at com.oracle.svm.hosted.NativeImageGenerator.run(NativeImageGenerator.java:491)
        at com.oracle.svm.hosted.NativeImageGeneratorRunner.buildImage(NativeImageGeneratorRunner.java:380)
        at com.oracle.svm.hosted.NativeImageGeneratorRunner.build(NativeImageGeneratorRunner.java:543)
        at com.oracle.svm.hosted.NativeImageGeneratorRunner.main(NativeImageGeneratorRunner.java:119)
        at com.oracle.svm.hosted.NativeImageGeneratorRunner$JDK9Plus.main(NativeImageGeneratorRunner.java:573)

How to Reproduce?

A reproducer git project may be found here; in summary the following is a minimal set required to reproduce the error:

  • Quarkus version 2.2.1.Final (does not occur on 2.1.2.Final)
  • Dependency quarkus-opentelemetry-exporter-otlp (I needed a few others to be included, perhaps to stimulate the otel SDK into doing something, but this appears to be the main trigger)
  • A config-mapped interface containing a property whose value is only resolved at runtime

e.g. the following interface

@ConfigMapping(prefix = "erewhon")
public interface ErewhonConfig {
  String user();
}

And application.yaml:

quarkus:
  application:
    name: otelnative
  opentelemetry:
    tracer:
      exporter:
        otlp:
          endpoint: http://erewhon.com:4317
  oidc:
    auth-server-url: https://erewhon.com

erewhon:
  user: ${runtime_username}

Its possible some other dependencies are required, but this seems to be the heart of the repro. FWIW these were the full list of dependencies I ended up with in the pom:

io.quarkus:quarkus-arc
io.quarkus:quarkus-kubernetes
io.quarkus:quarkus.config.yaml
io.quarkus:quarkus-container-image-jib
io.quarkus:quarkus-opentelemetry-exporter-otlp
io.quarkus:quarkus-security
io.quarkus:quarkus-oidc
io.quarkus:quarkus-oidc-token-propagation

Output of uname -a or ver

MINGW64_NT-10.0-18363 LAP1453 3.0.7-338.x86_64 2019-04-30 21:52 UTC x86_64 Msys

Output of java -version

openjdk version "11.0.11" 2021-04-20 OpenJDK Runtime Environment GraalVM CE 21.1.0 (build 11.0.11+8-jvmci-21.1-b05) OpenJDK 64-Bit Server VM GraalVM CE 21.1.0 (build 11.0.11+8-jvmci-21.1-b05, mixed mode, sharing)

GraalVM version (if different from Java)

OpenJDK Runtime Environment GraalVM CE 21.2.0 (build 11.0.12+6-jvmci-21.2-b08)

Quarkus version or git rev

2.2.1.Final

Build tool (ie. output of mvnw --version or gradlew --version)

Apache Maven 3.8.1 (05c21c65bdfed0f71a2f2ada8b84da59348c4c5d)

Additional information

No response
@wabrit wabrit added the kind/bug Something isn't working label Sep 2, 2021
@quarkus-bot
Copy link

quarkus-bot bot commented Sep 2, 2021

/cc @geoand, @iocanel, @kenfinnigan

@geoand
Copy link
Contributor

geoand commented Sep 3, 2021

There are two separate issues here.
The main issue about OpenTelemetry I have a fix more.
The other one is related to #19448 (cc @radcortez)

geoand added a commit to geoand/quarkus that referenced this issue Sep 3, 2021
@geoand
Fix use of OpenTelemetry OTLP exporter in native mode
d142b2f 
Essentially the fact that ClientTracingFilter was using a static
field for TextMapPropagator was causing GraalVM to initialize
the entire telemetry infrastructure at build time

Fixes: quarkusio#19877
@geoand geoand mentioned this issue Sep 3, 2021
Merged
@geoand
Copy link
Contributor

geoand commented Sep 3, 2021

#19897 is the fix for the main issue

@geoand geoand self-assigned this Sep 3, 2021
@quarkus-bot quarkus-bot bot added this to the 2.3 - main milestone Sep 3, 2021
kenfinnigan pushed a commit to kenfinnigan/quarkus that referenced this issue Sep 3, 2021
@geoand @kenfinnigan
Fix use of OpenTelemetry OTLP exporter in native mode
849b988 
Essentially the fact that ClientTracingFilter was using a static
field for TextMapPropagator was causing GraalVM to initialize
the entire telemetry infrastructure at build time

Fixes: quarkusio#19877
@gsmet gsmet modified the milestones: 2.3 - main, 2.2.2.Final Sep 6, 2021
gsmet pushed a commit to gsmet/quarkus that referenced this issue Sep 6, 2021
@geoand @gsmet
Fix use of OpenTelemetry OTLP exporter in native mode
0282ecb 
Essentially the fact that ClientTracingFilter was using a static
field for TextMapPropagator was causing GraalVM to initialize
the entire telemetry infrastructure at build time

Fixes: quarkusio#19877
(cherry picked from commit d142b2f)
gsmet pushed a commit to gsmet/quarkus that referenced this issue Sep 6, 2021
@geoand @gsmet
Fix use of OpenTelemetry OTLP exporter in native mode
1f99e82 
Essentially the fact that ClientTracingFilter was using a static
field for TextMapPropagator was causing GraalVM to initialize
the entire telemetry infrastructure at build time

Fixes: quarkusio#19877
(cherry picked from commit d142b2f)
gsmet pushed a commit to gsmet/quarkus that referenced this issue Sep 6, 2021
@geoand @gsmet
Fix use of OpenTelemetry OTLP exporter in native mode
b62d279 
Essentially the fact that ClientTracingFilter was using a static
field for TextMapPropagator was causing GraalVM to initialize
the entire telemetry infrastructure at build time

Fixes: quarkusio#19877
(cherry picked from commit d142b2f)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@geoand geoand

Labels
area/container-image area/kubernetes area/tracing kind/bug Something isn't working
Projects
None yet
Milestone
2.2.2.Final
Development

Successfully merging a pull request may close this issue.

Fix use of OpenTelemetry OTLP exporter in native mode geoand/quarkus
3 participants
@gsmet @wabrit @geoand