Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Applicaiton Insights java agent doesn't work correctly in Quarkus apps (custom attributes or getting SpanId or TraceId) #1980

Closed
guerrerotook opened this issue Nov 25, 2021 · 17 comments · Fixed by #2009

Comments

@guerrerotook
Copy link
Member

Expected behavior

Based on the documentation, I should be able to add custom attributes to the current http request in my app. That doesn't work.

Also, in the documentation it being mentioned that I should be able to get the current SpanId or TraceId but it's always zero.

Actual behavior

Telemetry is being sent to Application Insights but without my custom attributes or I can't get the current SpanId or TraceId.

This is the response of the API where I output these lines of code:

import io.opentelemetry.api.trace.Span;

String traceId = Span.current().getSpanContext().getTraceId();
String spanId = Span.current().getSpanContext().getSpanId();
@Path("/hello")
public class GreetingResource {

    @GET
    @Produces(MediaType.TEXT_PLAIN)
    public String hello() {
        Span.current().setAttribute("mycustomdimension", "myvalue1");
        String traceId = Span.current().getSpanContext().getTraceId();
        String spanId = Span.current().getSpanContext().getSpanId();
        return "Hello RESTEasy TraceId " + traceId + " SpanId " + spanId;
    }
}

This is the output of the API

Hello RESTEasy TraceId 00000000000000000000000000000000 SpanId 0000000000000000

To Reproduce

Create an empty Quarkus app using the RedHat code generator. Add the Application Insights java agent by adding these line of the code in the pom.xml in the section of the build plugin of quarkus maven.

 <plugin>
        <groupId>${quarkus.platform.group-id}</groupId>
        <artifactId>quarkus-maven-plugin</artifactId>
        <version>${quarkus.platform.version}</version>
        <extensions>true</extensions>
        <executions>
          <execution>
            <goals>
              <goal>build</goal>
              <goal>generate-code</goal>
              <goal>generate-code-tests</goal>
            </goals>
          </execution>
        </executions>
         <configuration>
          <jvmArgs>-javaagent:../src/main/resources/applicationinsights-agent-3.2.3.jar</jvmArgs>
        </configuration>
      </plugin>

Sample Application

Here you can find the same application https://github.com/guerrerotook/quarkus-telemetry-demo with the binary for the java agent. The only thing missing is a valid Application Insights key.

System information

Please provide the following information:

  • SDK Version: Java 11
  • OS type and version: Windows 11
  • Application Server type and version (if applicable): Quarkus app
  • Using spring-boot? No
  • Additional relevant libraries (with version, if applicable): No

Logs

2021-11-25 12:28:46.378+01:00 INFO  i.o.javaagent.tooling.VersionLogger - opentelemetry-javaagent - version: null
2021-11-25 12:28:46.444+01:00 DEBUG c.m.a.a.i.common.SystemInformation - Current PID: 7512
2021-11-25 12:28:46.779+01:00 DEBUG c.a.c.i.j.MemberNameConverterImpl - Failed to retrieve MethodHandles used to get naming strategy. Falling back to BeanUtils.
java.lang.ClassNotFoundException: com.fasterxml.jackson.databind.introspect.AccessorNamingStrategy.Provider
	at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:445)
	at io.opentelemetry.javaagent.bootstrap.AgentClassLoader.findClass(AgentClassLoader.java:152)
	at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:587)
	at io.opentelemetry.javaagent.bootstrap.AgentClassLoader.loadClass(AgentClassLoader.java:133)
	at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:520)
	at java.base/java.lang.Class.forName0(Native Method)
	at java.base/java.lang.Class.forName(Class.java:375)
	at com.azure.core.implementation.jackson.MemberNameConverterImpl.<clinit>(MemberNameConverterImpl.java:55)
	at com.azure.core.implementation.jackson.ObjectMapperShim.<init>(ObjectMapperShim.java:139)
	at com.azure.core.implementation.jackson.ObjectMapperShim.createPrettyPrintMapper(ObjectMapperShim.java:113)
	at com.azure.core.http.policy.HttpLoggingPolicy.<clinit>(HttpLoggingPolicy.java:39)
	at com.microsoft.applicationinsights.agent.internal.httpclient.LazyHttpClient.newHttpPipeLine(LazyHttpClient.java:134)
	at com.microsoft.applicationinsights.agent.internal.quickpulse.QuickPulse.initializeSync(QuickPulse.java:72)
	at com.microsoft.applicationinsights.agent.internal.quickpulse.QuickPulse.lambda$initialize$0(QuickPulse.java:53)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
	at java.base/java.lang.Thread.run(Thread.java:833)
2021-11-25 12:28:46.891+01:00 DEBUG reactor.util.Loggers - Using Slf4j logging framework
2021-11-25 12:28:47.798+01:00 INFO  c.m.applicationinsights.agent - ApplicationInsights Java Agent 3.2.3 started successfully (PID 7512)
2021-11-25 12:28:48.662+01:00 DEBUG c.m.a.a.internal.init.AppIdSupplier - appId retrieved: 3bea0ae3-0a7b-4386-a2fa-2c95db473b2d
2021-11-25 12:28:49.484+01:00 DEBUG c.m.a.a.internal.exporter.Exporter - exporting span: SpanData{spanContext=ImmutableSpanContext{traceId=6c563af8d3de288a73528e5c93ddd66b, spanId=d1d34a77cef9c396, traceFlags=01, traceState=ArrayBasedTraceState{entries=[ai-internal-sp, 100]}, remote=false, valid=true}, parentSpanContext=ImmutableSpanContext{traceId=00000000000000000000000000000000, spanId=0000000000000000, traceFlags=00, traceState=ArrayBasedTraceState{entries=[]}, remote=false, valid=false}, resource=Resource{schemaUrl=https://opentelemetry.io/schemas/1.7.0, attributes={host.arch="amd64", host.name="SurfaceBook2MS", os.description="Windows 10 10.0", os.type="windows", process.command_line="C:\Program Files\OpenJDK\jdk-17.0.1;bin;java.exe -javaagent:../src/main/resources/applicationinsights-agent-3.2.3.jar -Dio.quarkus.force-color-support=true -javaagent:C:\Users\luguerre\.m2\repository\io\quarkus\quarkus-class-change-agent\2.2.3.Final\quarkus-class-change-agent-2.2.3.Final.jar -XX:TieredStopAtLevel=1 -agentlib:jdwp=transport=dt_socket,address=localhost:5005,server=y,suspend=n -Djava.util.logging.manager=org.jboss.logmanager.LogManager", process.executable.path="C:\Program Files\OpenJDK\jdk-17.0.1;bin;java.exe", process.pid=7512, process.runtime.description="Oracle Corporation OpenJDK 64-Bit Server VM 17.0.1+12-39", process.runtime.name="OpenJDK Runtime Environment", process.runtime.version="17.0.1+12-39", service.name="unknown_service:java", telemetry.sdk.language="java", telemetry.sdk.name="opentelemetry", telemetry.sdk.version="1.7.0"}}, instrumentationLibraryInfo=InstrumentationLibraryInfo{name=io.opentelemetry.java-util-logging-spans, version=null, schemaUrl=null}, name=JBoss Threads version %s, kind=INTERNAL, startEpochNanos=1637839729481008400, endEpochNanos=1637839729483050900, attributes=AttributesMap{data={thread.name=main, thread.id=1, applicationinsights.internal.log=true, applicationinsights.internal.log_level=INFO, applicationinsights.internal.logger_name=org.jboss.threads}, capacity=128, totalAddedValues=5}, totalAttributeCount=5, events=[], totalRecordedEvents=0, links=[], totalRecordedLinks=0, status=ImmutableStatusData{statusCode=UNSET, description=}, hasEnded=true}
2021-11-25 12:28:52.259+01:00 WARN  muzzleMatcher - Instrumentation skipped, mismatched references were found: jaxrs [class io.opentelemetry.javaagent.instrumentation.jaxrs.v2_0.Resteasy30InstrumentationModule] on QuarkusClassLoader:Quarkus Base Runtime ClassLoader: DEV@29d25e65
2021-11-25 12:28:52.262+01:00 WARN  muzzleMatcher - -- io.opentelemetry.javaagent.instrumentation.jaxrs.v2_0.Resteasy30RequestContextInstrumentation$ContainerRequestContextAdvice:44 Missing class org.jboss.resteasy.core.interception.PostMatchContainerRequestContext
2021-11-25 12:28:52.562+01:00 WARN  muzzleMatcher - Instrumentation skipped, mismatched references were found: jaxrs [class io.opentelemetry.javaagent.instrumentation.jaxrs.v2_0.Resteasy30InstrumentationModule] on QuarkusClassLoader:Quarkus Base Runtime ClassLoader: DEV@29d25e65
2021-11-25 12:28:52.564+01:00 WARN  muzzleMatcher - -- io.opentelemetry.javaagent.instrumentation.jaxrs.v2_0.Resteasy30RequestContextInstrumentation$ContainerRequestContextAdvice:44 Missing class org.jboss.resteasy.core.interception.PostMatchContainerRequestContext
2021-11-25 12:28:52.581+01:00 WARN  muzzleMatcher - Instrumentation skipped, mismatched references were found: jaxrs [class io.opentelemetry.javaagent.instrumentation.jaxrs.v2_0.JerseyInstrumentationModule] on QuarkusClassLoader:Quarkus Base Runtime ClassLoader: DEV@29d25e65
2021-11-25 12:28:52.584+01:00 WARN  muzzleMatcher - -- io.opentelemetry.javaagent.instrumentation.jaxrs.v2_0.JerseyServletContainerInstrumentation$ServiceAdvice:49 Missing class javax.servlet.http.HttpServletRequest
2021-11-25 12:28:52.585+01:00 WARN  muzzleMatcher - -- io.opentelemetry.javaagent.instrumentation.jaxrs.v2_0.JerseySpanName:38 Missing class org.glassfish.jersey.uri.UriTemplate
2021-11-25 12:28:52.585+01:00 WARN  muzzleMatcher - -- io.opentelemetry.javaagent.instrumentation.jaxrs.v2_0.JerseySpanName:34 Missing class org.glassfish.jersey.server.ContainerRequest
2021-11-25 12:28:52.586+01:00 WARN  muzzleMatcher - -- io.opentelemetry.javaagent.instrumentation.jaxrs.v2_0.JerseySpanName:36 Missing class org.glassfish.jersey.server.ExtendedUriInfo
2021-11-25 12:28:52.587+01:00 WARN  muzzleMatcher - Instrumentation skipped, mismatched references were found: jaxrs [class io.opentelemetry.javaagent.instrumentation.jaxrs.v2_0.Resteasy30InstrumentationModule] on QuarkusClassLoader:Quarkus Base Runtime ClassLoader: DEV@29d25e65
2021-11-25 12:28:52.588+01:00 WARN  muzzleMatcher - -- io.opentelemetry.javaagent.instrumentation.jaxrs.v2_0.Resteasy30RequestContextInstrumentation$ContainerRequestContextAdvice:44 Missing class org.jboss.resteasy.core.interception.PostMatchContainerRequestContext
2021-11-25 12:28:52.602+01:00 WARN  muzzleMatcher - Instrumentation skipped, mismatched references were found: jaxrs [class io.opentelemetry.javaagent.instrumentation.jaxrs.v2_0.Resteasy30InstrumentationModule] on QuarkusClassLoader:Quarkus Base Runtime ClassLoader: DEV@29d25e65
2021-11-25 12:28:52.603+01:00 WARN  muzzleMatcher - -- io.opentelemetry.javaagent.instrumentation.jaxrs.v2_0.Resteasy30RequestContextInstrumentation$ContainerRequestContextAdvice:44 Missing class org.jboss.resteasy.core.interception.PostMatchContainerRequestContext
2021-11-25 12:28:52.713+01:00 DEBUG c.m.a.a.internal.exporter.Exporter - exporting span: SpanData{spanContext=ImmutableSpanContext{traceId=93461397fed75d80d9a0f0c10ef1f142, spanId=ecb69e34baba20aa, traceFlags=01, traceState=ArrayBasedTraceState{entries=[ai-internal-sp, 100]}, remote=false, valid=true}, parentSpanContext=ImmutableSpanContext{traceId=00000000000000000000000000000000, spanId=0000000000000000, traceFlags=00, traceState=ArrayBasedTraceState{entries=[]}, remote=false, valid=false}, resource=Resource{schemaUrl=https://opentelemetry.io/schemas/1.7.0, attributes={host.arch="amd64", host.name="SurfaceBook2MS", os.description="Windows 10 10.0", os.type="windows", process.command_line="C:\Program Files\OpenJDK\jdk-17.0.1;bin;java.exe -javaagent:../src/main/resources/applicationinsights-agent-3.2.3.jar -Dio.quarkus.force-color-support=true -javaagent:C:\Users\luguerre\.m2\repository\io\quarkus\quarkus-class-change-agent\2.2.3.Final\quarkus-class-change-agent-2.2.3.Final.jar -XX:TieredStopAtLevel=1 -agentlib:jdwp=transport=dt_socket,address=localhost:5005,server=y,suspend=n -Djava.util.logging.manager=org.jboss.logmanager.LogManager", process.executable.path="C:\Program Files\OpenJDK\jdk-17.0.1;bin;java.exe", process.pid=7512, process.runtime.description="Oracle Corporation OpenJDK 64-Bit Server VM 17.0.1+12-39", process.runtime.name="OpenJDK Runtime Environment", process.runtime.version="17.0.1+12-39", service.name="unknown_service:java", telemetry.sdk.language="java", telemetry.sdk.name="opentelemetry", telemetry.sdk.version="1.7.0"}}, instrumentationLibraryInfo=InstrumentationLibraryInfo{name=io.opentelemetry.java-util-logging-spans, version=null, schemaUrl=null}, name=JBoss Threads version %s, kind=INTERNAL, startEpochNanos=1637839732713815800, endEpochNanos=1637839732713833100, attributes=AttributesMap{data={thread.name=Quarkus Main Thread, thread.id=99, applicationinsights.internal.log=true, applicationinsights.internal.log_level=INFO, applicationinsights.internal.logger_name=org.jboss.threads}, capacity=128, totalAddedValues=5}, totalAttributeCount=5, events=[], totalRecordedEvents=0, links=[], totalRecordedLinks=0, status=ImmutableStatusData{statusCode=UNSET, description=}, hasEnded=true}
2021-11-25 12:28:53.574+01:00 DEBUG c.m.a.a.internal.exporter.Exporter - exporting span: SpanData{spanContext=ImmutableSpanContext{traceId=938d7da52f10f32b90a61022f6c938ca, spanId=2aa4f5fffac71ad4, traceFlags=01, traceState=ArrayBasedTraceState{entries=[ai-internal-sp, 100]}, remote=false, valid=true}, parentSpanContext=ImmutableSpanContext{traceId=00000000000000000000000000000000, spanId=0000000000000000, traceFlags=00, traceState=ArrayBasedTraceState{entries=[]}, remote=false, valid=false}, resource=Resource{schemaUrl=https://opentelemetry.io/schemas/1.7.0, attributes={host.arch="amd64", host.name="SurfaceBook2MS", os.description="Windows 10 10.0", os.type="windows", process.command_line="C:\Program Files\OpenJDK\jdk-17.0.1;bin;java.exe -javaagent:../src/main/resources/applicationinsights-agent-3.2.3.jar -Dio.quarkus.force-color-support=true -javaagent:C:\Users\luguerre\.m2\repository\io\quarkus\quarkus-class-change-agent\2.2.3.Final\quarkus-class-change-agent-2.2.3.Final.jar -XX:TieredStopAtLevel=1 -agentlib:jdwp=transport=dt_socket,address=localhost:5005,server=y,suspend=n -Djava.util.logging.manager=org.jboss.logmanager.LogManager", process.executable.path="C:\Program Files\OpenJDK\jdk-17.0.1;bin;java.exe", process.pid=7512, process.runtime.description="Oracle Corporation OpenJDK 64-Bit Server VM 17.0.1+12-39", process.runtime.name="OpenJDK Runtime Environment", process.runtime.version="17.0.1+12-39", service.name="unknown_service:java", telemetry.sdk.language="java", telemetry.sdk.name="opentelemetry", telemetry.sdk.version="1.7.0"}}, instrumentationLibraryInfo=InstrumentationLibraryInfo{name=io.opentelemetry.java-util-logging-spans, version=null, schemaUrl=null}, name=%s %s %s (powered by Quarkus %s) started in %ss. %s, kind=INTERNAL, startEpochNanos=1637839733574630100, endEpochNanos=1637839733574640000, attributes=AttributesMap{data={thread.name=Quarkus Main Thread, thread.id=99, applicationinsights.internal.log=true, applicationinsights.internal.log_level=INFO, applicationinsights.internal.logger_name=io.quarkus}, capacity=128, totalAddedValues=5}, totalAttributeCount=5, events=[], totalRecordedEvents=0, links=[], totalRecordedLinks=0, status=ImmutableStatusData{statusCode=UNSET, description=}, hasEnded=true}
2021-11-25 12:28:53.582+01:00 DEBUG c.m.a.a.internal.exporter.Exporter - exporting span: SpanData{spanContext=ImmutableSpanContext{traceId=b60388b2758cea7a29b4a766b97c1303, spanId=d081b4628ed530c8, traceFlags=01, traceState=ArrayBasedTraceState{entries=[ai-internal-sp, 100]}, remote=false, valid=true}, parentSpanContext=ImmutableSpanContext{traceId=00000000000000000000000000000000, spanId=0000000000000000, traceFlags=00, traceState=ArrayBasedTraceState{entries=[]}, remote=false, valid=false}, resource=Resource{schemaUrl=https://opentelemetry.io/schemas/1.7.0, attributes={host.arch="amd64", host.name="SurfaceBook2MS", os.description="Windows 10 10.0", os.type="windows", process.command_line="C:\Program Files\OpenJDK\jdk-17.0.1;bin;java.exe -javaagent:../src/main/resources/applicationinsights-agent-3.2.3.jar -Dio.quarkus.force-color-support=true -javaagent:C:\Users\luguerre\.m2\repository\io\quarkus\quarkus-class-change-agent\2.2.3.Final\quarkus-class-change-agent-2.2.3.Final.jar -XX:TieredStopAtLevel=1 -agentlib:jdwp=transport=dt_socket,address=localhost:5005,server=y,suspend=n -Djava.util.logging.manager=org.jboss.logmanager.LogManager", process.executable.path="C:\Program Files\OpenJDK\jdk-17.0.1;bin;java.exe", process.pid=7512, process.runtime.description="Oracle Corporation OpenJDK 64-Bit Server VM 17.0.1+12-39", process.runtime.name="OpenJDK Runtime Environment", process.runtime.version="17.0.1+12-39", service.name="unknown_service:java", telemetry.sdk.language="java", telemetry.sdk.name="opentelemetry", telemetry.sdk.version="1.7.0"}}, instrumentationLibraryInfo=InstrumentationLibraryInfo{name=io.opentelemetry.java-util-logging-spans, version=null, schemaUrl=null}, name=Profile %s activated. %s, kind=INTERNAL, startEpochNanos=1637839733582632100, endEpochNanos=1637839733582642800, attributes=AttributesMap{data={thread.name=Quarkus Main Thread, thread.id=99, applicationinsights.internal.log=true, applicationinsights.internal.log_level=INFO, applicationinsights.internal.logger_name=io.quarkus}, capacity=128, totalAddedValues=5}, totalAttributeCount=5, events=[], totalRecordedEvents=0, links=[], totalRecordedLinks=0, status=ImmutableStatusData{statusCode=UNSET, description=}, hasEnded=true}
2021-11-25 12:28:53.584+01:00 DEBUG c.m.a.a.internal.exporter.Exporter - exporting span: SpanData{spanContext=ImmutableSpanContext{traceId=d4ea3c7920c51f697d4fe7614a80b064, spanId=45cc6e060d88604a, traceFlags=01, traceState=ArrayBasedTraceState{entries=[ai-internal-sp, 100]}, remote=false, valid=true}, parentSpanContext=ImmutableSpanContext{traceId=00000000000000000000000000000000, spanId=0000000000000000, traceFlags=00, traceState=ArrayBasedTraceState{entries=[]}, remote=false, valid=false}, resource=Resource{schemaUrl=https://opentelemetry.io/schemas/1.7.0, attributes={host.arch="amd64", host.name="SurfaceBook2MS", os.description="Windows 10 10.0", os.type="windows", process.command_line="C:\Program Files\OpenJDK\jdk-17.0.1;bin;java.exe -javaagent:../src/main/resources/applicationinsights-agent-3.2.3.jar -Dio.quarkus.force-color-support=true -javaagent:C:\Users\luguerre\.m2\repository\io\quarkus\quarkus-class-change-agent\2.2.3.Final\quarkus-class-change-agent-2.2.3.Final.jar -XX:TieredStopAtLevel=1 -agentlib:jdwp=transport=dt_socket,address=localhost:5005,server=y,suspend=n -Djava.util.logging.manager=org.jboss.logmanager.LogManager", process.executable.path="C:\Program Files\OpenJDK\jdk-17.0.1;bin;java.exe", process.pid=7512, process.runtime.description="Oracle Corporation OpenJDK 64-Bit Server VM 17.0.1+12-39", process.runtime.name="OpenJDK Runtime Environment", process.runtime.version="17.0.1+12-39", service.name="unknown_service:java", telemetry.sdk.language="java", telemetry.sdk.name="opentelemetry", telemetry.sdk.version="1.7.0"}}, instrumentationLibraryInfo=InstrumentationLibraryInfo{name=io.opentelemetry.java-util-logging-spans, version=null, schemaUrl=null}, name=Installed features: [%s], kind=INTERNAL, startEpochNanos=1637839733584633300, endEpochNanos=1637839733584649800, attributes=AttributesMap{data={thread.name=Quarkus Main Thread, thread.id=99, applicationinsights.internal.log=true, applicationinsights.internal.log_level=INFO, applicationinsights.internal.logger_name=io.quarkus}, capacity=128, totalAddedValues=5}, totalAttributeCount=5, events=[], totalRecordedEvents=0, links=[], totalRecordedLinks=0, status=ImmutableStatusData{statusCode=UNSET, description=}, hasEnded=true}
2021-11-25 12:28:56.718+01:00 DEBUG c.m.a.a.internal.exporter.Exporter - exporting span: SpanData{spanContext=ImmutableSpanContext{traceId=a58a5e54eee45fd8e51bcdfc0382db43, spanId=555a48460d54abe5, traceFlags=01, traceState=ArrayBasedTraceState{entries=[ai-internal-sp, 100]}, remote=false, valid=true}, parentSpanContext=ImmutableSpanContext{traceId=00000000000000000000000000000000, spanId=0000000000000000, traceFlags=00, traceState=ArrayBasedTraceState{entries=[]}, remote=false, valid=false}, resource=Resource{schemaUrl=https://opentelemetry.io/schemas/1.7.0, attributes={host.arch="amd64", host.name="SurfaceBook2MS", os.description="Windows 10 10.0", os.type="windows", process.command_line="C:\Program Files\OpenJDK\jdk-17.0.1;bin;java.exe -javaagent:../src/main/resources/applicationinsights-agent-3.2.3.jar -Dio.quarkus.force-color-support=true -javaagent:C:\Users\luguerre\.m2\repository\io\quarkus\quarkus-class-change-agent\2.2.3.Final\quarkus-class-change-agent-2.2.3.Final.jar -XX:TieredStopAtLevel=1 -agentlib:jdwp=transport=dt_socket,address=localhost:5005,server=y,suspend=n -Djava.util.logging.manager=org.jboss.logmanager.LogManager", process.executable.path="C:\Program Files\OpenJDK\jdk-17.0.1;bin;java.exe", process.pid=7512, process.runtime.description="Oracle Corporation OpenJDK 64-Bit Server VM 17.0.1+12-39", process.runtime.name="OpenJDK Runtime Environment", process.runtime.version="17.0.1+12-39", service.name="unknown_service:java", telemetry.sdk.language="java", telemetry.sdk.name="opentelemetry", telemetry.sdk.version="1.7.0"}}, instrumentationLibraryInfo=InstrumentationLibraryInfo{name=io.opentelemetry.java-util-logging-spans, version=null, schemaUrl=null}, name=Files changed but restart not needed - notified extensions in: %ss , kind=INTERNAL, startEpochNanos=1637839736718696200, endEpochNanos=1637839736718707200, attributes=AttributesMap{data={thread.name=vert.x-worker-thread-0, thread.id=124, applicationinsights.internal.log=true, applicationinsights.internal.log_level=INFO, applicationinsights.internal.logger_name=io.quarkus.deployment.dev.RuntimeUpdatesProcessor}, capacity=128, totalAddedValues=5}, totalAttributeCount=5, events=[], totalRecordedEvents=0, links=[], totalRecordedLinks=0, status=ImmutableStatusData{statusCode=UNSET, description=}, hasEnded=true}
2021-11-25 12:28:56.814+01:00 WARN  muzzleMatcher - Instrumentation skipped, mismatched references were found: jaxrs [class io.opentelemetry.javaagent.instrumentation.jaxrs.v2_0.JerseyInstrumentationModule] on QuarkusClassLoader:Quarkus Base Runtime ClassLoader: DEV@29d25e65
2021-11-25 12:28:56.816+01:00 WARN  muzzleMatcher - -- io.opentelemetry.javaagent.instrumentation.jaxrs.v2_0.JerseyServletContainerInstrumentation$ServiceAdvice:49 Missing class javax.servlet.http.HttpServletRequest
2021-11-25 12:28:56.817+01:00 WARN  muzzleMatcher - -- io.opentelemetry.javaagent.instrumentation.jaxrs.v2_0.JerseySpanName:38 Missing class org.glassfish.jersey.uri.UriTemplate
2021-11-25 12:28:56.818+01:00 WARN  muzzleMatcher - -- io.opentelemetry.javaagent.instrumentation.jaxrs.v2_0.JerseySpanName:34 Missing class org.glassfish.jersey.server.ContainerRequest
2021-11-25 12:28:56.818+01:00 WARN  muzzleMatcher - -- io.opentelemetry.javaagent.instrumentation.jaxrs.v2_0.JerseySpanName:36 Missing class org.glassfish.jersey.server.ExtendedUriInfo
2021-11-25 12:28:56.819+01:00 WARN  muzzleMatcher - Instrumentation skipped, mismatched references were found: jaxrs [class io.opentelemetry.javaagent.instrumentation.jaxrs.v2_0.Resteasy30InstrumentationModule] on QuarkusClassLoader:Quarkus Base Runtime ClassLoader: DEV@29d25e65
2021-11-25 12:28:56.820+01:00 WARN  muzzleMatcher - -- io.opentelemetry.javaagent.instrumentation.jaxrs.v2_0.Resteasy30RequestContextInstrumentation$ContainerRequestContextAdvice:44 Missing class org.jboss.resteasy.core.interception.PostMatchContainerRequestContext
2021-11-25 12:28:56.828+01:00 WARN  muzzleMatcher - Instrumentation skipped, mismatched references were found: jaxrs [class io.opentelemetry.javaagent.instrumentation.jaxrs.v2_0.JerseyInstrumentationModule] on QuarkusClassLoader:Quarkus Base Runtime ClassLoader: DEV@29d25e65
2021-11-25 12:28:56.829+01:00 WARN  muzzleMatcher - -- io.opentelemetry.javaagent.instrumentation.jaxrs.v2_0.JerseyServletContainerInstrumentation$ServiceAdvice:49 Missing class javax.servlet.http.HttpServletRequest
2021-11-25 12:28:56.829+01:00 WARN  muzzleMatcher - -- io.opentelemetry.javaagent.instrumentation.jaxrs.v2_0.JerseySpanName:38 Missing class org.glassfish.jersey.uri.UriTemplate
2021-11-25 12:28:56.830+01:00 WARN  muzzleMatcher - -- io.opentelemetry.javaagent.instrumentation.jaxrs.v2_0.JerseySpanName:34 Missing class org.glassfish.jersey.server.ContainerRequest
2021-11-25 12:28:56.831+01:00 WARN  muzzleMatcher - -- io.opentelemetry.javaagent.instrumentation.jaxrs.v2_0.JerseySpanName:36 Missing class org.glassfish.jersey.server.ExtendedUriInfo
2021-11-25 12:28:56.831+01:00 WARN  muzzleMatcher - Instrumentation skipped, mismatched references were found: jaxrs [class io.opentelemetry.javaagent.instrumentation.jaxrs.v2_0.Resteasy30InstrumentationModule] on QuarkusClassLoader:Quarkus Base Runtime ClassLoader: DEV@29d25e65
2021-11-25 12:28:56.833+01:00 WARN  muzzleMatcher - -- io.opentelemetry.javaagent.instrumentation.jaxrs.v2_0.Resteasy30RequestContextInstrumentation$ContainerRequestContextAdvice:44 Missing class org.jboss.resteasy.core.interception.PostMatchContainerRequestContext
2021-11-25 12:28:56.859+01:00 WARN  muzzleMatcher - Instrumentation skipped, mismatched references were found: jaxrs [class io.opentelemetry.javaagent.instrumentation.jaxrs.v2_0.JerseyInstrumentationModule] on QuarkusClassLoader:Quarkus Base Runtime ClassLoader: DEV@29d25e65
2021-11-25 12:28:56.862+01:00 WARN  muzzleMatcher - -- io.opentelemetry.javaagent.instrumentation.jaxrs.v2_0.JerseyServletContainerInstrumentation$ServiceAdvice:49 Missing class javax.servlet.http.HttpServletRequest
2021-11-25 12:28:56.863+01:00 WARN  muzzleMatcher - -- io.opentelemetry.javaagent.instrumentation.jaxrs.v2_0.JerseySpanName:38 Missing class org.glassfish.jersey.uri.UriTemplate
2021-11-25 12:28:56.864+01:00 WARN  muzzleMatcher - -- io.opentelemetry.javaagent.instrumentation.jaxrs.v2_0.JerseySpanName:34 Missing class org.glassfish.jersey.server.ContainerRequest
2021-11-25 12:28:56.864+01:00 WARN  muzzleMatcher - -- io.opentelemetry.javaagent.instrumentation.jaxrs.v2_0.JerseySpanName:36 Missing class org.glassfish.jersey.server.ExtendedUriInfo
2021-11-25 12:28:56.865+01:00 WARN  muzzleMatcher - Instrumentation skipped, mismatched references were found: jaxrs [class io.opentelemetry.javaagent.instrumentation.jaxrs.v2_0.Resteasy30InstrumentationModule] on QuarkusClassLoader:Quarkus Base Runtime ClassLoader: DEV@29d25e65
2021-11-25 12:28:56.867+01:00 WARN  muzzleMatcher - -- io.opentelemetry.javaagent.instrumentation.jaxrs.v2_0.Resteasy30RequestContextInstrumentation$ContainerRequestContextAdvice:44 Missing class org.jboss.resteasy.core.interception.PostMatchContainerRequestContext
2021-11-25 12:28:57.112+01:00 WARN  muzzleMatcher - Instrumentation skipped, mismatched references were found: jaxrs [class io.opentelemetry.javaagent.instrumentation.jaxrs.v2_0.JerseyInstrumentationModule] on QuarkusClassLoader:Quarkus Base Runtime ClassLoader: DEV@29d25e65
2021-11-25 12:28:57.114+01:00 WARN  muzzleMatcher - -- io.opentelemetry.javaagent.instrumentation.jaxrs.v2_0.JerseyServletContainerInstrumentation$ServiceAdvice:49 Missing class javax.servlet.http.HttpServletRequest
2021-11-25 12:28:57.115+01:00 WARN  muzzleMatcher - -- io.opentelemetry.javaagent.instrumentation.jaxrs.v2_0.JerseySpanName:38 Missing class org.glassfish.jersey.uri.UriTemplate
2021-11-25 12:28:57.115+01:00 WARN  muzzleMatcher - -- io.opentelemetry.javaagent.instrumentation.jaxrs.v2_0.JerseySpanName:34 Missing class org.glassfish.jersey.server.ContainerRequest
2021-11-25 12:28:57.116+01:00 WARN  muzzleMatcher - -- io.opentelemetry.javaagent.instrumentation.jaxrs.v2_0.JerseySpanName:36 Missing class org.glassfish.jersey.server.ExtendedUriInfo
2021-11-25 12:28:57.117+01:00 WARN  muzzleMatcher - Instrumentation skipped, mismatched references were found: jaxrs [class io.opentelemetry.javaagent.instrumentation.jaxrs.v2_0.Resteasy30InstrumentationModule] on QuarkusClassLoader:Quarkus Base Runtime ClassLoader: DEV@29d25e65
2021-11-25 12:28:57.118+01:00 WARN  muzzleMatcher - -- io.opentelemetry.javaagent.instrumentation.jaxrs.v2_0.Resteasy30RequestContextInstrumentation$ContainerRequestContextAdvice:44 Missing class org.jboss.resteasy.core.interception.PostMatchContainerRequestContext
2021-11-25 12:28:57.183+01:00 DEBUG c.m.a.a.internal.exporter.Exporter - exporting span: SpanData{spanContext=ImmutableSpanContext{traceId=e1088f67818a6c292e7c456719bb61b0, spanId=42487aaa1ab1ad72, traceFlags=01, traceState=ArrayBasedTraceState{entries=[ai-internal-sp, 100]}, remote=false, valid=true}, parentSpanContext=ImmutableSpanContext{traceId=00000000000000000000000000000000, spanId=0000000000000000, traceFlags=00, traceState=ArrayBasedTraceState{entries=[]}, remote=false, valid=false}, resource=Resource{schemaUrl=https://opentelemetry.io/schemas/1.7.0, attributes={host.arch="amd64", host.name="SurfaceBook2MS", os.description="Windows 10 10.0", os.type="windows", process.command_line="C:\Program Files\OpenJDK\jdk-17.0.1;bin;java.exe -javaagent:../src/main/resources/applicationinsights-agent-3.2.3.jar -Dio.quarkus.force-color-support=true -javaagent:C:\Users\luguerre\.m2\repository\io\quarkus\quarkus-class-change-agent\2.2.3.Final\quarkus-class-change-agent-2.2.3.Final.jar -XX:TieredStopAtLevel=1 -agentlib:jdwp=transport=dt_socket,address=localhost:5005,server=y,suspend=n -Djava.util.logging.manager=org.jboss.logmanager.LogManager", process.executable.path="C:\Program Files\OpenJDK\jdk-17.0.1;bin;java.exe", process.pid=7512, process.runtime.description="Oracle Corporation OpenJDK 64-Bit Server VM 17.0.1+12-39", process.runtime.name="OpenJDK Runtime Environment", process.runtime.version="17.0.1+12-39", service.name="unknown_service:java", telemetry.sdk.language="java", telemetry.sdk.name="opentelemetry", telemetry.sdk.version="1.7.0"}}, instrumentationLibraryInfo=InstrumentationLibraryInfo{name=io.opentelemetry.netty-4.1, version=null, schemaUrl=null}, name=HTTP GET, kind=SERVER, startEpochNanos=1637839736619696800, endEpochNanos=1637839737183283900, attributes=AttributesMap{data={net.peer.ip=127.0.0.1, thread.name=vert.x-eventloop-thread-0, http.method=GET, http.scheme=http, http.host=localhost:8080, thread.id=105, http.status_code=200, http.target=/hello, http.user_agent=Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:94.0) Gecko/20100101 Firefox/94.0, http.flavor=1.1, net.peer.port=57491}, capacity=128, totalAddedValues=11}, totalAttributeCount=11, events=[], totalRecordedEvents=0, links=[], totalRecordedLinks=0, status=ImmutableStatusData{statusCode=UNSET, description=}, hasEnded=true}
2021-11-25 12:29:16.742+01:00 DEBUG c.m.a.a.internal.exporter.Exporter - exporting span: SpanData{spanContext=ImmutableSpanContext{traceId=b9efb7d0874e4bc46ea5e9fd780d0c46, spanId=3ce73f277d8ce0cb, traceFlags=01, traceState=ArrayBasedTraceState{entries=[ai-internal-sp, 100]}, remote=false, valid=true}, parentSpanContext=ImmutableSpanContext{traceId=00000000000000000000000000000000, spanId=0000000000000000, traceFlags=00, traceState=ArrayBasedTraceState{entries=[]}, remote=false, valid=false}, resource=Resource{schemaUrl=https://opentelemetry.io/schemas/1.7.0, attributes={host.arch="amd64", host.name="SurfaceBook2MS", os.description="Windows 10 10.0", os.type="windows", process.command_line="C:\Program Files\OpenJDK\jdk-17.0.1;bin;java.exe -javaagent:../src/main/resources/applicationinsights-agent-3.2.3.jar -Dio.quarkus.force-color-support=true -javaagent:C:\Users\luguerre\.m2\repository\io\quarkus\quarkus-class-change-agent\2.2.3.Final\quarkus-class-change-agent-2.2.3.Final.jar -XX:TieredStopAtLevel=1 -agentlib:jdwp=transport=dt_socket,address=localhost:5005,server=y,suspend=n -Djava.util.logging.manager=org.jboss.logmanager.LogManager", process.executable.path="C:\Program Files\OpenJDK\jdk-17.0.1;bin;java.exe", process.pid=7512, process.runtime.description="Oracle Corporation OpenJDK 64-Bit Server VM 17.0.1+12-39", process.runtime.name="OpenJDK Runtime Environment", process.runtime.version="17.0.1+12-39", service.name="unknown_service:java", telemetry.sdk.language="java", telemetry.sdk.name="opentelemetry", telemetry.sdk.version="1.7.0"}}, instrumentationLibraryInfo=InstrumentationLibraryInfo{name=io.opentelemetry.java-util-logging-spans, version=null, schemaUrl=null}, name=%s, line %d in %s, kind=INTERNAL, startEpochNanos=1637839756742956600, endEpochNanos=1637839756742967300, attributes=AttributesMap{data={thread.name=vert.x-worker-thread-0, thread.id=124, applicationinsights.internal.log=true, applicationinsights.internal.log_level=WARN, applicationinsights.internal.logger_name=io.quarkus.deployment.dev.JavaCompilationProvider}, capacity=128, totalAddedValues=5}, totalAttributeCount=5, events=[], totalRecordedEvents=0, links=[], totalRecordedLinks=0, status=ImmutableStatusData{statusCode=UNSET, description=}, hasEnded=true}
2021-11-25 12:29:16.756+01:00 DEBUG c.m.a.a.internal.exporter.Exporter - exporting span: SpanData{spanContext=ImmutableSpanContext{traceId=731e5e249a67f34f9d1cd81bbe7b8d9c, spanId=8e6a7937830820c9, traceFlags=01, traceState=ArrayBasedTraceState{entries=[ai-internal-sp, 100]}, remote=false, valid=true}, parentSpanContext=ImmutableSpanContext{traceId=00000000000000000000000000000000, spanId=0000000000000000, traceFlags=00, traceState=ArrayBasedTraceState{entries=[]}, remote=false, valid=false}, resource=Resource{schemaUrl=https://opentelemetry.io/schemas/1.7.0, attributes={host.arch="amd64", host.name="SurfaceBook2MS", os.description="Windows 10 10.0", os.type="windows", process.command_line="C:\Program Files\OpenJDK\jdk-17.0.1;bin;java.exe -javaagent:../src/main/resources/applicationinsights-agent-3.2.3.jar -Dio.quarkus.force-color-support=true -javaagent:C:\Users\luguerre\.m2\repository\io\quarkus\quarkus-class-change-agent\2.2.3.Final\quarkus-class-change-agent-2.2.3.Final.jar -XX:TieredStopAtLevel=1 -agentlib:jdwp=transport=dt_socket,address=localhost:5005,server=y,suspend=n -Djava.util.logging.manager=org.jboss.logmanager.LogManager", process.executable.path="C:\Program Files\OpenJDK\jdk-17.0.1;bin;java.exe", process.pid=7512, process.runtime.description="Oracle Corporation OpenJDK 64-Bit Server VM 17.0.1+12-39", process.runtime.name="OpenJDK Runtime Environment", process.runtime.version="17.0.1+12-39", service.name="unknown_service:java", telemetry.sdk.language="java", telemetry.sdk.name="opentelemetry", telemetry.sdk.version="1.7.0"}}, instrumentationLibraryInfo=InstrumentationLibraryInfo{name=io.opentelemetry.java-util-logging-spans, version=null, schemaUrl=null}, name=Restarting quarkus due to changes in %s., kind=INTERNAL, startEpochNanos=1637839756756920000, endEpochNanos=1637839756756930600, attributes=AttributesMap{data={thread.name=vert.x-worker-thread-0, thread.id=124, applicationinsights.internal.log=true, applicationinsights.internal.log_level=INFO, applicationinsights.internal.logger_name=io.quarkus.deployment.dev.RuntimeUpdatesProcessor}, capacity=128, totalAddedValues=5}, totalAttributeCount=5, events=[], totalRecordedEvents=0, links=[], totalRecordedLinks=0, status=ImmutableStatusData{statusCode=UNSET, description=}, hasEnded=true}
2021-11-25 12:29:16.775+01:00 DEBUG c.m.a.a.internal.exporter.Exporter - exporting span: SpanData{spanContext=ImmutableSpanContext{traceId=8bfca951c281fe6ba9d58f9486ad783d, spanId=1f18d68dcaf0e423, traceFlags=01, traceState=ArrayBasedTraceState{entries=[ai-internal-sp, 100]}, remote=false, valid=true}, parentSpanContext=ImmutableSpanContext{traceId=00000000000000000000000000000000, spanId=0000000000000000, traceFlags=00, traceState=ArrayBasedTraceState{entries=[]}, remote=false, valid=false}, resource=Resource{schemaUrl=https://opentelemetry.io/schemas/1.7.0, attributes={host.arch="amd64", host.name="SurfaceBook2MS", os.description="Windows 10 10.0", os.type="windows", process.command_line="C:\Program Files\OpenJDK\jdk-17.0.1;bin;java.exe -javaagent:../src/main/resources/applicationinsights-agent-3.2.3.jar -Dio.quarkus.force-color-support=true -javaagent:C:\Users\luguerre\.m2\repository\io\quarkus\quarkus-class-change-agent\2.2.3.Final\quarkus-class-change-agent-2.2.3.Final.jar -XX:TieredStopAtLevel=1 -agentlib:jdwp=transport=dt_socket,address=localhost:5005,server=y,suspend=n -Djava.util.logging.manager=org.jboss.logmanager.LogManager", process.executable.path="C:\Program Files\OpenJDK\jdk-17.0.1;bin;java.exe", process.pid=7512, process.runtime.description="Oracle Corporation OpenJDK 64-Bit Server VM 17.0.1+12-39", process.runtime.name="OpenJDK Runtime Environment", process.runtime.version="17.0.1+12-39", service.name="unknown_service:java", telemetry.sdk.language="java", telemetry.sdk.name="opentelemetry", telemetry.sdk.version="1.7.0"}}, instrumentationLibraryInfo=InstrumentationLibraryInfo{name=io.opentelemetry.java-util-logging-spans, version=null, schemaUrl=null}, name=%s stopped in %ss, kind=INTERNAL, startEpochNanos=1637839756775913000, endEpochNanos=1637839756775931400, attributes=AttributesMap{data={thread.name=Quarkus Main Thread, thread.id=99, applicationinsights.internal.log=true, applicationinsights.internal.log_level=INFO, applicationinsights.internal.logger_name=io.quarkus}, capacity=128, totalAddedValues=5}, totalAttributeCount=5, events=[], totalRecordedEvents=0, links=[], totalRecordedLinks=0, status=ImmutableStatusData{statusCode=UNSET, description=}, hasEnded=true}
2021-11-25 12:29:17.598+01:00 DEBUG c.m.a.a.internal.exporter.Exporter - exporting span: SpanData{spanContext=ImmutableSpanContext{traceId=bd7f2dbdce75926290222ca7e3990241, spanId=04962fc00fd1c1fb, traceFlags=01, traceState=ArrayBasedTraceState{entries=[ai-internal-sp, 100]}, remote=false, valid=true}, parentSpanContext=ImmutableSpanContext{traceId=00000000000000000000000000000000, spanId=0000000000000000, traceFlags=00, traceState=ArrayBasedTraceState{entries=[]}, remote=false, valid=false}, resource=Resource{schemaUrl=https://opentelemetry.io/schemas/1.7.0, attributes={host.arch="amd64", host.name="SurfaceBook2MS", os.description="Windows 10 10.0", os.type="windows", process.command_line="C:\Program Files\OpenJDK\jdk-17.0.1;bin;java.exe -javaagent:../src/main/resources/applicationinsights-agent-3.2.3.jar -Dio.quarkus.force-color-support=true -javaagent:C:\Users\luguerre\.m2\repository\io\quarkus\quarkus-class-change-agent\2.2.3.Final\quarkus-class-change-agent-2.2.3.Final.jar -XX:TieredStopAtLevel=1 -agentlib:jdwp=transport=dt_socket,address=localhost:5005,server=y,suspend=n -Djava.util.logging.manager=org.jboss.logmanager.LogManager", process.executable.path="C:\Program Files\OpenJDK\jdk-17.0.1;bin;java.exe", process.pid=7512, process.runtime.description="Oracle Corporation OpenJDK 64-Bit Server VM 17.0.1+12-39", process.runtime.name="OpenJDK Runtime Environment", process.runtime.version="17.0.1+12-39", service.name="unknown_service:java", telemetry.sdk.language="java", telemetry.sdk.name="opentelemetry", telemetry.sdk.version="1.7.0"}}, instrumentationLibraryInfo=InstrumentationLibraryInfo{name=io.opentelemetry.java-util-logging-spans, version=null, schemaUrl=null}, name=%s %s %s (powered by Quarkus %s) started in %ss. %s, kind=INTERNAL, startEpochNanos=1637839757598249200, endEpochNanos=1637839757598261500, attributes=AttributesMap{data={thread.name=Quarkus Main Thread, thread.id=160, applicationinsights.internal.log=true, applicationinsights.internal.log_level=INFO, applicationinsights.internal.logger_name=io.quarkus}, capacity=128, totalAddedValues=5}, totalAttributeCount=5, events=[], totalRecordedEvents=0, links=[], totalRecordedLinks=0, status=ImmutableStatusData{statusCode=UNSET, description=}, hasEnded=true}
2021-11-25 12:29:17.600+01:00 DEBUG c.m.a.a.internal.exporter.Exporter - exporting span: SpanData{spanContext=ImmutableSpanContext{traceId=5b48813652115684c2b1805256117cb9, spanId=46d12a89acc87039, traceFlags=01, traceState=ArrayBasedTraceState{entries=[ai-internal-sp, 100]}, remote=false, valid=true}, parentSpanContext=ImmutableSpanContext{traceId=00000000000000000000000000000000, spanId=0000000000000000, traceFlags=00, traceState=ArrayBasedTraceState{entries=[]}, remote=false, valid=false}, resource=Resource{schemaUrl=https://opentelemetry.io/schemas/1.7.0, attributes={host.arch="amd64", host.name="SurfaceBook2MS", os.description="Windows 10 10.0", os.type="windows", process.command_line="C:\Program Files\OpenJDK\jdk-17.0.1;bin;java.exe -javaagent:../src/main/resources/applicationinsights-agent-3.2.3.jar -Dio.quarkus.force-color-support=true -javaagent:C:\Users\luguerre\.m2\repository\io\quarkus\quarkus-class-change-agent\2.2.3.Final\quarkus-class-change-agent-2.2.3.Final.jar -XX:TieredStopAtLevel=1 -agentlib:jdwp=transport=dt_socket,address=localhost:5005,server=y,suspend=n -Djava.util.logging.manager=org.jboss.logmanager.LogManager", process.executable.path="C:\Program Files\OpenJDK\jdk-17.0.1;bin;java.exe", process.pid=7512, process.runtime.description="Oracle Corporation OpenJDK 64-Bit Server VM 17.0.1+12-39", process.runtime.name="OpenJDK Runtime Environment", process.runtime.version="17.0.1+12-39", service.name="unknown_service:java", telemetry.sdk.language="java", telemetry.sdk.name="opentelemetry", telemetry.sdk.version="1.7.0"}}, instrumentationLibraryInfo=InstrumentationLibraryInfo{name=io.opentelemetry.java-util-logging-spans, version=null, schemaUrl=null}, name=Profile %s activated. %s, kind=INTERNAL, startEpochNanos=1637839757600203800, endEpochNanos=1637839757600224100, attributes=AttributesMap{data={thread.name=Quarkus Main Thread, thread.id=160, applicationinsights.internal.log=true, applicationinsights.internal.log_level=INFO, applicationinsights.internal.logger_name=io.quarkus}, capacity=128, totalAddedValues=5}, totalAttributeCount=5, events=[], totalRecordedEvents=0, links=[], totalRecordedLinks=0, status=ImmutableStatusData{statusCode=UNSET, description=}, hasEnded=true}
2021-11-25 12:29:17.602+01:00 DEBUG c.m.a.a.internal.exporter.Exporter - exporting span: SpanData{spanContext=ImmutableSpanContext{traceId=5d825db4b9e3effbb8d0d1448125caed, spanId=63fe1893ebbaa85c, traceFlags=01, traceState=ArrayBasedTraceState{entries=[ai-internal-sp, 100]}, remote=false, valid=true}, parentSpanContext=ImmutableSpanContext{traceId=00000000000000000000000000000000, spanId=0000000000000000, traceFlags=00, traceState=ArrayBasedTraceState{entries=[]}, remote=false, valid=false}, resource=Resource{schemaUrl=https://opentelemetry.io/schemas/1.7.0, attributes={host.arch="amd64", host.name="SurfaceBook2MS", os.description="Windows 10 10.0", os.type="windows", process.command_line="C:\Program Files\OpenJDK\jdk-17.0.1;bin;java.exe -javaagent:../src/main/resources/applicationinsights-agent-3.2.3.jar -Dio.quarkus.force-color-support=true -javaagent:C:\Users\luguerre\.m2\repository\io\quarkus\quarkus-class-change-agent\2.2.3.Final\quarkus-class-change-agent-2.2.3.Final.jar -XX:TieredStopAtLevel=1 -agentlib:jdwp=transport=dt_socket,address=localhost:5005,server=y,suspend=n -Djava.util.logging.manager=org.jboss.logmanager.LogManager", process.executable.path="C:\Program Files\OpenJDK\jdk-17.0.1;bin;java.exe", process.pid=7512, process.runtime.description="Oracle Corporation OpenJDK 64-Bit Server VM 17.0.1+12-39", process.runtime.name="OpenJDK Runtime Environment", process.runtime.version="17.0.1+12-39", service.name="unknown_service:java", telemetry.sdk.language="java", telemetry.sdk.name="opentelemetry", telemetry.sdk.version="1.7.0"}}, instrumentationLibraryInfo=InstrumentationLibraryInfo{name=io.opentelemetry.java-util-logging-spans, version=null, schemaUrl=null}, name=Installed features: [%s], kind=INTERNAL, startEpochNanos=1637839757602209200, endEpochNanos=1637839757602222100, attributes=AttributesMap{data={thread.name=Quarkus Main Thread, thread.id=160, applicationinsights.internal.log=true, applicationinsights.internal.log_level=INFO, applicationinsights.internal.logger_name=io.quarkus}, capacity=128, totalAddedValues=5}, totalAttributeCount=5, events=[], totalRecordedEvents=0, links=[], totalRecordedLinks=0, status=ImmutableStatusData{statusCode=UNSET, description=}, hasEnded=true}
2021-11-25 12:29:17.604+01:00 DEBUG c.m.a.a.internal.exporter.Exporter - exporting span: SpanData{spanContext=ImmutableSpanContext{traceId=1ae32e759b983955704b02dc5ed46b94, spanId=fdcf25b3c484d10e, traceFlags=01, traceState=ArrayBasedTraceState{entries=[ai-internal-sp, 100]}, remote=false, valid=true}, parentSpanContext=ImmutableSpanContext{traceId=00000000000000000000000000000000, spanId=0000000000000000, traceFlags=00, traceState=ArrayBasedTraceState{entries=[]}, remote=false, valid=false}, resource=Resource{schemaUrl=https://opentelemetry.io/schemas/1.7.0, attributes={host.arch="amd64", host.name="SurfaceBook2MS", os.description="Windows 10 10.0", os.type="windows", process.command_line="C:\Program Files\OpenJDK\jdk-17.0.1;bin;java.exe -javaagent:../src/main/resources/applicationinsights-agent-3.2.3.jar -Dio.quarkus.force-color-support=true -javaagent:C:\Users\luguerre\.m2\repository\io\quarkus\quarkus-class-change-agent\2.2.3.Final\quarkus-class-change-agent-2.2.3.Final.jar -XX:TieredStopAtLevel=1 -agentlib:jdwp=transport=dt_socket,address=localhost:5005,server=y,suspend=n -Djava.util.logging.manager=org.jboss.logmanager.LogManager", process.executable.path="C:\Program Files\OpenJDK\jdk-17.0.1;bin;java.exe", process.pid=7512, process.runtime.description="Oracle Corporation OpenJDK 64-Bit Server VM 17.0.1+12-39", process.runtime.name="OpenJDK Runtime Environment", process.runtime.version="17.0.1+12-39", service.name="unknown_service:java", telemetry.sdk.language="java", telemetry.sdk.name="opentelemetry", telemetry.sdk.version="1.7.0"}}, instrumentationLibraryInfo=InstrumentationLibraryInfo{name=io.opentelemetry.java-util-logging-spans, version=null, schemaUrl=null}, name=Live reload total time: %ss , kind=INTERNAL, startEpochNanos=1637839757603205200, endEpochNanos=1637839757603216900, attributes=AttributesMap{data={thread.name=vert.x-worker-thread-0, thread.id=124, applicationinsights.internal.log=true, applicationinsights.internal.log_level=INFO, applicationinsights.internal.logger_name=io.quarkus.deployment.dev.RuntimeUpdatesProcessor}, capacity=128, totalAddedValues=5}, totalAttributeCount=5, events=[], totalRecordedEvents=0, links=[], totalRecordedLinks=0, status=ImmutableStatusData{statusCode=UNSET, description=}, hasEnded=true}
2021-11-25 12:29:17.617+01:00 DEBUG c.m.a.a.internal.exporter.Exporter - exporting span: SpanData{spanContext=ImmutableSpanContext{traceId=9d35735615cbfb231d1b630b59df2e1a, spanId=64caea9c61649f01, traceFlags=01, traceState=ArrayBasedTraceState{entries=[ai-internal-sp, 100]}, remote=false, valid=true}, parentSpanContext=ImmutableSpanContext{traceId=00000000000000000000000000000000, spanId=0000000000000000, traceFlags=00, traceState=ArrayBasedTraceState{entries=[]}, remote=false, valid=false}, resource=Resource{schemaUrl=https://opentelemetry.io/schemas/1.7.0, attributes={host.arch="amd64", host.name="SurfaceBook2MS", os.description="Windows 10 10.0", os.type="windows", process.command_line="C:\Program Files\OpenJDK\jdk-17.0.1;bin;java.exe -javaagent:../src/main/resources/applicationinsights-agent-3.2.3.jar -Dio.quarkus.force-color-support=true -javaagent:C:\Users\luguerre\.m2\repository\io\quarkus\quarkus-class-change-agent\2.2.3.Final\quarkus-class-change-agent-2.2.3.Final.jar -XX:TieredStopAtLevel=1 -agentlib:jdwp=transport=dt_socket,address=localhost:5005,server=y,suspend=n -Djava.util.logging.manager=org.jboss.logmanager.LogManager", process.executable.path="C:\Program Files\OpenJDK\jdk-17.0.1;bin;java.exe", process.pid=7512, process.runtime.description="Oracle Corporation OpenJDK 64-Bit Server VM 17.0.1+12-39", process.runtime.name="OpenJDK Runtime Environment", process.runtime.version="17.0.1+12-39", service.name="unknown_service:java", telemetry.sdk.language="java", telemetry.sdk.name="opentelemetry", telemetry.sdk.version="1.7.0"}}, instrumentationLibraryInfo=InstrumentationLibraryInfo{name=io.opentelemetry.netty-4.1, version=null, schemaUrl=null}, name=HTTP GET, kind=SERVER, startEpochNanos=1637839755859704700, endEpochNanos=1637839757617591500, attributes=AttributesMap{data={net.peer.ip=127.0.0.1, thread.name=vert.x-eventloop-thread-0, http.method=GET, http.scheme=http, http.host=localhost:8080, thread.id=105, http.status_code=200, http.target=/hello, http.user_agent=Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:94.0) Gecko/20100101 Firefox/94.0, http.flavor=1.1, net.peer.port=57491}, capacity=128, totalAddedValues=11}, totalAttributeCount=11, events=[], totalRecordedEvents=0, links=[], totalRecordedLinks=0, status=ImmutableStatusData{statusCode=UNSET, description=}, hasEnded=true}
2021-11-25 12:29:30.290+01:00 DEBUG c.m.applicationinsights.agent - running shutdown hook
2021-11-25 12:29:30.290+01:00 DEBUG c.m.applicationinsights.agent - flushing telemetry on shutdown completed successfully

As you can see on the logs, the last span exported doesn't include the custom attribute there.

@guerrerotook
Copy link
Member Author

I think I found the problem and why the http instrumentation is skipped.

In the logs the agent warns that it can't find the class javax.servlet.http.HttpServletRequest

WARN muzzleMatcher - -- io.opentelemetry.javaagent.instrumentation.jaxrs.v2_0.JerseyServletContainerInstrumentation$ServiceAdvice:49 Missing class javax.servlet.http.HttpServletRequest

So, it turns out that if you only have as a reference in maven resteasy you don't have a proper Servlet implementation only jaxrs. What I did was add quarkus-undertow as a dependency so the Java Agent can properly find all the dependencies, he needs to instrument the incoming http request.

Thanks!

@mkowa42
Copy link

mkowa42 commented Dec 2, 2021

@guerrerotook Confirmed. I can reproduce the same. The only thing is that in case you add quarkus-undertow as an additional dependency the RESTEasy JAX-RS implementation runs as a Servlet filter. Without Undertow it runs directly on Vertx (refer to https://quarkus.io/guides/http-reference). Not sure if it's a good idea (any downsides?) to add the undertow extension not for the purpose of using the Servlet API but only for having a working Application Insights instrumentation.

@trask
Copy link
Member

trask commented Dec 2, 2021

hi @guerrerotook @mkowa42 thanks for the repro and investigations! I agree that Application Insights should work without having to modify your application (though I'm very glad you have found a workaround for now). I'm planning to look into this next week to see what we can do to make this experience better, and will post back here.

@trask trask reopened this Dec 2, 2021
@mkowa42
Copy link

mkowa42 commented Dec 7, 2021

Hi @trask, One additional observation regarding outgoing requests (dependency).

If an outgoing JAX-RS request is implemented asynchronously. Then again it’s not correlated with the previous events. In my case it's implemented as a JAX-RS / Microprofile Client running inside the same Quarkus based application (which includes as the previously mentioned workaround still the quarkus-undertow extension).

Example JAX-RS/Microprofile client interface:

@ApplicationScoped
@RegisterRestClient
@Path("/")
public interface AsyncApi {
  @POST
  @Path("/path")
  @Consumes({"application/json"})
  @Produces({"application/json"})
  public CompletionStage<MyResponse> performAsync(Payload payload) throws ProcessingException;
}

It might be related to Quarkus which is heavily using Vert.x (not only) for implementing the asynchronous calls.
"Normal" JAX-RS requests (synchronous) are working fine (due to undertow as the Servlet API provider).

@trask
Copy link
Member

trask commented Dec 13, 2021

hey @guerrerotook and @mkowa42, I have sent a fix for this issue upstream to the OpenTelemetry Javaagent, and will pull that fix into our distribution once it's approved and merged upstream, I should be able to provide you with a SNAPSHOT build to test in a couple of days.

@mkowa42
Copy link

mkowa42 commented Dec 14, 2021

@trask Thanks for letting us know the progress. Does the upstream PR also address the issue for async client requests I mentioned at #1980 (comment)?

@trask
Copy link
Member

trask commented Dec 14, 2021

Does the upstream PR also address the issue for async client requests I mentioned at #1980 (comment)?

here's a SNAPSHOT build which includes the fix for server correlation: https://github.com/microsoft/ApplicationInsights-Java/suites/4658614061/artifacts/126616732

can you try it out with async client requests, and if it's still an issue, can you post a repro that we can use to debug?

@mkowa42
Copy link

mkowa42 commented Dec 14, 2021

Hi @trask, great work. The fix is solving both problems.

I tested my application after removing the servlet dependency (quarkus-undertow). Incoming requests as well as outgoing sync and async JAX-RS requests are now logged fine to App Insights having the expected correlations.

Can you already say, when you plan to release a version containing this fix?

Many thanks!

@ghost ghost added the Status: Fixed label Dec 15, 2021
@trask
Copy link
Member

trask commented Dec 15, 2021

Can you already say, when you plan to release a version containing this fix?

current target is mid-January

@mkowa42
Copy link

mkowa42 commented Jan 14, 2022

@trask, I saw the todays release of 3.2.5-BETA hoping to see the fix of this issue is part of that release. But it's not. I did also a quick test and it's not working.

The snapshot build you have provided in Dec. (#1980 (comment)) worked already fine.

Did I miss anything?

@heyams
Copy link
Contributor

heyams commented Jan 14, 2022

Thank you for letting us know. upstream PR is for opentelemetry repo.. we might not pull that change in yet.. will double check with @trask and let you know soon. We will make a GA release in 2 weeks.. definitely will include it in the GA if not available in our BETA release.

@trask
Copy link
Member

trask commented Jan 14, 2022

hi @mkowa42, I just ran 3.2.5-BETA against the repro in this issue and it's working for me. Can you confirm that the repro in this issue works for you? and if so, can you provide a separate repro for the issue you are seeing? thx!

@mkowa42
Copy link

mkowa42 commented Jan 14, 2022

Hi @trask - thanks for the reply.

If you ran the repro than it contains still the workaround as mentioned by @guerrerotook in comment (#1980 (comment)): Refer to https://github.com/guerrerotook/quarkus-telemetry-demo/blob/main/pom.xml#L36-L39

I think if you would try the same after removing the undertow extension you should get still the issue. This wasn't the case with the snapshot build in December (#1980 (comment) ).

@trask
Copy link
Member

trask commented Jan 14, 2022

@mkowa42 I don't have that workaround in my local test. Can you confirm that the repro in this issue works for you? and if so, can you provide a separate repro for the issue you are seeing? thx!

@trask
Copy link
Member

trask commented Jan 19, 2022

@mkowa42 please let us know, hoping to isolate the issue you are seeing and get that fixed before 3.2.5 GA if possible, thx

@mkowa42
Copy link

mkowa42 commented Jan 19, 2022

Hi @trask - I downloaded a fresh copy of the 3.2.5-BETA and tested again my code. Now it's working! So it looks that I made a mistake the last time. Thank you again!

@trask
Copy link
Member

trask commented Jan 20, 2022

awesome 🎉

@ghost ghost locked as resolved and limited conversation to collaborators Feb 19, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants