OpenTelemetry-agent conflicts with skywalking-agent,OpenTelemetry-agent overrode the modification logic of skywalking-agent to bytecode

[oswin886]

Describe the bug
OpenTelemetry-agent conflicts with skywalking-agent,OpenTelemetry-agent overrode the modification logic of skywalking-agent to bytecode.

Steps to reproduce
Through pre-research, it is found that openTelemetry-agent and skywalking-agent have also enhanced the bytecode of the org.eclipse.jetty.server.HttpChannel file.Therefore, a test application jetty-demo based on jetty9 has been developed here.

1. Only install skywalking-agent, and then monitor the contents of the bytecode file of org.eclipse.jetty.server.HttpChannel in real time through arthas, as shown in the following figure. -javaagent:D:\data\skywalking-agent\skywalking-agent.jar
   

2. Only install OpenTelemetry-agent, and then monitor the contents of the bytecode file of org.eclipse.jetty.server.HttpChannel in real time through arthas, as shown in the following figure. -javaagent:D:\data\opentelemetry-javaagent.jar
   

3. Install openTelemetry-agent and skywalking-agent at the same time, and then monitor the contents of the bytecode file of org.eclipse.jetty.server.HttpChannel in real time through arthas, as shown in the following figure. -javaagent:D:\data\skywalking-agent\skywalking-agent.jar -javaagent:D:\data\opentelemetry-javaagent.jar.
   

4.After the third step is completed, request the /test api and then monitor the contents of the bytecode file of org.eclipse.jetty.server.HttpChannel in real time through arthas, as shown in the following figure.

If possible, provide a recipe for reproducing the error.

What did you expect to see?
OpenTelemetry-agent and skywalking-agent do not affect each other.

What did you see instead?
The jetty9.x plugin of skywalking-agent does not work.

What version are you using?
opentelemetry: v1.20.0
skywalking: v8.10.0

Environment
Compiler: JDK 1.8
OS: Windows10
test app: Jetty9

Additional context
other env: idea+maven3.8.6+arthas3.6.2.

Other plugins have similar situations, such as Tomcat9.x, Mysql5.x.

I(@oswin886) have put the test application and tools required for the reproduction in the opentelemetry-new-issue.zip package. In addition, I have communicated with the development engineer of the skywalking community. He(@wu-sheng) suggested that I submit an issue to the opentelemetry community. If you have any questions or other things I need to provide, please feed me or he(@wu-sheng) back as soon as possible.

opentelemetry-new-issue.zip

[mateuszrzeszutek]

Hey @oswin886 ,

In general, we don't recommend or support using other javaagents together with the OTel javaagent. However, we will accept patches that fix these kind of issues (as long as they don't break anything important).

[wu-sheng]

@mateuszrzeszutek An interested behavior of OTel javaagent is, it seems intentional to retransfer the codes in the runtime, even SkyWalking's agent has done everything after OTel javaagent did in the booting stage.
Do you know any mechanism or setup to control this?

[mateuszrzeszutek]

I think this might be the code responsible for enabling it:

opentelemetry-java-instrumentation/javaagent-tooling/src/main/java/io/opentelemetry/javaagent/tooling/AgentInstaller.java

Lines 136 to 137 in 0ace238

	.with(AgentBuilder.RedefinitionStrategy.RETRANSFORMATION)
	.with(new RedefinitionDiscoveryStrategy())

(+ the jar manifest entries)

[trask]

(just linking to related #1534)

[oliver-zhang]

Hey @mateuszrzeszutek ,
When I disable retransformation use the following way

My springboot application fail to start , bellow is the error message

Is there any other way to deal this problem ?

[laurit]

@oliver-zhang the default value for redefinition strategy is DISABLED which will break the agent. Besides RETRANSFORMATION the other option is REDEFINITION.

[laurit]

@oswin886 @oliver-zhang @wu-sheng try https://oss.sonatype.org/content/repositories/snapshots/io/opentelemetry/javaagent/opentelemetry-javaagent/1.24.0-SNAPSHOT/opentelemetry-javaagent-1.24.0-20230302.081319-55.jar and let us know whether this fixes the issue you had

[oliver-zhang]

@oswin886 @oliver-zhang @wu-sheng try https://oss.sonatype.org/content/repositories/snapshots/io/opentelemetry/javaagent/opentelemetry-javaagent/1.24.0-SNAPSHOT/opentelemetry-javaagent-1.24.0-20230302.081319-55.jar and let us know whether this fixes the issue you had

Yes, It's fixes