Agent instrumentation fails on sybase jdbc instrumentation #2644
Comments
|
@pparapatics What OS do you use? What version of JVM and from what vendor? |
|
Tried two setups:
1)
JVM:
openjdk version "11.0.8" 2020-07-14
OpenJDK Runtime Environment 18.9 (build 11.0.8+10)
OpenJDK 64-Bit Server VM 18.9 (build 11.0.8+10, mixed mode)
Operating system:
NAME="Linux Mint"
VERSION="20 (Ulyana)"
2)
CentOS 8 with
openjdk 11.0.10 2021-01-19 LTS
OpenJDK Runtime Environment 18.9 (build 11.0.10+9-LTS)
OpenJDK 64-Bit Server VM 18.9 (build 11.0.10+9-LTS, mixed mode, sharing)
…On Fri, 26 Mar 2021 at 13:11, Nikita Salnikov-Tarnovski < ***@***.***> wrote:
@pparapatics <https://github.com/pparapatics> What OS do you use? What
version of JVM and from what vendor?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#2644 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AK7QSWFIPWXHWII7O2ILAZTTFR2YFANCNFSM4Z3IBJ5A>
.
|
|
@pparapatics can you post all the JVM command-line options you are using (including any other also, is there a stack trace associated with the error you posted above? |
|
Sorry for the late reply. NOTE: Picked up JDK_JAVA_OPTIONS: --add-opens=java.base/java.lang=ALL-UNNAMED --add-opens=java.base/java.io=ALL-UNNAMED --add-opens=java.rmi/sun.rmi.transport=ALL-UNNAMED |
Seems like the reason for failure is somewhere inside your application then... Can you tell us what libraries/frameworks does it use? Any chance that you can provide us with a minimal application that reproduces this failure? |
|
Yes, that makes it more difficult. There are plenty of libraries involved. Is there a possibility I can debug it on my machine to provide you more information? |
looks like JVM crash, not sure if debugging will help here... Can you try to add Also can you try to create a jvm crash log file and share it with us? See https://docs.oracle.com/javase/8/docs/technotes/guides/troubleshoot/felog001.html for explanation |
|
You said that tomcat does not start up correctly. Did the jvm process crash (by crash I mean that the process has died) if so look for hs_err_pid log file in tomcat bin dir, if you can't find it then see the link that @iNikem gave and set the path where the file is generated. Usually when jvm crashes there is some info in output that describes the cash and includes the path to hs_err_pid log. |
|
Thank you for the debugging hints. I started the application with the debug setting on and found the following exception which relates to a stack overflow. It is related to the the jdbc instrumentation. I tried to verify that this is the root cause by deactivating the jdbc instrumenation using the flag: -Dotel.integration.jdbc.enabled=false `[opentelemetry.auto.trace 2021-04-07 17:51:54:472 +0200] [localhost-startStop-1] DEBUG io.opentelemetry.javaagent.bootstrap.ExceptionLogger - Failed to handle exception in instrumentation for com.sybase.jdbc4.jdbc.SybCallableStatement on ParallelWebappClassLoader^M java.lang.StackOverflowError |
|
Try using |
|
Yes setting -Dotel.instrumentation.jdbc.enabled=false works. The error is gone and tracing works. (An additional remark. My initial description was incorrect. The error does not prevent the startup. It takes much longer and therefore it just looked like it was hanging) |
pparapatics
changed the title
Yes, I believe this gives a very good indication to the source of this error. |
Describe the bug
Starting from version 0.17.0 the agent instrumentation fails on a tomcat 8 application
Error message:
*** java.lang.instrument ASSERTION FAILED ***: "!errorOutstanding" with message transform method call failed at ./src/java.instrument/share/native/libinstrument/JPLISAgent.c line: 873
Versions failing: 0.17.0, 1.0.0, 1.0.1
Versions working: 0.10.1
Steps to reproduce
Start tomcat 8 with javagent
What did you expect to see?
For version 0.10.1 the application starts up, agent works as expected (propagation, exporting, ...)
What did you see instead?
Version 0.17.0 fails with message above
*** java.lang.instrument ASSERTION FAILED ***: "!errorOutstanding" with message transform method call failed at ./src/java.instrument/share/native/libinstrument/JPLISAgent.c line: 873
After that no application logs are show anymore
Tomcat does not start up correctly
What version are you using?
Versions failing: 0.17.0, 1.0.0, 1.0.1
Versions working: 0.10.1
Environment
Tomcat version: Apache Tomcat/8.5.32
Additional context
The application running in tomcat is a quite complex monolithical application.
The text was updated successfully, but these errors were encountered: