Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Move ShutdownHelper to agent-bootstrap… #3913

Merged
merged 1 commit into from
Sep 29, 2022

Conversation

mcculls
Copy link
Contributor

@mcculls mcculls commented Sep 29, 2022

…to avoid having to inject it into the bootstrap classloader later on.

Types under agent-bootstrap are automatically made available from the bootclasspath as part of installing the javaagent.

This avoids the need to register them as helper classes via HelperInjector. It is especially important to avoid using HelperInjector when instrumentating JDK classes because the only way we can inject types into the bootclasspath after the javaagent has been installed is to use a temporary jar file (unfortunately the only approach allowed by the JVM's instrumentation API)

Use of a temporary jar file causes issues in serverless environments, but it's worse when instrumenting j.l.Shutdown because we use File.deleteOnExit to attempt to clean up these temporary jars when the process ends (we cannot remove them while the JVM is alive because of how the JVM maps the classes into memory) and File.deleteOnExit interacts with j.l.Shutdown to register its own shutdown hook.

This results in an exception while trying to inject the helper

java.lang.NoClassDefFoundError: Could not initialize class java.io.DeleteOnExitHook
        at java.io.File.deleteOnExit(File.java:1090)
        at datadog.trace.agent.tooling.HelperInjector.deleteTempDir(HelperInjector.java:242)
        at datadog.trace.agent.tooling.HelperInjector.injectBootstrapClassLoader(HelperInjector.java:185)
        at datadog.trace.agent.tooling.HelperInjector.transform(HelperInjector.java:135)
        at net.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer.doTransform(AgentBuilder.java:11897)
        at net.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer.transform(AgentBuilder.java:11834)
        at net.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer.access$1700(AgentBuilder.java:11551)
        at net.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer$LegacyVmDispatcher.run(AgentBuilder.java:12234)
        at net.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer$LegacyVmDispatcher.run(AgentBuilder.java:12174)
        at java.security.AccessController.doPrivileged(Native Method)
        at net.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer.doPrivileged(AgentBuilder.java)
        at net.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer.transform(AgentBuilder.java:11743)
        at datadog.trace.agent.tooling.bytebuddy.DDClassFileTransformer.transform(DDClassFileTransformer.java:44)
        at sun.instrument.TransformerManager.transform(TransformerManager.java:188)
        at sun.instrument.InstrumentationImpl.transform(InstrumentationImpl.java:428)
        at java.lang.ApplicationShutdownHooks.<clinit>(ApplicationShutdownHooks.java:42)
        at java.lang.Runtime.addShutdownHook(Runtime.java:211)
        at datadog.trace.util.AgentTaskScheduler.scheduleTarget(AgentTaskScheduler.java:171)
        at datadog.trace.util.AgentTaskScheduler.schedule(AgentTaskScheduler.java:85)
        at datadog.trace.util.AgentTaskScheduler.scheduleWithJitter(AgentTaskScheduler.java:114)
        at datadog.trace.util.AgentTaskScheduler.scheduleWithJitter(AgentTaskScheduler.java:119)
        at datadog.trace.bootstrap.Agent.scheduleJmxStart(Agent.java:479)
        at datadog.trace.bootstrap.Agent.start(Agent.java:229)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at datadog.trace.bootstrap.AgentBootstrap.agentmain(AgentBootstrap.java:60)
        at datadog.trace.bootstrap.AgentBootstrap.premain(AgentBootstrap.java:47)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at sun.instrument.InstrumentationImpl.loadClassAndStartAgent(InstrumentationImpl.java:386)
        at sun.instrument.InstrumentationImpl.loadClassAndCallPremain(InstrumentationImpl.java:401)

which then leads to the following exception at shutdown:

Exception in thread "main" java.lang.NoClassDefFoundError: Could not initialize class java.io.DeleteOnExitHook
        at java.io.File.deleteOnExit(Unknown Source)

…nto the bootstrap classloader later on.

Types under agent-bootstrap are automatically made available from the bootclasspath as part of installing the javaagent.

This avoids the need to register them as helper classes via HelperInjector. It is especially important to avoid using
HelperInjector when instrumentating JDK classes because the only way we can inject types into the bootclasspath after
the javaagent has been installed is to use a temporary jar file (unfortunately the only approach allowed by the JVM's
instrumentation API)

Use of a temporary jar file causes issues in serverless environments, but it's worse when instrumenting j.l.Shutdown
because we use File.deleteOnExit to attempt to clean up these temporary jars when the process ends (we cannot remove
them while the JVM is alive because of how the JVM maps the classes into memory) and File.deleteOnExit interacts with
j.l.Shutdown to register its own shutdown hook.

This results in an exception while trying to inject the helper which then leads to the following exception at shutdown:

Exception in thread "main" java.lang.NoClassDefFoundError: Could not initialize class java.io.DeleteOnExitHook
        at java.io.File.deleteOnExit(Unknown Source)
@mcculls mcculls merged commit b03bba8 into master Sep 29, 2022
@mcculls mcculls deleted the mcculls/fixShutdownDeleteOnExitNCDFE branch September 29, 2022 06:45
@github-actions github-actions bot added this to the 0.110.0 milestone Sep 29, 2022
am312 pushed a commit that referenced this pull request Sep 29, 2022
…nto the bootstrap classloader later on. (#3913)

Types under agent-bootstrap are automatically made available from the bootclasspath as part of installing the javaagent.

This avoids the need to register them as helper classes via HelperInjector. It is especially important to avoid using
HelperInjector when instrumentating JDK classes because the only way we can inject types into the bootclasspath after
the javaagent has been installed is to use a temporary jar file (unfortunately the only approach allowed by the JVM's
instrumentation API)

Use of a temporary jar file causes issues in serverless environments, but it's worse when instrumenting j.l.Shutdown
because we use File.deleteOnExit to attempt to clean up these temporary jars when the process ends (we cannot remove
them while the JVM is alive because of how the JVM maps the classes into memory) and File.deleteOnExit interacts with
j.l.Shutdown to register its own shutdown hook.

This results in an exception while trying to inject the helper which then leads to the following exception at shutdown:

Exception in thread "main" java.lang.NoClassDefFoundError: Could not initialize class java.io.DeleteOnExitHook
        at java.io.File.deleteOnExit(Unknown Source)
@ValentinZakharov ValentinZakharov added the inst: others All other instrumentations label Oct 4, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
inst: others All other instrumentations
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants