feat(attach): implement basic dynamic attach #234
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
github-actions
bot
added
the
needs-triage
andrewazores
added
feat
andrewazores
force-pushed
the
dynamic-attach
branch
from
66441c2 to
b08711f
Compare
andrewazores
force-pushed
the
dynamic-attach
branch
from
9fa7ae6 to
9d3409f
Compare
ebaron
reviewed
Feb 26, 2024
There was a problem hiding this comment.
Is there a suggested way to run this after #311? As expected, when running with the plain jar, I get:
$ java -jar target/cryostat-agent-0.4.0-SNAPSHOT.jar
Exception in thread "main" java.lang.NoClassDefFoundError: org/slf4j/LoggerFactory
at io.cryostat.agent.Agent.<clinit>(Agent.java:73)
Caused by: java.lang.ClassNotFoundException: org.slf4j.LoggerFactory
at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:641)
at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:188)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:525)
... 1 more
With the shaded jar, I get:
$ java -jar target/cryostat-agent-0.4.0-SNAPSHOT-shaded.jar
2024-02-26 15:59:23:538 -0500 [main] INFO io.cryostat.agent.Agent - Attaching to VM: io.cryostat.Cryostat 19841
2024-02-26 15:59:23:542 -0500 [main] INFO io.cryostat.agent.Agent - Injecting agent into PID 19841
2024-02-26 15:59:23:542 -0500 [main] INFO io.cryostat.agent.Agent - /home/ebaron/git/cryostat-agent/target/cryostat-agent-0.4.0-SNAPSHOT-shaded.jar
com.sun.tools.attach.AgentLoadException: Agent JAR not found or no Agent-Class attribute
at jdk.attach/sun.tools.attach.HotSpotVirtualMachine.loadAgent(HotSpotVirtualMachine.java:160)
at io.cryostat.agent.Agent.call(Agent.java:136)
at io.cryostat.agent.Agent.call(Agent.java:63)
at io.cryostat.agent.shaded.picocli.CommandLine.executeUserObject(CommandLine.java:2041)
at io.cryostat.agent.shaded.picocli.CommandLine.access$1500(CommandLine.java:148)
at io.cryostat.agent.shaded.picocli.CommandLine$RunLast.executeUserObjectOfLastSubcommandWithSameParent(CommandLine.java:2461)
at io.cryostat.agent.shaded.picocli.CommandLine$RunLast.handle(CommandLine.java:2453)
at io.cryostat.agent.shaded.picocli.CommandLine$RunLast.handle(CommandLine.java:2415)
at io.cryostat.agent.shaded.picocli.CommandLine$AbstractParseResultHandler.execute(CommandLine.java:2273)
at io.cryostat.agent.shaded.picocli.CommandLine$RunLast.execute(CommandLine.java:2417)
at io.cryostat.agent.shaded.picocli.CommandLine.execute(CommandLine.java:2170)
at io.cryostat.agent.Agent.main(Agent.java:109)
I expected the shaded jar to work. It appears to find the correct path to the shaded jar, and the manifest contains the correct Agent-Class attribute, yet the VM still fails to load it.
|
Hmm, I'll have to take a look. |
andrewazores
force-pushed
the
dynamic-attach
branch
from
9d3409f to
5afbcbc
Compare
and Not sure what's going on - from my end it looks like it's just about working as intended. I haven't tried supplying the injected Agent with its required configuration options yet, but it's at least getting to that point. |
ebaron
approved these changes
Apr 5, 2024
Open
13 tasks
andrewazores
force-pushed
the
dynamic-attach
branch
from
5afbcbc to
037d748
Compare
5 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Based on #224
Related to #233
This implements basic dynamic attach. The Agent can be launched as a standalone Java process like:
$ java -jar target/cryostat-agent-0.4.0-SNAPSHOT.jar 1234 # 1234 is another JVM's pidThis will cause the Agent process to look up PID
1234using its Attach providers. If1234is found as a valid PID then the Agent process will attach to PID1234and attempt to load the Agent's own JAR into that running JVM, which will then bootstrap into the normal Agent launch process. If1234is not found then nothing is bootstrapped. Instead of1234the wildcard*can also be specified, which will lead to the Agent JAR attempting to bootstrap itself into every JVM it finds. Finally, if the PID is not specified or is0, then the Agent will check if exactly one candidate JVM is available and attempt to bootstrap into that, failing if there are too many JVM candidates or none.In this mode the Agent still tries to load its configuration entirely from the JVM it is bootstrapping into, which does not really make sense for a late-attaching Agent - the host application probably does not have Cryostat Agent-specific configuration properties if it was not launched with the Cryostat Agent to begin with. This will be addressed some time later in a follow-up PR.Additional late-binding configuration options can be specified to the Agent launcher with command line options:
$ java -jar target/cryostat-agent-0.4.0-SNAPSHOT.jar \ -Dcryostat.agent.baseuri=http://cryostat.local \ --smartTrigger=[ProcessCpuLoad>0.2]~profile \ @/deployment/app/moreAgentArgs \ 1234Try
$ java -jar target/cryostat-agent-0.4.0-SNAPSHOT.jar -hand see picocli for a more complete explanation of the available options and their behaviour. These late-binding configuration options are picked up by the Agent launcher process and serialized into a singleStringargument that is passed into the injected Agent instances, which they receive on theiragentmain(String arg)and then deserialize again. System properties specified with-Dare set onto the host JVM before the injected Agent attempts to read its configuration values, so from the Agent implementation POV this is identical to setting the-Dor equivalent environment variable on the host JVM process itself. This mechanism does allow the user who controls the agent injection to override other arbitrary system properties of the host JVM at runtime, however if the user has enough access and ability to inject an agent then they can already exploit this without using the Cryostat Agent specifically.To test:
Things to investigate:
ByteArrayInputStreamor something so we can send it the JAR contents directly?*PID wildcard, it might also be useful to have a--watchoption where the agent launcher becomes a daemon, injecting agent instances into new JVMs as they appear.