Java SDK basic OTEL Support

[adinauer]

Description

More details tbd

Our plan is to create a SpanProcessor (for sending spans to Sentry) as well as probably a TextMapPropagator (for dealing with tracing across systems).

We'll probably also need to provide an agent JAR for people to auto instrument their applications.

There may be problems with initializing Sentry in combination with the agent, needs investigation.

Implementing it should address #1348 .

Here's issues on other SDKs implementing OTEL support:

• Ruby SDK basic OTEL Support sentry-ruby#1907
• Basic OpenTelemetry (OTEL) Support sentry-python#1687
• Add OpenTelemetry Node Package sentry-javascript#6000
• [GO] Add basic OpenTelemetry support sentry-go#486

Challenges faced

Accessing the initialized Sentry instance of the target application from the OTEL Java Agent

Asked on OTEL GH and got some suggestions:

Use reflection

This would require us to perform every interaction with Sentry using reflection, not just retrieving the instance but every invocation which would lead to very difficult maintenance and brittle code

Add Sentry to the bootstrap classloader

so the same instance can be accessed from the target application and the OTEL Java Agent: This requires us to create our own custom distro of the OTEL Java Agent as shown in their samples. We first tried producing an agent JAR with SpanProcessor added as extension but that didn't allow us to add to the bootstrap classloader.

TODOs

• Merge feature into main
• Release
• Document
• Create new modules for OTEL
• Add new modules to release registry
• Add instrumenter to options and make startTransaction and createChild no-op if the instrumenter performing the operation does not match the configured one.
• Implement SpanProcessor
• Add otel to context
• Implement TextMapPropagator
• Allow configuring Sentry when only using the Java Agent so it's not needed in the target application
• Make sure null classloader (i.e. bootstrap classloader) doesn't cause issues
• NoOpTransaction attaches Empty sentry-trace and baggage headers. #2376
• Add logging to SentrySpanProcessor, SentryPropagator etc.
• Add Sentry OTEL Java Agent to OTEL demo
• Upgrade to latest OTEL release(s)
• Add automated tests for OTEL
• Add README to sentry-opentelemetry modules
• Use HubAdapter instead of Sentry
• Do not override default exporter config but rather document how to get rid of log spam if only using Sentry without OTEL exporter
• Link errors to transactions created via OTEL
• Make it possible to disable auto init in Java Agent so the target application can still use ENV vars and properties file for init without triggering auto init

[dalbani]

I've looked at the documentation and through the code, but I couldn't find an answer to this simple question: say that I'm using the Sentry OpenTelemetry Java agent, what am I supposed to see on my Sentry dashboard?
What gets actually pushed there when OTEL_TRACES_EXPORTER=none and OTEL_METRICS_EXPORTER=none?

To give some context, I'm trying to instrument an off-the-shelf application (Keycloak to be precise) with the Java agent, with the goal of seeing generated exceptions on my Sentry dashboard. Is this use case supposed to be (already) possible?

[adinauer]

Hello @dalbani. We're not using exporters to send the Spans to Sentry but instead rely on a SpanProcessor which converts OpenTelemetry Spans into Sentry Transactions / Spans. I haven't tried instrumenting keycloak myself and I don't know what underlying frameworks (if any) it uses so I can't say for sure whether something would show up. I wasn't able to find any direct instrumentation for it in the OpenTelemetry repository (https://github.com/open-telemetry/opentelemetry-java-instrumentation/tree/main/instrumentation). If it does work you should be able to see transactions in the Sentry UI under "Performance".

[adinauer]

I guess you should at least be able to see some database queries showing up in Sentry.

[dalbani]

Thanks @adinauer for the response.
Yes, I do see database activity in the Performance section.
Although a very interesting "bycatch", as I wrote above, I'm above all interested in tracking Java exceptions being thrown by Keycloak.
Basically the same behavior that I get with my other apps using the Sentry SDK.
But actually, if my understanding is correct of how things work, I guess this capability is handled by io.sentry:sentry-logback which I use in my apps??

I suppose what I'm looking for is for the Java agent to act as a big try/catch { Sentry.captureException(); } wrapper.
Does that make sense? 😄

[adinauer]

Yes for now you'd have to rely on one of the logging integrations (logback, log4j2, jul) for capturing errors. We have #2480 to explore capturing errors via the Java Agent. I guess we'd have to find the place where unhandled exceptions are caught for every framework we want to support and capture it there. We don't have plans for doing this for things like keycloak though. It would be for commonly used frameworks.

[dalbani]

For the record, I've stumbled upon an extension for Quarkus (the framework Keycloak is based on) for an integration with Sentry: https://github.com/quarkiverse/quarkus-logging-sentry. [*]
I just need to find out how to properly use in my setup and it should do exactly what I want.

[*] It's your call obviously, but having an official support for Quarkus would be great, I think.

[adinauer]

@dalbani thanks for the feedback. We've had multiple requests for a quarkus integration (e.g. #2254). We may look into adding it at some point but it's not on our immediate roadmap (yet).