Add Instrumentation Library for Apache Pulsar

[riferrei]

Context

As of today there is no support for Apache Pulsar in the OTel Java agent. Pulsar has been growing fast as an open source alternative for streaming data applications and soon enough there will be a need from developers to instrument their applications using its APIs. As such, I wonder if it wouldn't be nice to add this support to the OTel Java agent? I am a couple weeks away to have a complete branch with this implementation. I would love to include this code into the project if this sounds like a good idea.

How?

The support for Pulsar would initially focus on adding support for its Java client version 2.7.0, which includes support for both its producer and consumer APIs. This would promote a good coverage in most of the scenarios found today from developers building applications with Pulsar. Thereafter there would be support for the Pulsar Functions API that works as a processing layer to mutate messages passing through Pulsar. Once the support for the Pulsar Functions API is in place we would automatically be providing support for the Pulsar I/O framework — a framework from Pulsar that allows the deployment of specialized connectors that perform data sourcing and sinking to/from Pulsar. FYI, the Pulsar I/O framework was written on top of the Pulsar Functions API.

Alternative

Currently I have been solving this problem to add support for OTel in Pulsar by leveraging its interceptors API to which I wrote two of them (one for the producer and another for the consumer) to enhance the code with the instrumentation. However, it would be much easier for developers to not have to add these interceptors in their code but — rely on the Java agent to provide the instrumentation automatically.

[iNikem]

@riferrei This is a terrific idea :) Your contribution will be much appreciated!

In general, we try to provide our users with both options: with or without javaagent. Take a look at spring-web-3.1 instrumentations for example. library instrumentations provide some kind of library-specific interceptor or callback that end-user can apply manually to their code. javaagent instrumentations in this case just automatically apply library instrumentations using bytecode instrumentation. If you could follow such approach, it would be fantastic :)

[tjiuming]

@trask @mateuszrzeszutek I’ll handle this issue, please assign to me