pulsar-client-instrumentation can't export receive span when call batchReceive() / batchReceiveAsync() method

[coderzc]

Describe the bug

When I use batchReceive()/batchReceiveAsync() method to receive messages on pulsar-client, can't export the "receive" span.

Steps to reproduce

    public static void main(String[] args) throws PulsarClientException {
        String serviceUrl = "pulsar://localhost:6650";
        String topic = "my-topic";

        PulsarClient client = PulsarClient.builder()
                .serviceUrl(serviceUrl)
                .build();

        Consumer<String> consumer = client.newConsumer(Schema.STRING)
                .topic(topic)
                .subscriptionType(SubscriptionType.Shared)
                .subscriptionName("subs")
                .subscribe();

        Producer<String> producer = client.newProducer(Schema.STRING)
                .enableBatching(false)
                .topic(topic)
                .create();

        final int N = 1;

        for (int i = 0; i < N ; i++) {
            producer.newMessage().key("key-" + i).value("message-" + i).send();
        }

        for (int i = 0; i < N ; i++) {
            Messages<String> messages = consumer.batchReceive();
            for (Message<String> message : messages) {
                System.out.println("Received message: " + message.getValue());
                consumer.acknowledge(message);
            }
        }

        consumer.close();
        producer.close();
        client.close();
    }

java -javaagent:/path/opentelemetry-javaagent.jar \
     -Dotel.resource.attributes=service.name=myapp \
     -Dotel.traces.exporter=otlp \
     -Dotel.metrics.exporter=none \
     -jar myapp.jar

Expected behavior

Export the "receive" span when using batchReceive() method to receive messages on pulsar-client.

Actual behavior

Can't export the "receive" span

This may be related to otel.instrumentation.messaging.experimental.receive-telemetry.enabled, since when enable it, this span can be exported but traceID has changed

Javaagent or library instrumentation version

v1.32.1

Environment

JDK: JDK17
OS: macOS M1

Additional context

[coderzc]

cc @laurit I see you support batchReceive, PTAL
#8173

[coderzc]

@dao-jun Do you have some ideas?

[dao-jun]

@coderzc Try add an env -Dotel.instrumentation.messaging.experimental.receive-telemetry.enabled=true

[coderzc]

@coderzc Try add an env -Dotel.instrumentation.messaging.experimental.receive-telemetry.enabled=true

@dao-jun Yes, when enabling this feature, this span can be exported, but traceID has changed.

How do I want to receive span and publish span to be under the same traceID?

[dao-jun]

@coderzc Try add an env -Dotel.instrumentation.messaging.experimental.receive-telemetry.enabled=true

@dao-jun Yes, when enabling this feature, this span can be exported, but traceID has changed.

How do I want to receive span and publish span to be under the same traceID?

If you are using batchReceive, it is an expected behavior.
For example, we received the 3 messages by calling batchReceive:

1. traceId: 1, spanId: 1
2. traceId: 2, spanId: 2
3. traceId: 3, spanId: 3

For the batchReceive span, which traceId should it extends? So, create a traceId is expected.

[coderzc]

@coderzc Try add an env -Dotel.instrumentation.messaging.experimental.receive-telemetry.enabled=true

@dao-jun Yes, when enabling this feature, this span can be exported, but traceID has changed.
How do I want to receive span and publish span to be under the same traceID?

If you are using batchReceive, it is an expected behavior. For example, we received the 3 messages by calling batchReceive:

1. traceId: 1, spanId: 1
2. traceId: 2, spanId: 2
3. traceId: 3, spanId: 3

For the batchReceive span, which traceId should it extends? So, create a traceId is expected.

Makes sense