Re-create Span with existing span ID

[RohitRanjanMS]

Hi,

We have specific scenarios in Azure Functions where we aim to re-create the Activity with same span ID as the remote span.

In the out-of-process scenario, the SERVER span is generated in the host process, while the user's code is executed in the worker process. Our objective is to conceal the implementation details and explore possibilities for recreating the remote span, thereby invoking the user's code within this context.

In the Durable Functions, the same work undergoes pausing and resuming on the worker. Upon each resumption of the work, our goal is to establish the same span as the current span.

We attempted to utilize reflection for this purpose but are currently exploring whether there is an officially supported method to achieve the desired outcome.

[RohitRanjanMS]

@trask @lmolkova @jviau @cijothomas

[lmolkova]

to add context: Java allows this with something like

// extract span context
SpanContext remote = SpanContext.createFromRemoteParent("<traceId>", "<spanId>", TraceFlags.getDefault(), null);

// create parent Context from SpanContext
io.opentelemetry.context.Context parent = io.opentelemetry.context.Context.root().with(Span.wrap(remote));

// now we can make it current and it'll become a parent to any spans created in it's scope
Scope scope = parent.makeCurrent();
Span child = tracer.spanBuilder("child").startSpan();

// do child work

child.end();
scope.close();

I.e. it allows to make remote span context to be current/active and act as an ambient parent to any child spans.

I don't think API spec explicitly mentions it should be possible, and it'd be great to create a spec issue to fix it.

[cijothomas]

There is nothing in the spec that allows to re-create a Span, reusing same SpanId as an existing span, right?
Even the java example -it does not look like spanid is re-used...

[lmolkova]

Java does not re-create span, it creates a current context that can be used a parent.

I.e. functions host will create a span, then context would flow to function worker, where span context can act as an ambient parent.

@RohitRanjanMS to confirm that re-creating full Activity with an existing span-id is not a requirement.
The requirement (based on my understanding) is to

• be able to create children from this ambient semi-activity represented with remote ActivityContext and nothing else
• potentially record attributes/events on this semi-activity and pass them back to the host (this is a future extensibility)

My understanding that it needs BCL work, but we'd like to hear otel-dotnet thoughts on this.

[cijothomas]

This is likely to be relevant : dotnet/runtime#86966

[jviau]

@cijothomas that is mostly what we need. It is missing one thing we are considering but have not settled on as mandatory: capturing tags in the worker process and flowing back to the host. We would then add those tags to the host's Activity before recording it.

Related For the original title of this issue, there is another use case for this: durable task.

Without getting into the implementation details of how durable orchestrations work, suffice it to say that we re-create the same activity multiple times and only want to record it when the orchestration completes. This is because the same orchestration instance is loaded in and out of the application multiple times through its life ("replayed"). Representing each replay as its own span is noisy a not usable from a customer standpoint. Today we use reflection to set the span ID each time we rehydrate the activity. We don't need to worry about manually persisting and rehydrating tags as the orchestration replay will naturally rehydrate those.

We can discuss this in a separate issue (or even in the dotnet repo?) if need be.

[cijothomas]

@cijothomas that is mostly what we need.

When you said this, did you meant dotnet/runtime#86966 is what you'll mostly need?

[jviau]

@cijothomas that is mostly what we need.

When you said this, did you meant dotnet/runtime#86966 is what you'll mostly need?

Yes, that issue is mostly what we need. It is only having a way to capture tags and propagate them back to the host that wouldn't be there. But as I said, we are not convinced we will even have a feature like that.

[cijothomas]

Then we can close this issue, as this issue is asking for "re-create Span with existing span id", which is infeasible, and won't be supported without backing spec first.

[jviau]

@cijothomas let me clarify: for functions that issue will work for us. For durable task, we do have a need to recreate the same span repeatedly. This framework follows a record -> replay system, so we are replaying customer actions back to them. Part of that involves setting up the same span for them to interact with during the replay.

[cijothomas]

For durable task, we do have a need to recreate the same span repeatedly.

Got it. This should be opened as a spec issue as well.

[jviau]

Where would we create this spec issue?

[cijothomas]

Before opening a feature request against this repo, consider whether the feature
should/could be implemented in the other OpenTelemetry client
libraries. If so, please open an issue on
opentelemetry-specification
first.

[martinjt]

I'm closing this issue as not planned since it would need to be part of the spec if we were going to implement something like this.