Guidelines on preventing infinite traces?

[gao-artur]

Consider the following example method that starts the background Task with a loop that starts a new activity on every iteration:

private Task StartLoop()
{
    return Task.Run(async () =>
    {
        while (true)
        {
            using var loopActivity = _activitySource.StartActivity();
            await Task.Delay(5_000);
        }
    });
}

Now, consider this method is called from a method where Activity.Current != null

public Task StartAsync()
{
    using var startupActivity = _activitySource.StartActivity();
    _loopTask = StartLoop();
    return Task.CompletedTask;
}

Every loopActivity instance has the startupActivity as its parent. It means if the startupActivity was created during the application startup, and the loop runs for the entire application's lifetime, we get an "infinite" trace because all the loopActivity will be added to the same trace.

The simple solution is setting Activity.Current = null before the while loop, but it looks very error-prone (I have to remember to do the same every time I add another background loop in the application). Also, in the example code, it's obvious there is a loop, but it's not always the case. For example, I can use a 3rd party library to subscribe to messages that may use a polling loop internally. If this library supports OTel, it will create an activity having loopActivity as a parent.

Are there any general guidelines on how to prevent infinite traces?

[cijothomas]

Same as #984 ?

[gao-artur]

Indeed, these issues have a common topic. However, my question was how to detect situations where we must reset the parent activity. Currently, we monitor our traces and actively look for too long traces. The problem is Tempo (the UI) fails to retrieve such big traces and it's usually not easy to understand where they come from. More than that, sometimes, the root span stays open forever, so we don't have this information at all.