Report datadog.tracer.abandoned_spans health metric

[mtoffl01]

What does this PR do?

Report the count of unfinished spans as a metric, along with tags about the span name and span integration.

This PR also makes the following small changes:

• Amends abandonedSpansCandidate type to contain an Integration field
• Modifies the TestStatsdClient for ease of use

Motivation

Abandoned spans info is already available in tracer logs, but making the information visible as a metric that we can track alongside other data points in the Datadog platform will make debugging easier for customer and support.

Reviewer's Checklist

• Changed code has unit tests for its functionality at or near 100% coverage.
• System-Tests covering this feature have been added and enabled with the va.b.c-dev version tag.
• There is a benchmark for any new code, or changes to existing code.
• If this interacts with the agent in a new way, a system test has been added.
• Add an appropriate team label so this PR gets put in the right place for the release notes.
• Non-trivial go.mod changes, e.g. adding new modules, are reviewed by @DataDog/dd-trace-go-guild.
• For internal contributors, a matching PR should be created to the v2-dev branch and reviewed by @DataDog/apm-go.

Unsure? Have a question? Request a review!

[datadog-datadog-prod-us1]

Datadog Report

Branch report: mtoff/health-metrics-3
Commit report: e129ede
Test service: dd-trace-go

✅ 0 Failed, 5121 Passed, 70 Skipped, 2m 54.52s Total Time

[pr-commenter]

Benchmarks

Benchmark execution time: 2024-12-23 16:30:12

Comparing candidate commit 286ed77 in PR branch mtoff/health-metrics-3 with baseline commit d393324 in branch main.

Found 3 performance improvements and 1 performance regressions! Performance is the same for 55 metrics, 0 unstable metrics.

scenario:BenchmarkSetTagMetric-24

• 🟩 execution_time [-7.943ns; -5.317ns] or [-6.119%; -4.096%]

scenario:BenchmarkSetTagString-24

• 🟩 execution_time [-7.771ns; -4.049ns] or [-6.073%; -3.165%]

scenario:BenchmarkSetTagStringer-24

• 🟩 execution_time [-8.327ns; -5.393ns] or [-5.282%; -3.421%]

scenario:BenchmarkTracerAddSpans-24

• 🟥 execution_time [+100.110ns; +185.690ns] or [+2.498%; +4.634%]

[darccio]

LGTM, just left a non-blocking question

[darccio]

@mtoffl01 @hannahkm QQ: wouldn't a blank integration already indicate it's a span created manually? Is this required somewhere else to have manual as value?

[hannahkm]

I believe the main concern is that leaving it blank might give the impression of being misconfigured/missing information. If we manually set it to manual here, it'll give a little more clarity.

[mtoffl01]

Yes, exactly what Hannah said.
From being on the troubleshooting side, an explicit value (e.g. "manual") is a good way to differentiate from an unset/missing value.

[darccio]

LGTM then