Add .NET network + HTTP connection spans #1192

lmolkova · 2024-06-27T23:20:31Z

This PR introduces additional network-level spans to-be emitted by .NET 9 🤞

Related to #454, #1226

In particular:

DNS, TLS, and socket connect spans that track individual stages of connection establishment
wrapper HTTP connection setup span for above spans
wait-for-connection span that signals when a connection is not immediately available for the request
- link that connects wait-for-connection to the connection setup
- is expected to be off by default

On .NET every ActivitySource (aka tracer) is opt-in, so users have full control over what they are enabling.

The volume of new telemetry is expected to be low (with caveats):

The connection-level spans (groups 1 and 2 above) are rare - connections are created rarely.
Wait-for-connection spans are only reported when connection is not readily available and would be present when connection pool is overloaded.
Links to connection setup are most concerning since we don't know how backends are handling incoming links. See Link visualizations concerns: too many incoming links, showing transitive links opentelemetry-specification#4130 for the context. This is off by default docs)?

Overall, the feature is intended to help debug issues like:

misconfigured connection pool
DNS/TLS
abandoned response streams (or slow response stream reading) that result in connection staying busy
...

docs/dotnet/dotnet-connection-traces.md

docs/attributes-registry/dns.md

docs/dotnet/dotnet-connection-traces.md

docs/dotnet/dotnet-network-traces.md

antonfirsov

Sorry for the delay, I wanted to do it in tandem with dotnet/docs#43872 to make sure the review goes deep enough.

docs/dotnet/dotnet-network-traces.md

lmolkova · 2024-12-10T07:01:29Z

@antonfirsov I think I addressed you comments (all except listing error codes), could you please take another look?

.NET 9 introduced additions to `System.Net` distributed tracing which define a contract following OTel recommendations: - dotnet/runtime#104251 adjusted the HTTP client request activity so it natively complies with OTel recommendations. This included adding standard tags. - dotnet/runtime#103922 introduced activities breaking down the HTTP connection setup. This PR documents the activities together with their attributes in a manner similar to the metrics docs added by #37213. The matching semconv PR is open-telemetry/semantic-conventions#1192. Conceptual docs are being introduced in #42830. Moreover, the PR also adjusts certain parts of the metrics doc in order to make it more accurate, synchronize it with the activities doc and incorporate the changes from #42810. --------- Co-authored-by: Genevieve Warren <[email protected]> Co-authored-by: Miha Zupan <[email protected]>

lmolkova commented Jun 27, 2024

View reviewed changes

docs/dotnet/dotnet-connection-traces.md Outdated Show resolved Hide resolved

antonfirsov reviewed Jun 28, 2024

View reviewed changes

docs/dotnet/dotnet-connection-traces.md Outdated Show resolved Hide resolved

docs/dotnet/dotnet-connection-traces.md Outdated Show resolved Hide resolved

docs/dotnet/dotnet-connection-traces.md Outdated Show resolved Hide resolved

lmolkova force-pushed the dotnet-http-connection-spans branch from 7d7c232 to 9a4ba14 Compare July 3, 2024 00:59