[otelgin]: Catch and record panics in Middleware

[asecondo]

Problem Statement

Today, the otelgin middleware does not handle panics. In the event that there's some bug in some service code or lower middleware layer that panics, this information is not present in the resulting span. It would be incredibly useful to leverage this information to help troubleshooting in development and production environments.

Proposed Solution

Catch, record, and "rethrow" any panics. This solution would likely look very similar to the otelgin.HTML(...) handler here.

Alternatives

• Do nothing, as is currently done today
• Add panic recovery handling to this middleware. This feels incorrect, as we'd be going beyond the expected responsibilities of this middleware.

Prior Art

As mentioned above, we would likely leverage a similar solution used in the same package here.

Additional Context

N/A.

[pellared]

None of our instrumentation libraries handles panics. Exception is (as you mentioned) otelgin.HTML, which does not look right to me. I would propose removing panic handling from otelgin.HTML.

The SDK is responsible for handling panics. span.End recovers in
https://github.com/open-telemetry/opentelemetry-go/blob/8b510c36ed26fafe151df35a251f4589f5959802/sdk/trace/span.go#L420

Therefore, any panic happening should already be caught by the defer span.End().

We could consider setting error span status in span.End when a panic occurs. However, this should be a separate issue in https://github.com/open-telemetry/opentelemetry-go. @MrAlias, was it ever proposed or discussed?

Similar comments:

• feat(otelgin): add panic recovery and test panic handling #6366 (comment)
• feat(otelgin): add support for recording panics #5090 (comment)

[flc1125]

From the code, although it handles panic, there is no situation where Status is set to Error. However, in actual application, it is very necessary.

open-telemetry/opentelemetry-go@8b510c3/sdk/trace/span.go#L420