-
Notifications
You must be signed in to change notification settings - Fork 562
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Ignore context cancel error in otelgrpc #2644
Comments
Re I'm in the same position, where I'd like to ignore Likewise, the server interceptor should ignore Since this logic is not trivial, I'd suggest making it optional to avoid hiding exceptional cases. In the rest of my code, whenever I see a cancellation, I set a |
With this enabled, context.Canceled and gprc_codes.Canceled are considered OK. A separate attribute is set to true if a cancellation was detected. To avoid conflating situations where some downstream server component issued a cancel of a child context, and that is now propagated over gRPC, we first check if the parent context is canceled. It could be argued that status.Convert/FromError should use errors.Unwrap, since the cancelation that is being returned to a server interceptor was received from a downstream gRPC client. For the simplicity of this commit, this is left unchanged from before. Closes open-telemetry#2644.
This is still a real issue. It is very difficult to arrange a gRPC stream without making use of cancelation. The cancelation support works fine, but the resulting spans should not register as errors. Related to open-telemetry/otel-arrow#227. |
…34175) **Description:** We have been using the OTel-Arrow components in production and begun to monitor the component's health using span data. There are spans coming from otelgrpc instrumentation and there are explicit spans in both components. This adds a new test that exercises multiple streams between the exporter and receiver components. At the end of the test, it checks for no span errors, with one exception discussed in open-telemetry/opentelemetry-go-contrib#2644. We will not modify the logic of these components to avoid stream cancelation, we will lobby for stream cancelation not to register as span errors. The receiver is modified to avoid setting Span errors for EOF and Cancelation following its existing logic for setting log severity for the same condition. As a result, the `otel_arrow_stream_inflight` span will not show errors for the final `Recv()` as a stream shuts down. **Link to tracking Issue:** Part of open-telemetry/otel-arrow#227. **Testing:** A new test is added.
We usually close gRPC streams by canceling the context. The problem is the
otelgrpc
fill the error part of the span(context canceled) and the backend(X-Ray in my case) renders it as an error while it is just a simple close.One may argue that you should close the stream gracefully by sending
CloseSend
to prevent that. While it is true, it complicates the closing process and I think it is much easier to cancel the context to close the stream.Is there any way to ignore the
context cancel
error being reported?The text was updated successfully, but these errors were encountered: