fix: Restore transient error detection for canceled hyper requests #840
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description:
The crate used to, but now fails to, detect gRPC requests that failed with a tonic error wrapping a hyper error where the hyper error was in a "cancelled" state.
This detection mechanism uses
downcast
to inspect a source (inner) error, and broke whenhyper
was upgraded from0.14.x
to1.x
on 4/26 in 2445034.cc @RickyLB / @mehcode
tonic 0.11
, the current dependency, andtonic 0.9
, the previous dependency before that commit, both depend onhyper 0.14
, and the downcast to a hyper 1.xError
type will not return the hyper 0.14.xError
type wrapped by tonic.This PR modifies
is_hyper_canceled
to check for theError
type from both versions of hyper.Alternatives Considered
hyper
crate maintainers to not provide public constructors for error variantsFixes #
Notes for reviewer:
This fixes the backoff/retry mechanism in the context of connection failures, such as:
That error should be retried, but is not, because the hyper error in a canceled state is not detected.
Logs after this PR is applied (and before the broken hyper upgrade linked above was applied):
Checklist