Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix issue with workers connecting in high latency conditions #4535

Merged
merged 1 commit into from
Mar 15, 2024

Conversation

jefferai
Copy link
Member

At some point we migrated to grpc ConnectParams to control how often retries were happening. One value that we didn't set was the MinConnectTimeout. It turns out that if you don't set this, it won't use the default; instead if will actually use a value of zero, which gRPC internally ups to something that seems to be around 25ms to meet a minimum threshold. This is not documented in the function comments.

This commit simply sets the value of MinConnectTimeout to the unexported gRPC default.

At some point we migrated to grpc ConnectParams to control how often
retries were happening. One value that we didn't set was the
MinConnectTimeout. It turns out that if you don't set this, it won't use
the default; instead if will actually use a value of zero, which gRPC
internally ups to something that seems to be around 25ms to meet a
minimum threshold. This is not documented in the function comments.

This commit simply sets the value of MinConnectTimeout to the unexported
gRPC default.
@jefferai jefferai added this to the 0.15.x milestone Mar 15, 2024
@jefferai jefferai requested a review from jimlambrt March 15, 2024 19:30
Copy link
Collaborator

@jimlambrt jimlambrt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ty

@jefferai jefferai merged commit 7e21402 into main Mar 15, 2024
46 checks passed
@jefferai jefferai deleted the jefferai-fix-high-latency-worker-connections branch March 15, 2024 19:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants