Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

runtime/committee/client: reduce gRPC max backoff timeout #3035

Merged
merged 1 commit into from
Jun 19, 2020
Merged

runtime/committee/client: reduce gRPC max backoff timeout #3035

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .changelog/3035.bugfix.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
runtime/committee/client: Reduce gRPC max backoff timeout

Committee nodes are expected to be available and this timeout is more in line
with timeouts used in the clients using these connections.
20 changes: 19 additions & 1 deletion go/runtime/committee/client.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ import (
"time"

"google.golang.org/grpc"
"google.golang.org/grpc/backoff"
"google.golang.org/grpc/resolver"
"google.golang.org/grpc/resolver/manual"

Expand All @@ -22,7 +23,14 @@ import (
"github.com/oasisprotocol/oasis-core/go/common/pubsub"
)

const defaultCloseDelay = 5 * time.Second
const (
defaultCloseDelay = 5 * time.Second

// gRPC backoff max delay when establishing connections (default: 120).
grpcBackoffMaxDelay = 10 * time.Second
// Default.
grpcMinConnectTimeout = 20 * time.Second
)

// NodeSelectionFeedback is feedback to the node selection policy.
type NodeSelectionFeedback struct {
Expand Down Expand Up @@ -333,13 +341,23 @@ func (cc *committeeClient) updateConnectionLocked(n *node.Node) error {
cs.resolver = manual.NewBuilderWithScheme("oasis-core-resolver")
cs.resolver.InitialState(resolver.State{})

// Backoff config.
backoffConfig := backoff.DefaultConfig
backoffConfig.MaxDelay = grpcBackoffMaxDelay

// Create a virtual connection to the given node.
conn, err := cmnGrpc.Dial(
"oasis-core-resolver:///",
grpc.WithTransportCredentials(creds),
// https://github.com/grpc/grpc-go/issues/3003
grpc.WithDefaultServiceConfig(`{"loadBalancingPolicy":"round_robin"}`),
grpc.WithResolvers(cs.resolver),
grpc.WithConnectParams(
grpc.ConnectParams{
Backoff: backoffConfig,
MinConnectTimeout: grpcMinConnectTimeout,
},
),
)
if err != nil {
cc.logger.Warn("failed to dial node",
Expand Down