client: Introduce `ServiceClient` #7489

CabinfeverB · 2023-12-04T03:12:46Z

What problem does this PR solve?

Issue Number: ref #7431 ref #7576

What is changed and how does it work?

This PR introduce the interface ServiceClient which can get the gRPC client of a specific gRPC server in a PD cluster. It can send request to the follower including follower-handle or forward.
And also a balancer is introduced.
Some tools have been added to client to make it possible to run integration tests in the client package. The reason for adding it to the client package is that ServiceClient should be invisible to the caller, so it cannot be tested well in tests/integrations.

Check List

Tests

Unit test
Integration test

Code changes

Has the configuration change
Has HTTP API interfaces changed (Don't forget to add the declarative for the new API)
Has persistent data change

Side effects

Possible performance regression
Increased code complexity
Breaking backward compatibility

Related changes

PR to update pingcap/docs/pingcap/docs-cn:
PR to update pingcap/tiup:
Need to cherry-pick to the release branch

Release note

None.

Signed-off-by: Cabinfever_B <[email protected]>

ti-chi-bot · 2023-12-04T03:12:50Z

[REVIEW NOTIFICATION]

This pull request has been approved by:

JmPotato
rleungx

To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment.
After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.

The full list of commands accepted by this bot can be found here.

Reviewer can indicate their review by submitting an approval review.
Reviewer can cancel approval by submitting a request changes review.

CabinfeverB · 2023-12-04T03:13:09Z

/cc @niubell

ti-chi-bot · 2023-12-04T03:13:12Z

@CabinfeverB: GitHub didn't allow me to request PR reviews from the following users: niubell.

Note that only tikv members and repo collaborators can review this PR, and authors cannot review their own PRs.

In response to this:

/cc @niubell

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

Signed-off-by: Cabinfever_B <[email protected]>

codecov · 2023-12-04T03:29:54Z

Codecov Report

Merging #7489 (970cc22) into master (f1870c1) will decrease coverage by 0.01%.
The diff coverage is 73.22%.

❗ Current head 970cc22 differs from pull request most recent head cd364e2. Consider uploading reports for the commit cd364e2 to get more accurate results

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #7489      +/-   ##
==========================================
- Coverage   74.75%   74.75%   -0.01%     
==========================================
  Files         457      459       +2     
  Lines       50415    50541     +126     
==========================================
+ Hits        37686    37780      +94     
- Misses       9394     9412      +18     
- Partials     3335     3349      +14

Flag	Coverage Δ
unittests	`74.75% <73.22%> (-0.01%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

client/grpcutil/grpcutil.go

rleungx · 2023-12-04T03:21:51Z

client/pd_service_discovery.go

+	Available() bool
+	// CheckAvailable checks the status about network connection or other availability of the current service client
+	CheckAvailable()


What is the difference?

CheckAvailable is used to try to mark the client as available if it is in unavailable state. It's not quite the right name. Any suggestion?

Then, why not merge these two?

let me have a try

rleungx · 2023-12-04T03:29:57Z

client/pd_service_discovery.go

+	// CheckNetworkAvailable checks if the network connection for the current service client is available
+	CheckNetworkAvailable(context.Context)
+	// RespToErr checks if client need to retry based on the PD server error response.
+	RespToErr(*pdpb.Error, error) bool


Is it necessary?

I need a method to mark the follower client as unavailable

How about just NeedRetry/IsRetryable or something similar?

Signed-off-by: Cabinfever_B <[email protected]>

client/grpcutil/grpcutil.go

client/pd_service_discovery.go

client/testutil/check_env_linux.go

rleungx · 2023-12-11T09:30:00Z

client/pd_service_discovery.go

+	i := 0
+	defer c.mu.Unlock()


exchange these two?

rleungx · 2023-12-11T09:32:27Z

client/pd_service_discovery.go

+		return nil
+	}
+	for ; i < c.totalNode; i++ {
+		if c.now.Available() {


If it's not available, will it be excluded in the next round?

No. I don't remove unavailable nodes from the queue. The simple approach is taken now and can be optimized in the future if needed

rleungx · 2023-12-11T09:33:19Z

client/pd_service_discovery.go

+	// mustLeader: whether must send to leader.
+	BuildGRPCContext(ctx context.Context, mustLeader bool) context.Context
+	// IsLeader returns whether the target PD server is leader.
+	IsLeader() bool


What if the leader changes?

If leader changes, it will create new ServiceClient and store it in leader

How about targetIsLeader or something else?

Signed-off-by: Cabinfever_B <[email protected]>

rleungx · 2023-12-14T08:53:42Z

client/pd_service_discovery.go

+}
+
+// BuildGRPCContext implements ServiceClient.
+func (c *pdServiceClient) BuildGRPCContext(ctx context.Context, toLeader bool) context.Context {


What is the difference between toLeader and isLeader?

The function of the ServiceClient is to interact with a PD server, which may be a follower or leader. When a client sends a request to a server, mutLeader is true if the request is forwarded to the leader. Otherwise, it is sent to the server to process the request, and mutLeader is false.

rleungx · 2023-12-14T08:59:39Z

client/pd_service_discovery.go

+
+// pdServiceBalancer is a load balancer for PD service clients.
+// It is used to balance the request to all servers and manage the connections to multiple PD service nodes.
+type pdServiceBalancer struct {


How about supporting multiple policies? And make the round-robin a common library?

I am a bit confused here because it seems that there is forwarding logic in the internal ServiceClient, and now we have introduced an additional layer of the load balancer. Will these two things conflict with each other?

I am a bit confused here because it seems that there is forwarding logic in the internal ServiceClient, and now we have introduced an additional layer of the load balancer. Will these two things conflict with each other?

There will be no conflict. In the past, followers were randomly selected for forwarding, but now it is in turn

How about supporting multiple policies? And make the round-robin a common library?

I've thought about it. Since it is observed that there is now only one Round-Robin balancer in /pkg/balancer, I haven't implemented multiple policies yet

CabinfeverB · 2023-12-18T08:17:50Z

@JmPotato PTAL~

JmPotato · 2023-12-18T08:39:47Z

client/pd_service_discovery.go

+	// BuildGRPCContext builds a context object with a gRPC context.
+	// ctx: the original context object.
+	// mustLeader: whether must send to leader.
+	BuildGRPCContext(ctx context.Context, mustLeader bool) context.Context


According to the name, it's hard to tell what this build is for. Suggest using a more specified name like BuildGRPCTargetContext.

JmPotato · 2023-12-18T08:40:21Z

client/pd_service_discovery.go

+var _ ServiceClient = (*pdServiceClient)(nil)
+var _ ServiceClient = (*pdServiceAPIClient)(nil)


Suggested change

var _ ServiceClient = (*pdServiceClient)(nil)

var _ ServiceClient = (*pdServiceAPIClient)(nil)

var (

_ ServiceClient = (*pdServiceClient)(nil)

_ ServiceClient = (*pdServiceAPIClient)(nil)

)

JmPotato · 2023-12-18T08:41:42Z

client/pd_service_discovery.go

+	networkFailure atomic.Bool
+}
+
+func newPDServiceClient(addr, leaderAddr string, conn *grpc.ClientConn, isLeader bool) *pdServiceClient {


Maybe using the interface as the returned value is better.

Suggested change

func newPDServiceClient(addr, leaderAddr string, conn *grpc.ClientConn, isLeader bool) *pdServiceClient {

func newPDServiceClient(addr, leaderAddr string, conn *grpc.ClientConn, isLeader bool) ServiceClient {

JmPotato · 2023-12-18T08:42:45Z

client/pd_service_discovery.go

+}
+
+// NetworkAvailable implements ServiceClient.
+func (c *pdServiceClient) Available() bool {


Since the judgment condition is whether there is a network error, would Reachable be a better name?

pdServiceAPIClient checks the network and availability

JmPotato · 2023-12-18T08:47:47Z

client/pd_service_discovery.go

+
+// pdServiceBalancer is a load balancer for PD service clients.
+// It is used to balance the request to all servers and manage the connections to multiple PD service nodes.
+type pdServiceBalancer struct {


I am a bit confused here because it seems that there is forwarding logic in the internal ServiceClient, and now we have introduced an additional layer of the load balancer. Will these two things conflict with each other?

Signed-off-by: Cabinfever_B <[email protected]>

rleungx

The rest LGTM.

rleungx · 2023-12-20T08:39:39Z

client/go.mod

@@ -19,6 +20,8 @@ require (
 	google.golang.org/grpc v1.54.0
 )

+require google.golang.org/genproto/googleapis/rpc v0.0.0-20231106174013-bbf56f31fb17 // indirect


need format

rleungx · 2023-12-20T08:40:30Z

client/pd_service_discovery.go

+	// mustLeader: whether must send to leader.
+	BuildGRPCContext(ctx context.Context, mustLeader bool) context.Context
+	// IsLeader returns whether the target PD server is leader.
+	IsLeader() bool


How about targetIsLeader or something else?

Signed-off-by: Cabinfever_B <[email protected]>

JmPotato · 2023-12-21T07:18:11Z

client/testutil/check_env.go

Maybe the name check_env_dummy.go is better.

Also renamed the file in /pkg/utils/tempurl. PTAL

Signed-off-by: Cabinfever_B <[email protected]>

…clinet

Signed-off-by: Cabinfever_B <[email protected]>

CabinfeverB · 2023-12-21T09:43:45Z

/merge

ti-chi-bot · 2023-12-21T09:43:46Z

@CabinfeverB: It seems you want to merge this PR, I will help you trigger all the tests:

/run-all-tests

You only need to trigger /merge once, and if the CI test fails, you just re-trigger the test that failed and the bot will merge the PR for you after the CI passes.

If you have any questions about the PR merge process, please refer to pr process.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

ti-chi-bot · 2023-12-21T09:43:48Z

This pull request has been accepted and is ready to merge.

Commit hash: 970cc22

ti-chi-bot · 2023-12-21T09:44:01Z

@CabinfeverB: Your PR was out of date, I have automatically updated it for you.

If the CI test fails, you just re-trigger the test that failed and the bot will merge the PR for you after the CI passes.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

add service client

7e725fa

Signed-off-by: Cabinfever_B <[email protected]>

CabinfeverB requested review from lhy1024 and rleungx December 4, 2023 03:12

ti-chi-bot bot added release-note-none Denotes a PR that doesn't merit a release note. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. labels Dec 4, 2023

add service client

889c0eb

Signed-off-by: Cabinfever_B <[email protected]>

rleungx reviewed Dec 4, 2023

View reviewed changes

CabinfeverB added 5 commits December 4, 2023 13:47

address comment

6315793

Signed-off-by: Cabinfever_B <[email protected]>

address comment

6535c6e

Signed-off-by: Cabinfever_B <[email protected]>

address comment

423717c

Signed-off-by: Cabinfever_B <[email protected]>

Merge branch 'master' into follower/service_clinet

013fbce

address comment

0891cf1

Signed-off-by: Cabinfever_B <[email protected]>

CabinfeverB mentioned this pull request Dec 6, 2023

*: follower support to handle GetRegion and other region api #7432

Merged

rleungx reviewed Dec 11, 2023

View reviewed changes

address comment

f0c694c

Signed-off-by: Cabinfever_B <[email protected]>

rleungx reviewed Dec 14, 2023

View reviewed changes

CabinfeverB requested a review from JmPotato December 18, 2023 08:17

JmPotato reviewed Dec 18, 2023

View reviewed changes

CabinfeverB added 2 commits December 19, 2023 11:13

address comment

defcfa6

Signed-off-by: Cabinfever_B <[email protected]>

Merge branch 'master' into follower/service_clinet

42dc707

rleungx approved these changes Dec 20, 2023

View reviewed changes

ti-chi-bot bot added the status/LGT1 Indicates that a PR has LGTM 1. label Dec 20, 2023

CabinfeverB added 2 commits December 20, 2023 17:39

address comment

3cbc33e

Signed-off-by: Cabinfever_B <[email protected]>

Merge branch 'master' into follower/service_clinet

6327e14

address comment

fe10b28

Signed-off-by: Cabinfever_B <[email protected]>

JmPotato reviewed Dec 21, 2023

View reviewed changes

CabinfeverB added 4 commits December 21, 2023 16:25

address comment

e9f71d8

Signed-off-by: Cabinfever_B <[email protected]>

address comment

c200dad

Signed-off-by: Cabinfever_B <[email protected]>

Merge remote-tracking branch 'upstream/master' into follower/service_…

ea95d4f

…clinet

fix test

970cc22

Signed-off-by: Cabinfever_B <[email protected]>

JmPotato approved these changes Dec 21, 2023

View reviewed changes

ti-chi-bot bot added status/LGT2 Indicates that a PR has LGTM 2. and removed status/LGT1 Indicates that a PR has LGTM 1. labels Dec 21, 2023

ti-chi-bot bot added the status/can-merge Indicates a PR has been approved by a committer. label Dec 21, 2023

Merge branch 'master' into follower/service_clinet

cd364e2

ti-chi-bot bot merged commit 74ef91d into tikv:master Dec 21, 2023
22 checks passed

This was referenced Dec 22, 2023

Enhance ServiceDiscovery to improve client high availability and load balance #7576

Open

Activate Follower-Handle #7431

Closed

		var _ ServiceClient = (*pdServiceClient)(nil)
		var _ ServiceClient = (*pdServiceAPIClient)(nil)

	func newPDServiceClient(addr, leaderAddr string, conn grpc.ClientConn, isLeader bool) pdServiceClient {
	func newPDServiceClient(addr, leaderAddr string, conn *grpc.ClientConn, isLeader bool) ServiceClient {

client: Introduce ServiceClient #7489

client: Introduce ServiceClient #7489

Conversation

CabinfeverB commented Dec 4, 2023 • edited Loading

What problem does this PR solve?

What is changed and how does it work?

Check List

Release note

ti-chi-bot bot commented Dec 4, 2023 • edited Loading

CabinfeverB commented Dec 4, 2023

ti-chi-bot bot commented Dec 4, 2023

codecov bot commented Dec 4, 2023 • edited Loading

Codecov Report

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

CabinfeverB commented Dec 18, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rleungx left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

CabinfeverB commented Dec 21, 2023

ti-chi-bot bot commented Dec 21, 2023

ti-chi-bot bot commented Dec 21, 2023

ti-chi-bot bot commented Dec 21, 2023

client: Introduce `ServiceClient` #7489

client: Introduce `ServiceClient` #7489

CabinfeverB commented Dec 4, 2023 •

edited

Loading

ti-chi-bot bot commented Dec 4, 2023 •

edited

Loading

codecov bot commented Dec 4, 2023 •

edited

Loading