pool-coordinator implementation of yurthub #1073

Congrool · 2022-11-23T07:03:01Z

What type of PR is this?

/kind feature

What this PR does / why we need it:

Yurthub portion of pool-coordinator implementation, which includes the following functions:

sync pool-scoped resources(currently endpointslices and endpoints) to pool-coordinator
send portion of local cache(currently pods and nodes) to pool-coordinator
proxy heartbeat(kube-node-lease) to cloud site for follower yurthubs when their connections to cloud have broken
cache responses of requests to pool-coordinator which ask the cloud for pool-scoped resources
proxy requests to pool-coordinator when follower yurthub offline

Which issue(s) this PR fixes:

Fixes #778

Special notes for your reviewer:

Does this PR introduce a user-facing change?

other Note

openyurt-bot · 2022-11-23T07:03:05Z

@Congrool: GitHub didn't allow me to assign the following users: your_reviewer.

Note that only openyurtio members, repo collaborators and people who have commented on this issue/PR can be assigned. Additionally, issues/PRs can only have 10 assignees at the same time.
For more information please see the contributor guide

In response to this:

What type of PR is this?

/kind feature

What this PR does / why we need it:

Yurthub portion of pool-coordinator implementation, which includes the following functions:

sync pool-scoped resources(currently endpointslices and endpoints) to pool-coordinator

send portion of local cache(currently pods and nodes) to pool-coordinator

proxy kube-node-lease for follower yurthubs to cloud site

cache responses of requests to pool-coordinator which ask the cloud for pool-scoped resources

proxy requests to pool-coordinator when follower yurthub offline

Which issue(s) this PR fixes:

Fixes #

Special notes for your reviewer:

Does this PR introduce a user-facing change?
other Note

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

openyurt-bot · 2022-11-23T07:03:12Z

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: Congrool
To complete the pull request process, please assign kadisi
You can assign the PR to them by writing /assign @kadisi in a comment when ready.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

Congrool · 2022-11-23T07:06:50Z

The implementation currently has not been tested.

pkg/yurthub/poolcoordinator/coordinator.go

rambohe-ch · 2022-11-23T12:22:24Z

pkg/yurthub/storage/etcd/storage.go

+	values := make([][]byte, 0, len(getResp.Kvs))
+	for _, kv := range getResp.Kvs {
+		values = append(values, kv.Value)
+	}


Line 275~277 should be as following:

for i, kv := range getResp.Kvs { values[i] = append(values[i], kv.Value...) }

rambohe-ch · 2022-11-23T12:45:54Z

pkg/yurthub/poolcoordinator/coordinator.go

+	if time.Now().After(renewTime.Add(defaultPoolCacheStaleDuration)) {
+		coordinator.Lock()
+		defer coordinator.Unlock()
+		coordinator.isPoolCacheSynced = false


coordinator.isPoolCacheSynced = true?

pkg/yurthub/proxy/remote/loadbalancer.go

Congrool · 2022-11-30T02:35:23Z

fixes #778 , need to link it

rambohe-ch · 2022-11-30T02:38:28Z

fixes #778 , need to link it

@Congrool done

Congrool · 2022-12-01T10:04:32Z

TODO List:

Distinguish where logs/exec request is from, Cloud APIServer or Coodinator.
Need a way to record the watch request, and return err when proxy handler switch happens. To avoid using rv of pool-coordinator to list/watch cloud apiserver or vise versa.
Need a GracePeriod to tolerate pool-coordinator being unavailable for a short period.

Test:

Automatically setup etcd before running etcd storage test
Test for Coordinator component.
New unit tests to cover the logic of Proxy and CacheResponse

Signed-off-by: Congrool <[email protected]>

rambohe-ch · 2022-12-07T09:15:54Z

@Congrool I will merge this pull request for integration test, and you can complete works in the todo list when making integration test.

Signed-off-by: Congrool <[email protected]>

…er of this client is healthy Signed-off-by: Congrool <[email protected]>

Signed-off-by: Congrool <[email protected]>

…esources Signed-off-by: Congrool <[email protected]>

Signed-off-by: Congrool <[email protected]>

…ool when pool coordinator becomes ready Signed-off-by: Congrool <[email protected]>

Signed-off-by: Congrool <[email protected]>

Congrool · 2022-12-07T09:25:07Z

OK, please go ahead @rambohe-ch

* pool-coordinator implementation of yurthub Signed-off-by: Congrool <[email protected]>

openyurt-bot added the kind/feature kind/feature label Nov 23, 2022

openyurt-bot requested review from huangyuqi and rambohe-ch November 23, 2022 07:03

openyurt-bot added the size/XXL label Nov 23, 2022

Congrool force-pushed the poolcoordinator branch from 435527d to f1c3f6f Compare November 23, 2022 08:05

rambohe-ch reviewed Nov 23, 2022

View reviewed changes

pkg/yurthub/poolcoordinator/coordinator.go Show resolved Hide resolved

rambohe-ch reviewed Nov 23, 2022

View reviewed changes

pkg/yurthub/proxy/remote/loadbalancer.go Outdated Show resolved Hide resolved

rambohe-ch reviewed Nov 23, 2022

View reviewed changes

pkg/yurthub/proxy/remote/loadbalancer.go Outdated Show resolved Hide resolved

rambohe-ch reviewed Nov 24, 2022

View reviewed changes

pkg/yurthub/proxy/remote/loadbalancer.go Outdated Show resolved Hide resolved

rambohe-ch reviewed Nov 24, 2022

View reviewed changes

pkg/yurthub/proxy/remote/loadbalancer.go Outdated Show resolved Hide resolved

rambohe-ch linked an issue Nov 30, 2022 that may be closed by this pull request

[feature request]NodePool Governance Capability: pool-coordinator supports share pool scope data in the nodepool #778

Closed

Congrool added 4 commits December 7, 2022 17:05

init etcd storage

c33f71f

Signed-off-by: Congrool <[email protected]>

add ut for etcdStorage

897fcc5

Signed-off-by: Congrool <[email protected]>

add unit tests for etcd storage

6b28ef6

Signed-off-by: Congrool <[email protected]>

improve lifecycle context management of yurthub and etcd storage

c94d3a2

Signed-off-by: Congrool <[email protected]>

Congrool force-pushed the poolcoordinator branch from 947dc5d to 4d9e05c Compare December 7, 2022 09:06

Congrool added 5 commits December 7, 2022 17:22

add pool coordinator proxy

d10ad4b

Signed-off-by: Congrool <[email protected]>

implement coordinator

e48264c

Signed-off-by: Congrool <[email protected]>

integrate coordinator into yurthub

8950261

Signed-off-by: Congrool <[email protected]>

avoid race condition when elector status changes

ef97f73

Signed-off-by: Congrool <[email protected]>

use rest manager to generate cloud kubeclient to ensure that the serv…

75aca26

…er of this client is healthy Signed-off-by: Congrool <[email protected]>

Congrool added 13 commits December 7, 2022 17:23

use separated user-agent for pool-scoped resources list/watch request

2b77999

Signed-off-by: Congrool <[email protected]>

only allow leader yurthub update pool-scoped cache

fedd1cc

Signed-off-by: Congrool <[email protected]>

remove old pool-scoped resources before sync

511f801

Signed-off-by: Congrool <[email protected]>

cache response to local when proxy to pool-coordinator

2f82744

Signed-off-by: Congrool <[email protected]>

add comments and fix lint

dd4e835

Signed-off-by: Congrool <[email protected]>

only save cache to poolcoordinator when leader asks for pool-scoped r…

53f8adb

…esources Signed-off-by: Congrool <[email protected]>

optimize proxy handler for different resources

b7d60d2

Signed-off-by: Congrool <[email protected]>

fix cache response and add GZipReaderCloser

d58dc4b

Signed-off-by: Congrool <[email protected]>

add ability of switching pool-scoped resource request from cloud to p…

262d9ec

…ool when pool coordinator becomes ready Signed-off-by: Congrool <[email protected]>

stop write to pool-coordinator when giving up leadership

eb45555

Signed-off-by: Congrool <[email protected]>

cancel local watch when pool-coordinator is ready

82948b9

Signed-off-by: Congrool <[email protected]>

only cache node and pod resources to pool-coordinator

fcb0c31

Signed-off-by: Congrool <[email protected]>

add options for coordinator storage

9306fa0

Signed-off-by: Congrool <[email protected]>

Congrool force-pushed the poolcoordinator branch from 4d9e05c to 9306fa0 Compare December 7, 2022 09:23

rambohe-ch merged commit 61b6716 into openyurtio:pool-coordinator-dev Dec 7, 2022

Congrool added a commit to Congrool/openyurt that referenced this pull request Jan 17, 2023

pool-coordinator implementation of yurthub (openyurtio#1073)

3e69468

* pool-coordinator implementation of yurthub Signed-off-by: Congrool <[email protected]>

Congrool mentioned this pull request Jan 17, 2023

cherry-pick yurthub code of pool-coordinator-dev #1156

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

pool-coordinator implementation of yurthub #1073

pool-coordinator implementation of yurthub #1073

Congrool commented Nov 23, 2022 •

edited

Loading

openyurt-bot commented Nov 23, 2022

What type of PR is this?

What this PR does / why we need it:

Which issue(s) this PR fixes:

Special notes for your reviewer:

Does this PR introduce a user-facing change?

other Note

openyurt-bot commented Nov 23, 2022

Congrool commented Nov 23, 2022

rambohe-ch Nov 23, 2022

rambohe-ch Nov 23, 2022

Congrool commented Nov 30, 2022

rambohe-ch commented Nov 30, 2022

Congrool commented Dec 1, 2022 •

edited

Loading

rambohe-ch commented Dec 7, 2022

Congrool commented Dec 7, 2022

pool-coordinator implementation of yurthub #1073

pool-coordinator implementation of yurthub #1073

Conversation

Congrool commented Nov 23, 2022 • edited Loading

What type of PR is this?

What this PR does / why we need it:

Which issue(s) this PR fixes:

Special notes for your reviewer:

Does this PR introduce a user-facing change?

other Note

openyurt-bot commented Nov 23, 2022

What type of PR is this?

What this PR does / why we need it:

Which issue(s) this PR fixes:

Special notes for your reviewer:

Does this PR introduce a user-facing change?

other Note

openyurt-bot commented Nov 23, 2022

Congrool commented Nov 23, 2022

rambohe-ch Nov 23, 2022

Choose a reason for hiding this comment

rambohe-ch Nov 23, 2022

Choose a reason for hiding this comment

Congrool commented Nov 30, 2022

rambohe-ch commented Nov 30, 2022

Congrool commented Dec 1, 2022 • edited Loading

rambohe-ch commented Dec 7, 2022

Congrool commented Dec 7, 2022

Congrool commented Nov 23, 2022 •

edited

Loading

Congrool commented Dec 1, 2022 •

edited

Loading