[feature request]NodePool Governance Capability: pool-coordinator supports share pool scope data in the nodepool

[Peeknut]

What would you like to be added:
As described in proposal: #804, Yurthub needs to add the following features:

• Leader yurthub start to list/watch pool scope data(endpoints, endpointslice) from cloud kube-apiserver, and store the response into pool-coordinator
• When all of pool scope data is synced, Leader yurthub should update a mark(like a lease) into pool-coordinator in order to specify that clients(like kube-proxy, coredns) can list/watch pool scope data from pool-coordinator instead of cloud kube-apiserver.
• Yurthub should switch requests of list/watch the pool-scope data between pool-coordinator and cloud kube-apiserver in terms of the status of pool-coordinator.

Notes: For details, please refer to the proposal: #804

Why is this needed:
As mentioned in the proposal(#804), the network traffic between cloud-edge will be reduced for more than 90%, because the pool-scope data is shared in nodePool.

others
/kind feature

[Congrool]

Hi，I want to take over this work.
/assign

[Congrool]

This work is almost finished, including:

1. add a client to send objs in response to etcd in pool-coordinator.
2. add a reverse proxy to re-direct requests to apiserver in pool-coordinator when the node cannot reach the cloud.
3. kubectl logs/exec/auth/api-versions/api-resources/get/list/watch to pool-coordinator has been tested

Some problems still unsolved:

1. cannot cache secret, which will be used when using InClusterConfig to setup pod.
2. unit tests have not been revised
3. resource gc in pool-coordinator. Considering that when an edge component uses GET request to get a resource, this resource will be cached into the pool-coordinator, but when to delete it?
4. support node-scope and pool-scope cache at the same time

[Congrool]

For ReverseProxy, the processing logic of request from edge side is something like:

switch{
case cloud.IsHealthy:
   if poolCoordinator.IsHealthy && ! LeaderYurthub && IsPoolScopeResource(req.Resource) {
    fallthrough
  }
  if err := cloud.Handle(req); err != nil {
    fallthrough
  }
case poolCoordinator.IsHealthy:
  if err := poolCoordinator.Handle(req); err != nil {
    fallthrough
  }
default:
  localCache.Handle(req)
}

So I need a way to determine whether this yurthub is leader yurthub.
What do you think? @rambohe-ch

[rambohe-ch]

For ReverseProxy, the processing logic of request from edge side is something like:

switch{
case cloud.IsHealthy:
   if poolCoordinator.IsHealthy && ! LeaderYurthub && IsPoolScopeResource(req.Resource) {
    fallthrough
  }
  if err := cloud.Handle(req); err != nil {
    fallthrough
  }
case poolCoordinator.IsHealthy:
  if err := poolCoordinator.Handle(req); err != nil {
    fallthrough
  }
default:
  localCache.Handle(req)
}

So I need a way to determine whether this yurthub is leader yurthub. What do you think? @rambohe-ch

@Congrool please wait me for a little time.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[Congrool]

/reopen

This feature is not completely implemented. So we need to reopen it.

[openyurt-bot]

@Congrool: Reopened this issue.

In response to this:

/reopen

This feature is not completely implemented. So we need to reopen it.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[Congrool]

I'll submit pr of pool-coordinator cache implementation. But I think we need to rebase the pool-coordinator-dev on #882 first.

@rambohe-ch What do you think?

[rambohe-ch]

I'll submit pr of pool-coordinator cache implementation. But I think we need to rebase the pool-coordinator-dev on #882 first.

@rambohe-ch What do you think?

@Congrool agree with you.

[Congrool]

/close

[openyurt-bot]

@Congrool: Closing this issue.

In response to this:

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.