Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update Precedence config and resources output #3930

Merged
merged 1 commit into from
Jun 23, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/multicluster/quick-start.md
Original file line number Diff line number Diff line change
Expand Up @@ -56,7 +56,7 @@ Namepsace `kube-system`.
```bash
$kubectl apply -f https://github.com/antrea-io/antrea/releases/download/$TAG/antrea-multicluster-leader-global.yml
$kubectl create ns antrea-multicluster
$kubectl apply -f https://github.com/antrea-io/antrea/releases/download/$TAG/antrea-multicluster-leader-namespaced.yml > antrea-multicluster-leader-namespaced.yml
$kubectl apply -f https://github.com/antrea-io/antrea/releases/download/$TAG/antrea-multicluster-leader-namespaced.yml
$kubectl apply -f https://github.com/antrea-io/antrea/releases/download/$TAG/antrea-multicluster-member.yml
```

Expand Down
51 changes: 18 additions & 33 deletions docs/multicluster/user-guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -96,7 +96,7 @@ To deploy Multi-cluster Controller in a dual-role cluster, please refer to

```bash
kubectl create ns antrea-multicluster
kubectl apply -f https://github.com/antrea-io/antrea/releases/download/$TAG/antrea-multicluster-leader-namespaced.yml > antrea-multicluster-leader-namespaced.yml
kubectl apply -f https://github.com/antrea-io/antrea/releases/download/$TAG/antrea-multicluster-leader-namespaced.yml
```

The Multi-cluster Controller in the leader cluster will be deployed in Namespace `antrea-multicluster`
Expand Down Expand Up @@ -396,18 +396,12 @@ $kubectl annotate node node-1 multicluster.antrea.io/gateway=true
```

Multi-cluster Controller in the member cluster will detect the Gateway Node, and
create a `Gateway` CR with the same name as the Node. You can check the `Gateway`
with: `kubectl get gateway node-1 -o yaml`, which should show the Gateway
information like:
create a `Gateway` CR with the same name as the Node. You can check it with command:

```yaml
apiVersion: multicluster.crd.antrea.io/v1alpha1
kind: Gateway
metadata:
name: node-1
namespace: kube-system
gatewayIP: 10.17.27.55
internalIP: 10.17.27.55
```bash
$kubectl get gateway -n kube-system
NAME GATEWAY IP INTERNAL IP AGE
node-1 10.17.27.55 10.17.27.55 10s
```

`internalIP` of the Gateway is used for the tunnels between the Gateway Node and
Expand All @@ -419,7 +413,7 @@ there are several possibilities:

* By default, the K8s Node's `InternalIP` is used as `gatewayIP` too.
* You can choose to use the K8s Node's `ExternalIP` as `gatewayIP`, by changing
the configuration option `gatewayIPPrecedence` to value: `public`, when
the configuration option `gatewayIPPrecedence` to value: `external`, when
deploying the member Multi-cluster Controller. The configration option is
defined in ConfigMap `antrea-mc-controller-config-***` in `antrea-multicluster-member.yml`.
* When the Gateway Node has a separate IP for external communication or is
Expand All @@ -442,21 +436,12 @@ ConfigMap `antrea-mc-controller-config-***`. In other member clusters, a
ClusterInfoImport CR will be created for the cluster which includes the
exported network information. For example, in cluster `test-cluster-west`, you
you can see a ClusterInfoImport CR with name `test-cluster-east-clusterinfo`
is created for cluster `test-cluster-east`. You can check the
`ClusterInfoImport` with command: `kubectl get clusterinfoimport -o yaml`,
which should show information like:
is created for cluster `test-cluster-east`:

```yaml
apiVersion: multicluster.crd.antrea.io/v1alpha1
kind: ClusterInfoImport
metadata:
name: test-cluster-east-clusterinfo
namespace: kube-system
spec:
clusterID: test-cluster-east
gatewayInfos:
- gatewayIP: 10.17.27.55
serviceCIDR: 110.96.0.0/20
```bash
$kubectl get clusterinfoimport -n kube-system
NAME CLUSTER ID SERVICE CIDR AGE
test-cluster-east-clusterinfo test-cluster-east 110.96.0.0/20 10s
```

Make sure you repeat the same step to assign a Gateway Node in all member
Expand Down Expand Up @@ -505,14 +490,14 @@ ResourceImport CRs. You can check them in the leader cluster with commands:

```bash
$kubectl get resourceexport -n antrea-multicluster
luolanzone marked this conversation as resolved.
Show resolved Hide resolved
NAME AGE
test-cluster-west-default-nginx-endpoints 30s
test-cluster-west-default-nginx-service 30s
NAME CLUSTER ID KIND NAMESPACE NAME AGE
test-cluster-west-default-nginx-endpoints test-cluster-west Endpoints default nginx 30s
test-cluster-west-default-nginx-service test-cluster-west Service default nginx 30s

$kubectl get resourceimport -n antrea-multicluster
NAME AGE
default-nginx-endpoints 99s
default-nginx-service 99s
NAME KIND NAMESPACE NAME AGE
default-nginx-endpoints Endpoints default nginx 99s
default-nginx-service ServiceImport default nginx 99s
```

When there is any new change on the exported Service, the imported multi-cluster
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -25,8 +25,10 @@ import (
type Precedence string

const (
PrecedencePrivate = "private"
PrecedencePublic = "public"
PrecedencePrivate = "private"
PrecedencePublic = "public"
PrecedenceInternal = "internal"
PrecedenceExternal = "external"
)

//+kubebuilder:object:root=true
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -158,14 +158,14 @@ func (r *LeaderClusterSetReconciler) runBackgroundTasks() {
}

// updateStatus updates ClusterSet Status as follows:
// 1. TotalClusters is the number of Leader and Member clusters
// in the ClusterSet resource last processed.
// 1. TotalClusters is the number of member clusters in the
// ClusterSet resource last processed.
// 2. ObservedGeneration is the Generation from the last processed
// ClusterSet resource.
// 3. Individual cluster status is obtained from MemberClusterAnnounce
// controller.
// 3. ReadyClusters is the number of clusters with "Ready" = "True"
// 4. Overall condition of the ClusterSet is also computed as follows:
// 4. ReadyClusters is the number of member clusters with "Ready" = "True"
// 5. Overall condition of the ClusterSet is also computed as follows:
// a. "Ready" = "True" if all clusters have "Ready" = "True".
// Message & Reason will be absent.
// b. "Ready" = "Unknown" if all clusters have "Ready" = "Unknown".
Expand All @@ -184,7 +184,7 @@ func (r *LeaderClusterSetReconciler) updateStatus() {
}

status := multiclusterv1alpha1.ClusterSetStatus{}
status.TotalClusters = int32(len(r.clusterSetConfig.Spec.Members) + len(r.clusterSetConfig.Spec.Leaders))
status.TotalClusters = int32(len(r.clusterSetConfig.Spec.Members))
status.ObservedGeneration = r.clusterSetConfig.Generation
clusterStatues := r.StatusManager.GetMemberClusterStatuses()
status.ClusterStatuses = clusterStatues
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -233,7 +233,7 @@ func TestLeaderClusterStatus(t *testing.T) {
actualStatus := clusterSet.Status
expectedStatus := mcsv1alpha1.ClusterSetStatus{
ObservedGeneration: 1,
TotalClusters: 3,
TotalClusters: 2,
ClusterStatuses: statues,
Conditions: []mcsv1alpha1.ClusterSetCondition{
{
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -241,7 +241,7 @@ func (r *MemberClusterSetReconciler) updateStatus() {
}

status := multiclusterv1alpha1.ClusterSetStatus{}
status.TotalClusters = int32(len(r.clusterSetConfig.Spec.Members) + len(r.clusterSetConfig.Spec.Leaders))
status.TotalClusters = int32(len(r.clusterSetConfig.Spec.Members))
status.ObservedGeneration = r.clusterSetConfig.Generation
status.ClusterStatuses = r.remoteCommonAreaManager.GetMemberClusterStatues()

Expand Down
7 changes: 4 additions & 3 deletions multicluster/controllers/multicluster/node_controller.go
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,7 @@ func NewNodeReconciler(
namespace string,
precedence mcsv1alpha1.Precedence) *NodeReconciler {
if string(precedence) == "" {
precedence = mcsv1alpha1.PrecedencePrivate
precedence = mcsv1alpha1.PrecedenceInternal
}
reconciler := &NodeReconciler{
Client: client,
Expand Down Expand Up @@ -138,12 +138,13 @@ func (r *NodeReconciler) getGatawayNodeIP(node *corev1.Node) (string, string, er
var gatewayIP, internalIP string
for _, addr := range node.Status.Addresses {
if addr.Type == corev1.NodeInternalIP {
if r.precedence == mcsv1alpha1.PrecedencePrivate {
if r.precedence == mcsv1alpha1.PrecedencePrivate || r.precedence == mcsv1alpha1.PrecedenceInternal {
gatewayIP = addr.Address
}
internalIP = addr.Address
}
if r.precedence == mcsv1alpha1.PrecedencePublic && addr.Type == corev1.NodeExternalIP {
if (r.precedence == mcsv1alpha1.PrecedencePublic || r.precedence == mcsv1alpha1.PrecedenceExternal) &&
addr.Type == corev1.NodeExternalIP {
gatewayIP = addr.Address
}
}
Expand Down