Skip to content

Commit

Permalink
update yurtcluster operator proposal (openyurtio#429)
Browse files Browse the repository at this point in the history 
Signed-off-by: SataQiu <[email protected]>
  • Loading branch information
@SataQiu
SataQiu authored Sep 10, 2021
1 parent d30c897 commit 9c87226
Showing 1 changed file with 170 additions and 30 deletions.
200 changes: 170 additions & 30 deletions docs/proposals/20210722-yurtcluster-operator.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -3,20 +3,21 @@ title: YurtCluster Operator
authors:
- "@gnunu"
- "@lindayu17"
- "@SataQiu"
reviewers:
- "@rambohe-ch"
- "@guofei"
- "@kadisi"
- "@wenjun93"
creation-date: 2021-07-22
last-updated: 2021-07-22
last-updated: 2021-08-18
status: provisional
---

# Convert Vanilla Kubernetes cluster to OpenYurt cluster through Operator

## Table of Contents
* [Convert Vanilla Kubernetes cluster to OpenYurt cluster through Opertaor](#convert-vanilla-kubernetes-cluster-to-openyurt-cluster-through-operator)
* [Convert Vanilla Kubernetes cluster to OpenYurt cluster through Operator](#convert-vanilla-kubernetes-cluster-to-openyurt-cluster-through-operator)
* [Table of Contents](#table-of-contents)
* [Glossary](#glossary)
* [Summary](#summary)
Expand Down Expand Up @@ -63,7 +64,8 @@ To make OpenYurt translation easy for user, i.e., as automative as possible.

### Non-Goals/Future Work

This is NOT a replacement of yurtctl command.
This is NOT a replacement of yurtctl command, but the convert and revert functions of yurtctl will be deprecated in the future.
And we recommend that you do the node conversion based on the declarative API of YurtCluster Operator.

## Proposal

Expand All @@ -72,43 +74,180 @@ This is an operator, with CRD and Controller.
CRD's definition:

```go
import (
corev1 "k8s.io/api/core/v1"
)

// ImageMeta allows to customize the image used for components that are not
// originated from the openyurtio/openyurt release process
type ImageMeta struct {
// Repository sets the container registry to pull images from.
// if not set, the ImageRepository defined in YurtClusterSpec will be used instead.
// +optional
Repository string `json:"repository,omitempty"`
// Tag allows to specify a tag for the image.
// if not set, the tag releated to the YurtVersion defined in YurtClusterSpec will be used instead.
// +optional
Tag string `json:"tag,omitempty"`
}

// ComponentConfig defines the common config for the yurt components
type ComponentConfig struct {
Enabled bool `json:"enebled,omitempty"`
Version string `json:"version,omitempty"`
// Enabled indicates whether the yurt component has been enabled
// +optional
Enabled *bool `json:"enabled,omitempty"`
// ImageMeta allows to customize the image used for the yurt component
// +optional
ImageMeta `json:",inline"`
}

// YurtHubSpec defines the configuration for yurthub component
type YurtHubSpec struct {
// ComponentConfig defines the common config for the yurt components
// +optional
ComponentConfig `json:",inline"`
// EnableResourceFilter enables to filter response that comes back from reverse proxy
// +optional
EnableResourceFilter *bool `json:"enableResourceFilter,omitempty"`
// AccessServerThroughHub enables pods access kube-apiserver through yurthub or not
// +optional
AccessServerThroughHub *bool `json:"accessServerThroughHub,omitempty"`
// AutoRestartNodePod represents whether to automatically restart the pod on the node after it was converted
// This will trigger the re-injection of environment variables (e.g. KUBERNETES_SERVICE_HOST and KUBERNETES_SERVICE_PORT)
// This should only be set when EnableResourceFilter is true.
// +optional
AutoRestartNodePod *bool `json:"autoRestartNodePod,omitempty"`
// PodManifestPath defines the path to the directory on edge node containing static pod files.
// +optional
PodManifestPath string `json:"podManifestPath,omitempty"`
// KubeadmConfPath defines the path to kubelet service conf that is used by kubelet component to join the cluster on the edge node.
// +optional
KubeadmConfPath string `json:"kubeadmConfPath,omitempty"`
}

type NodeSpec struct {
Names []string `json:"names,omitempty"`
Pattern string `json:"pattern,omitempty"`
ExcludedNames []string `json:"excludedNames,omitempty"`
ExcludedPattern string `json:"excludedPattern,omitempty"`
// YurtTunnelSpec defines the configuration for yurt tunnel component
type YurtTunnelSpec struct {
// ComponentConfig defines the common config for the yurt components
// +optional
ComponentConfig `json:",inline"`
// ServerCount defines the replicas for the tunnel server Pod.
// Its value should be greater than or equal to the number of API Server.
// Operator will automatically override this value if it is less than the number of API Server.
// +optional
ServerCount int `json:"serverCount,omitempty"`
// PublicIP defines the public IP for tunnel server listen on.
// If this field is empty, the tunnel agent will use NodePort Service to connect to the tunnel server.
// +optional
PublicIP string `json:"publicIP,omitempty"`
// PublicPort defines the public port for tunnel server listen on.
// +optional
PublicPort int `json:"publicPort,omitempty"`
}

type FailedNode struct {
Name string `json:"name,omitempty"`
Info string `json:"info,omitempty"`
// NodeSet defines a set of Kubernetes nodes.
// It will merge the nodes that selected by Names, NamePattern, and Selector,
// and then remove the nodes that match ExcludedNames and ExcludedNamePattern as the final set of nodes.
type NodeSet struct {
// Names defines the node names to be selected
// +optional
Names []string `json:"names,omitempty"`
// NamePattern defines the regular expression to select nodes based on node name
// +optional
NamePattern string `json:"namePattern,omitempty"`
// Selector defines the label selector to select nodes
// +optional
Selector *corev1.NodeSelector `json:"selector,omitempty"`
// ExcludedNames defines the node names to be excluded
// +optional
ExcludedNames []string `json:"excludedNames,omitempty"`
// ExcludedNamePattern defines the regular expression to exclude nodes based on node name
// +optional
ExcludedNamePattern string `json:"excludedNamePattern,omitempty"`
}

// YurtClusterSpec defines the desired state of YurtCluster
type YurtClusterSpec struct {
CloudNodes NodeSpec `json:"cloudNodes,omitempty"`
EdgeNodes NodeSpec `json:"edgeNodes,omitempty"`
YurtHub ComponentConfig `json:"yurtHub,omitempty"`
YurtControllerManager ComponentConfig `json:"yurtControllerManager,omitempty"`
TunnelService ComponentConfig `json:"yurtTunnel,omitempty"`
AppManager ComponentConfig `json:"appManger,omitempty"`
// ImageRepository sets the container registry to pull images from.
// If empty, `docker.io/openyurt` will be used by default
// +optional
ImageRepository string `json:"imageRepository,omitempty"`
// YurtVersion is the target version of OpneYurt.
// +optional
YurtVersion string `json:"yurtVersion,omitempty"`
// CloudNodes defines the node set with cloud role.
// +optional
CloudNodes NodeSet `json:"cloudNodes,omitempty"`
// EdgeNodes defines the node set with edge role.
// +optional
EdgeNodes NodeSet `json:"edgeNodes,omitempty"`
// YurtHubSpec defines the configuration for yurthub component.
// +optional
YurtHubSpec YurtHubSpec `json:"yurtHubSpec,omitempty"`
// YurtTunnelSpec defines the configuration for yurt tunnel component.
// +optional
YurtTunnelSpec YurtTunnelSpec `json:"yurtTunnelSpec,omitempty"`
}

// Phase is a string representation of a YurtCluster Phase.
type Phase string

const (
// PhaseInvalid is the state when the YurtCluster is invalid
PhaseInvalid = Phase("Invalid")
// PhaseConverting is the state when the YurtCluster is converting
PhaseConverting = Phase("Converting")
// PhaseDeleting is the state when the YurtCluster is deleting
PhaseDeleting = Phase("Deleting")
// PhaseSucceed is the state when the YurtCluster is ready
PhaseSucceed = Phase("Succeed")
)

// YurtClusterStatusFailure defines errors states for YurtCluster objects.
type YurtClusterStatusFailure string

const (
// InvalidConfigurationYurtClusterError indicates a state that must be fixed before progress can be made.
InvalidConfigurationYurtClusterError YurtClusterStatusFailure = "InvalidConfiguration"
)

// NodeCondition describes the state of a node at a certain point
type NodeCondition struct {
// The status for the condition's last transition.
// +optional
Status string `json:"status,omitempty"`
// The reason for the condition's last transition.
// +optional
Reason string `json:"reason,omitempty"`
// A human readable message indicating details about the transition.
// +optional
Message string `json:"message,omitempty"`
// The last time this condition was updated.
// +optional
LastTransitionTime metav1.Time `json:"lastTransitionTime,omitempty"`
// The generation observed by the node agent controller.
// +optional
ObservedGeneration int64 `json:"observedGeneration,omitempty"`
}

// YurtClusterStatus defines the observed state of YurtCluster
type YurtClusterStatus struct {
ConvertedCloudNodes []string `json:"convertedCloudNodes,omitempty"`
ConvertedEdgeNodes []string `json:"convertedEdgeNodes,omitempty"`
YurtHub ComponentConfig `json:"yurtHub,omitempty"`
YurtControllerManager ComponentConfig `json:"yurtControllerManager,omitempty"`
YurtTunnel ComponentConfig `json:"yurtTunnel,omitempty"`
AppManager ComponentConfig `json:"appManger,omitempty"`
ExcludedNodes []string `json:"excludedNodes,omitempty"`
FailedNodes []FailedNode `json:"failedNodes,omitempty"`
// Phase represents the current phase of the yurt cluster
// +optional
Phase Phase `json:"phase,omitempty"`
// FailureReason indicates that there is a problem reconciling the state, and
// will be set to a token value suitable for programmatic interpretation.
// +optional
FailureReason *YurtClusterStatusFailure `json:"failureReason,omitempty"`
// FailureMessage indicates that there is a fatal problem reconciling the
// state, and will be set to a descriptive error message.
// +optional
FailureMessage *string `json:"failureMessage,omitempty"`
// NodeConditions holds the info about node conditions
// +optional
NodeConditions map[string]NodeCondition `json:"nodeConditions,omitempty"`
// The generation observed by the operator controller.
// +optional
ObservedGeneration int64 `json:"observedGeneration,omitempty"`
}
```

Expand Down Expand Up @@ -143,7 +282,7 @@ The Operator would do these things:

##### FR1
Analyze requests:
Through the YurtCluster CR, the controller would figure out what nodes to convert, what nodes to revert, and if enabling addons like YurtTunel and AppManger.
Through the YurtCluster CR, the controller would figure out what nodes to convert, what nodes to revert, and if enabling addons like YurtHub.
Th nodes to be converted or reverted can be specified using Regular expressions, the controller would process the nodes set, including those coming in the future.

##### FR2
Expand All @@ -169,7 +308,7 @@ This operator should keep cluster serects safe.

The procedure of coverting/reverting a node will refer to yurtctl's implementation.

Wheteher or not use helm is to be discovered.
Whether or not use helm is to be discovered.

### Risks and Mitigations

Expand All @@ -183,7 +322,7 @@ manual: user should convert/revert nodes manually.

## Upgrade Strategy

The Operator supports OpenYurt components upgrade, such as YuntTunnel and AppManager.
The Operator supports OpenYurt components upgrade, such as YuntHub.

For Operator itself, upgrade would have no side effects, idempotent semantics would be kept.

Expand All @@ -199,3 +338,4 @@ test Operator upgrade
## Implementation History

[x] 03/27/2021: Proposed idea in an issue (https://github.com/openyurtio/openyurt/issues/328).
[x] 08/18/2021: Refine and update the YurtCluster Spec definition.

0 comments on commit 9c87226

Please sign in to comment.