From 39919b5eb2ade2ab5bef28c55f1a0e16d9854f38 Mon Sep 17 00:00:00 2001 From: Naresh Koduru Date: Thu, 29 Aug 2024 17:02:29 +0530 Subject: [PATCH] =?UTF-8?q?=F0=9F=93=96=20Move=20Control=20Plane=20contrac?= =?UTF-8?q?ts=20doc=20to=20right=20chapter?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit add new file Fix cluster.md path move the line upwards --- docs/book/src/SUMMARY.md | 1 + .../architecture/controllers/control-plane.md | 255 ------------------ .../book/src/developer/providers/contracts.md | 2 +- .../src/developer/providers/control-plane.md | 254 +++++++++++++++++ 4 files changed, 256 insertions(+), 256 deletions(-) create mode 100644 docs/book/src/developer/providers/control-plane.md diff --git a/docs/book/src/SUMMARY.md b/docs/book/src/SUMMARY.md index 24c09d956358..1e33c79f9ad1 100644 --- a/docs/book/src/SUMMARY.md +++ b/docs/book/src/SUMMARY.md @@ -100,6 +100,7 @@ - [v1.7 to v1.8](./developer/providers/migrations/v1.7-to-v1.8.md) - [Provider contracts](./developer/providers/contracts.md) - [Cluster Infrastructure](./developer/providers/cluster-infrastructure.md) + - [Control Plane](./developer/providers/control-plane.md) - [Machine Infrastructure](./developer/providers/machine-infrastructure.md) - [Bootstrap](./developer/providers/bootstrap.md) - [Implementer's Guide](./developer/providers/implementers-guide/overview.md) diff --git a/docs/book/src/developer/architecture/controllers/control-plane.md b/docs/book/src/developer/architecture/controllers/control-plane.md index 54c73cfdc687..69bd4b6a74c8 100644 --- a/docs/book/src/developer/architecture/controllers/control-plane.md +++ b/docs/book/src/developer/architecture/controllers/control-plane.md @@ -13,261 +13,6 @@ A reference implementation is managed within the core Cluster API project as the Kubeadm control plane controller (`KubeadmControlPlane`). In this document, we refer to an example `ImplementationControlPlane` where not otherwise specified. -## Contracts - -### Control Plane Provider - -The general expectation of a control plane controller is to instantiate a -Kubernetes control plane consisting of the following services: - -#### Required Control Plane Services - -* etcd -* Kubernetes API Server -* Kubernetes Controller Manager -* Kubernetes Scheduler - -#### Optional Control Plane Services - -* Cloud controller manager -* Cluster DNS (e.g. CoreDNS) -* Service proxy (e.g. kube-proxy) - -#### Prohibited Services - -* CNI - should be left to user to apply once control plane is instantiated. - -### Relationship to other Cluster API types - -The Cluster controller will set an OwnerReference on the Control Plane. The Control Plane controller should normally take no action during reconciliation until it sees the ownerReference. - -A Control Plane controller implementation must either supply a controlPlaneEndpoint (via its own `spec.controlPlaneEndpoint` field), -or rely on `spec.controlPlaneEndpoint` in its parent [Cluster](./cluster.md) object. - -If an endpoint is not provided, the implementer should exit reconciliation until it sees `cluster.spec.controlPlaneEndpoint` populated. - -A Control Plane controller can optionally provide a `controlPlaneEndpoint` - -The Cluster controller bubbles up `status.ready` into `status.controlPlaneReady` and `status.initialized` into a `controlPlaneInitialized` condition from the Control Plane CR. - -### CRD contracts - -The CRD name must have the format produced by `sigs.k8s.io/cluster-api/util/contract.CalculateCRDName(Group, Kind)`. -The same applies for the name of the corresponding ControlPlane template CRD. - -#### Required `spec` fields for implementations using replicas - -* `replicas` - is an integer representing the number of desired - replicas. In the KubeadmControlPlane, this represents the desired - number of control plane machines. - -* `scale` subresource with the following signature: - -``` yaml -scale: - labelSelectorPath: .status.selector - specReplicasPath: .spec.replicas - statusReplicasPath: .status.replicas -status: {} -``` - -More information about the [scale subresource can be found in the Kubernetes -documentation][scale]. - -#### Required `spec` fields for implementations using version - -* `version` - is a string representing the Kubernetes version to be used - by the control plane machines. The value must be a valid semantic version; - also if the value provided by the user does not start with the v prefix, it - must be added. - -#### Required `spec` fields for implementations using Machines - -* `machineTemplate` - is a struct containing details of the control plane - machine template. - -* `machineTemplate.metadata` - is a struct containing info about metadata for control plane - machines. - -* `machineTemplate.metadata.labels` - is a map of string keys and values that can be used - to organize and categorize control plane machines. - -* `machineTemplate.metadata.annotations` - is a map of string keys and values containing - arbitrary metadata to be applied to control plane machines. - -* `machineTemplate.infrastructureRef` - is a corev1.ObjectReference to a custom resource - offered by an infrastructure provider. The namespace in the ObjectReference must - be in the same namespace of the control plane object. - -* `machineTemplate.nodeDrainTimeout` - is a *metav1.Duration defining the total amount of time - that the controller will spend on draining a control plane node. - The default value is 0, meaning that the node can be drained without any time limitations. - -* `machineTemplate.nodeVolumeDetachTimeout` - is a *metav1.Duration defining how long the controller - will spend on waiting for all volumes to be detached. - The default value is 0, meaning that the volume can be detached without any time limitations. - -* `machineTemplate.nodeDeletionTimeout` - is a *metav1.Duration defining how long the controller - will attempt to delete the Node that is hosted by a Machine after the Machine is marked for - deletion. A duration of 0 will retry deletion indefinitely. It defaults to 10 seconds on the - Machine. - -#### Optional `spec` fields for implementations providing endpoints - -The `ImplementationControlPlane` object may provide a `spec.controlPlaneEndpoint` field to inform the Cluster -controller where the endpoint is located. - -Implementers might opt to choose the `APIEndpoint` struct exposed by Cluster API types, or the following: - - - - - - - - - - - - - - - - - -
Field Type Description
hostString - The hostname on which the API server is serving. -
portInteger - The port on which the API server is serving. -
- -#### Required `status` fields - -The `ImplementationControlPlane` object **must** have a `status` object. - -The `status` object **must** have the following fields defined: - - - - - - - - - - - - - - - - - - - - -
Field Type Description Implementation in Kubeadm Control Plane Controller
initializedBoolean - a boolean field that is true when the target cluster has - completed initialization such that at least once, the - target's control plane has been contactable. - - Transitions to initialized when the controller detects that kubeadm has uploaded - a kubeadm-config configmap, which occurs at the end of kubeadm provisioning. -
readyBoolean - Ready denotes that the target API Server is ready to receive requests. -
- -#### Required `status` fields for implementations using replicas - -Where the `ImplementationControlPlane` has a concept of replicas, e.g. most -high availability control planes, then the `status` object **must** have the -following fields defined: - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Field Type Description Implementation in Kubeadm Control Plane Controller
readyReplicasIntegerTotal number of fully running and ready control plane instances.Is equal to the number of fully running and ready control plane machines
replicasIntegerTotal number of non-terminated control plane instances, - i.e. the state machine for this instance - of the control plane is able to transition to ready.Is equal to the number of non-terminated control plane machines
selectorString`selector` is the label selector in string format to avoid - introspection by clients, and is used to provide the CRD-based integration - for the scale subresource and additional integrations for things like - kubectl describe. The string will be in the same format as the query-param - syntax. More info about label selectors: https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/#label-selectors -
unavailableReplicasInteger - Total number of unavailable control plane instances targeted by this control plane, equal - to the desired number of control plane instances - ready instances. - - Total number of unavailable machines targeted by this control - plane. This is the total number of machines that are still required - for the deployment to have 100% available capacity. They may either - be machines that are running but not yet ready or machines that still - have not been created. -
- updatedReplicas - integer - Total number of non-terminated machines targeted by this - control plane that have the desired template spec. - - Total number of non-terminated machines targeted by this - control plane that have the desired template spec. -
- -#### Required `status` fields for implementations using version - -* `version` - is a string representing the minimum Kubernetes version for the - control plane machines in the cluster. - NOTE: The minimum Kubernetes version, and more specifically the API server - version, will be used to determine when a control plane is fully upgraded - (`spec.version == status.version`) and for enforcing [Kubernetes version - skew policies](https://kubernetes.io/releases/version-skew-policy/) in managed topologies. - -#### Optional `status` fields - -The `status` object **may** define several fields: - -* `failureReason` - is a string that explains why an error has occurred, if possible. -* `failureMessage` - is a string that holds the message contained by the error. -* `externalManagedControlPlane` - is a bool that should be set to true if the Node objects do not - exist in the cluster. For example, managed control plane providers for AKS, EKS, GKE, etc, should - set this to `true`. Leaving the field undefined is equivalent to setting the value to `false`. - -Note: once any of `failureReason` or `failureMessage` surface on the cluster who is referencing the control plane object, -they cannot be restored anymore (it is considered a terminal error; the only way to recover is to delete and recreate the cluster). - ## Example usage ```yaml diff --git a/docs/book/src/developer/providers/contracts.md b/docs/book/src/developer/providers/contracts.md index 6aee482880fe..1d374c6e25c4 100644 --- a/docs/book/src/developer/providers/contracts.md +++ b/docs/book/src/developer/providers/contracts.md @@ -6,7 +6,7 @@ Cluster API defines the following contracts: - [Infrastructure provider contract](./cluster-infrastructure.md) - [Bootstrap provider contract](./bootstrap.md) -- [Control Plane provider contract](../../developer/architecture/controllers/control-plane.md#crd-contracts) +- [Control Plane provider contract](./control-plane.md) - [Machine provider contract](./machine-infrastructure.md) - [clusterctl provider contract](../../clusterctl/provider-contract.md#clusterctl-provider-contract) - [Multi tenancy contract](../../developer/architecture/controllers/multi-tenancy.md#contract) diff --git a/docs/book/src/developer/providers/control-plane.md b/docs/book/src/developer/providers/control-plane.md new file mode 100644 index 000000000000..ce5cefcd6861 --- /dev/null +++ b/docs/book/src/developer/providers/control-plane.md @@ -0,0 +1,254 @@ +# Control Plane Provider Specification + +## Overview + +The general expectation of a control plane controller is to instantiate a +Kubernetes control plane consisting of the following services: + +#### Required Control Plane Services + +* etcd +* Kubernetes API Server +* Kubernetes Controller Manager +* Kubernetes Scheduler + +#### Optional Control Plane Services + +* Cloud controller manager +* Cluster DNS (e.g. CoreDNS) +* Service proxy (e.g. kube-proxy) + +#### Prohibited Services + +* CNI - should be left to user to apply once control plane is instantiated. + +### Relationship to other Cluster API types + +The Cluster controller will set an OwnerReference on the Control Plane. The Control Plane controller should normally take no action during reconciliation until it sees the ownerReference. + +A Control Plane controller implementation must either supply a controlPlaneEndpoint (via its own `spec.controlPlaneEndpoint` field), +or rely on `spec.controlPlaneEndpoint` in its parent [Cluster](../architecture/controllers/cluster.md) object. + +If an endpoint is not provided, the implementer should exit reconciliation until it sees `cluster.spec.controlPlaneEndpoint` populated. + +A Control Plane controller can optionally provide a `controlPlaneEndpoint` + +The Cluster controller bubbles up `status.ready` into `status.controlPlaneReady` and `status.initialized` into a `controlPlaneInitialized` condition from the Control Plane CR. + +### CRD contracts + +The CRD name must have the format produced by `sigs.k8s.io/cluster-api/util/contract.CalculateCRDName(Group, Kind)`. +The same applies for the name of the corresponding ControlPlane template CRD. + +#### Required `spec` fields for implementations using replicas + +* `replicas` - is an integer representing the number of desired + replicas. In the KubeadmControlPlane, this represents the desired + number of control plane machines. + +* `scale` subresource with the following signature: + +``` yaml +scale: + labelSelectorPath: .status.selector + specReplicasPath: .spec.replicas + statusReplicasPath: .status.replicas +status: {} +``` + +More information about the [scale subresource can be found in the Kubernetes +documentation][scale]. + +#### Required `spec` fields for implementations using version + +* `version` - is a string representing the Kubernetes version to be used + by the control plane machines. The value must be a valid semantic version; + also if the value provided by the user does not start with the v prefix, it + must be added. + +#### Required `spec` fields for implementations using Machines + +* `machineTemplate` - is a struct containing details of the control plane + machine template. + +* `machineTemplate.metadata` - is a struct containing info about metadata for control plane + machines. + +* `machineTemplate.metadata.labels` - is a map of string keys and values that can be used + to organize and categorize control plane machines. + +* `machineTemplate.metadata.annotations` - is a map of string keys and values containing + arbitrary metadata to be applied to control plane machines. + +* `machineTemplate.infrastructureRef` - is a corev1.ObjectReference to a custom resource + offered by an infrastructure provider. The namespace in the ObjectReference must + be in the same namespace of the control plane object. + +* `machineTemplate.nodeDrainTimeout` - is a *metav1.Duration defining the total amount of time + that the controller will spend on draining a control plane node. + The default value is 0, meaning that the node can be drained without any time limitations. + +* `machineTemplate.nodeVolumeDetachTimeout` - is a *metav1.Duration defining how long the controller + will spend on waiting for all volumes to be detached. + The default value is 0, meaning that the volume can be detached without any time limitations. + +* `machineTemplate.nodeDeletionTimeout` - is a *metav1.Duration defining how long the controller + will attempt to delete the Node that is hosted by a Machine after the Machine is marked for + deletion. A duration of 0 will retry deletion indefinitely. It defaults to 10 seconds on the + Machine. + +#### Optional `spec` fields for implementations providing endpoints + +The `ImplementationControlPlane` object may provide a `spec.controlPlaneEndpoint` field to inform the Cluster +controller where the endpoint is located. + +Implementers might opt to choose the `APIEndpoint` struct exposed by Cluster API types, or the following: + + + + + + + + + + + + + + + + + +
Field Type Description
hostString + The hostname on which the API server is serving. +
portInteger + The port on which the API server is serving. +
+ +#### Required `status` fields + +The `ImplementationControlPlane` object **must** have a `status` object. + +The `status` object **must** have the following fields defined: + + + + + + + + + + + + + + + + + + + + +
Field Type Description Implementation in Kubeadm Control Plane Controller
initializedBoolean + a boolean field that is true when the target cluster has + completed initialization such that at least once, the + target's control plane has been contactable. + + Transitions to initialized when the controller detects that kubeadm has uploaded + a kubeadm-config configmap, which occurs at the end of kubeadm provisioning. +
readyBoolean + Ready denotes that the target API Server is ready to receive requests. +
+ +#### Required `status` fields for implementations using replicas + +Where the `ImplementationControlPlane` has a concept of replicas, e.g. most +high availability control planes, then the `status` object **must** have the +following fields defined: + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Field Type Description Implementation in Kubeadm Control Plane Controller
readyReplicasIntegerTotal number of fully running and ready control plane instances.Is equal to the number of fully running and ready control plane machines
replicasIntegerTotal number of non-terminated control plane instances, + i.e. the state machine for this instance + of the control plane is able to transition to ready.Is equal to the number of non-terminated control plane machines
selectorString`selector` is the label selector in string format to avoid + introspection by clients, and is used to provide the CRD-based integration + for the scale subresource and additional integrations for things like + kubectl describe. The string will be in the same format as the query-param + syntax. More info about label selectors: https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/#label-selectors +
unavailableReplicasInteger + Total number of unavailable control plane instances targeted by this control plane, equal + to the desired number of control plane instances - ready instances. + + Total number of unavailable machines targeted by this control + plane. This is the total number of machines that are still required + for the deployment to have 100% available capacity. They may either + be machines that are running but not yet ready or machines that still + have not been created. +
+ updatedReplicas + integer + Total number of non-terminated machines targeted by this + control plane that have the desired template spec. + + Total number of non-terminated machines targeted by this + control plane that have the desired template spec. +
+ +#### Required `status` fields for implementations using version + +* `version` - is a string representing the minimum Kubernetes version for the + control plane machines in the cluster. + NOTE: The minimum Kubernetes version, and more specifically the API server + version, will be used to determine when a control plane is fully upgraded + (`spec.version == status.version`) and for enforcing [Kubernetes version + skew policies](https://kubernetes.io/releases/version-skew-policy/) in managed topologies. + +#### Optional `status` fields + +The `status` object **may** define several fields: + +* `failureReason` - is a string that explains why an error has occurred, if possible. +* `failureMessage` - is a string that holds the message contained by the error. +* `externalManagedControlPlane` - is a bool that should be set to true if the Node objects do not + exist in the cluster. For example, managed control plane providers for AKS, EKS, GKE, etc, should + set this to `true`. Leaving the field undefined is equivalent to setting the value to `false`. + +Note: once any of `failureReason` or `failureMessage` surface on the cluster who is referencing the control plane object, +they cannot be restored anymore (it is considered a terminal error; the only way to recover is to delete and recreate the cluster).