From a2251ad1e5a9ed84b7165af211dc795a9b72ff8d Mon Sep 17 00:00:00 2001 From: Scott Lowe Date: Tue, 16 Mar 2021 15:03:06 -0600 Subject: [PATCH] Add information on using kustomize Add examples of using kustomize with CAPI. Include information on modifying kustomize transformer configuration. Signed-off-by: Scott Lowe --- docs/book/src/tasks/using-kustomize.md | 154 +++++++++++++++++++++++++ 1 file changed, 154 insertions(+) create mode 100644 docs/book/src/tasks/using-kustomize.md diff --git a/docs/book/src/tasks/using-kustomize.md b/docs/book/src/tasks/using-kustomize.md new file mode 100644 index 000000000000..5ab81ea3a14b --- /dev/null +++ b/docs/book/src/tasks/using-kustomize.md @@ -0,0 +1,154 @@ +# Using Kustomize with Workload Cluster Manifests + +Although the `clusterctl config cluster` command exposes a number of different configuration values +for customizing workload cluster YAML manifests, some users may need additional flexibility above +and beyond what `clusterctl config cluster` or the example "flavor" templates that some CAPI providers +supply (as an example, see [these flavor templates](https://github.com/kubernetes-sigs/cluster-api-provider-azure/tree/master/templates/flavors) +for the Cluster API Provider for Azure). In the future, a [templating solution](https://github.com/kubernetes-sigs/cluster-api/issues/3252) +may be integrated into `clusterctl` to help address this need, but in the meantime users can use +`kustomize` as a solution to this need. + +This document provides a few examples of using `kustomize` with Cluster API. All of these examples +assume that the "base" (unmodified) Cluster API configuration (perhaps generated using +`clusterctl config cluster`) is in a directory whose relative path is `../../base`. + +## Example: Using Kustomize to Specify Custom Images + +Users can use `kustomize` to specify custom OS images for Cluster API nodes. Using the Cluster API +Provider for AWS (CAPA) as an example, the following `kustomization.yaml` would leverage a JSON 6902 patch +to modify the AMI for nodes in a workload cluster: + +```yaml +--- +apiVersion: kustomize.config.k8s.io/v1beta1 +kind: Kustomization +resources: + - ../../base +patchesJson6902: + - path: custom-ami.json + target: + group: infrastructure.cluster.x-k8s.io + kind: AWSMachineTemplate + name: ".*" + version: v1alpha3 +``` + +The referenced JSON 6902 patch looks like this: + +```json +[ + { "op": "add", "path": "/spec/template/spec/ami", "value": "ami-042db61632f72f145"} +] +``` + +This configuration assumes that the workload cluster _only_ uses MachineDeployments. Since +MachineDeployments and the KubeadmControlPlane both leverage AWSMachineTemplates, this `kustomize` +configuration would catch all nodes in the workload cluster. + +## Example: Adding a MachineHealthCheck for a Workload Cluster + +Users could also use `kustomize` to combine additional resources, like a MachineHealthCheck (MHC), with the +base Cluster API manifest. In an overlay, specify the following in `kustomization.yaml`: + +```yaml +--- +apiVersion: kustomize.config.k8s.io/v1beta1 +kind: Kustomization +resources: + - ../../base + - workload-mhc.yaml +``` + +The content of the `workload-mhc.yaml` file would be the definition of a standard MHC: + +```yaml +apiVersion: cluster.x-k8s.io/v1alpha3 +kind: MachineHealthCheck +metadata: + name: md-0-mhc +spec: + clusterName: test + # maxUnhealthy: 40% + nodeStartupTimeout: 10m + selector: + matchLabels: + cluster.x-k8s.io/deployment-name: md-0 + unhealthyConditions: + - type: Ready + status: Unknown + timeout: 300s + - type: Ready + status: "False" + timeout: 300s +``` + +Running `kustomize build .` with this configuration would append the MHC to the base +Cluster API manifest, thus creating the MHC at the same time as the workload cluster. + +## Configuring Kustomize for Cluster API + +Although `kustomize` can be used with Cluster API without any changes, modifying the transformer +configurations for `kustomize` can make it more effective with Cluster API. For example, changes to +the `nameReference` transformer in `kustomize` will enable `kustomize` to know about the references +between Cluster API objects in a manifest. See [here](https://github.com/kubernetes-sigs/kustomize/tree/master/examples/transformerconfigs) +for more information on transformer configurations. + +Add the following content to the `namereference.yaml` transformer configuration: + +```yaml +- kind: Cluster + group: cluster.x-k8s.io + version: v1alpha3 + fieldSpecs: + - path: spec/clusterName + kind: MachineDeployment + - path: spec/template/spec/clusterName + kind: MachineDeployment + +- kind: AWSCluster + group: infrastructure.cluster.x-k8s.io + version: v1alpha3 + fieldSpecs: + - path: spec/infrastructureRef/name + kind: Cluster + +- kind: KubeadmControlPlane + group: controlplane.cluster.x-k8s.io + version: v1alpha3 + fieldSpecs: + - path: spec/controlPlaneRef/name + kind: Cluster + +- kind: AWSMachine + group: infrastructure.cluster.x-k8s.io + version: v1alpha3 + fieldSpecs: + - path: spec/infrastructureRef/name + kind: Machine + +- kind: KubeadmConfig + group: bootstrap.cluster.x-k8s.io + version: v1alpha3 + fieldSpecs: + - path: spec/bootstrap/configRef/name + kind: Machine + +- kind: AWSMachineTemplate + group: infrastructure.cluster.x-k8s.io + version: v1alpha3 + fieldSpecs: + - path: spec/template/spec/infrastructureRef/name + kind: MachineDeployment + - path: spec/infrastructureTemplate/name + kind: KubeadmControlPlane + +- kind: KubeadmConfigTemplate + group: bootstrap.cluster.x-k8s.io + version: v1alpha3 + fieldSpecs: + - path: spec/template/spec/bootstrap/configRef/name + kind: MachineDeployment +``` + +This will enable the use of the `namePrefix` and `nameSuffix` directives in `kustomization.yaml` +and ensure that _all_ references to the names are properly updated.