Skip to content

Commit

Permalink
Add information on using kustomize
Browse files Browse the repository at this point in the history
Add examples of using kustomize with CAPI. Include information on modifying kustomize transformer configuration.

Signed-off-by: Scott Lowe <[email protected]>
  • Loading branch information
scottslowe committed Mar 16, 2021
1 parent c080446 commit a2251ad
Showing 1 changed file with 154 additions and 0 deletions.
154 changes: 154 additions & 0 deletions docs/book/src/tasks/using-kustomize.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,154 @@
# Using Kustomize with Workload Cluster Manifests

Although the `clusterctl config cluster` command exposes a number of different configuration values
for customizing workload cluster YAML manifests, some users may need additional flexibility above
and beyond what `clusterctl config cluster` or the example "flavor" templates that some CAPI providers
supply (as an example, see [these flavor templates](https://github.com/kubernetes-sigs/cluster-api-provider-azure/tree/master/templates/flavors)
for the Cluster API Provider for Azure). In the future, a [templating solution](https://github.com/kubernetes-sigs/cluster-api/issues/3252)
may be integrated into `clusterctl` to help address this need, but in the meantime users can use
`kustomize` as a solution to this need.

This document provides a few examples of using `kustomize` with Cluster API. All of these examples
assume that the "base" (unmodified) Cluster API configuration (perhaps generated using
`clusterctl config cluster`) is in a directory whose relative path is `../../base`.

## Example: Using Kustomize to Specify Custom Images

Users can use `kustomize` to specify custom OS images for Cluster API nodes. Using the Cluster API
Provider for AWS (CAPA) as an example, the following `kustomization.yaml` would leverage a JSON 6902 patch
to modify the AMI for nodes in a workload cluster:

```yaml
---
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- ../../base
patchesJson6902:
- path: custom-ami.json
target:
group: infrastructure.cluster.x-k8s.io
kind: AWSMachineTemplate
name: ".*"
version: v1alpha3
```
The referenced JSON 6902 patch looks like this:
```json
[
{ "op": "add", "path": "/spec/template/spec/ami", "value": "ami-042db61632f72f145"}
]
```

This configuration assumes that the workload cluster _only_ uses MachineDeployments. Since
MachineDeployments and the KubeadmControlPlane both leverage AWSMachineTemplates, this `kustomize`
configuration would catch all nodes in the workload cluster.

## Example: Adding a MachineHealthCheck for a Workload Cluster

Users could also use `kustomize` to combine additional resources, like a MachineHealthCheck (MHC), with the
base Cluster API manifest. In an overlay, specify the following in `kustomization.yaml`:

```yaml
---
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- ../../base
- workload-mhc.yaml
```
The content of the `workload-mhc.yaml` file would be the definition of a standard MHC:

```yaml
apiVersion: cluster.x-k8s.io/v1alpha3
kind: MachineHealthCheck
metadata:
name: md-0-mhc
spec:
clusterName: test
# maxUnhealthy: 40%
nodeStartupTimeout: 10m
selector:
matchLabels:
cluster.x-k8s.io/deployment-name: md-0
unhealthyConditions:
- type: Ready
status: Unknown
timeout: 300s
- type: Ready
status: "False"
timeout: 300s
```

Running `kustomize build .` with this configuration would append the MHC to the base
Cluster API manifest, thus creating the MHC at the same time as the workload cluster.

## Configuring Kustomize for Cluster API

Although `kustomize` can be used with Cluster API without any changes, modifying the transformer
configurations for `kustomize` can make it more effective with Cluster API. For example, changes to
the `nameReference` transformer in `kustomize` will enable `kustomize` to know about the references
between Cluster API objects in a manifest. See [here](https://github.com/kubernetes-sigs/kustomize/tree/master/examples/transformerconfigs)
for more information on transformer configurations.

Add the following content to the `namereference.yaml` transformer configuration:

```yaml
- kind: Cluster
group: cluster.x-k8s.io
version: v1alpha3
fieldSpecs:
- path: spec/clusterName
kind: MachineDeployment
- path: spec/template/spec/clusterName
kind: MachineDeployment
- kind: AWSCluster
group: infrastructure.cluster.x-k8s.io
version: v1alpha3
fieldSpecs:
- path: spec/infrastructureRef/name
kind: Cluster
- kind: KubeadmControlPlane
group: controlplane.cluster.x-k8s.io
version: v1alpha3
fieldSpecs:
- path: spec/controlPlaneRef/name
kind: Cluster
- kind: AWSMachine
group: infrastructure.cluster.x-k8s.io
version: v1alpha3
fieldSpecs:
- path: spec/infrastructureRef/name
kind: Machine
- kind: KubeadmConfig
group: bootstrap.cluster.x-k8s.io
version: v1alpha3
fieldSpecs:
- path: spec/bootstrap/configRef/name
kind: Machine
- kind: AWSMachineTemplate
group: infrastructure.cluster.x-k8s.io
version: v1alpha3
fieldSpecs:
- path: spec/template/spec/infrastructureRef/name
kind: MachineDeployment
- path: spec/infrastructureTemplate/name
kind: KubeadmControlPlane
- kind: KubeadmConfigTemplate
group: bootstrap.cluster.x-k8s.io
version: v1alpha3
fieldSpecs:
- path: spec/template/spec/bootstrap/configRef/name
kind: MachineDeployment
```

This will enable the use of the `namePrefix` and `nameSuffix` directives in `kustomization.yaml`
and ensure that _all_ references to the names are properly updated.

0 comments on commit a2251ad

Please sign in to comment.