Skip to content

Commit

Permalink
Add documentation for CAS support in gen1 and gen2 clusters (#381)
Browse files Browse the repository at this point in the history 
* Adding documentation for cluster autoscaler support in gen1 clusters

* Add documentation for gen2 CAS support

* Addressed review comments

* Addressed further review comments

* Added repo-revision in bundle create

* Renamed basecluster to cluster template

* Changing to v0.10 in repo-revision as it is a stable version

* Adding v0.10 in repo-revision
  • Loading branch information
@ShaunakJoshi1407
ShaunakJoshi1407 authored Dec 20, 2022
1 parent 10a724a commit 1974305
Show file tree
Hide file tree
Showing 2 changed files with 150 additions and 2 deletions.
71 changes: 69 additions & 2 deletions docs/gen2_Tutorial.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -10,8 +10,6 @@ typically contains multiple related resources that together define an arbitraril
If you make subsequent changes to the base cluster, workload clusters originally created from it
will automatically acquire the changes.

**NB: Base clusters only support dynamic profiles.**

## Creating Cluster-API cluster manifest

Note: The CAPA version used here is v2.0 and the manifests created here are in accordance with this version.
Expand Down Expand Up @@ -358,6 +356,8 @@ arlon basecluster validategit --repo-alias prod --repo-path <pathToDirectory> [-

## gen2 cluster creation

**Note: Base clusters only support dynamic profiles.**

To create a gen2 workload cluster from the base cluster:

```shell
Expand Down Expand Up @@ -429,6 +429,73 @@ and clean them up.

If the cluster which which is being deleted is a cluster created using patch files, the controller first cleans the git repo where the respective patch files of the cluster are present and then it destroys all the related ArgoCD applications and clean them up.

## Enabling Cluster Autoscaler in the workload cluster
To create a gen2 cluster with autoscaler, we need:

- bundle pointing to the bundle/capi-cluster-autoscaler in the arlon repository.
- dynamic profile that contains the above bundle.
- a cluster template manifest(that makes use of MachineDeployment and not MachinePools) which has the CAPI annotations for min and max nodes set ( as a part of `preparegit` or manually add it ).
- run `arlon cluster create` with the repo-path pointing to the cluster template manifest described in the step above, set the profile to be the one created in step 2 and pass the `autoscaler` flag.

### Bundle creation

Register a dynamic bundle pointing to the bundles/capi-cluster-autoscaler in the Arlon repo.

To enable the cluster-autoscaler bundle, add one more parameter during cluster creation: `srcType`. This is the ArgoCD-defined application source type (Helm, Kustomize, Directory). In addition to this, the `repo-revision` parameter should also be set to a stable arlon release branch ( in this case v0.10 ).

This example creates a bundle pointing to the bundles/capi-cluster-autoscaler in Arlon repo

```shell
arlon bundle create cas-bundle --tags cas,devel,test --desc "CAS Bundle" --repo-url https://github.com/arlonproj/arlon.git --repo-path bundles/capi-cluster-autoscaler --srctype helm --repo-revision v0.10
```

### Profile creation

Create a profile that contains this capi-cluster-autoscaler bundle.

```shell
arlon profile create dynamic-cas --repo-url <repoUrl> --repo-base-path profiles --bundles cas-bundle --desc "dynamic cas profile" --tags examples
```

### manifest directory preparation

Two additional properties `cas-max` and `cas-min` are used to set 2 annotations for Max/Min nodes on MachineDeployment required by the cluster autoscaler for CAPI as a part of the manifest directory preparation. These are the annotations required by the MachineDeployment for autoscaling.

Note: These are the default values for the `cas-min` and `cas-max` properties
```shell
annotations:
cluster.x-k8s.io/cluster-api-autoscaler-node-group-min-size: '1'
cluster.x-k8s.io/cluster-api-autoscaler-node-group-max-size: '9'
```

These annotations are added during the preparegit step. If these annotations are already present in the manifest file, then they will not be added again as a part of `preparegit`

Preparing the git directory to serve as the cluster template:

```shell
arlon basecluster preparegit --repo-path <pathToDirectory> --cas-min 1 --cas-max 9 --repo-url <repoUrl>
```

### manifest directory validation

To determine if a git directory is eligible to serve as cluster template, run the `basecluster validategit` command:

```shell
arlon basecluster validategit --repo-path <pathToDirectory> --repo-url <repoUrl>
```

### gen2 cluster creation with autoscaling enabled

To add CAS support for gen2 clusters, the cluster create sub-command of the arlon CLI has a `autoscaler` flag which deploys the capi-cluster-autoscaler helm chart on the management cluster.

To create a gen2 workload with a cluster-autoscaler pod running, from the cluster template, run this command:

Note: Use the dynamic profile that was created in the previous steps

```shell
arlon cluster create --cluster-name <clusterName> --repo-url <repoUrl> --repo-path <pathToDirectory> --profile dynamic-cas --autoscaler
```

## Known issues and limitations

Gen2 clusters are powerful because the base cluster can be arbitrarily complex and feature rich. Since they are fairly
Expand Down
81 changes: 81 additions & 0 deletions docs/tutorial.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -453,3 +453,84 @@ This example updates a cluster to use a new profile `my-new-profile`:
```shell
arlon cluster update eks-1 --profile my-new-profile
```
## Enabling Cluster Autoscaler in the workload cluster:

### Bundle creation:
Register a dynamic bundle pointing to the bundles/capi-cluster-autoscaler in the Arlon repo.

To enable the cluster-autoscaler bundle, add one more parameter during cluster creation: `srcType`. This is the ArgoCD-defined application source type (Helm, Kustomize, Directory). In addition to this, the `repo-revision` parameter should also be set to a stable arlon release branch ( in this case v0.10 ).

This example creates a bundle pointing to the bundles/capi-cluster-autoscaler in Arlon repo

```shell
arlon bundle create cas-bundle --tags cas,devel,test --desc "CAS Bundle" --repo-url https://github.com/arlonproj/arlon.git --repo-path bundles/capi-cluster-autoscaler --srctype helm --repo-revision v0.10.0
```

### Profile creation:
Create a profile that contains this capi-cluster-autoscaler bundle.

```shell
arlon profile create dynamic-cas --repo-url ${WORKSPACE_REPO_URL} --repo-base-path profiles --bundles cas-bundle --desc "dynamic cas profile" --tags examples
```

### Clusterspec creation:
Create a clusterspec with CAPI as ApiProvider and autoscaling enabled.In addition to this, the ClusterAutoscaler(Min|Max)Nodes properties are used to set 2 annotations on MachineDeployment required by the cluster autoscaler for CAPI.

```shell
arlon clusterspec create cas-spec --api capi --cloud aws --type eks --kubeversion v1.21.10 --nodecount 2 --nodetype t2.medium --tags devel,test --desc "dev/test" --region ${REGION} --sshkey ${SSH_KEY_NAME} --casenabled
```

### Cluster creation:
Deploy a cluster from this cluster spec and profile created in the previous steps.

```shell
arlon cluster deploy --repo-url ${WORKSPACE_REPO_URL} --cluster-name cas-cluster --profile dynamic-cas --cluster-spec cas-spec
```
Consequently, this example produces the directory `clusters/cas-cluster/` in the repo. This will contain the capi-autoscaler subchart and manifests `mgmt/charts/`.
To verify its contents:

```shell
$ cd ${WORKSPACE_REPO_URL}
$ tree clusters/cas-cluster
clusters/cas-cluster
└── mgmt
├── Chart.yaml
├── charts
│   ├── capi-aws-eks
│   │   ├── Chart.yaml
│   │   └── templates
│   │   └── cluster.yaml
│   ├── capi-aws-kubeadm
│   │   ├── Chart.yaml
│   │   └── templates
│   │   └── cluster.yaml
│   ├── capi-cluster-autoscaler
│   │   ├── Chart.yaml
│   │   └── templates
│   │   ├── callhomeconfig.yaml
│   │   └── rbac.yaml
│   └── xplane-aws-eks
│   ├── Chart.yaml
│   └── templates
│   ├── cluster.yaml
│   └── network.yaml
├── templates
│   ├── clusterregistration.yaml
│   ├── ns.yaml
│   ├── profile.yaml
│   └── rbac.yaml
└── values.yaml
```

At this point, the cluster is provisioning. To monitor the progress of the cluster deployment, check the status of
the ArgoCD app of the same name. Eventually, the ArgoCD apps will be synced and healthy.

```shell
$ argocd app list

NAME CLUSTER NAMESPACE PROJECT STATUS HEALTH SYNCPOLICY CONDITIONS REPO PATH TARGET
cas-cluster https://kubernetes.default.svc default default Synced Healthy Auto-Prune <none> ${WORKSPACE_REPO_URL} clusters/cas-cluster/mgmt main
cas-cluster-cas-bundle cas-cluster default default Synced Healthy Auto-Prune <none> https://github.com/arlonproj/arlon.git bundles/capi-cluster-autoscaler HEAD
cas-cluster-profile-dynamic-cas https://kubernetes.default.svc argocd default Synced Healthy Auto-Prune <none> ${WORKSPACE_REPO_URL} profiles/dynamic-cas/mgmt

```

0 comments on commit 1974305

Please sign in to comment.