Skip to content

Commit

Permalink
Renaming base cluster to cluster template (#383)
Browse files Browse the repository at this point in the history
* Renamed all the base clusters to cluster manifest and added alias to the basemanifest

* refactored the docs

* Renaming to clustertemplate in cas gen2 docs

* Renaming to clustertemplate

* Addressed the reviews(Doc refactoring)

* updated the cluster overrides command

* Refactored the docs

Co-authored-by: Jayanth Reddy <[email protected]>
Co-authored-by: Trilok Khairnar <[email protected]>
Co-authored-by: Shaunak Joshi <[email protected]>
  • Loading branch information
4 people authored Dec 21, 2022
1 parent b72499e commit a9cf12f
Show file tree
Hide file tree
Showing 13 changed files with 114 additions and 111 deletions.
1 change: 1 addition & 0 deletions cmd/basecluster/basecluster.go
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ func NewCommand() *cobra.Command {
Use: "basecluster",
Short: "Manage base clusters",
Long: "Manage base clusters",
Aliases: []string{"clustertemplate"},
DisableAutoGenTag: true,
Run: func(c *cobra.Command, args []string) {
c.Usage()
Expand Down
61 changes: 30 additions & 31 deletions docs/baseclusters.md → docs/clustertemplate.md
Original file line number Diff line number Diff line change
@@ -1,17 +1,16 @@
# Next-gen Cluster Provisioning using Base Clusters
# Next-gen Cluster Provisioning using Cluster templates

This proposal describes a new way of provisioning workload clusters in Arlon.
The most significant change is the *Base Cluster* construct, which replaces the current ClusterSpec.
To distinguish them from current generation
clusters, the ones deployed from a base cluster are called next-gen clusters.
This document describes a new way of provisioning workload clusters in Arlon.
The most significant change is the *Cluster Template* construct, which replaces the older ClusterSpec from gen1 clusters.
To distinguish them from the older gen1 clusters, the ones deployed from a cluster template are called next-gen clusters or gen2 clusters.

## Goals

- Allow users to deploy arbitrarily complex clusters using the full Cluster API feature set.
- Fully declarative and gitops compatible: a cluster deployment should be composed of one or more
self-sufficient manifests that the user can choose to either apply directly (via kubectl) or store in
git for later-stage deployment by a gitops tool (mainly ArgoCD).
- Support Linked Mode update: an update to the the base cluster should
- Support Linked Mode update: an update to the the cluster template should
automatically propagate to all workload clusters deployed from it.

## Profile support
Expand All @@ -24,24 +23,24 @@ One can be attached later.

## Architecture diagram

This example shows a base cluster named `capi-quickstart` used to deploy two workload
This example shows a cluster template named `capi-quickstart` used to deploy two workload
clusters `cluster-a` and `cluster-b`. Additionally, `cluster-a` is given profile `xxx`,
while `cluster-b` is given profile `yyy`.

![architecture](./images/arlon_gen2.png)

## Base Cluster
## Cluster Template

A base cluster serves as a base for creating new workload clusters. The workload clusters
are all exact copies of the base cluster, meaning that they acquire all unmodified resources of the
base cluster, except for:
A cluster template serves as a base for creating new workload clusters. The workload clusters
are all exact copies of the cluster template, meaning that they acquire all unmodified resources of the
cluster template, except for:

- resource names, which are prefixed during the cluster creation process to make them unique to avoid conflicts
- the namespace, which is set to a new namespace unique to the workload cluster

### Preparation

- To create a base cluster, a user first creates a single YAML file containing the desired Cluster API
- To create a cluster template, a user first creates a single YAML file containing the desired Cluster API
cluster and all related resources (e.g. MachineDeployments, etc...), using whatever tool the user
chooses (e.g. `clusterctl generate cluster`). The user is responsible for the correctness of the file
and resources within. Arlon will not check for errors. For example, the specified Kubernetes version
Expand All @@ -53,20 +52,20 @@ cluster creation. The directory should be unique to the file, and not contain an
- If not already registered, the git repository should also be registered in ArgoCD with
the proper credentials for read/write access.

To check whether the git directory is a compliant Arlon base cluster,
To check whether the git directory is a compliant Arlon cluster template,
the user runs:

```shell
arlon basecluster validategit --repo-url <repoUrl> --repo-path <pathToDirectory> [--repo-revision revision]
arlon clustertemplate validategit --repo-url <repoUrl> --repo-path <pathToDirectory> [--repo-revision revision]
```

*Note: if --repo-revision is not specified, it defaults to main.*

The command produces an error the first time because the git directory has not yet been "prepped".
To "prep" the directory to become a compliant Arlon base cluster, the user runs:
To "prep" the directory to become a compliant Arlon cluster template, the user runs:

```shell
arlon basecluster preparegit --repo-url <repoUrl> --repo-path <pathToDirectory> [--repo-revision revision]
arlon clustertemplate preparegit --repo-url <repoUrl> --repo-path <pathToDirectory> [--repo-revision revision]
```

This pushes a commit to the repo with these changes:
Expand All @@ -77,13 +76,13 @@ mechanism. The content of the file is sourced from this [Scott Lowe blog article
- All `namespace` properties in the cluster manifest are removed to allow Kustomize to override the
namespace of all resources.

If prep is successful, another invocation of `arlon basecluster validategit` should succeed as well.
If prep is successful, another invocation of `arlon clustertemplate validategit` should succeed as well.

## Workload clusters

### Creation

Use `arlon cluster create` to create a next-gen workload cluster from a base cluster
Use `arlon cluster create` to create a next-gen workload cluster from a cluster template
(*this is different from* `arlon cluster deploy` *for creating current-generation clusters*).
The command creates between 2 and 3 (depending on whether a profile is used)
ArgoCD application resources that together
Expand All @@ -103,25 +102,25 @@ The `--profile` flag is optional; a cluster can be created with no profile.
### Composition

A workload cluster is composed of 2 to 3 ArgoCD application resources, which are named
based on the name of the base cluster and the workload cluster. For illustration purposes,
the following discussion assumes that the base cluster is named `capi-quickstart`, the
based on the name of the cluster template and the workload cluster. For illustration purposes,
the following discussion assumes that the cluster template is named `capi-quickstart`, the
workload cluster is named `cluster-a`, and the optional profile is named `xxx`.

#### Cluster app

The `cluster-a` application is the *cluster app* for the cluster.
It is responsible for deploying the base cluster resources, meaning the Cluster API manifests.
It is responsible for deploying the cluster template resources, meaning the Cluster API manifests.
It is named directly from the workload cluster name.

The application's spec uses a ApplicationSourceKustomize that points to the base cluster's git
The application's spec uses a ApplicationSourceKustomize that points to the cluster template's git
directory. The spec ensures that all deployed resources are configured to:

- Reside in the `cluster-a` namespace, which is deployed by the *arlon app* (see below).
This achieved by setting `app.Spec.Destination.Namespace` to the workload cluster's name
(*this only works if the resources do not specify an explicit namespace; this requirement is
taken care of by the "prep" step on the base cluster*).
taken care of by the "prep" step on the *cluster template*).
- Be named `cluster-a-capi-quickstart`, meaning the workload cluster name followed by the
base cluster name. This is achieved by setting `app.Spec.Source.Kustomize.NamePrefix` to
cluster template name. This is achieved by setting `app.Spec.Source.Kustomize.NamePrefix` to
the workload cluster name plus a hyphen.

#### Arlon app
Expand Down Expand Up @@ -177,33 +176,33 @@ cluster and cleans up all related applications.

## Update Semantics

A base cluster lives in git and is shared by all workload clusters created from it.
A cluster template lives in git and is shared by all workload clusters created from it.
This is sometimes referred to as *Linked Mode*.
Any git update to the cluster can affect the associated workload clusters, therefore
such updates must be planned and managed with care; there is a real risk of such an
update breaking existing clusters.

- By default, a workload's cluster *cluster app* is configured with auto-sync, meaning ArgoCD will
immediately apply any changes in the base cluster to the deployed Cluster API cluster resources.
- In general, a base cluster **does not need to be "prepped" again** after a modification to its main
immediately apply any changes in the cluster template to the deployed Cluster API cluster resources.
- In general, a cluster template **does not need to be "prepped" again** after a modification to its main
manifest file (the one containing the Cluster API resources). So the user is free to edit the manifest
directly, commit/push the changes, and expect to see immediate changes to already-deployed clusters
created from that base cluster.
created from that cluster template.

### Unsupported changes

The controllers for Cluster API and its providers disallow changes to some fields belonging
to already-deployed resources.

- For example, changing the base cluster name (`medata.Name` of the `Cluster` resource) will have disastrous consequences on already-deployed
- For example, changing the cluster template name (`medata.Name` of the `Cluster` resource) will have disastrous consequences on already-deployed
clusters, causing many resources to enter the OutOfSync state and never recover because ArgoCD
fails to apply the changes (they are rejected by the controllers). Consequently, a user should never
change the name of a base cluster.
change the name of a cluster template.
- Besides the cluster name, other fields cannot change (this has been observed anecdotally, we don't
yet have an exhaustive list).
- Changing the Kubernetes version of the control plane or data plane *is* supported, so long as the new version
is supported by the relevant providers. If accepted, such a change will result in a rolling update
of the corresponding plane.
- Specific to AWS: the `AWSMachineTemplate.spec` is immutable and a CAPI webhook disallows such updates. The user is advised to not make such modifications to a basecluster manifest.
- Specific to AWS: the `AWSMachineTemplate.spec` is immutable and a CAPI webhook disallows such updates. The user is advised to not make such modifications to a cluster template manifest.
In the event that such an event does happen, the user is advised to not manually sync in those changes via `argocd`. If a new cluster with a different `AWSMachineTemplate.spec` is desired,
the recommended approach is to make a copy of the manifests in the workspace repository and then issue an `arlon cluster create` command which would then consume this manifest.
4 changes: 2 additions & 2 deletions docs/concepts.md
Original file line number Diff line number Diff line change
Expand Up @@ -99,6 +99,6 @@ They currently include:
- The initial (worker) node count
- The Kubernetes version

## Base cluster
## Cluster Template

To know more about basecluster (Arlon gen2 clusters), read it [here](./baseclusters.md)
To know more about cluster template (Arlon gen2 clusters), read it [here](./clustertemplate.md)
8 changes: 4 additions & 4 deletions docs/design.md
Original file line number Diff line number Diff line change
Expand Up @@ -77,9 +77,9 @@ Clusterspec currently includes:

#### gen2

for gen2 clusters, the Cluster Specification is called the base cluster, which is described in detail [here](https://github.com/arlonproj/arlon/blob/main/docs/baseclusters.md).
for gen2 clusters, the Cluster Specification is called the cluster template, which is described in detail [here](https://github.com/arlonproj/arlon/blob/main/docs/clustertemplate.md).

A base cluster manifest consists of:
A cluster template manifest consists of:

- A predefined list of Cluster API objects: Cluster, Machines, Machine Deployments, etc. to be deployed in the current namespace
- The specific infrastructure provider to be used (e.g aws).ß
Expand All @@ -96,7 +96,7 @@ Once the clusterspec is created for a gen-1 cluster, there is no need to prepare

#### gen2

Once the base cluster manifest is created, the next step is to preare the workspace repository directory in which this base cluster manifest is present. This is explained in detail [here](https://github.com/arlonproj/arlon/blob/main/docs/baseclusters.md#preparation)
Once the cluster template manifest is created, the next step is to preare the workspace repository directory in which this cluster template manifest is present. This is explained in detail [here](https://github.com/arlonproj/arlon/blob/main/docs/clustertemplate.md#preparation)

### Cluster Creation

Expand All @@ -123,4 +123,4 @@ Cluster deployment is explained [here](https://github.com/arlonproj/arlon/blob/m

#### gen2

Base cluster creation is explained [here](https://github.com/arlonproj/arlon/blob/main/docs/baseclusters.md#creation)
cluster template creation is explained [here](https://github.com/arlonproj/arlon/blob/main/docs/clustertemplate.md#creation)
2 changes: 1 addition & 1 deletion docs/e2e_testing.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@

- First, we run the `testing/e2e_setup.sh` script, which helps us setup a kind management cluster, a git-server [gitea](https://gitea.io/en-us) based workspace repository and installs services like argocd, arlon, capi in the management cluster. It also installs other required tools like kubectl, docker, kind, kuttl, clusterctl, helm and gitea.

- In addition to this, the script also creates a capi-eks cluster manifest which serves as the basecluster manifest. This cluster is created as a part of the e2e tests and is pushed to the workspace repository created in the previous step.
- In addition to this, the script also creates a capi-eks cluster manifest which serves as the cluster template manifest. This cluster is created as a part of the e2e tests and is pushed to the workspace repository created in the previous step.

- This script also adds a xenial bundle manifest to the workspace repository which is required for creating a xenial bundle and a corresponding profile which is consumed by the test.

Expand Down
Loading

0 comments on commit a9cf12f

Please sign in to comment.