Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] 📖 Add initial guide to deploying addons with tilt #7154

Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
49 changes: 49 additions & 0 deletions docs/book/src/developer/tilt.md
Original file line number Diff line number Diff line change
Expand Up @@ -378,6 +378,55 @@ Set to `false` if your provider does not have a ./config folder or you do not wa
in tilt version >= v0.22.2 (see https://blog.tilt.dev/2021/08/09/resource-grouping.html); as a convention,
provider abbreviation should be used (CAPD, KCP etc.).
Copy link
Member

@sbueringer sbueringer Sep 23, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(unrelated to this file just had to add it somewhere)

We're adding a Tips & Tricks section to the implementation doc in #7270.

There was a suggestion to also mention the tilt setup there:

What if we add a note about being possible to develop and test with tilt (assuming that #7154 merge soon, this note could close the circle)
#7270 (comment)



### tilt-addon configuration
The Cluster API Tilt environment can be used to deploy additional Go projects.
An addon must supply a `tilt-addon.yaml` or `tilt-addon.json` file describing how to build it. Here is an example:

```yaml
name: test-extension
config:
image: "gcr.io/k8s-staging-cluster-api/test-extension"
container_name: "extension"
live_reload_deps: ["main.go", "handlers",]
label: test-extension
resource_deps: ["capi_controller"]
additional_resources: [
"extensionconfig.yaml"
]
```

The tilt-addon configuration has the following fields:

**name**: the name of the extension. Must match the name listed in `enable-addons` in `tilt-settings.yaml`

**image**: the image for this provider, as referenced in the kustomize files. This must match; otherwise, Tilt won't build it.

**live_reload_deps**: a list of files/directories to watch. If any of them changes, Tilt rebuilds the manager binary for the provider and performs a live update of the running container.

**additional_docker_helper_commands** (String, default=""): Additional commands to be run in the helper image docker build.

**additional_docker_build_commands** (String, default=""): Additional commands to be appended to
the dockerfile.

**go_main** (String, default="main.go"): The go main file if not located at the root of the folder

**container_name**: The name of the container tilt should target for deploying the binary.

**label** (String, default=provider name): The label to be used to group provider components in the tilt UI
in tilt version >= v0.22.2 (see https://blog.tilt.dev/2021/08/09/resource-grouping.html); as a convention,
provider abbreviation should be used (CAPD, KCP etc.).

**resource_deps**: Tilt resources that must be deployed before the addon is deployed.

**additional_resources**: a list of Kubernetes yamls that will be deployed the cluster alongside the addon.

The initial use case for addons in tilt is for developing Runtime Extensions. More information about developing a Runtime Extension with Tilt [can be found here](../tasks/experimental-features/runtime-sdk/deploy-runtime-extension.md#developing-runtime-extensions-with-tilt)

Additionally, debug and extra args can be added to addons in the principle `tilt-settings.yaml` as described in exactly the same way it can be added to a provider.



## Customizing Tilt

If you need to customize Tilt's behavior, you can create files in cluster-api's `tilt.d` directory. This file is ignored
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -45,3 +45,80 @@ Alternative deployment methods can be used as long as the HTTPs endpoint is acce
- deploying the HTTPS Server outside the Management Cluster.

In those cases recommendations about availability and identity and access management still apply.

## Developing Runtime Extensions with Tilt

[The Cluster API Tilt environment](../../../developer/tilt.md) can be used to enable easier deployment and testing of Runtime Extensions in a dev workflow. The following steps for deploying an Extension can be automated with tilt:

1. Enabling both the `CLUSTER_TOPOLOGY` and `EXP_RUNTIME_SDK` flags for Cluster API.
2. Building a binary and Docker image for the Extension.
3. Deploying Kubernetes manifests including Service, RBAC, Deployment,
4. Deploying ExtensionConfig for registering a runtime extension.
5. Enabling live debugging.

The Test Extension is a Runtime Extension that's used for Cluster API end-to-end testing. It registers every available Hook - Patches and Lifecycle hooks and performs a number of patches on templates created for the Cluster.
The test extension is built into the CAPI Tilt workflow, and can be enabled with b

To deploy it with Tilt simply modify the tilt-settings.yaml to:

```yaml
default_registry: localhost:5000
deploy_kind_cluster: false
deploy_cert_manager: true
enable_providers:
- docker
- kubeadm-bootstrap
- kubeadm-control-plane
kustomize_substitutions:
EXP_CLUSTER_RESOURCE_SET: 'true'
EXP_MACHINE_POOL: 'true'
CLUSTER_TOPOLOGY: 'true'
EXP_RUNTIME_SDK: 'true'
enable_addons:
- "test-extension"
```
The test extension will now be deployed. The deployment includes the ExtensionConfig required for registering the extension with Cluster API.

The test extension implements Lifecycle hooks by reading the desired response directly from a configmap. This behaviour is part of the test extension implementation and is not shared by other runtime extensions.

To deploy a Runtime Extension that's external to the CAPI repo, for example a Runtime Extension called `sample-extension`, the following is needed:
1) `tilt-settings.yaml` must point to the `sample-extension` directory.
2) `tilt-settings.yaml` must enable the `sample-extension`, similar to how the `test-extension` is enabled above.
3) The `sample-extension` directory must contain a `tilt-addon.yaml` file.

After making these changes the `tilt-settings.yaml` will look like the below:

```yaml
default_registry: localhost:5000
deploy_kind_cluster: false
deploy_cert_manager: true
enable_providers:
- docker
- kubeadm-bootstrap
- kubeadm-control-plane
kustomize_substitutions:
EXP_CLUSTER_RESOURCE_SET: 'true'
EXP_MACHINE_POOL: 'true'
CLUSTER_TOPOLOGY: 'true'
EXP_RUNTIME_SDK: 'true'
addon_repos:
- "~/cluster-api-sample-runtime-extension"
enable_addons:
- "test-extension"
- "sample-extension"
```

Inside the `cluster-api-sample-runtime-extension` directory we will have a `tilt-addon.yaml` that looks like:

```yaml
name: sample-extension
config:
image: "gcr.io/k8s-staging-cluster-api/sample-extension"
container_name: "extension"
live_reload_deps: ["main.go", "handlers",]
label: sample-extension
resource_deps: ["capi_controller"]
additional_resources: [
"extensionconfig.yaml"
]
```