Use Skaffold + hack/ scripts for dev/test loops #226
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
hack/dev-up.sh uses the hcloud CLI and alexellis/k3sup to quickly bring up an n node cluster in Hetzner Cloud for testing purposes. hack/dev-down.sh cleans up resources created by hack/dev-up.sh hack/run-e2e-tests.sh runs the CSI e2e test suite against the active cluster. Skaffold handles building, tagging and deploying the manifests for both local dev, and CI test automation. In local development, it also does live reload, rebuilding+repushing images, and redeploying containers.
|
Hey @samcday, thank you for the MR. Unfortunately, we don't plan to change the Testsuite (or add more tools to support/enhance the test suite). We decided to be "bare metal" here and keep the needed knowledge to a minimum. With the current test suite, the handling between our own k8s integrations is completely identically and the knowledge needed to start to test it is quite low (atm you just need to run |
|
Hey @LKaemmerling Alright, I'll just keep this work in my fork :) Just curious - did you try out the |
samcday
added a commit
to hetznercloud/hcloud-cloud-controller-manager
that referenced
this pull request
Apr 6, 2023
This PR simplifies and unifies the process of bringing up a hcloud k8s environment that is suitable for development/testing purposes. It builds off the work originally started in hetznercloud/csi-driver#226 The central script is `hack/dev-up.sh`. The script requires a `HCLOUD_TOKEN`, and will build a 1+ k8s cluster using `k3sup` + `hcloud` CLI tools. It's typically quite fast: a cold execution of the script should take less <1min before a fully operational cluster is ready for use. The `dev-down.sh` script will delete all resources created by `dev-up.sh`. One of the primary objectives in this work was to simplify the e2e testing process. Julian and I decided to remove the matrix of kubeadm clusters, and also remove flannel as an option for CNI. Instead, our test suite now runs on the latest 3 stable releases of k3s. Further, the option to create a cluster with/without hcloud networking support has been removed. Created clusters *always* have a network and an install of Cilium configured for direct routing mode (traffic bound for pods on other nodes is handed to hcloud network fabric). Skaffold is plumbed into the e2e test pipelines on GitHub. This way, the process of building and deploying a test build of hccm to a cluster is unified for development and test. Once this work has landed and stabilized here, we expect to package it up a bit further and use it in csi-driver and anywhere else we operate Kubernetes integrations that need automated testing. --------- Co-authored-by: Julian Tölle <[email protected]>
apricote
added a commit
that referenced
this pull request
Apr 6, 2023
This PR simplifies and unifies the process of bringing up a hcloud k8s environment that is suitable for development/testing purposes. It builds off the work originally started in #226 The central script is `hack/dev-up.sh`. The script requires a `HCLOUD_TOKEN`, and will build a 1+ k8s cluster using `k3sup` + `hcloud` CLI tools. It's typically quite fast: a cold execution of the script should take less <1min before a fully operational cluster is ready for use. The `dev-down.sh` script will delete all resources created by `dev-up.sh`. One of the primary objectives in this work was to simplify the e2e testing process. Skaffold is plumbed into the e2e test pipelines on GitHub. This way, the process of building and deploying a test build of csi-driver to a cluster is unified for development and test. Once this work has landed and stabilized here, we expect to package it up a bit further and use it in csi-driver and anywhere else we operate Kubernetes integrations that need automated testing. Co-authored-by: Sam Day <[email protected]>
apricote
added a commit
that referenced
this pull request
Apr 6, 2023
This PR simplifies and unifies the process of bringing up a hcloud k8s environment that is suitable for development/testing purposes. It builds off the work originally started in #226 The central script is `hack/dev-up.sh`. The script requires a `HCLOUD_TOKEN`, and will build a 1+ k8s cluster using `k3sup` + `hcloud` CLI tools. It's typically quite fast: a cold execution of the script should take less <1min before a fully operational cluster is ready for use. The `dev-down.sh` script will delete all resources created by `dev-up.sh`. One of the primary objectives in this work was to simplify the e2e testing process. Skaffold is plumbed into the e2e test pipelines on GitHub. This way, the process of building and deploying a test build of csi-driver to a cluster is unified for development and test. Once this work has landed and stabilized here, we expect to package it up a bit further and use it in csi-driver and anywhere else we operate Kubernetes integrations that need automated testing. Co-authored-by: Sam Day <[email protected]>
lukasmetzner
pushed a commit
that referenced
this pull request
Oct 10, 2024
This PR simplifies and unifies the process of bringing up a hcloud k8s environment that is suitable for development/testing purposes. It builds off the work originally started in #226 The central script is `hack/dev-up.sh`. The script requires a `HCLOUD_TOKEN`, and will build a 1+ k8s cluster using `k3sup` + `hcloud` CLI tools. It's typically quite fast: a cold execution of the script should take less <1min before a fully operational cluster is ready for use. The `dev-down.sh` script will delete all resources created by `dev-up.sh`. One of the primary objectives in this work was to simplify the e2e testing process. Skaffold is plumbed into the e2e test pipelines on GitHub. This way, the process of building and deploying a test build of csi-driver to a cluster is unified for development and test. Once this work has landed and stabilized here, we expect to package it up a bit further and use it in csi-driver and anywhere else we operate Kubernetes integrations that need automated testing. Co-authored-by: Sam Day <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
hack/dev-up.sh uses the hcloud CLI and alexellis/k3sup to quickly bring
up an n node cluster in Hetzner Cloud for testing purposes.
hack/dev-down.sh cleans up resources created by hack/dev-up.sh
hack/run-e2e-tests.sh runs the CSI e2e test suite against the active
cluster.
Skaffold handles building, tagging and deploying the manifests for both
local dev, and CI test automation. In local development, it also does
live reload, rebuilding+repushing images, and redeploying containers.