feat(deployments) add Helm chart for kuma #916

austince · 2020-07-21T01:49:07Z

Summary

Chart

Add a Helm chart (compatible with Helm 3) for the Kuma Control Plane. As discussed with @nickolaev + @jakubdyszkiewicz, this initial Helm chart is hard-coded but we would like to eventually merge the kumactl install commands with generating charts.

Based on kumactl install control-plane, this chart parameterizes all the possible flags and supports CNI, installing CRDs, and all modes.

I'll write more docs when we get closer to "ready to merge". For testing, I'll also need some help, but I think an e2e test using the current framework and Helm 3 as a library to deploy to the testing cluster should work.

CI/CD

Use helm/chart-releaser to publish charts to GitHub Releases and then host them on the gh-pages branch.

Full changelog

Add Helm 3 chart
Add Helm support in e2e tests
Add chart-releaser to CI

Issues resolved

Fix #852
Fix #706

Documentation

N/A for now

deployments/charts/kuma-cp/templates/cp-webhooks-and-secrets.yaml

nickolaev · 2020-07-22T06:02:30Z

deployments/charts/kuma-cp/values.yaml

+  #  resources:
+  #     requests:
+  #       cpu: 100m
+  #       memory: 256Mi


Do we need these?

More for documentation purposes. Usually when installing a new chart, I'll copy the chart's values.yaml file and modify from there. We can certainly replace these comments with more structured docs in the README.

nickolaev · 2020-07-22T06:02:39Z

deployments/charts/kuma-cp/values.yaml

+  #      key: ""
+  #    kds:
+  #      cert: ""
+  #      key: ""


nickolaev · 2020-07-22T10:48:11Z

deployments/charts/kuma-cp/create-namespace.sh

+
+kubectl patch namespace/"$NAMESPACE" \
+  --type merge \
+  --patch '{ "metadata": { "labels": { "kuma.io/system-namespace": "true" } } }'


Why do you need this one?
Can you just apply this one -> https://github.com/kumahq/kuma/blob/master/app/kumactl/data/install/k8s/control-plane/namespace.yaml

Is there a way with kumactl to install only the namespace? Helm 3 requires a namespace to exist before deployment:

$ helm3 install kuma-cp . --namespace kuma-system Error: create: failed to create: namespaces "kuma-system" not found

The other way would be to make users specify the namespace in values.yaml and ignore the Helm .Release.Namespace entirely, though this is not ideal in terms of fitting with the Helm conventions and would likely lead to confusion.

no ! right now, there is no way to install namespace only through kumactl. Can we
keep the kuma-system as a default value in the namespace ?

Hey @tharun208, can you elaborate on the ask? I'm not sure I understand. The issue w/ Helm 3 is that you must create the namespace, whether it is kuma-system or another, before the actual deployment if you need to modify it (ex: add the system label).

Following up from slack, we've tried to use the Helm 3.2.0 option --create-namespace but results in the same error.

Another hacky option here would be to run a Job w/ a pre-install hook that updates the namespace with the label using kubectl directly.

@nickolaev, what do you think about this approach of adding the label with a pre-install job?

Why would this be "hacky"? It looks legitimate to me? I am surprised they did not make that the default behaviour.

Just not something I've seen before, but happy to give it a shot. A bit like the "jump host" solution you were talking about

It worked! Added in the latest commit

deployments/charts/kuma-cp/values.yaml

Signed-off-by: austin ce <[email protected]>

austince · 2020-07-29T21:23:27Z

Don't know why the tests are failing, seems to be something with the GUI server

nickolaev · 2020-07-30T11:44:09Z

deployments/charts/kuma-cp/templates/pre-install-patch-namespace-job.yaml

+  name: {{ $serviceAccountName }}
+  annotations:
+    "helm.sh/hook": "pre-install"
+    "helm.sh/hook-delete-policy": "hook-succeeded,hook-failed"


Hmm does this guarantee a proper cleanup when charts are deleted?

Yes, it should be deleted either when the pre-install hook (i.e. running that job) succeeds or fails. Are you seeing differently?

…plates Signed-off-by: Nikolay Nikolaev <[email protected]>

Signed-off-by: Nikolay Nikolaev <[email protected]>

…expected naming Signed-off-by: Nikolay Nikolaev <[email protected]>

Signed-off-by: Nikolay Nikolaev <[email protected]>

Signed-off-by: austin ce <[email protected]>

austince · 2020-07-30T15:56:55Z

Still a bit of flakiness with the e2e test, but works up until deletion where we then get an RBAC error:

Test App deployment with Helm chart Should deploy two apps 2020-07-30T15:45:19Z logger.go:66: Error: uninstallation completed with 3 error(s): admission webhook "secret.validator.kuma-admission.kuma.io" denied the request: secrets "kuma-kds-tls-cert" is forbidden: User "system:serviceaccount:kuma-system:kuma-control-plane" cannot get resource "secrets" in API group "" in the namespace "kuma-system"; admission webhook "secret.validator.kuma-admission.kuma.io" denied the request: secrets "kuma-admission-server-tls-cert" is forbidden: User "system:serviceaccount:kuma-system:kuma-control-plane" cannot get resource "secrets" in API group "" in the namespace "kuma-system"; admission webhook "secret.validator.kuma-admission.kuma.io" denied the request: secrets "kuma-sds-tls-cert" is forbidden: User "system:serviceaccount:kuma-system:kuma-control-plane" cannot get resource "secrets" in API group "" in the namespace "kuma-system"

Might pass on a re-run, and could also be that the namespace is cleaned up before the release is done deleting? Happy to tackle this in either this PR or another

slonka · 2023-04-13T17:49:50Z

tools/releases/helm.sh

+
+function usage {
+  echo "Usage: $0 [--package|--release]"
+  exit 0


@nickolaev shouldn't do that ;)

nickolaev mentioned this pull request Jul 21, 2020

feat(kuma-charts) Helm charts for kuma #852

Closed

austince force-pushed the feat/helm/initial-chart branch 3 times, most recently from 319c6ff to 545762f Compare July 21, 2020 22:09

nickolaev reviewed Jul 22, 2020

View reviewed changes

deployments/charts/kuma-cp/templates/cp-webhooks-and-secrets.yaml Outdated Show resolved Hide resolved

nickolaev reviewed Jul 22, 2020

View reviewed changes

deployments/charts/kuma-cp/values.yaml Outdated

# key: ""

# kds:

# cert: ""

# key: ""

Copy link

Contributor

nickolaev Jul 22, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And these?

nickolaev marked this pull request as ready for review July 22, 2020 09:48

nickolaev requested a review from a team as a code owner July 22, 2020 09:48

nickolaev reviewed Jul 22, 2020

View reviewed changes

deployments/charts/kuma-cp/values.yaml Outdated Show resolved Hide resolved

austince force-pushed the feat/helm/initial-chart branch 11 times, most recently from d259c8c to f7e9703 Compare July 23, 2020 19:01

austince requested review from jakubdyszkiewicz and nickolaev July 23, 2020 19:02

austince force-pushed the feat/helm/initial-chart branch 4 times, most recently from 9fc48fe to 064f6e1 Compare July 24, 2020 12:44

austince added 2 commits July 24, 2020 09:56

feat(deployments) add Helm chart for kuma-cp

b8e6b73

Signed-off-by: austin ce <[email protected]>

ci(helm) add Helm 3 to install dev tools

d712ce2

Signed-off-by: austin ce <[email protected]>

austince added 2 commits July 29, 2020 15:20

Merge branch 'master' into feat/helm/initial-chart

1abcfbf

Signed-off-by: austin ce <[email protected]>

ci(helm) add Helm lint to check

ce63546

Signed-off-by: austin ce <[email protected]>

austince force-pushed the feat/helm/initial-chart branch 2 times, most recently from bfa168d to 3cf0749 Compare July 29, 2020 20:21

austince added 2 commits July 29, 2020 16:23

style(e2e) fmt changes

ab92017

Signed-off-by: austin ce <[email protected]>

docs(deployments) update kuma-cp chart README

e736f77

Signed-off-by: austin ce <[email protected]>

austince force-pushed the feat/helm/initial-chart branch from 3cf0749 to e736f77 Compare July 29, 2020 20:23

Merge remote-tracking branch 'origin/master' into feat/helm-initial-char

4fb8cb7

nickolaev reviewed Jul 30, 2020

View reviewed changes

Nikolay Nikolaev and others added 9 commits July 30, 2020 15:42

fix(*) sync charts with the latest development in kumactl install tem…

7e5a9f8

…plates Signed-off-by: Nikolay Nikolaev <[email protected]>

docs(helm) document nameOverride

b6b9c19

Signed-off-by: Nikolay Nikolaev <[email protected]>

fix(test) helm test should use annotations

767a34d

Signed-off-by: Nikolay Nikolaev <[email protected]>

fix(test) helm chart path constant

44a774e

Signed-off-by: Nikolay Nikolaev <[email protected]>

fix(test) helm test use Build version for images

e1cd220

Signed-off-by: Nikolay Nikolaev <[email protected]>

fix(test) helm test use nameOverride to ensure consistency with test …

13e7b14

…expected naming Signed-off-by: Nikolay Nikolaev <[email protected]>

fix(test) helm test delete kuma-system namespace when deleting helm

87c969c

Signed-off-by: Nikolay Nikolaev <[email protected]>

feat(deployments) rename kuma-cp chart to kuma

9664ecc

Signed-off-by: austin ce <[email protected]>

ci(helm) comment out releasing helm charts

cc913ca

Signed-off-by: austin ce <[email protected]>

austince force-pushed the feat/helm/initial-chart branch from d302cf4 to cc913ca Compare July 30, 2020 15:23

docs(deployments) update kuma chart README

35eb107

Signed-off-by: austin ce <[email protected]>

austince force-pushed the feat/helm/initial-chart branch from d9813d1 to 35eb107 Compare July 30, 2020 15:25

austince requested a review from nickolaev July 30, 2020 15:54

austince changed the title ~~feat(deployments) add Helm chart for kuma-cp~~ feat(deployments) add Helm chart for kuma Jul 30, 2020

nickolaev approved these changes Jul 31, 2020

View reviewed changes

nickolaev merged commit 7a6d6dd into kumahq:master Jul 31, 2020

austince deleted the feat/helm/initial-chart branch July 31, 2020 14:36

nickolaev mentioned this pull request Oct 2, 2020

chore(*) adding austince as a maintainer #1047

Merged

slonka reviewed Apr 13, 2023

View reviewed changes

tools/releases/helm.sh

function usage {

echo "Usage: $0 [--package|--release]"

exit 0

Copy link

Contributor

slonka Apr 13, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@nickolaev shouldn't do that ;)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(deployments) add Helm chart for kuma #916

feat(deployments) add Helm chart for kuma #916

austince commented Jul 21, 2020 •

edited

Loading

nickolaev Jul 22, 2020

austince Jul 22, 2020

nickolaev Jul 22, 2020

nickolaev Jul 22, 2020

austince Jul 22, 2020

tharun208 Jul 22, 2020

austince Jul 22, 2020 •

edited

Loading

austince Jul 22, 2020

austince Jul 23, 2020 •

edited

Loading

austince Jul 28, 2020

nickolaev Jul 28, 2020

austince Jul 28, 2020

austince Jul 28, 2020

austince commented Jul 29, 2020

nickolaev Jul 30, 2020

austince Jul 30, 2020

austince commented Jul 30, 2020 •

edited

Loading

slonka Apr 13, 2023

feat(deployments) add Helm chart for kuma #916

feat(deployments) add Helm chart for kuma #916

Conversation

austince commented Jul 21, 2020 • edited Loading

Summary

Chart

CI/CD

Full changelog

Issues resolved

Documentation

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

austince Jul 22, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

austince Jul 23, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

austince commented Jul 29, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

austince commented Jul 30, 2020 • edited Loading

Choose a reason for hiding this comment

austince commented Jul 21, 2020 •

edited

Loading

austince Jul 22, 2020 •

edited

Loading

austince Jul 23, 2020 •

edited

Loading

austince commented Jul 30, 2020 •

edited

Loading