From e089a5dd58dd39dee5f38583f4ccb2de891dd269 Mon Sep 17 00:00:00 2001 From: Meg McRoberts Date: Thu, 2 Mar 2023 01:03:02 -0800 Subject: [PATCH 01/39] k8s doc references on landing page Signed-off-by: Meg McRoberts --- docs/content/en/docs/crd-ref/_index.md | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) diff --git a/docs/content/en/docs/crd-ref/_index.md b/docs/content/en/docs/crd-ref/_index.md index 23e517ce9c..b1838abd9c 100644 --- a/docs/content/en/docs/crd-ref/_index.md +++ b/docs/content/en/docs/crd-ref/_index.md @@ -13,3 +13,21 @@ that are defined for the Keptn Lifecycle Toolkit. Information that is published here has been reviewed for technical accuracy but the format and content is still evolving. We welcome your input!** + +Each CRD is an object of an API library. +Keptn APIs follow the Kubernetes API versioning scheme. +and is itself composed of objects and sub-objects. + +For more information, see the Kubernetes documentation: + +* [API Overview](https://kubernetes.io/docs/reference/using-api/) + +* [Custom Resources](https://kubernetes.io/docs/concepts/extend-kubernetes/api-extension/custom-resources/#:~:text=A%20resource%20is%20an%20endpoint,in%20a%20default%20Kubernetes%20installation.) + +* [API versioning](https://kubernetes.io/docs/reference/using-api/#api-versioning) + +* [Understanding Kubernetes Objects](https://kubernetes.io/docs/concepts/overview/working-with-objects/kubernetes-objects/) + + + + From 3246ac609d6b424125873304c508f351e4e3e32a Mon Sep 17 00:00:00 2001 From: Meg McRoberts Date: Thu, 2 Mar 2023 01:24:22 -0800 Subject: [PATCH 02/39] process go.mod file --- docs/go.mod | 5 +---- 1 file changed, 1 insertion(+), 4 deletions(-) diff --git a/docs/go.mod b/docs/go.mod index 7ae5fbaa06..bb69a32b84 100644 --- a/docs/go.mod +++ b/docs/go.mod @@ -4,7 +4,4 @@ go 1.19 require github.com/keptn-sandbox/lifecycle-toolkit-docs v0.0.0-20230209144724-01b35a6cfc44 // indirect= -require ( - github.com/google/docsy/dependencies v0.6.0 // indirect - github.com/keptn-sandbox/lifecycle-toolkit-docs v0.0.0-20230209144724-01b35a6cfc44 // indirect -) +require github.com/google/docsy/dependencies v0.6.0 // indirect From 35bdd33599b704640b1afb321d0f98495c10c780 Mon Sep 17 00:00:00 2001 From: Meg McRoberts Date: Thu, 6 Apr 2023 22:21:10 -0700 Subject: [PATCH 03/39] modified apiVersion in synopsis Signed-off-by: Meg McRoberts --- .../en/docs/yaml-crd-ref/taskdefinition.md | 282 ++++++++++++++++++ 1 file changed, 282 insertions(+) diff --git a/docs/content/en/docs/yaml-crd-ref/taskdefinition.md b/docs/content/en/docs/yaml-crd-ref/taskdefinition.md index 877444e902..da729c4cfc 100644 --- a/docs/content/en/docs/yaml-crd-ref/taskdefinition.md +++ b/docs/content/en/docs/yaml-crd-ref/taskdefinition.md @@ -3,3 +3,285 @@ title: KeptnTaskDefinition description: Define tasks that can be run pre- or post-deployment weight: 89 --- + +A `KeptnTaskDefinition` defines tasks +that are run by the Keptn Lifecycle Toolkit +as part of the pre- and post-deployment phases of a `KeptnApp`. + +## Yaml Synopsis + +```yaml +apiVersion: lifecycle.keptn.sh/v?alpha? +kind: KeptnTaskDefinition +metadata: + name: +spec: + function: + inline | httpRef | functionRef: + [parameters: + map: + textMessage: "This is my configuration"] + [secureParameters: + secret: slack-token] +``` + +## Fields + +* **apiVersion** -- API version being used. +` +* **kind** -- Resource type. Must be set to `KeptnTaskDefinition` + +* **name** -- Unique name of this task. + * Must be an alphanumeric string and, by convention, is all lowercase. + * Can include the special characters `_`, `-`, . + * Should not include spaces. + +* **function** -- Code to be executed, + expressed as a [Deno](https://deno.land/) script. + Refer to [function runtime](https://github.com/keptn/lifecycle-toolkit/tree/main/functions-runtime) + for more information about the runtime. + + The `function` can be defined as one of the following: + + * **inline** - Include the actual executable code to execute. + This can be written as a full-fledged Deno script + that is included in this file. + For example: + + ```yaml + function: + inline: + code: | + console.log("Deployment Task has been executed"); + ``` + + * **httpRef** - Specify a Deno script to be executed at runtime + from the remote webserver that is specified. + For example: + + ```yaml + name: hello-keptn-http + spec: + function: + httpRef: + url: "https://www.example.com/yourscript.js" + ``` + + * **functionRef** -- Execute another `KeptnTaskDefinition` that has been defined. + Populate this field with the value of the `name` field + for the `KeptnTaskDefinition` to be called. + This is commonly used to call a general function + that is used in multiple places, possibly with different parameters. + An example is: + + ```yaml + spec: + function: + functionRef: + name: slack-notification + ``` + + This can also be used to group a set of tasks + into a single `KeptnTaskDefinition`, + such as defining a `KeptnTaskDefinition` for testing. + In this case, it other, existing `KeptnTaskDefinition`s + for each type of test to be run, + specifying each by the value of the `name` field. + +* **parameters** - An optional field to supply input parameters to a function. + The Lifecycle Toolkit passes the values defined inside the `map` field + as a JSON object. + For example: + + ```yaml + spec: + parameters: + map: + textMessage: "This is my configuration" + + The JSON object can be read + through the `DATA` environment variable using `Deno.env.get("DATA");`. + + Multi-level maps are not supported at this time. + + Currently only one secret can be passed. + The secret must have a `key` called `SECURE_DATA`. + It can be accessed via the environment variable `Deno.env.get("SECURE_DATA")`. + See [Context](#context) for details. + + +* **secureParameters** -- An optional field used to pass a Kubernetes secret. + The `secret` value is the Kubernetes secret name + that is mounted into the runtime and made available to functions + using the `SECURE_DATA` environment variable. + For example: + + ```yaml + secureParameters: + secret: slack-token + + See [Create secret text](#create-secret-text) for details. + +## Usage + +A Task is responsible for executing the TaskDefinition of a workload. +The execution is done by spawning a Kubernetes Job to handle a single Task. +In its state, it tracks the current status of this Kubernetes Job. + +### Context + +A context environment variable is available via `Deno.env.get("CONTEXT")`. +It can be used like this: + +```javascript +let context = Deno.env.get("CONTEXT"); + +if (contextdata.objectType == "Application") { + let application_name = contextdata.appName; + let application_version = contextdata.appVersion; +} + +if (contextdata.objectType == "Workload") { + let application_name = contextdata.appName; + let workload_name = contextdata.workloadName; + let workload_version = contextdata.workloadVersion; +} +``` + +### Create secret text + +```yaml +# kubectl create secret generic my-secret --from-literal=SECURE_DATA=foo + +apiVersion: lifecycle.keptn.sh/v1alpha3 +kind: KeptnTaskDefinition +metadata: + name: dummy-task + namespace: "default" +spec: + function: + secureParameters: + secret: my-secret + inline: + code: | + let secret_text = Deno.env.get("SECURE_DATA"); + // secret_text = "foo" +``` + +This methodology supports multiple variables +by creating a Kubernetes secret with a JSON string: + + +```yaml +# kubectl create secret generic my-secret \ +# --from-literal=SECURE_DATA="{\"foo\": \"bar\", \"foo2\": \"bar2\"}" + +apiVersion: lifecycle.keptn.sh/v1alpha3 +kind: KeptnTaskDefinition +metadata: + name: dummy-task + namespace: "default" +spec: + function: + secureParameters: + secret: my-secret + inline: + code: | + let secret_text = Deno.env.get("SECURE_DATA"); + let secret_text_obj = JSON.parse(secret_text); + // secret_text_obj["foo"] = "bar" + // secret_text_obj["foo2"] = "bar2" +``` + +## Examples + +**Example 1** defines a full-fledged Deno script +within the `KeptnTaskDefinition` YAML file: + +```yaml +apiVersion: lifecycle.keptn.sh/v1alpha3 +kind: KeptnTaskDefinition +metadata: + name: hello-keptn-inline +spec: + function: + inline: + code: | + let text = Deno.env.get("DATA"); + let data; + let name; + data = JSON.parse(text); + + name = data.name + console.log("Hello, " + name + " new"); +``` + +**Example 2** fetches the Deno script from a remote webserver at runtime: + +```yaml +apiVersion: lifecycle.keptn.sh/v1alpha3 +kind: KeptnTaskDefinition +metadata: + name: hello-keptn-http +spec: + function: + httpRef: + url: +``` + +See the +[sample-app/version-1](https://github.com/keptn-sandbox/lifecycle-toolkit-examples/blob/main/sample-app/version-1/app-pre-deploy.yaml) +PodtatoHead example for a more complete example. + +**Example 3** calls another defined task, +illustrating how one `KeptnTaskDefinition` can build +on top of other `KeptnTaskDefinition`s. +In this case, it calls `slack-notification-dev`, +passing `parameters` and `secureParameters` to that other task: + +```yaml +apiVersion: lifecycle.keptn.sh/v1alpha3 +kind: KeptnTaskDefinition +metadata: + name: slack-notification-dev +spec: + function: + functionRef: + name: slack-notification + parameters: + map: + textMessage: "This is my configuration" + secureParameters: + secret: slack-token +``` + +**More examples** + +* See the [operator/config/samples](https://github.com/keptn/lifecycle-toolkit/tree/main/operator/config/samples) +directory for more example `KeptnTaskDefinition` YAML files. +Separate examples are provided for each API version. +For example, the `lifecycle_v1alpha3_keptntaskdefinition` file +contains examples for the `v1alpha3` version of the lifecycle API group. +## Files + +API Reference: + +* [KeptnTaskDefinition](../../api-ref/lifecycle/v1alpha3/#keptntaskdefinition) +* [KeptnTaskDefinitionList](../../api-ref/lifecycle/v1alpha3/#keptntaskdefinitionlist) +* [KeptnTaskDefinitionSpec](../../api-ref/lifecycle/v1alpha3/#keptntaskdefinitionspec) +* [FunctionReference](../../api-ref/lifecycle/v1alpha3/#functionreference) +* [FunctionSpec](../../api-ref/lifecycle/v1alpha3/#functionspec) +* [FunctionStatus](../../api-ref/lifecycle/v1alpha3/#functionstatus) +* [HttpReference](../../api-ref/lifecycle/v1alpha3/#httpreference) +* [Inline](../../api-ref/lifecycle/v1alpha3/#inline) + +## Differences between versions + +The `KeptnTaskDefinition` is the same for +all `v1alpha?` library versions. + +## See also + +* Link to "use-case" guide pages that do something interesting with this CRD +* Link to reference pages for any related CRDs + From 3e6260cfde58ec34352e3231f6acfef46a3ca3e9 Mon Sep 17 00:00:00 2001 From: Meg McRoberts Date: Fri, 7 Apr 2023 00:43:42 -0700 Subject: [PATCH 04/39] markdownlint-fix Signed-off-by: Meg McRoberts --- docs/content/en/docs/crd-ref/_index.md | 2 -- docs/content/en/docs/yaml-crd-ref/taskdefinition.md | 4 +--- 2 files changed, 1 insertion(+), 5 deletions(-) diff --git a/docs/content/en/docs/crd-ref/_index.md b/docs/content/en/docs/crd-ref/_index.md index 13ab97bf81..b96035fe53 100644 --- a/docs/content/en/docs/crd-ref/_index.md +++ b/docs/content/en/docs/crd-ref/_index.md @@ -30,5 +30,3 @@ For more information, see the Kubernetes documentation: * [API versioning](https://kubernetes.io/docs/reference/using-api/#api-versioning) * [Understanding Kubernetes Objects](https://kubernetes.io/docs/concepts/overview/working-with-objects/kubernetes-objects/) - - diff --git a/docs/content/en/docs/yaml-crd-ref/taskdefinition.md b/docs/content/en/docs/yaml-crd-ref/taskdefinition.md index da729c4cfc..a56f553fd8 100644 --- a/docs/content/en/docs/yaml-crd-ref/taskdefinition.md +++ b/docs/content/en/docs/yaml-crd-ref/taskdefinition.md @@ -109,7 +109,6 @@ spec: It can be accessed via the environment variable `Deno.env.get("SECURE_DATA")`. See [Context](#context) for details. - * **secureParameters** -- An optional field used to pass a Kubernetes secret. The `secret` value is the Kubernetes secret name that is mounted into the runtime and made available to functions @@ -171,7 +170,6 @@ spec: This methodology supports multiple variables by creating a Kubernetes secret with a JSON string: - ```yaml # kubectl create secret generic my-secret \ # --from-literal=SECURE_DATA="{\"foo\": \"bar\", \"foo2\": \"bar2\"}" @@ -262,6 +260,7 @@ directory for more example `KeptnTaskDefinition` YAML files. Separate examples are provided for each API version. For example, the `lifecycle_v1alpha3_keptntaskdefinition` file contains examples for the `v1alpha3` version of the lifecycle API group. + ## Files API Reference: @@ -284,4 +283,3 @@ all `v1alpha?` library versions. * Link to "use-case" guide pages that do something interesting with this CRD * Link to reference pages for any related CRDs - From 148824f0ac27f7955718931ea9fe83a8e3e8ad44 Mon Sep 17 00:00:00 2001 From: Meg McRoberts Date: Mon, 10 Apr 2023 23:10:16 -0700 Subject: [PATCH 05/39] Fix formatting, Examples Signed-off-by: Meg McRoberts --- .../en/docs/yaml-crd-ref/taskdefinition.md | 31 +++++++++++++++---- 1 file changed, 25 insertions(+), 6 deletions(-) diff --git a/docs/content/en/docs/yaml-crd-ref/taskdefinition.md b/docs/content/en/docs/yaml-crd-ref/taskdefinition.md index a56f553fd8..062f807e3c 100644 --- a/docs/content/en/docs/yaml-crd-ref/taskdefinition.md +++ b/docs/content/en/docs/yaml-crd-ref/taskdefinition.md @@ -6,7 +6,8 @@ weight: 89 A `KeptnTaskDefinition` defines tasks that are run by the Keptn Lifecycle Toolkit -as part of the pre- and post-deployment phases of a `KeptnApp`. +as part of the pre- and post-deployment phases of a +[KeptnApp](./app). ## Yaml Synopsis @@ -98,6 +99,7 @@ spec: parameters: map: textMessage: "This is my configuration" + ``` The JSON object can be read through the `DATA` environment variable using `Deno.env.get("DATA");`. @@ -123,12 +125,20 @@ spec: ## Usage -A Task is responsible for executing the TaskDefinition of a workload. -The execution is done by spawning a Kubernetes Job to handle a single Task. +A Task is responsible for executing the TaskDefinition of a +[workload](https://kubernetes.io/docs/concepts/workloads/).. +The execution is done by spawning a Kubernetes +[Job](https://kubernetes.io/docs/concepts/workloads/controllers/job/) +to handle a single Task. In its state, it tracks the current status of this Kubernetes Job. ### Context +A Kubernetes context is a set of access parameters +that contains a Kubernetes cluster, a user, and a namespace. +For more information, see +[Configure Access to Multiple Clusters](https://kubernetes.io/docs/tasks/access-application-cluster/configure-access-multiple-clusters/). + A context environment variable is available via `Deno.env.get("CONTEXT")`. It can be used like this: @@ -193,7 +203,9 @@ spec: ## Examples -**Example 1** defines a full-fledged Deno script +### Example 1: inline script + +This example defines a full-fledged Deno script within the `KeptnTaskDefinition` YAML file: ```yaml @@ -214,7 +226,9 @@ spec: console.log("Hello, " + name + " new"); ``` -**Example 2** fetches the Deno script from a remote webserver at runtime: +### Example 2: httpRef script + +This example fetches the Deno script from a remote webserver at runtime: ```yaml apiVersion: lifecycle.keptn.sh/v1alpha3 @@ -227,11 +241,16 @@ spec: url: ``` +For another example, see the +[sample-app](https://github.com/keptn-sandbox/lifecycle-toolkit-examples/blob/main/sample-app/version-1/app-pre-deploy.yaml). + See the [sample-app/version-1](https://github.com/keptn-sandbox/lifecycle-toolkit-examples/blob/main/sample-app/version-1/app-pre-deploy.yaml) PodtatoHead example for a more complete example. -**Example 3** calls another defined task, +### Example 3: functionRef + +This example calls another defined task, illustrating how one `KeptnTaskDefinition` can build on top of other `KeptnTaskDefinition`s. In this case, it calls `slack-notification-dev`, From 9ffe1c3222be33e8738b6b06a133c84a5ac99653 Mon Sep 17 00:00:00 2001 From: Meg McRoberts Date: Tue, 11 Apr 2023 01:33:04 -0700 Subject: [PATCH 06/39] removed concepts/tasks, moved write-tasks to integrating Signed-off-by: Meg McRoberts --- docs/content/en/docs/concepts/tasks/_index.md | 172 ------------------ docs/content/en/docs/implementing/tasks.md | 99 ++++++++++ .../en/docs/tasks/write-tasks/_index.md | 113 ------------ .../en/docs/yaml-crd-ref/taskdefinition.md | 2 +- 4 files changed, 100 insertions(+), 286 deletions(-) delete mode 100644 docs/content/en/docs/concepts/tasks/_index.md create mode 100644 docs/content/en/docs/implementing/tasks.md delete mode 100644 docs/content/en/docs/tasks/write-tasks/_index.md diff --git a/docs/content/en/docs/concepts/tasks/_index.md b/docs/content/en/docs/concepts/tasks/_index.md deleted file mode 100644 index f7236be9a9..0000000000 --- a/docs/content/en/docs/concepts/tasks/_index.md +++ /dev/null @@ -1,172 +0,0 @@ ---- -title: Tasks -description: Learn what Keptn Tasks are and how to use them -icon: concepts -layout: quickstart -weight: 10 -hidechildren: true # this flag hides all sub-pages in the sidebar-multicard.html ---- - -### Keptn Task Definition - -A `KeptnTaskDefinition` is a CRD used to define tasks that can be run by the Keptn Lifecycle Toolkit -as part of pre- and post-deployment phases of a deployment. -The task definition is a [Deno](https://deno.land/) script -Please, refer to the [function runtime](https://github.com/keptn/lifecycle-toolkit/tree/main/functions-runtime) for more -information about the runtime. -In the future, we also intend to support other runtimes, especially running a container image directly. - -A task definition can be configured in three different ways: - -- inline -- referring to an HTTP script -- referring to another `KeptnTaskDefinition` - -An inline task definition looks like the following: - -```yaml -apiVersion: lifecycle.keptn.sh/v1alpha2 -kind: KeptnTaskDefinition -metadata: - name: deployment-hello -spec: - function: - inline: - code: | - console.log("Deployment Task has been executed"); -``` - -In the code section, it is possible to define a full-fletched Deno script. - -```yaml -apiVersion: lifecycle.keptn.sh/v1alpha2 -kind: KeptnTaskDefinition -metadata: - name: hello-keptn-inline -spec: - function: - inline: - code: | - let text = Deno.env.get("DATA"); - let data; - let name; - data = JSON.parse(text); - - name = data.name - console.log("Hello, " + name + " new"); -``` - -The runtime can also fetch the script on the fly from a remote webserver. For this, the CRD should look like the -following: - -```yaml -apiVersion: lifecycle.keptn.sh/v1alpha2 -kind: KeptnTaskDefinition -metadata: - name: hello-keptn-http -spec: - function: - httpRef: - url: -``` - -An example is -available [here](https://github.com/keptn-sandbox/lifecycle-toolkit-examples/blob/main/sample-app/version-1/app-pre-deploy.yaml) -. - -Finally, `KeptnTaskDefinition` can build on top of other `KeptnTaskDefinition`s. -This is a common use case where a general function can be re-used in multiple places with different parameters. - -```yaml -apiVersion: lifecycle.keptn.sh/v1alpha2 -kind: KeptnTaskDefinition -metadata: - name: slack-notification-dev -spec: - function: - functionRef: - name: slack-notification - parameters: - map: - textMessage: "This is my configuration" - secureParameters: - secret: slack-token -``` - -## Context - -A context environment variable is available via `Deno.env.get("CONTEXT")`. It can be used like this: - -```javascript -let context = Deno.env.get("CONTEXT"); - -if (contextdata.objectType == "Application") { - let application_name = contextdata.appName; - let application_version = contextdata.appVersion; -} - -if (contextdata.objectType == "Workload") { - let application_name = contextdata.appName; - let workload_name = contextdata.workloadName; - let workload_version = contextdata.workloadVersion; -} -``` - -## Input Parameters and Secret Handling - -As you might have noticed, Task Definitions also have the possibility to use input parameters. -The Lifecycle Toolkit passes the values defined inside the `map` field as a JSON object. -At the moment, multi-level maps are not supported. -The JSON object can be read through the environment variable `DATA` using `Deno.env.get("DATA");`. -K8s secrets can also be passed to the function using the `secureParameters` field. -Currently only one secret can be passed. The secret must have a `key` called `SECURE_DATA`. -It can be accessed via the environment variable `Deno.env.get("SECURE_DATA")`. - -For example: - -```yaml -# kubectl create secret generic my-secret --from-literal=SECURE_DATA=foo - -apiVersion: lifecycle.keptn.sh/v1alpha1 -kind: KeptnTaskDefinition -metadata: - name: dummy-task - namespace: "default" -spec: - function: - secureParameters: - secret: my-secret - inline: - code: | - let secret_text = Deno.env.get("SECURE_DATA"); - // secret_text = "foo" -``` - -This methodology supports multiple variables by creating a K8s secret with a JSON string: - -```yaml -# kubectl create secret generic my-secret \ -# --from-literal=SECURE_DATA="{\"foo\": \"bar\", \"foo2\": \"bar2\"}" - -apiVersion: lifecycle.keptn.sh/v1alpha1 -kind: KeptnTaskDefinition -metadata: - name: dummy-task - namespace: "default" -spec: - function: - secureParameters: - secret: my-secret - inline: - code: | - let secret_text = Deno.env.get("SECURE_DATA"); - let secret_text_obj = JSON.parse(secret_text); - // secret_text_obj["foo"] = "bar" - // secret_text_obj["foo2"] = "bar2" -``` - -### Keptn Task - -A Task is responsible for executing the TaskDefinition of a workload. -The execution is done spawning a K8s Job to handle a single Task. -In its state, it keeps track of the current status of the K8s Job created. diff --git a/docs/content/en/docs/implementing/tasks.md b/docs/content/en/docs/implementing/tasks.md new file mode 100644 index 0000000000..fa1d48fc97 --- /dev/null +++ b/docs/content/en/docs/implementing/tasks.md @@ -0,0 +1,99 @@ +--- +title: Working with Keptn tasks +description: Learn how to work with Keptn tasks +icon: concepts +layout: quickstart +weight: 20 +hidechildren: false # this flag hides all sub-pages in the sidebar-multicard.html +--- + +## Keptn Task Definition + +Keptn tasks are defined in a +[KeptnTaskDefinition](../yaml-crd-ref/taskdefinition) +CRD. +A task definition includes a function +that defines the action taken by that task. +It can be configured in one of three different ways: + +- inline +- referring to an HTTP script +- referring to another `KeptnTaskDefinition` + +## Parameterized functions + +`KeptnTaskDefinition`s can use input parameters. +Consider the following example: + +```yaml +apiVersion: lifecycle.keptn.sh/v1alpha2 +kind: KeptnTaskDefinition +metadata: + name: slack-notification-dev +spec: + function: + functionRef: + name: slack-notification + parameters: + map: + textMessage: "This is my configuration" + secureParameters: + secret: slack-token +``` + +Note the following about using parameters with functions: + +- The Lifecycle Toolkit passes the values + defined inside the `map` field as a JSON object. +- Multi-level maps are not currently supported. +- The JSON object can be read through the environment variable `DATA` + using `Deno.env.get("DATA");`. + +## Passing secrets to a function + +In the previous example, you see that +Kubernetes +[secrets](https://kubernetes.io/docs/concepts/configuration/secret/) +can be passed to the function +using the `secureParameters` field. + +Here, the `secret` value is the name of the Kubernetes secret, +which contains a field with the key `SECURE_DATA`. +The value of that field is then available to the function's runtime +via an environment variable called `SECURE_DATA`. + +For example, if you have a task function that should make use of secret data, +you must first ensure that the secret containing the `SECURE_DATA` key exists +For example: + +```yaml +apiVersion: v1 +kind: Secret +metadata: + name: deno-demo-secret + namespace: default +type: Opaque +data: + SECURE_DATA: YmFyCg== # base64 encoded string, e.g. 'bar' +``` + +Then, you can make use of that secret as follows: + +```yaml +apiVersion: lifecycle.keptn.sh/v1alpha3 +kind: KeptnTaskDefinition +metadata: + name: deployment-hello + namespace: "default" +spec: + function: + secureParameters: + secret: deno-demo-secret + inline: + code: | + console.log("Deployment Hello Task has been executed"); + + let foo = Deno.env.get('SECURE_DATA'); + console.log(foo); + Deno.exit(0); +``` diff --git a/docs/content/en/docs/tasks/write-tasks/_index.md b/docs/content/en/docs/tasks/write-tasks/_index.md deleted file mode 100644 index 5480c12ce1..0000000000 --- a/docs/content/en/docs/tasks/write-tasks/_index.md +++ /dev/null @@ -1,113 +0,0 @@ ---- -title: Write Keptn Tasks -description: Learn how to use the Keptn Lifecycle Toolkit and explore basic features. -icon: concepts -layout: quickstart -weight: 20 -hidechildren: true # this flag hides all sub-pages in the sidebar-multicard.html ---- - -## Keptn Task Definition - -A `KeptnTaskDefinition` is a CRD used to define tasks that can be run by the Keptn Lifecycle Toolkit -as part of pre- and post-deployment phases of a deployment. The task definition is a [Deno](https://deno.land/) script. -In the future, we also intend to support other runtimes, especially running a container image directly. - -A task definition can be configured in three different ways: - -- inline -- referring to an HTTP script -- referring to another `KeptnTaskDefinition` - -An inline task definition looks like the following: - -```yaml -apiVersion: lifecycle.keptn.sh/v1alpha2 -kind: KeptnTaskDefinition -metadata: - name: deployment-hello -spec: - function: - inline: - code: | - console.log("Deployment Task has been executed"); -``` - -In the code section, it is possible to define a full-fletched Deno script. - -The runtime can also fetch the script on the fly from a remote webserver. For this, the CRD should look like the -following: - -```yaml -apiVersion: lifecycle.keptn.sh/v1alpha2 -kind: KeptnTaskDefinition -metadata: - name: hello-keptn-http -spec: - function: - httpRef: - url: -``` - -Finally, `KeptnTaskDefinition` can build on top of other `KeptnTaskDefinition`s. -This is a common use case where a general function can be re-used in multiple places with different parameters. - -```yaml -apiVersion: lifecycle.keptn.sh/v1alpha2 -kind: KeptnTaskDefinition -metadata: - name: slack-notification-dev -spec: - function: - functionRef: - name: slack-notification - parameters: - map: - textMessage: "This is my configuration" - secureParameters: - secret: slack-token -``` - -As you might have noticed, Task Definitions also have the possibility to use input parameters. -The Lifecycle Toolkit passes the values defined inside the `map` field as a JSON object. -At the moment, multi-level maps are not supported. -The JSON object can be read through the environment variable `DATA` using `Deno.env.get("DATA");`. -Kubernetes secrets can also be passed to the function using the `secureParameters` field. - -Here, the `secret` value is the name of the K8s secret containing a field with the key `SECURE_DATA`. -The value of that field will then be available to the functions runtime via an environment variable called `SECURE_DATA`. - -For example, if you have a task function that should make use of secret data, you must first ensure that the secret -containing the `SECURE_DATA` key exists, as e.g.: - -```yaml -apiVersion: v1 -kind: Secret -metadata: - name: deno-demo-secret - namespace: default -type: Opaque -data: - SECURE_DATA: YmFyCg== # base64 encoded string, e.g. 'bar' -``` - -Then, you can make use of that secret as follows: - -```yaml -apiVersion: lifecycle.keptn.sh/v1alpha3 -kind: KeptnTaskDefinition -metadata: - name: deployment-hello - namespace: "default" -spec: - function: - secureParameters: - secret: deno-demo-secret - inline: - code: | - console.log("Deployment Hello Task has been executed"); - - let foo = Deno.env.get('SECURE_DATA'); - console.log(foo); - Deno.exit(0); -``` diff --git a/docs/content/en/docs/yaml-crd-ref/taskdefinition.md b/docs/content/en/docs/yaml-crd-ref/taskdefinition.md index 062f807e3c..945ba2c2ea 100644 --- a/docs/content/en/docs/yaml-crd-ref/taskdefinition.md +++ b/docs/content/en/docs/yaml-crd-ref/taskdefinition.md @@ -300,5 +300,5 @@ all `v1alpha?` library versions. ## See also -* Link to "use-case" guide pages that do something interesting with this CRD +* [Working with tasks](../implementing/tasks) * Link to reference pages for any related CRDs From 7b78c312bd72475b0b45c87f5b3955ebe32e47d3 Mon Sep 17 00:00:00 2001 From: Meg McRoberts Date: Tue, 11 Apr 2023 01:37:46 -0700 Subject: [PATCH 07/39] markdownlint error Signed-off-by: Meg McRoberts --- docs/content/en/docs/yaml-crd-ref/taskdefinition.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/content/en/docs/yaml-crd-ref/taskdefinition.md b/docs/content/en/docs/yaml-crd-ref/taskdefinition.md index 945ba2c2ea..ac47b38d23 100644 --- a/docs/content/en/docs/yaml-crd-ref/taskdefinition.md +++ b/docs/content/en/docs/yaml-crd-ref/taskdefinition.md @@ -272,9 +272,9 @@ spec: secret: slack-token ``` -**More examples** +### More examples -* See the [operator/config/samples](https://github.com/keptn/lifecycle-toolkit/tree/main/operator/config/samples) +See the [operator/config/samples](https://github.com/keptn/lifecycle-toolkit/tree/main/operator/config/samples) directory for more example `KeptnTaskDefinition` YAML files. Separate examples are provided for each API version. For example, the `lifecycle_v1alpha3_keptntaskdefinition` file From 9fedc0ede2a00d554bcf7c8f6df13f5c030c56f0 Mon Sep 17 00:00:00 2001 From: Meg McRoberts Date: Tue, 11 Apr 2023 01:45:00 -0700 Subject: [PATCH 08/39] CI fixes Signed-off-by: Meg McRoberts meg.mcroberts@dynatrace.com --- docs/content/en/docs/implementing/tasks.md | 2 +- docs/content/en/docs/yaml-crd-ref/taskdefinition.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/content/en/docs/implementing/tasks.md b/docs/content/en/docs/implementing/tasks.md index fa1d48fc97..2325538e12 100644 --- a/docs/content/en/docs/implementing/tasks.md +++ b/docs/content/en/docs/implementing/tasks.md @@ -10,7 +10,7 @@ hidechildren: false # this flag hides all sub-pages in the sidebar-multicard.htm ## Keptn Task Definition Keptn tasks are defined in a -[KeptnTaskDefinition](../yaml-crd-ref/taskdefinition) +[KeptnTaskDefinition](../yaml-crd-ref/taskdefinition/) CRD. A task definition includes a function that defines the action taken by that task. diff --git a/docs/content/en/docs/yaml-crd-ref/taskdefinition.md b/docs/content/en/docs/yaml-crd-ref/taskdefinition.md index ac47b38d23..1898134ab2 100644 --- a/docs/content/en/docs/yaml-crd-ref/taskdefinition.md +++ b/docs/content/en/docs/yaml-crd-ref/taskdefinition.md @@ -34,7 +34,7 @@ spec: * **name** -- Unique name of this task. * Must be an alphanumeric string and, by convention, is all lowercase. - * Can include the special characters `_`, `-`, . + * Can include the special characters `_`, `-`, (others?) * Should not include spaces. * **function** -- Code to be executed, From 24e631387dde50008cdea64140723f510db5ab85 Mon Sep 17 00:00:00 2001 From: Meg McRoberts Date: Tue, 11 Apr 2023 02:22:34 -0700 Subject: [PATCH 09/39] restore crd-ref as home for autogenerated files Signed-off-by: Meg McRoberts --- .../en/docs/yaml-crd-ref/taskdefinition.md | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/docs/content/en/docs/yaml-crd-ref/taskdefinition.md b/docs/content/en/docs/yaml-crd-ref/taskdefinition.md index 1898134ab2..b7b3f4d58f 100644 --- a/docs/content/en/docs/yaml-crd-ref/taskdefinition.md +++ b/docs/content/en/docs/yaml-crd-ref/taskdefinition.md @@ -284,14 +284,14 @@ contains examples for the `v1alpha3` version of the lifecycle API group. API Reference: -* [KeptnTaskDefinition](../../api-ref/lifecycle/v1alpha3/#keptntaskdefinition) -* [KeptnTaskDefinitionList](../../api-ref/lifecycle/v1alpha3/#keptntaskdefinitionlist) -* [KeptnTaskDefinitionSpec](../../api-ref/lifecycle/v1alpha3/#keptntaskdefinitionspec) -* [FunctionReference](../../api-ref/lifecycle/v1alpha3/#functionreference) -* [FunctionSpec](../../api-ref/lifecycle/v1alpha3/#functionspec) -* [FunctionStatus](../../api-ref/lifecycle/v1alpha3/#functionstatus) -* [HttpReference](../../api-ref/lifecycle/v1alpha3/#httpreference) -* [Inline](../../api-ref/lifecycle/v1alpha3/#inline) +* [KeptnTaskDefinition](../../crd-ref/lifecycle/v1alpha3/#keptntaskdefinition) +* [KeptnTaskDefinitionList](../../crd-ref/lifecycle/v1alpha3/#keptntaskdefinitionlist) +* [KeptnTaskDefinitionSpec](../../crd-ref/lifecycle/v1alpha3/#keptntaskdefinitionspec) +* [FunctionReference](../../crd-ref/lifecycle/v1alpha3/#functionreference) +* [FunctionSpec](../../crd-ref/lifecycle/v1alpha3/#functionspec) +* [FunctionStatus](../../crd-ref/lifecycle/v1alpha3/#functionstatus) +* [HttpReference](../../crd-ref/lifecycle/v1alpha3/#httpreference) +* [Inline](../../crd-ref/lifecycle/v1alpha3/#inline) ## Differences between versions From 199d5b38d9c370d5894667fcca78be5f64735f96 Mon Sep 17 00:00:00 2001 From: Meg McRoberts Date: Mon, 17 Apr 2023 02:50:43 -0700 Subject: [PATCH 10/39] delete concepts/overview again Signed-off-by: Meg McRoberts --- .../concepts/architecture/cert-manager.md | 64 ++++++++++++ .../overview/klc-cert-manager/_index.md | 25 ----- docs/content/en/docs/crd-ref/_index.md | 2 - docs/content/en/docs/install/_index.md | 3 + docs/content/en/docs/install/cert-manager.md | 99 +++++++++++++++++++ docs/content/en/docs/install/reqs.md | 9 ++ 6 files changed, 175 insertions(+), 27 deletions(-) create mode 100644 docs/content/en/docs/concepts/architecture/cert-manager.md delete mode 100644 docs/content/en/docs/concepts/overview/klc-cert-manager/_index.md create mode 100644 docs/content/en/docs/install/cert-manager.md diff --git a/docs/content/en/docs/concepts/architecture/cert-manager.md b/docs/content/en/docs/concepts/architecture/cert-manager.md new file mode 100644 index 0000000000..95eda8bf7e --- /dev/null +++ b/docs/content/en/docs/concepts/architecture/cert-manager.md @@ -0,0 +1,64 @@ +--- +title: Keptn Certificate Manager +description: Learn how the cert-manager works +icon: concepts +layout: quickstart +weight: 100 +hidechildren: false # this flag hides all sub-pages in the sidebar-multicard.html +--- + +### Keptn Cert Manager + +The Lifecycle Toolkit includes a Mutating Webhook +that requires TLS certificates to be mounted as a volume in its pod. +In version 0.6.0 and later, the certificate creation +is handled automatically by +the [klt-cert-manager](https://github.com/keptn/lifecycle-toolkit/blob/main/klt-cert-manager/README.md). + +How it works: + +* The certificate is created as a secret +in the `keptn-lifecycle-toolkit-system` namespace +with a renewal threshold of 12 hours. +* If the certificate expires, +the [klt-cert-manager](https://github.com/keptn/lifecycle-toolkit/blob/main/klt-cert-manager/README.md) +renews it. +* The Lifecycle Toolkit operator waits for a valid certificate to be ready. +* When the certificate is ready, + it is mounted on an empty dir volume in the operator. + +`klt-cert-manager` is a customized certificate manager +that is installed with the Lifecycle Toolkit by default. +It is included to simplify installation for new users +and because it is much smaller than most standard certificate managers. +However, KLT is compatible with most certificate managers +and can be configured to use another certificate manager if you prefer. +See [Use your own cert-manager](../../install/cert-manager) +for instructions. + +## Invalid certificate errors + +When a certificate is left over from an older version, +the webhook or the operator may generate errors +because of an invalid certificate. +To solve this, delete the certificate and restart the operator. + +The KLT cert-manager certificate is stored as a secret in the `klt` namespace. +To retrieve it: + +```shell +kubectl get secrets -n keptn-lifecycle-toolkit-system +``` + +This returns something like: +```shell +NAME TYPE DATA AGE +klt-certs Opaque 5 4d23h +``` + +Specify the `NAME` of the KLT certificate (`klt-certs` in this case) +to delete the KLT certificate: + +```shell +kubectl delete secret klt-certs -n keptn-lifecycle-toolkit-system +``` diff --git a/docs/content/en/docs/concepts/overview/klc-cert-manager/_index.md b/docs/content/en/docs/concepts/overview/klc-cert-manager/_index.md deleted file mode 100644 index b935a31f16..0000000000 --- a/docs/content/en/docs/concepts/overview/klc-cert-manager/_index.md +++ /dev/null @@ -1,25 +0,0 @@ ---- -title: Keptn Certificate Manager -icon: concepts -layout: quickstart -weight: 5 -hidechildren: true # this flag hides all sub-pages in the sidebar-multicard.html ---- - -### Keptn Cert Manager - -The Lifecycle Toolkit includes a Mutating Webhook which requires TLS certificates to be mounted as a volume in its pod. -In version 0.6.0 and later, the certificate creation -is handled automatically by -the [klt-cert-manager](https://github.com/keptn/lifecycle-toolkit/blob/main/klt-cert-manager/README.md). - -The certificate is created as a secret in the `keptn-lifecycle-toolkit-system` namespace with a renewal threshold of 12 -hours. -If it expires, the [klt-cert-manager](https://github.com/keptn/lifecycle-toolkit/blob/main/klt-cert-manager/README.md) -renews it. -The Lifecycle Toolkit operator waits for a valid certificate to be ready. -The certificate is mounted on an empty dir volume in the operator. - -When a certificate is left over from an older version, the webhook or the operator may generate errors because of an -invalid certificate. -To solve this, delete the certificate and restart the operator. diff --git a/docs/content/en/docs/crd-ref/_index.md b/docs/content/en/docs/crd-ref/_index.md index 13ab97bf81..b96035fe53 100644 --- a/docs/content/en/docs/crd-ref/_index.md +++ b/docs/content/en/docs/crd-ref/_index.md @@ -30,5 +30,3 @@ For more information, see the Kubernetes documentation: * [API versioning](https://kubernetes.io/docs/reference/using-api/#api-versioning) * [Understanding Kubernetes Objects](https://kubernetes.io/docs/concepts/overview/working-with-objects/kubernetes-objects/) - - diff --git a/docs/content/en/docs/install/_index.md b/docs/content/en/docs/install/_index.md index 786b2a2372..efd5c6c776 100644 --- a/docs/content/en/docs/install/_index.md +++ b/docs/content/en/docs/install/_index.md @@ -19,6 +19,9 @@ or as part of an existing production cluster. 1. Understand the [Software versions and resources](reqs.md) that are required 1. [Bring or create your Kubernetes cluster](k8s.md) +1. [Replace the default cert-manager](cert-manager.md) (optional). + This step is only required if you want to replace the default KLT cert-manager + with another cert-manager. 1. [Install the Keptn Lifecycle Controller](install.md) 1. [Integrate the Keptn Lifecycle Controller into your Kubernetes cluster](integrate.md) 1. [Upgrade](upgrade.md) to a new version of the Keptn Lifecycle Toolkit diff --git a/docs/content/en/docs/install/cert-manager.md b/docs/content/en/docs/install/cert-manager.md new file mode 100644 index 0000000000..f78913c415 --- /dev/null +++ b/docs/content/en/docs/install/cert-manager.md @@ -0,0 +1,99 @@ +--- +title: Use your own cert-manager (optional) +description: Replace the default KLT cert-manager +weight: 30 +hidechildren: false # this flag hides all sub-pages in the sidebar-multicard.html +--- + +The Keptn Lifecycle Toolkit includes +a light-weight, customized cert-manager +that is used to register Webhooks to the [KubeAPI](https://kubernetes.io/docs/reference/access-authn-authz/extensible-admission-controllers/). +Bundling the cert-manager simplifies the installation for new users +and provides the functionality KLT needs +without the overhead of other cert-managers. +For a description of the architecture, see +[Keptn Certificate Manager](../concepts/architecture/cert-manager.md). + +KLT, however, works well with standard cert-managers. +The KLT cert-manager can also coexist with another cert-manager. +If you are already using a different cert-manager, +you can continue to use that cert-manager for other components +and use the KLT cert-manager just for KLT activities +or you can configure KLT to use that cert-manager. + +If you want KLT to use your cert-manager, +you must configure it *before* you install KLT. +The steps are: + +* Install the cert-manager of your choice + if it is not already installed. +* Modify the `Deployment` manifest of each KLT operator component. +* Add the `Certificate` CRD for the cert-manager you are using. + +## Modify the KLT manifest + +You must modify the KLT manifest for each KLT operator component +to make it aware of the cert-manager you are using. +These instructions implement +[cert-manager.io](https://cert-manager.io/); +the process is similar for other cert-managers. + +To configure KLT to use your cert-manager, +change the `Deployment` manifest of each KLT operator component +and **replace** the following `volumes` definition + + ```yaml + - emptyDir: {} + name: certs-dir + ``` + + with + + ```yaml + - name: cert + secret: + defaultMode: 420 + secretName: webhook-server-cert + ``` + +Each manifest must have the following special annotation: + +```yaml +cert-manager.io/inject-ca-from=klt-serving-cert/keptn-lifecycle-toolkit-system +``` + +The value of the annotation must match the +`name/namespace` of the cert-manager CRD discussed below. + +## Add the CRD for your cert-manager + +This is the CRD for `cert-manager.io`: + +```yaml +apiVersion: cert-manager.io/v1 +kind: Certificate +metadata: + name: klt-serving-cert + namespace: keptn-lifecycle-toolkit-system +spec: + dnsNames: + - lifecycle-webhook-service.keptn-lifecycle-toolkit-system.svc + - lifecycle-webhook-service.keptn-lifecycle-toolkit-system.svc.cluster.local + issuerRef: + kind: Issuer + name: klt-selfsigned-issuer + secretName webhook-server-cert +``` + +Note the following about these fields: + +* The `apiVersion` field refers to the API for the cert-manager. +* The `metadata` section includes two fields. + The value of these fields must match the annotations + used in the KLT operator manifests. +* The value of the `secretName` field + must match the value of the `secretName` field used + in the `volumes` definition section of the KLT operator manifests above. + +See the [CA Injector](https://cert-manager.io/docs/concepts/ca-injector/) +documentation for more details. diff --git a/docs/content/en/docs/install/reqs.md b/docs/content/en/docs/install/reqs.md index eb20b5793f..e9ec16d46b 100644 --- a/docs/content/en/docs/install/reqs.md +++ b/docs/content/en/docs/install/reqs.md @@ -12,3 +12,12 @@ hidechildren: false # this flag hides all sub-pages in the sidebar-multicard.htm The Keptn Lifecycle Controller requires Kubernetes v1.24.0 or later. ## Resource requirements + +## cert-manager + +KLT includes a lightweight cert-manager +that is used for installation and Webhooks. +You can configure a different cert-manager +before you install KLT. +See [Implement your own cert-manager](cert-manager.md) +for instructions. From a32a02b2a5b96efe80ca0673e2fa3415843c1559 Mon Sep 17 00:00:00 2001 From: Simon Schrottner Date: Tue, 18 Apr 2023 11:23:07 +0200 Subject: [PATCH 11/39] fixing links Signed-off-by: Simon Schrottner --- .../en/docs/yaml-crd-ref/taskdefinition.md | 20 +++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/docs/content/en/docs/yaml-crd-ref/taskdefinition.md b/docs/content/en/docs/yaml-crd-ref/taskdefinition.md index b7b3f4d58f..68090e2bef 100644 --- a/docs/content/en/docs/yaml-crd-ref/taskdefinition.md +++ b/docs/content/en/docs/yaml-crd-ref/taskdefinition.md @@ -7,7 +7,7 @@ weight: 89 A `KeptnTaskDefinition` defines tasks that are run by the Keptn Lifecycle Toolkit as part of the pre- and post-deployment phases of a -[KeptnApp](./app). +[KeptnApp](./app.md). ## Yaml Synopsis @@ -284,14 +284,14 @@ contains examples for the `v1alpha3` version of the lifecycle API group. API Reference: -* [KeptnTaskDefinition](../../crd-ref/lifecycle/v1alpha3/#keptntaskdefinition) -* [KeptnTaskDefinitionList](../../crd-ref/lifecycle/v1alpha3/#keptntaskdefinitionlist) -* [KeptnTaskDefinitionSpec](../../crd-ref/lifecycle/v1alpha3/#keptntaskdefinitionspec) -* [FunctionReference](../../crd-ref/lifecycle/v1alpha3/#functionreference) -* [FunctionSpec](../../crd-ref/lifecycle/v1alpha3/#functionspec) -* [FunctionStatus](../../crd-ref/lifecycle/v1alpha3/#functionstatus) -* [HttpReference](../../crd-ref/lifecycle/v1alpha3/#httpreference) -* [Inline](../../crd-ref/lifecycle/v1alpha3/#inline) +* [KeptnTaskDefinition](../crd-ref/lifecycle/v1alpha3/_index.md#keptntaskdefinition) +* [KeptnTaskDefinitionList](../crd-ref/lifecycle/v1alpha3/_index.md#keptntaskdefinitionlist) +* [KeptnTaskDefinitionSpec](../crd-ref/lifecycle/v1alpha3/_index.md#keptntaskdefinitionspec) +* [FunctionReference](../crd-ref/lifecycle/v1alpha3/_index.md#functionreference) +* [FunctionSpec](../crd-ref/lifecycle/v1alpha3/_index.md#functionspec) +* [FunctionStatus](../crd-ref/lifecycle/v1alpha3/_index.md#functionstatus) +* [HttpReference](../crd-ref/lifecycle/v1alpha3/_index.md#httpreference) +* [Inline](../crd-ref/lifecycle/v1alpha3/_index.md#inline) ## Differences between versions @@ -300,5 +300,5 @@ all `v1alpha?` library versions. ## See also -* [Working with tasks](../implementing/tasks) +* [Working with tasks](../implementing/tasks.md) * Link to reference pages for any related CRDs From 4cab5c16f4ff7e3af4501aee09bbe89ba1788891 Mon Sep 17 00:00:00 2001 From: Meg McRoberts Date: Thu, 2 Mar 2023 01:03:02 -0800 Subject: [PATCH 12/39] k8s doc references on landing page Signed-off-by: Meg McRoberts --- docs/content/en/docs/crd-ref/_index.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/docs/content/en/docs/crd-ref/_index.md b/docs/content/en/docs/crd-ref/_index.md index b96035fe53..cdf84aba36 100644 --- a/docs/content/en/docs/crd-ref/_index.md +++ b/docs/content/en/docs/crd-ref/_index.md @@ -18,7 +18,8 @@ We welcome your input!** Each CRD is an object of an API library. Keptn APIs follow the Kubernetes API versioning scheme. and are themselves composed of objects and sub-objects. -By introducing CRDs, Keptn is extending the base Kubernetes API with new objects and functionality. +By introducing CRDs, Keptn is extending the base Kubernetes API +with new objects and functionality. Keptn APIs follow API versioning conventions recommended by Kubernetes. For more information, see the Kubernetes documentation: @@ -30,3 +31,4 @@ For more information, see the Kubernetes documentation: * [API versioning](https://kubernetes.io/docs/reference/using-api/#api-versioning) * [Understanding Kubernetes Objects](https://kubernetes.io/docs/concepts/overview/working-with-objects/kubernetes-objects/) + From b912b422d51ff03b3d4dbbfea39b8ab27da1bbab Mon Sep 17 00:00:00 2001 From: Meg McRoberts Date: Thu, 2 Mar 2023 01:24:22 -0800 Subject: [PATCH 13/39] process go.mod file Signed-off-by: Meg McRoberts meg.mcroberts@dynatrace.com --- docs/go.mod | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/docs/go.mod b/docs/go.mod index ca8bf79cd6..2601707167 100644 --- a/docs/go.mod +++ b/docs/go.mod @@ -2,8 +2,14 @@ module github.com/keptn/keptn-lifecycle-toolkit/docs go 1.19 +<<<<<<< HEAD require ( github.com/google/docsy/dependencies v0.6.0 // indirect github.com/keptn/community v0.0.0-20230412061900-85b0a5576ec5 // indirect github.com/keptn/docs-tooling v0.1.1 // indirect ) +======= +require github.com/keptn-sandbox/lifecycle-toolkit-docs v0.0.0-20230209144724-01b35a6cfc44 // indirect= + +require github.com/google/docsy/dependencies v0.6.0 // indirect +>>>>>>> 3246ac6 (process go.mod file) From e67fd5360f75ee12543deb3e04b17285d1daf4b2 Mon Sep 17 00:00:00 2001 From: Meg McRoberts Date: Thu, 2 Mar 2023 01:03:02 -0800 Subject: [PATCH 14/39] k8s doc references on landing page Signed-off-by: Meg McRoberts --- docs/content/en/docs/crd-ref/_index.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/docs/content/en/docs/crd-ref/_index.md b/docs/content/en/docs/crd-ref/_index.md index b96035fe53..cdf84aba36 100644 --- a/docs/content/en/docs/crd-ref/_index.md +++ b/docs/content/en/docs/crd-ref/_index.md @@ -18,7 +18,8 @@ We welcome your input!** Each CRD is an object of an API library. Keptn APIs follow the Kubernetes API versioning scheme. and are themselves composed of objects and sub-objects. -By introducing CRDs, Keptn is extending the base Kubernetes API with new objects and functionality. +By introducing CRDs, Keptn is extending the base Kubernetes API +with new objects and functionality. Keptn APIs follow API versioning conventions recommended by Kubernetes. For more information, see the Kubernetes documentation: @@ -30,3 +31,4 @@ For more information, see the Kubernetes documentation: * [API versioning](https://kubernetes.io/docs/reference/using-api/#api-versioning) * [Understanding Kubernetes Objects](https://kubernetes.io/docs/concepts/overview/working-with-objects/kubernetes-objects/) + From 03ffad46cccbe153d8955cbd6e848f329aaa67f1 Mon Sep 17 00:00:00 2001 From: Meg McRoberts Date: Thu, 2 Mar 2023 01:24:22 -0800 Subject: [PATCH 15/39] process go.mod file Signed-off-by: Meg McRoberts meg.mcroberts@dynatrace.com --- docs/go.mod | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/docs/go.mod b/docs/go.mod index ca8bf79cd6..2601707167 100644 --- a/docs/go.mod +++ b/docs/go.mod @@ -2,8 +2,14 @@ module github.com/keptn/keptn-lifecycle-toolkit/docs go 1.19 +<<<<<<< HEAD require ( github.com/google/docsy/dependencies v0.6.0 // indirect github.com/keptn/community v0.0.0-20230412061900-85b0a5576ec5 // indirect github.com/keptn/docs-tooling v0.1.1 // indirect ) +======= +require github.com/keptn-sandbox/lifecycle-toolkit-docs v0.0.0-20230209144724-01b35a6cfc44 // indirect= + +require github.com/google/docsy/dependencies v0.6.0 // indirect +>>>>>>> 3246ac6 (process go.mod file) From bea8543a1068ba522b318a0e12a3645307f75780 Mon Sep 17 00:00:00 2001 From: Meg McRoberts Date: Thu, 2 Mar 2023 01:03:02 -0800 Subject: [PATCH 16/39] rebase cleanup Signed-off-by: Meg McRoberts --- docs/content/en/docs/crd-ref/_index.md | 5 +---- 1 file changed, 1 insertion(+), 4 deletions(-) diff --git a/docs/content/en/docs/crd-ref/_index.md b/docs/content/en/docs/crd-ref/_index.md index cdf84aba36..77e22ffdc8 100644 --- a/docs/content/en/docs/crd-ref/_index.md +++ b/docs/content/en/docs/crd-ref/_index.md @@ -17,10 +17,7 @@ We welcome your input!** Each CRD is an object of an API library. Keptn APIs follow the Kubernetes API versioning scheme. -and are themselves composed of objects and sub-objects. -By introducing CRDs, Keptn is extending the base Kubernetes API -with new objects and functionality. -Keptn APIs follow API versioning conventions recommended by Kubernetes. +and are composed of objects and sub-objects. For more information, see the Kubernetes documentation: From 384fe5a0020e9f0a7926f75d27f7180db5639cf9 Mon Sep 17 00:00:00 2001 From: Meg McRoberts Date: Thu, 20 Apr 2023 05:43:08 -0700 Subject: [PATCH 17/39] add metrics sections, 0.8.0 updates Signed-off-by: Meg McRoberts --- docs/content/en/docs/crd-ref/_index.md | 1 - docs/content/en/docs/implementing/_index.md | 14 ++ .../en/docs/implementing/evaluatemetrics.md | 5 - docs/content/en/docs/implementing/metrics.md | 169 ++++++++++++++++++ .../docs/yaml-crd-ref/evaluationdefinition.md | 110 ++++++++++++ .../docs/yaml-crd-ref/evaluationdefnition.md | 5 - .../docs/yaml-crd-ref/evaluationprovider.md | 49 ++++- docs/content/en/docs/yaml-crd-ref/metric.md | 115 ++++++++++++ .../en/docs/yaml-crd-ref/metricsprovider.md | 122 ++++++++++++- docs/go.mod | 6 - 10 files changed, 577 insertions(+), 19 deletions(-) delete mode 100644 docs/content/en/docs/implementing/evaluatemetrics.md create mode 100644 docs/content/en/docs/implementing/metrics.md create mode 100644 docs/content/en/docs/yaml-crd-ref/evaluationdefinition.md delete mode 100644 docs/content/en/docs/yaml-crd-ref/evaluationdefnition.md diff --git a/docs/content/en/docs/crd-ref/_index.md b/docs/content/en/docs/crd-ref/_index.md index 77e22ffdc8..e4b173fd3b 100644 --- a/docs/content/en/docs/crd-ref/_index.md +++ b/docs/content/en/docs/crd-ref/_index.md @@ -28,4 +28,3 @@ For more information, see the Kubernetes documentation: * [API versioning](https://kubernetes.io/docs/reference/using-api/#api-versioning) * [Understanding Kubernetes Objects](https://kubernetes.io/docs/concepts/overview/working-with-objects/kubernetes-objects/) - diff --git a/docs/content/en/docs/implementing/_index.md b/docs/content/en/docs/implementing/_index.md index a8d31f3f49..68c8f181fb 100644 --- a/docs/content/en/docs/implementing/_index.md +++ b/docs/content/en/docs/implementing/_index.md @@ -12,3 +12,17 @@ This section is under development. Information that is published here has been reviewed for technical accuracy but the format and content is still evolving. We welcome your input!** + +This section provides information about how to implement +various features and functionality with the Keptn Lifecycle Toolkit. +The following topics are covered: + +* Workloads, Applications, and Deployments + +* Observability + + * Evaluations + * [Site metrics](metrics.md) that provides a single entry-point + to site metrics at the application or workload level, + based on one or more standard data providers + * Tracing diff --git a/docs/content/en/docs/implementing/evaluatemetrics.md b/docs/content/en/docs/implementing/evaluatemetrics.md deleted file mode 100644 index 0c70c43172..0000000000 --- a/docs/content/en/docs/implementing/evaluatemetrics.md +++ /dev/null @@ -1,5 +0,0 @@ ---- -title: Evaluate metrics -description: Define all workloads and checks associated with an application -weight: 130 ---- diff --git a/docs/content/en/docs/implementing/metrics.md b/docs/content/en/docs/implementing/metrics.md new file mode 100644 index 0000000000..3bcd345072 --- /dev/null +++ b/docs/content/en/docs/implementing/metrics.md @@ -0,0 +1,169 @@ +--- +title: Keptn Metrics +description: Implement Keptn site metrics +weight: 130 +--- + +Kubernetes provides two metrics servers, +one that is custom, the other external. +However, the Kubernetes metrics servers +only allow you to release a single service on a API; +you cannot use multiple observability platforms +in one namespace for one API. + +Keptn Metrics provides a single entry point to all metrics in the cluster +and allows you to use multiple observability platforms +and, beginning with V0.8.0, +multiple instances of any observability platform. +Keptn Metrics are also application aware +so report data for all workflows that are included in the Keptn application. + +This data can be presented on Grafana +or any standard dashboard application that you configure. + +[More introductory info to come] + +## Keptn metrics basics + +Keptn metrics are implemented with two CRDs: + +* [KeptnMetric](../yaml-crd-ref/metric.md) -- + define the metric to report +* [KeptnMetricsProvider](../yaml-crd-ref/metricsprovider.md) -- + define the data provider to be used for this metric + +## Using OpenTelemetry with Keptn metrics + +Keptn metrics can be exposed as OpenTelemetry (OTel) metrics +via port `9999` of the KLT metrics-operator. + +To expose OTel metrics, +be sure that the `EXPOSE_KEPTN_METRICS` environment variable +in the `metrics-operator` manifest is set to `true`, +which is the default value. + +To access the metrics, use the following command: + +```shell +kubectl port-forward deployment/metrics-operator 9999 -n keptn-lifecycle-toolkit-system +``` + +You can access the metrics from your browser at: + +```http://localhost:9999/metrics``` + +## Accessing Metrics via the Kubernetes Custom Metrics API + +`KeptnMetrics` can also be retrieved via the Kubernetes Custom Metrics API. + +### Using the HorizontalPodAutoscaler + +Use the Kubernetes Custom Metrics API +to refer to Keptnmetrics via the +[Kubernetes HorizontalPodAutoscaler](https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale/) +(HPA), +as in the following example: + +```yaml +apiVersion: autoscaling/v2 +kind: HorizontalPodAutoscaler +metadata: + name: podtato-head-entry + namespace: podtato-kubectl +spec: + scaleTargetRef: + apiVersion: apps/v1 + kind: Deployment + name: podtato-head-entry + minReplicas: 1 + maxReplicas: 10 + metrics: + - type: Object + object: + metric: + name: keptnmetric-sample + describedObject: + apiVersion: metrics.keptn.sh/v1alpha1 + kind: KeptnMetric + name: keptnmetric-sample + target: + type: Value + value: "10" +``` + +See the [Scaling Kubernetes Workloads based on Dynatrace Metrics](https://www.linkedin.com/pulse/scaling-kubernetes-workloads-based-dynatrace-metrics-keptnproject/) +blog post +for a detailed discussion of doing this with Dynatrace metrics. +A similar approach could be used to implement HPA with other data providers. + +### Retrieve KeptnMetric values with kubectl raw + +Use the `kubectl raw` command +to retrieve the values of a `KeptnMetric`, as in the following example: + +```shell +$ kubectl get --raw "/apis/custom.metrics.k8s.io/v1beta2/namespaces/podtato-kubectl/keptnmetrics.metrics.sh/keptnmetric-sample/keptnmetric-sample" | jq . + +{ + "kind": "MetricValueList", + "apiVersion": "custom.metrics.k8s.io/v1beta2", + "metadata": {}, + "items": [ + { + "describedObject": { + "kind": "KeptnMetric", + "namespace": "podtato-kubectl", + "name": "keptnmetric-sample", + "apiVersion": "metrics.keptn.sh/v1alpha1" + }, + "metric": { + "name": "keptnmetric-sample", + "selector": { + "matchLabels": { + "app": "frontend" + } + } + }, + "timestamp": "2023-01-25T09:26:15Z", + "value": "10" + } + ] +} +``` + +### Filter on matching labels + +You can filter based on matching labels. +For example, to retrieve all metrics +that are labelled with `app=frontend`, +use the following command: + +```shell +$ kubectl get --raw "/apis/custom.metrics.k8s.io/v1beta2/namespaces/podtato-kubectl/keptnmetrics.metrics.sh/*/*?labelSelector=app%3Dfrontend" | jq . + +{ + "kind": "MetricValueList", + "apiVersion": "custom.metrics.k8s.io/v1beta2", + "metadata": {}, + "items": [ + { + "describedObject": { + "kind": "KeptnMetric", + "namespace": "keptn-lifecycle-toolkit-system", + "name": "keptnmetric-sample", + "apiVersion": "metrics.keptn.sh/v1alpha1" + }, + "metric": { + "name": "keptnmetric-sample", + "selector": { + "matchLabels": { + "app": "frontend" + } + } + }, + "timestamp": "2023-01-25T09:26:15Z", + "value": "10" + } + ] +} +``` diff --git a/docs/content/en/docs/yaml-crd-ref/evaluationdefinition.md b/docs/content/en/docs/yaml-crd-ref/evaluationdefinition.md new file mode 100644 index 0000000000..55f94d9f4a --- /dev/null +++ b/docs/content/en/docs/yaml-crd-ref/evaluationdefinition.md @@ -0,0 +1,110 @@ +--- +title: KeptnEvaluationDefinition +description: Define all workloads and checks associated with an application +weight: 20 +--- +--- + +title: KeptnEvaluationDefinition +description: Define all evaluations associated with an application +weight: 30 +--- + +A `KeptnEvaluationDefinition` defines evaluation tasks +that can be run by the Keptn Lifecycle Toolkit +as part of pre- and post-analysis phases of a workload or application. + +## Yaml Synopsis + +```yaml +apiVersion: lifecycle.keptn.sh/v1alpha3 +kind: KeptnEvaluationDefinition +metadata: + name: pre-deployment-hello +spec: + objectives: + - evaluationTarget: ">1" + keptnMetricRef: + name: available-cpus + namespace: some-namespace +``` + +## Fields + +* **apiVersion** -- API version being used. +` +* **kind** -- Resource type. + Must be set to `KeptnEvaluationDefinition` + +* **metadata** + * **name** -- Unique name of this evaluation + such as `pre-deploy-eval` or `post-deploy-resource-eval`. + * Must be an alphanumeric string and, by convention, is all lowercase. + * Can include the special characters `_`, `-`, (others?) + * Should not include spaces. + +* **spec** + * **source** -- Name of the data provider being used for this evaluation. + The value of the `source` field must match + the string used for the `name` field + in the corresponding [KeptnEvaluationProvider](evaluationprovider.md) CRD. + + Each `KeptnEvaluationDefinition` CRD can use only one data provider; + if you are using multiple data provider, you must create + `KeptnEvaluationProvider` and `KeptnEvaluationDefinition` CRDs for each. + + Currently, you can only access one occurrance of each type of data provider + in your KLT cluster. + + * **objectives** -- define the evaluations to be performed. + Each objective is expressed as a `query` and an `evaluationTarget` value. + + * **query** -- Any query that is supported by the data provider. + * **value** -- Desired value of the query, + expressed as an arithmatic formula, + usually less than (`<`) or greater than (`>`) + +## Usage + +## Examples + +## Files + +API Reference: + +## Differences between versions + +In the `v1alpha1` and `v1alpha2` API versions, +`KeptnEvaluationDefinition` references the `KeptnEvaluationProvider` CRD +to identify the data source associated with this definition +and itself contained the queries +that are now taken from the specified [KeptnMetric](metric.md) CRD. +The synopsis was: + +```yaml +apiVersion: lifecycle.keptn.sh/v?alpha? +kind: KeptnEvaluationDefinition +metadata: + name: +spec: + source: prometheus | dynatrace | datadog + objectives: + - name: query-1 + query: "xxxx" + evaluationTarget: <20 + - name: query-2 + query: "yyyy" + evaluationTarget: >4 +``` + +Beginning with `v1alpha3` API version, +`KeptnEvaluationDefinition` references the data source defined +in the [KeptnMetricsProvider](metricsprovider.md) +and the queries are specified in the corresponding +[KeptnMetric](metric.md) CRD +although the `evaluationTarget` is defined in this CRD. + +## See also + +* [KeptnMetricsProvider](metricsprovider.md) +* [KeptnMetric](metric.md) diff --git a/docs/content/en/docs/yaml-crd-ref/evaluationdefnition.md b/docs/content/en/docs/yaml-crd-ref/evaluationdefnition.md deleted file mode 100644 index e054a6e08a..0000000000 --- a/docs/content/en/docs/yaml-crd-ref/evaluationdefnition.md +++ /dev/null @@ -1,5 +0,0 @@ ---- -title: KeptnEvaluationDefinition -description: Define all workloads and checks associated with an application -weight: 20 ---- diff --git a/docs/content/en/docs/yaml-crd-ref/evaluationprovider.md b/docs/content/en/docs/yaml-crd-ref/evaluationprovider.md index 1f2ad4bc43..421bbba55e 100644 --- a/docs/content/en/docs/yaml-crd-ref/evaluationprovider.md +++ b/docs/content/en/docs/yaml-crd-ref/evaluationprovider.md @@ -1,5 +1,52 @@ --- -title: KeptnEvaluationProvider +title: KeptnEvaluationProvider (deprecated) description: Define the evaluation provider weight: 13 --- + +In earlier releases of the Lifecycle Toolkit, +`KeptnEvaluationProvider` defined the data provider +used by [KeptnEvaluationDefinition]( + +## Yaml Synopsis + +```yaml +apiVersion: lifecycle.keptn.sh/v?alpha? +kind: KeptnTaskDefinition +metadata: + name: +``` + +## Fields + +* **apiVersion** -- API version being used. +` +* **kind** -- Resource type. + Must be set to `KeptnTaskDefinition` + +* **name** -- Unique name of this task. + * Must be an alphanumeric string and, by convention, is all lowercase. + * Can include the special characters `_`, `-`, (others?) + * Should not include spaces. + +## Usage + +### Create secret text + +## Examples + +## Files + +API Reference: + +* [KeptnTaskDefinition](../../crd-ref/lifecycle/v1alpha3/#keptntaskdefinition) + +## Differences between versions + +The `KeptnEvaluationProvider` is deprecated in the v1alpha3 API version. +`KeptnEvaluationDefinition` now gets provider information from the +[KeptnMetricsProvider](metricsprovider.md) CR. + +## See also + +* [KeptnEvaluationDefinition](evaluationdefinition) diff --git a/docs/content/en/docs/yaml-crd-ref/metric.md b/docs/content/en/docs/yaml-crd-ref/metric.md index 7d48b8222f..a1156e2ac7 100644 --- a/docs/content/en/docs/yaml-crd-ref/metric.md +++ b/docs/content/en/docs/yaml-crd-ref/metric.md @@ -3,3 +3,118 @@ title: KeptnMetric description: Define all workloads and checks associated with an application weight: 50 --- + +A `KeptnMetric` represents a metric that is collected from a provider. +Providing the metrics as a CRD into a Kubernetes cluster +facilitates the reusability of this data across multiple components +and allows using multiple observability platforms +for different metrics at the same time. + +A `KeptnMetric` looks like the following: + +## Yaml Synopsis + +```yaml +apiVersion: metrics.keptn.sh/v?alpha? +kind: KeptnMetric +metadata: + name: + namespace: +spec: + provider: + name: "" + query: "" + fetchIntervalSeconds: <#-seconds> +``` + +## Fields + +* **apiVersion** -- API version being used. +` +* **kind** -- Resource type. + Must be set to `KeptnTaskDefinition` + +* **metadata** + * **name** -- Unique name of this metric. + * Must be an alphanumeric string and, by convention, is all lowercase. + * Can include the special characters `_`, `-`, (others?) + * Should not include spaces. + * **namespace** -- namespace of the application using this metric + +* **spec** + * **provider.name** -- + Name of this instance of the data source + from which the metric is collected. + This value must match the value of the `spec.provider.name` field + of the corresponding [KeptnMetricsProvider](metricsprovider.md) CRD + Assigning your own name to the provider + rather than just the type of provider + enables you to support multiple instances of a data provider. + For example, you might have `dev-prometheus` + as the name of the Prometheus server that monitors the dev deployment + and `prod-prometheus` as the name of the Prometheus server + that monitors the production deployment + * **query** -- String in the provider-specific query language, + used to obtain a metric. + * **fetchIntervalSeconds** -- Number of seconds between ?? + +## Usage + +## Example + +This example pulls metrics from the data provider +defined as `my-provider` in the `spec.provider.name` field +of the corresponding `KeptnMetricsProvider` CRD. + +```yaml +apiVersion: metrics.keptn.sh/v1alpha3 +kind: KeptnMetric +metadata: + name: keptnmetric-sample + namespace: podtato-kubectl +spec: + provider: + name: "my-provider" + query: "sum(kube_pod_container_resource_limits{resource='cpu'})" + fetchIntervalSeconds: 5 + + +## Files + +API Reference: + +* [KeptnTaskDefinition](../../crd-ref/lifecycle/v1alpha3/#keptntaskdefinition) + +## Differences between versions + +Beginning with the `v1alpha3` API version, +Keptn allows you to define multiple instances of the same data source. +In earlier versions, you could use multiple data sources +but only one instance of each. +Consequently the `v1alpha1` and `v1alpha2` library versions +define the `provider` field with the name of the data provider +(`prometheus`, `dynatrace`, or `dql`) +rather than the particular name assigned +to the instance of the data provider +that is assigned in the +[KeptnMetricsProvider](metricsprovider.md) CRD. + +So the `v1alpha1` and `v1alpha2` synopsis +of the `spec` field is: + +```yaml +... +spec: + provider: + name: "prometheus | dynatrace | dql" + fetchIntervalSeconds: + query: >- + "" +``` + +## See also + +* [KeptnEvaluationDefinition](evaluationdefinition.md) +* [KeptnMetricsProvider](metricsprovider.md) +* Implementing [Keptn Metrics](../implementing/metrics.md) +* Architecture of the [Keptn Metrics Operator](../concepts/architecture/components/metrics-operator/_index.md) diff --git a/docs/content/en/docs/yaml-crd-ref/metricsprovider.md b/docs/content/en/docs/yaml-crd-ref/metricsprovider.md index 0ca9078a11..c319c56357 100644 --- a/docs/content/en/docs/yaml-crd-ref/metricsprovider.md +++ b/docs/content/en/docs/yaml-crd-ref/metricsprovider.md @@ -1,5 +1,125 @@ --- title: KeptnMetricsProvider -description: Define all workloads and checks associated with an application +description: Define data provider used for metrics and evaluations weight: 55 --- + +`KeptnMetricsProvider` defines an instance of the data provider +(such as Prometheus, Dynatrace, or Datadog) +that is used by the [KeptnMetric](metric.md) +and [KeptnEvaluationDefinition](evaluationdefinition.md) CRDs. +One Keptn application can perform evaluations and metrics +from more than one data provider +and, beginning in V0.8.0, +more than one instance of each data provider. +To implement this, create a `KeptnMetricsProvider` CRD +for each instance of each data provider being used +then reference the appropriate provider +for each evaluation or metric definition. + +## Yaml Synopsis + +```yaml +apiVersion: lifecycle.keptn.sh/v?alpha? +kind: KeptnMetricsProvider +metadata: + name: + namespace: +spec: + type: prometheus | dynatrace | dql + targetServer: "" + secretKeyRef: + name: + key: + + +## Fields + +* **apiVersion** -- API version being used. +` +* **kind** -- Resource type. Must be set to `KeptnMetricsProvider` + +* **metadata** + * **name** -- Unique name of this provider, + used to reference the provider for the + [KeptnEvaluationDefinition](evaluationdefinition) + and [KeptnMetric](metric.md) CRs. + * Must be an alphanumeric string and, by convention, is all lowercase. + * Can include the special characters `_`, `-`, (others?) + * Should not include spaces. + + * **namespace** -- Namespace where this provider is used. + +* **spec** + + * **type** -- The type of data provider for this instance + * **targetServer** -- URL of the data provider, enclosed in double quotes + * **secretKeyRef** + * **name:** -- Name of the token for this data provider + * **key:** -- Key for this data provider + + +## Usage + + +## Examples + +### Example 1: Dynatrace data provider + +```yaml +apiVersion: metrics.keptn.sh/v1alpha2 +kind: KeptnMetricsProvider +metadata: + name: dynatrace + namespace: podtato-kubectl +spec: + targetServer: "" + secretKeyRef: + name: dt-api-token + key: DT_TOKEN +``` + +## Files + +API Reference: + +* [KeptnTaskDefinition](../../crd-ref/lifecycle/v1alpha3/#keptntaskdefinition) + +## Differences between versions + +For the `v1alpha1` and `v1alpha2` API versions, +Keptn did not support +using more than one instance of a particular data provider +in the same namespace. +In other words, one namespace could support one instance each +of Prometheus, Dynatrace, and Datadog +but could not support, for example, two instances of Prometheus. + +The synopsis in those older API versions +only specified the `metadata.name` field +that identified the data provider (`prometheus`, `dynatrace`, or `dql`): + +```yaml +apiVersion: metrics.keptn.sh/v1alpha2 +kind: KeptnMetricsProvider +metadata: + name: prometheus | dynatrace |dql + namespace: +spec: + targetServer: "" + secretKeyRef: + name: dt-api-token + key: DT_TOKEN +``` + +Also note that, for the v1alpha1 and v1alpha2 API versions, +`KeptnMetricsProvider` only specifies the provider +for the `KeptnMetrics` CR. +Beginning with `v1alpha3` API version, +`KeptnMetricsProvider` is also used to specify the provider +for the `KeptnEvaluationDefinition` CR. + +## See also + +* [KeptnEvaluationDefinition](evaluationdefinition.md) +* [KeptnMetric](metric.md) diff --git a/docs/go.mod b/docs/go.mod index 2601707167..ca8bf79cd6 100644 --- a/docs/go.mod +++ b/docs/go.mod @@ -2,14 +2,8 @@ module github.com/keptn/keptn-lifecycle-toolkit/docs go 1.19 -<<<<<<< HEAD require ( github.com/google/docsy/dependencies v0.6.0 // indirect github.com/keptn/community v0.0.0-20230412061900-85b0a5576ec5 // indirect github.com/keptn/docs-tooling v0.1.1 // indirect ) -======= -require github.com/keptn-sandbox/lifecycle-toolkit-docs v0.0.0-20230209144724-01b35a6cfc44 // indirect= - -require github.com/google/docsy/dependencies v0.6.0 // indirect ->>>>>>> 3246ac6 (process go.mod file) From 005c785f90180445a0934c8ed36265cfce15ba42 Mon Sep 17 00:00:00 2001 From: Meg McRoberts Date: Mon, 17 Apr 2023 02:50:43 -0700 Subject: [PATCH 18/39] merge/rebase screwup Signed-off-by: Meg McRoberts --- docs/content/en/docs/install/_index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/content/en/docs/install/_index.md b/docs/content/en/docs/install/_index.md index 2c71a4fced..efd5c6c776 100644 --- a/docs/content/en/docs/install/_index.md +++ b/docs/content/en/docs/install/_index.md @@ -19,7 +19,7 @@ or as part of an existing production cluster. 1. Understand the [Software versions and resources](reqs.md) that are required 1. [Bring or create your Kubernetes cluster](k8s.md) -1. [Replace the default cert-manager](cert-manager.md) (optional) +1. [Replace the default cert-manager](cert-manager.md) (optional). This step is only required if you want to replace the default KLT cert-manager with another cert-manager. 1. [Install the Keptn Lifecycle Controller](install.md) From 8650bf4563a7758a4967c37e98f567728133a08f Mon Sep 17 00:00:00 2001 From: Meg McRoberts Date: Thu, 20 Apr 2023 15:10:56 -0700 Subject: [PATCH 19/39] go.mod Signed-off-by: Meg McRoberts --- docs/go.mod | 6 ------ 1 file changed, 6 deletions(-) diff --git a/docs/go.mod b/docs/go.mod index 2601707167..ca8bf79cd6 100644 --- a/docs/go.mod +++ b/docs/go.mod @@ -2,14 +2,8 @@ module github.com/keptn/keptn-lifecycle-toolkit/docs go 1.19 -<<<<<<< HEAD require ( github.com/google/docsy/dependencies v0.6.0 // indirect github.com/keptn/community v0.0.0-20230412061900-85b0a5576ec5 // indirect github.com/keptn/docs-tooling v0.1.1 // indirect ) -======= -require github.com/keptn-sandbox/lifecycle-toolkit-docs v0.0.0-20230209144724-01b35a6cfc44 // indirect= - -require github.com/google/docsy/dependencies v0.6.0 // indirect ->>>>>>> 3246ac6 (process go.mod file) From 7ad3dca290d7a593f10f63f3428a7c2ba5bb3ea7 Mon Sep 17 00:00:00 2001 From: Meg McRoberts Date: Thu, 2 Mar 2023 01:03:02 -0800 Subject: [PATCH 20/39] k8s doc references on landing page Signed-off-by: Meg McRoberts --- docs/content/en/docs/crd-ref/_index.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/docs/content/en/docs/crd-ref/_index.md b/docs/content/en/docs/crd-ref/_index.md index b96035fe53..cdf84aba36 100644 --- a/docs/content/en/docs/crd-ref/_index.md +++ b/docs/content/en/docs/crd-ref/_index.md @@ -18,7 +18,8 @@ We welcome your input!** Each CRD is an object of an API library. Keptn APIs follow the Kubernetes API versioning scheme. and are themselves composed of objects and sub-objects. -By introducing CRDs, Keptn is extending the base Kubernetes API with new objects and functionality. +By introducing CRDs, Keptn is extending the base Kubernetes API +with new objects and functionality. Keptn APIs follow API versioning conventions recommended by Kubernetes. For more information, see the Kubernetes documentation: @@ -30,3 +31,4 @@ For more information, see the Kubernetes documentation: * [API versioning](https://kubernetes.io/docs/reference/using-api/#api-versioning) * [Understanding Kubernetes Objects](https://kubernetes.io/docs/concepts/overview/working-with-objects/kubernetes-objects/) + From 7c886f92045a1a8c604c6bfbdd066c74d0253659 Mon Sep 17 00:00:00 2001 From: Meg McRoberts Date: Thu, 2 Mar 2023 01:24:22 -0800 Subject: [PATCH 21/39] process go.mod file Signed-off-by: Meg McRoberts meg.mcroberts@dynatrace.com --- docs/go.mod | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/docs/go.mod b/docs/go.mod index ca8bf79cd6..2601707167 100644 --- a/docs/go.mod +++ b/docs/go.mod @@ -2,8 +2,14 @@ module github.com/keptn/keptn-lifecycle-toolkit/docs go 1.19 +<<<<<<< HEAD require ( github.com/google/docsy/dependencies v0.6.0 // indirect github.com/keptn/community v0.0.0-20230412061900-85b0a5576ec5 // indirect github.com/keptn/docs-tooling v0.1.1 // indirect ) +======= +require github.com/keptn-sandbox/lifecycle-toolkit-docs v0.0.0-20230209144724-01b35a6cfc44 // indirect= + +require github.com/google/docsy/dependencies v0.6.0 // indirect +>>>>>>> 3246ac6 (process go.mod file) From d2800d3b0aa7a56224b550c1276df09409e07d44 Mon Sep 17 00:00:00 2001 From: Meg McRoberts Date: Thu, 2 Mar 2023 01:03:02 -0800 Subject: [PATCH 22/39] rebase cleanup Signed-off-by: Meg McRoberts --- docs/content/en/docs/crd-ref/_index.md | 5 +---- 1 file changed, 1 insertion(+), 4 deletions(-) diff --git a/docs/content/en/docs/crd-ref/_index.md b/docs/content/en/docs/crd-ref/_index.md index cdf84aba36..77e22ffdc8 100644 --- a/docs/content/en/docs/crd-ref/_index.md +++ b/docs/content/en/docs/crd-ref/_index.md @@ -17,10 +17,7 @@ We welcome your input!** Each CRD is an object of an API library. Keptn APIs follow the Kubernetes API versioning scheme. -and are themselves composed of objects and sub-objects. -By introducing CRDs, Keptn is extending the base Kubernetes API -with new objects and functionality. -Keptn APIs follow API versioning conventions recommended by Kubernetes. +and are composed of objects and sub-objects. For more information, see the Kubernetes documentation: From ebe666731df0bc93f8440e739d7d5563f854831c Mon Sep 17 00:00:00 2001 From: Meg McRoberts Date: Thu, 20 Apr 2023 05:43:08 -0700 Subject: [PATCH 23/39] add metrics sections, 0.8.0 updates Signed-off-by: Meg McRoberts --- docs/content/en/docs/crd-ref/_index.md | 1 - docs/content/en/docs/implementing/_index.md | 14 ++ .../en/docs/implementing/evaluatemetrics.md | 5 - docs/content/en/docs/implementing/metrics.md | 169 ++++++++++++++++++ .../docs/yaml-crd-ref/evaluationdefinition.md | 110 ++++++++++++ .../docs/yaml-crd-ref/evaluationdefnition.md | 5 - .../docs/yaml-crd-ref/evaluationprovider.md | 49 ++++- docs/content/en/docs/yaml-crd-ref/metric.md | 115 ++++++++++++ .../en/docs/yaml-crd-ref/metricsprovider.md | 122 ++++++++++++- docs/go.mod | 6 - 10 files changed, 577 insertions(+), 19 deletions(-) delete mode 100644 docs/content/en/docs/implementing/evaluatemetrics.md create mode 100644 docs/content/en/docs/implementing/metrics.md create mode 100644 docs/content/en/docs/yaml-crd-ref/evaluationdefinition.md delete mode 100644 docs/content/en/docs/yaml-crd-ref/evaluationdefnition.md diff --git a/docs/content/en/docs/crd-ref/_index.md b/docs/content/en/docs/crd-ref/_index.md index 77e22ffdc8..e4b173fd3b 100644 --- a/docs/content/en/docs/crd-ref/_index.md +++ b/docs/content/en/docs/crd-ref/_index.md @@ -28,4 +28,3 @@ For more information, see the Kubernetes documentation: * [API versioning](https://kubernetes.io/docs/reference/using-api/#api-versioning) * [Understanding Kubernetes Objects](https://kubernetes.io/docs/concepts/overview/working-with-objects/kubernetes-objects/) - diff --git a/docs/content/en/docs/implementing/_index.md b/docs/content/en/docs/implementing/_index.md index a8d31f3f49..68c8f181fb 100644 --- a/docs/content/en/docs/implementing/_index.md +++ b/docs/content/en/docs/implementing/_index.md @@ -12,3 +12,17 @@ This section is under development. Information that is published here has been reviewed for technical accuracy but the format and content is still evolving. We welcome your input!** + +This section provides information about how to implement +various features and functionality with the Keptn Lifecycle Toolkit. +The following topics are covered: + +* Workloads, Applications, and Deployments + +* Observability + + * Evaluations + * [Site metrics](metrics.md) that provides a single entry-point + to site metrics at the application or workload level, + based on one or more standard data providers + * Tracing diff --git a/docs/content/en/docs/implementing/evaluatemetrics.md b/docs/content/en/docs/implementing/evaluatemetrics.md deleted file mode 100644 index 0c70c43172..0000000000 --- a/docs/content/en/docs/implementing/evaluatemetrics.md +++ /dev/null @@ -1,5 +0,0 @@ ---- -title: Evaluate metrics -description: Define all workloads and checks associated with an application -weight: 130 ---- diff --git a/docs/content/en/docs/implementing/metrics.md b/docs/content/en/docs/implementing/metrics.md new file mode 100644 index 0000000000..3bcd345072 --- /dev/null +++ b/docs/content/en/docs/implementing/metrics.md @@ -0,0 +1,169 @@ +--- +title: Keptn Metrics +description: Implement Keptn site metrics +weight: 130 +--- + +Kubernetes provides two metrics servers, +one that is custom, the other external. +However, the Kubernetes metrics servers +only allow you to release a single service on a API; +you cannot use multiple observability platforms +in one namespace for one API. + +Keptn Metrics provides a single entry point to all metrics in the cluster +and allows you to use multiple observability platforms +and, beginning with V0.8.0, +multiple instances of any observability platform. +Keptn Metrics are also application aware +so report data for all workflows that are included in the Keptn application. + +This data can be presented on Grafana +or any standard dashboard application that you configure. + +[More introductory info to come] + +## Keptn metrics basics + +Keptn metrics are implemented with two CRDs: + +* [KeptnMetric](../yaml-crd-ref/metric.md) -- + define the metric to report +* [KeptnMetricsProvider](../yaml-crd-ref/metricsprovider.md) -- + define the data provider to be used for this metric + +## Using OpenTelemetry with Keptn metrics + +Keptn metrics can be exposed as OpenTelemetry (OTel) metrics +via port `9999` of the KLT metrics-operator. + +To expose OTel metrics, +be sure that the `EXPOSE_KEPTN_METRICS` environment variable +in the `metrics-operator` manifest is set to `true`, +which is the default value. + +To access the metrics, use the following command: + +```shell +kubectl port-forward deployment/metrics-operator 9999 -n keptn-lifecycle-toolkit-system +``` + +You can access the metrics from your browser at: + +```http://localhost:9999/metrics``` + +## Accessing Metrics via the Kubernetes Custom Metrics API + +`KeptnMetrics` can also be retrieved via the Kubernetes Custom Metrics API. + +### Using the HorizontalPodAutoscaler + +Use the Kubernetes Custom Metrics API +to refer to Keptnmetrics via the +[Kubernetes HorizontalPodAutoscaler](https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale/) +(HPA), +as in the following example: + +```yaml +apiVersion: autoscaling/v2 +kind: HorizontalPodAutoscaler +metadata: + name: podtato-head-entry + namespace: podtato-kubectl +spec: + scaleTargetRef: + apiVersion: apps/v1 + kind: Deployment + name: podtato-head-entry + minReplicas: 1 + maxReplicas: 10 + metrics: + - type: Object + object: + metric: + name: keptnmetric-sample + describedObject: + apiVersion: metrics.keptn.sh/v1alpha1 + kind: KeptnMetric + name: keptnmetric-sample + target: + type: Value + value: "10" +``` + +See the [Scaling Kubernetes Workloads based on Dynatrace Metrics](https://www.linkedin.com/pulse/scaling-kubernetes-workloads-based-dynatrace-metrics-keptnproject/) +blog post +for a detailed discussion of doing this with Dynatrace metrics. +A similar approach could be used to implement HPA with other data providers. + +### Retrieve KeptnMetric values with kubectl raw + +Use the `kubectl raw` command +to retrieve the values of a `KeptnMetric`, as in the following example: + +```shell +$ kubectl get --raw "/apis/custom.metrics.k8s.io/v1beta2/namespaces/podtato-kubectl/keptnmetrics.metrics.sh/keptnmetric-sample/keptnmetric-sample" | jq . + +{ + "kind": "MetricValueList", + "apiVersion": "custom.metrics.k8s.io/v1beta2", + "metadata": {}, + "items": [ + { + "describedObject": { + "kind": "KeptnMetric", + "namespace": "podtato-kubectl", + "name": "keptnmetric-sample", + "apiVersion": "metrics.keptn.sh/v1alpha1" + }, + "metric": { + "name": "keptnmetric-sample", + "selector": { + "matchLabels": { + "app": "frontend" + } + } + }, + "timestamp": "2023-01-25T09:26:15Z", + "value": "10" + } + ] +} +``` + +### Filter on matching labels + +You can filter based on matching labels. +For example, to retrieve all metrics +that are labelled with `app=frontend`, +use the following command: + +```shell +$ kubectl get --raw "/apis/custom.metrics.k8s.io/v1beta2/namespaces/podtato-kubectl/keptnmetrics.metrics.sh/*/*?labelSelector=app%3Dfrontend" | jq . + +{ + "kind": "MetricValueList", + "apiVersion": "custom.metrics.k8s.io/v1beta2", + "metadata": {}, + "items": [ + { + "describedObject": { + "kind": "KeptnMetric", + "namespace": "keptn-lifecycle-toolkit-system", + "name": "keptnmetric-sample", + "apiVersion": "metrics.keptn.sh/v1alpha1" + }, + "metric": { + "name": "keptnmetric-sample", + "selector": { + "matchLabels": { + "app": "frontend" + } + } + }, + "timestamp": "2023-01-25T09:26:15Z", + "value": "10" + } + ] +} +``` diff --git a/docs/content/en/docs/yaml-crd-ref/evaluationdefinition.md b/docs/content/en/docs/yaml-crd-ref/evaluationdefinition.md new file mode 100644 index 0000000000..55f94d9f4a --- /dev/null +++ b/docs/content/en/docs/yaml-crd-ref/evaluationdefinition.md @@ -0,0 +1,110 @@ +--- +title: KeptnEvaluationDefinition +description: Define all workloads and checks associated with an application +weight: 20 +--- +--- + +title: KeptnEvaluationDefinition +description: Define all evaluations associated with an application +weight: 30 +--- + +A `KeptnEvaluationDefinition` defines evaluation tasks +that can be run by the Keptn Lifecycle Toolkit +as part of pre- and post-analysis phases of a workload or application. + +## Yaml Synopsis + +```yaml +apiVersion: lifecycle.keptn.sh/v1alpha3 +kind: KeptnEvaluationDefinition +metadata: + name: pre-deployment-hello +spec: + objectives: + - evaluationTarget: ">1" + keptnMetricRef: + name: available-cpus + namespace: some-namespace +``` + +## Fields + +* **apiVersion** -- API version being used. +` +* **kind** -- Resource type. + Must be set to `KeptnEvaluationDefinition` + +* **metadata** + * **name** -- Unique name of this evaluation + such as `pre-deploy-eval` or `post-deploy-resource-eval`. + * Must be an alphanumeric string and, by convention, is all lowercase. + * Can include the special characters `_`, `-`, (others?) + * Should not include spaces. + +* **spec** + * **source** -- Name of the data provider being used for this evaluation. + The value of the `source` field must match + the string used for the `name` field + in the corresponding [KeptnEvaluationProvider](evaluationprovider.md) CRD. + + Each `KeptnEvaluationDefinition` CRD can use only one data provider; + if you are using multiple data provider, you must create + `KeptnEvaluationProvider` and `KeptnEvaluationDefinition` CRDs for each. + + Currently, you can only access one occurrance of each type of data provider + in your KLT cluster. + + * **objectives** -- define the evaluations to be performed. + Each objective is expressed as a `query` and an `evaluationTarget` value. + + * **query** -- Any query that is supported by the data provider. + * **value** -- Desired value of the query, + expressed as an arithmatic formula, + usually less than (`<`) or greater than (`>`) + +## Usage + +## Examples + +## Files + +API Reference: + +## Differences between versions + +In the `v1alpha1` and `v1alpha2` API versions, +`KeptnEvaluationDefinition` references the `KeptnEvaluationProvider` CRD +to identify the data source associated with this definition +and itself contained the queries +that are now taken from the specified [KeptnMetric](metric.md) CRD. +The synopsis was: + +```yaml +apiVersion: lifecycle.keptn.sh/v?alpha? +kind: KeptnEvaluationDefinition +metadata: + name: +spec: + source: prometheus | dynatrace | datadog + objectives: + - name: query-1 + query: "xxxx" + evaluationTarget: <20 + - name: query-2 + query: "yyyy" + evaluationTarget: >4 +``` + +Beginning with `v1alpha3` API version, +`KeptnEvaluationDefinition` references the data source defined +in the [KeptnMetricsProvider](metricsprovider.md) +and the queries are specified in the corresponding +[KeptnMetric](metric.md) CRD +although the `evaluationTarget` is defined in this CRD. + +## See also + +* [KeptnMetricsProvider](metricsprovider.md) +* [KeptnMetric](metric.md) diff --git a/docs/content/en/docs/yaml-crd-ref/evaluationdefnition.md b/docs/content/en/docs/yaml-crd-ref/evaluationdefnition.md deleted file mode 100644 index e054a6e08a..0000000000 --- a/docs/content/en/docs/yaml-crd-ref/evaluationdefnition.md +++ /dev/null @@ -1,5 +0,0 @@ ---- -title: KeptnEvaluationDefinition -description: Define all workloads and checks associated with an application -weight: 20 ---- diff --git a/docs/content/en/docs/yaml-crd-ref/evaluationprovider.md b/docs/content/en/docs/yaml-crd-ref/evaluationprovider.md index 1f2ad4bc43..421bbba55e 100644 --- a/docs/content/en/docs/yaml-crd-ref/evaluationprovider.md +++ b/docs/content/en/docs/yaml-crd-ref/evaluationprovider.md @@ -1,5 +1,52 @@ --- -title: KeptnEvaluationProvider +title: KeptnEvaluationProvider (deprecated) description: Define the evaluation provider weight: 13 --- + +In earlier releases of the Lifecycle Toolkit, +`KeptnEvaluationProvider` defined the data provider +used by [KeptnEvaluationDefinition]( + +## Yaml Synopsis + +```yaml +apiVersion: lifecycle.keptn.sh/v?alpha? +kind: KeptnTaskDefinition +metadata: + name: +``` + +## Fields + +* **apiVersion** -- API version being used. +` +* **kind** -- Resource type. + Must be set to `KeptnTaskDefinition` + +* **name** -- Unique name of this task. + * Must be an alphanumeric string and, by convention, is all lowercase. + * Can include the special characters `_`, `-`, (others?) + * Should not include spaces. + +## Usage + +### Create secret text + +## Examples + +## Files + +API Reference: + +* [KeptnTaskDefinition](../../crd-ref/lifecycle/v1alpha3/#keptntaskdefinition) + +## Differences between versions + +The `KeptnEvaluationProvider` is deprecated in the v1alpha3 API version. +`KeptnEvaluationDefinition` now gets provider information from the +[KeptnMetricsProvider](metricsprovider.md) CR. + +## See also + +* [KeptnEvaluationDefinition](evaluationdefinition) diff --git a/docs/content/en/docs/yaml-crd-ref/metric.md b/docs/content/en/docs/yaml-crd-ref/metric.md index 7d48b8222f..a1156e2ac7 100644 --- a/docs/content/en/docs/yaml-crd-ref/metric.md +++ b/docs/content/en/docs/yaml-crd-ref/metric.md @@ -3,3 +3,118 @@ title: KeptnMetric description: Define all workloads and checks associated with an application weight: 50 --- + +A `KeptnMetric` represents a metric that is collected from a provider. +Providing the metrics as a CRD into a Kubernetes cluster +facilitates the reusability of this data across multiple components +and allows using multiple observability platforms +for different metrics at the same time. + +A `KeptnMetric` looks like the following: + +## Yaml Synopsis + +```yaml +apiVersion: metrics.keptn.sh/v?alpha? +kind: KeptnMetric +metadata: + name: + namespace: +spec: + provider: + name: "" + query: "" + fetchIntervalSeconds: <#-seconds> +``` + +## Fields + +* **apiVersion** -- API version being used. +` +* **kind** -- Resource type. + Must be set to `KeptnTaskDefinition` + +* **metadata** + * **name** -- Unique name of this metric. + * Must be an alphanumeric string and, by convention, is all lowercase. + * Can include the special characters `_`, `-`, (others?) + * Should not include spaces. + * **namespace** -- namespace of the application using this metric + +* **spec** + * **provider.name** -- + Name of this instance of the data source + from which the metric is collected. + This value must match the value of the `spec.provider.name` field + of the corresponding [KeptnMetricsProvider](metricsprovider.md) CRD + Assigning your own name to the provider + rather than just the type of provider + enables you to support multiple instances of a data provider. + For example, you might have `dev-prometheus` + as the name of the Prometheus server that monitors the dev deployment + and `prod-prometheus` as the name of the Prometheus server + that monitors the production deployment + * **query** -- String in the provider-specific query language, + used to obtain a metric. + * **fetchIntervalSeconds** -- Number of seconds between ?? + +## Usage + +## Example + +This example pulls metrics from the data provider +defined as `my-provider` in the `spec.provider.name` field +of the corresponding `KeptnMetricsProvider` CRD. + +```yaml +apiVersion: metrics.keptn.sh/v1alpha3 +kind: KeptnMetric +metadata: + name: keptnmetric-sample + namespace: podtato-kubectl +spec: + provider: + name: "my-provider" + query: "sum(kube_pod_container_resource_limits{resource='cpu'})" + fetchIntervalSeconds: 5 + + +## Files + +API Reference: + +* [KeptnTaskDefinition](../../crd-ref/lifecycle/v1alpha3/#keptntaskdefinition) + +## Differences between versions + +Beginning with the `v1alpha3` API version, +Keptn allows you to define multiple instances of the same data source. +In earlier versions, you could use multiple data sources +but only one instance of each. +Consequently the `v1alpha1` and `v1alpha2` library versions +define the `provider` field with the name of the data provider +(`prometheus`, `dynatrace`, or `dql`) +rather than the particular name assigned +to the instance of the data provider +that is assigned in the +[KeptnMetricsProvider](metricsprovider.md) CRD. + +So the `v1alpha1` and `v1alpha2` synopsis +of the `spec` field is: + +```yaml +... +spec: + provider: + name: "prometheus | dynatrace | dql" + fetchIntervalSeconds: + query: >- + "" +``` + +## See also + +* [KeptnEvaluationDefinition](evaluationdefinition.md) +* [KeptnMetricsProvider](metricsprovider.md) +* Implementing [Keptn Metrics](../implementing/metrics.md) +* Architecture of the [Keptn Metrics Operator](../concepts/architecture/components/metrics-operator/_index.md) diff --git a/docs/content/en/docs/yaml-crd-ref/metricsprovider.md b/docs/content/en/docs/yaml-crd-ref/metricsprovider.md index 0ca9078a11..c319c56357 100644 --- a/docs/content/en/docs/yaml-crd-ref/metricsprovider.md +++ b/docs/content/en/docs/yaml-crd-ref/metricsprovider.md @@ -1,5 +1,125 @@ --- title: KeptnMetricsProvider -description: Define all workloads and checks associated with an application +description: Define data provider used for metrics and evaluations weight: 55 --- + +`KeptnMetricsProvider` defines an instance of the data provider +(such as Prometheus, Dynatrace, or Datadog) +that is used by the [KeptnMetric](metric.md) +and [KeptnEvaluationDefinition](evaluationdefinition.md) CRDs. +One Keptn application can perform evaluations and metrics +from more than one data provider +and, beginning in V0.8.0, +more than one instance of each data provider. +To implement this, create a `KeptnMetricsProvider` CRD +for each instance of each data provider being used +then reference the appropriate provider +for each evaluation or metric definition. + +## Yaml Synopsis + +```yaml +apiVersion: lifecycle.keptn.sh/v?alpha? +kind: KeptnMetricsProvider +metadata: + name: + namespace: +spec: + type: prometheus | dynatrace | dql + targetServer: "" + secretKeyRef: + name: + key: + + +## Fields + +* **apiVersion** -- API version being used. +` +* **kind** -- Resource type. Must be set to `KeptnMetricsProvider` + +* **metadata** + * **name** -- Unique name of this provider, + used to reference the provider for the + [KeptnEvaluationDefinition](evaluationdefinition) + and [KeptnMetric](metric.md) CRs. + * Must be an alphanumeric string and, by convention, is all lowercase. + * Can include the special characters `_`, `-`, (others?) + * Should not include spaces. + + * **namespace** -- Namespace where this provider is used. + +* **spec** + + * **type** -- The type of data provider for this instance + * **targetServer** -- URL of the data provider, enclosed in double quotes + * **secretKeyRef** + * **name:** -- Name of the token for this data provider + * **key:** -- Key for this data provider + + +## Usage + + +## Examples + +### Example 1: Dynatrace data provider + +```yaml +apiVersion: metrics.keptn.sh/v1alpha2 +kind: KeptnMetricsProvider +metadata: + name: dynatrace + namespace: podtato-kubectl +spec: + targetServer: "" + secretKeyRef: + name: dt-api-token + key: DT_TOKEN +``` + +## Files + +API Reference: + +* [KeptnTaskDefinition](../../crd-ref/lifecycle/v1alpha3/#keptntaskdefinition) + +## Differences between versions + +For the `v1alpha1` and `v1alpha2` API versions, +Keptn did not support +using more than one instance of a particular data provider +in the same namespace. +In other words, one namespace could support one instance each +of Prometheus, Dynatrace, and Datadog +but could not support, for example, two instances of Prometheus. + +The synopsis in those older API versions +only specified the `metadata.name` field +that identified the data provider (`prometheus`, `dynatrace`, or `dql`): + +```yaml +apiVersion: metrics.keptn.sh/v1alpha2 +kind: KeptnMetricsProvider +metadata: + name: prometheus | dynatrace |dql + namespace: +spec: + targetServer: "" + secretKeyRef: + name: dt-api-token + key: DT_TOKEN +``` + +Also note that, for the v1alpha1 and v1alpha2 API versions, +`KeptnMetricsProvider` only specifies the provider +for the `KeptnMetrics` CR. +Beginning with `v1alpha3` API version, +`KeptnMetricsProvider` is also used to specify the provider +for the `KeptnEvaluationDefinition` CR. + +## See also + +* [KeptnEvaluationDefinition](evaluationdefinition.md) +* [KeptnMetric](metric.md) diff --git a/docs/go.mod b/docs/go.mod index 2601707167..ca8bf79cd6 100644 --- a/docs/go.mod +++ b/docs/go.mod @@ -2,14 +2,8 @@ module github.com/keptn/keptn-lifecycle-toolkit/docs go 1.19 -<<<<<<< HEAD require ( github.com/google/docsy/dependencies v0.6.0 // indirect github.com/keptn/community v0.0.0-20230412061900-85b0a5576ec5 // indirect github.com/keptn/docs-tooling v0.1.1 // indirect ) -======= -require github.com/keptn-sandbox/lifecycle-toolkit-docs v0.0.0-20230209144724-01b35a6cfc44 // indirect= - -require github.com/google/docsy/dependencies v0.6.0 // indirect ->>>>>>> 3246ac6 (process go.mod file) From b98aa0357ae7fd8dc00f0b9f34b67cd8b217421b Mon Sep 17 00:00:00 2001 From: Meg McRoberts Date: Mon, 17 Apr 2023 02:50:43 -0700 Subject: [PATCH 24/39] merge/rebase screwup Signed-off-by: Meg McRoberts --- docs/content/en/docs/install/_index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/content/en/docs/install/_index.md b/docs/content/en/docs/install/_index.md index 2c71a4fced..efd5c6c776 100644 --- a/docs/content/en/docs/install/_index.md +++ b/docs/content/en/docs/install/_index.md @@ -19,7 +19,7 @@ or as part of an existing production cluster. 1. Understand the [Software versions and resources](reqs.md) that are required 1. [Bring or create your Kubernetes cluster](k8s.md) -1. [Replace the default cert-manager](cert-manager.md) (optional) +1. [Replace the default cert-manager](cert-manager.md) (optional). This step is only required if you want to replace the default KLT cert-manager with another cert-manager. 1. [Install the Keptn Lifecycle Controller](install.md) From d593911e8b66ea6b7bd5005a0b4e810cb5c630ca Mon Sep 17 00:00:00 2001 From: Meg McRoberts Date: Thu, 2 Mar 2023 01:03:02 -0800 Subject: [PATCH 25/39] rebase conflicts Signed-off-by: Meg McRoberts --- docs/content/en/docs/crd-ref/_index.md | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/docs/content/en/docs/crd-ref/_index.md b/docs/content/en/docs/crd-ref/_index.md index e4b173fd3b..cdf84aba36 100644 --- a/docs/content/en/docs/crd-ref/_index.md +++ b/docs/content/en/docs/crd-ref/_index.md @@ -17,7 +17,10 @@ We welcome your input!** Each CRD is an object of an API library. Keptn APIs follow the Kubernetes API versioning scheme. -and are composed of objects and sub-objects. +and are themselves composed of objects and sub-objects. +By introducing CRDs, Keptn is extending the base Kubernetes API +with new objects and functionality. +Keptn APIs follow API versioning conventions recommended by Kubernetes. For more information, see the Kubernetes documentation: @@ -28,3 +31,4 @@ For more information, see the Kubernetes documentation: * [API versioning](https://kubernetes.io/docs/reference/using-api/#api-versioning) * [Understanding Kubernetes Objects](https://kubernetes.io/docs/concepts/overview/working-with-objects/kubernetes-objects/) + From 5fb18e78b6eb64b530f43e1a82c16aa6330d6da8 Mon Sep 17 00:00:00 2001 From: Meg McRoberts Date: Thu, 2 Mar 2023 01:24:22 -0800 Subject: [PATCH 26/39] process go.mod file Signed-off-by: Meg McRoberts meg.mcroberts@dynatrace.com --- docs/go.mod | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/docs/go.mod b/docs/go.mod index ca8bf79cd6..2601707167 100644 --- a/docs/go.mod +++ b/docs/go.mod @@ -2,8 +2,14 @@ module github.com/keptn/keptn-lifecycle-toolkit/docs go 1.19 +<<<<<<< HEAD require ( github.com/google/docsy/dependencies v0.6.0 // indirect github.com/keptn/community v0.0.0-20230412061900-85b0a5576ec5 // indirect github.com/keptn/docs-tooling v0.1.1 // indirect ) +======= +require github.com/keptn-sandbox/lifecycle-toolkit-docs v0.0.0-20230209144724-01b35a6cfc44 // indirect= + +require github.com/google/docsy/dependencies v0.6.0 // indirect +>>>>>>> 3246ac6 (process go.mod file) From 082ee325b61727f06e455581bd998b7b4db7b5dc Mon Sep 17 00:00:00 2001 From: Meg McRoberts Date: Thu, 2 Mar 2023 01:24:22 -0800 Subject: [PATCH 27/39] process go.mod file --- docs/go.mod | 6 ------ 1 file changed, 6 deletions(-) diff --git a/docs/go.mod b/docs/go.mod index 2601707167..ca8bf79cd6 100644 --- a/docs/go.mod +++ b/docs/go.mod @@ -2,14 +2,8 @@ module github.com/keptn/keptn-lifecycle-toolkit/docs go 1.19 -<<<<<<< HEAD require ( github.com/google/docsy/dependencies v0.6.0 // indirect github.com/keptn/community v0.0.0-20230412061900-85b0a5576ec5 // indirect github.com/keptn/docs-tooling v0.1.1 // indirect ) -======= -require github.com/keptn-sandbox/lifecycle-toolkit-docs v0.0.0-20230209144724-01b35a6cfc44 // indirect= - -require github.com/google/docsy/dependencies v0.6.0 // indirect ->>>>>>> 3246ac6 (process go.mod file) From 57d6897007beaf4aaac57919f963269b325c533c Mon Sep 17 00:00:00 2001 From: Meg McRoberts Date: Mon, 17 Apr 2023 02:50:43 -0700 Subject: [PATCH 28/39] more rebase Signed-off-by: Meg McRoberts --- docs/content/en/docs/crd-ref/_index.md | 1 - 1 file changed, 1 deletion(-) diff --git a/docs/content/en/docs/crd-ref/_index.md b/docs/content/en/docs/crd-ref/_index.md index cdf84aba36..8fd909997d 100644 --- a/docs/content/en/docs/crd-ref/_index.md +++ b/docs/content/en/docs/crd-ref/_index.md @@ -31,4 +31,3 @@ For more information, see the Kubernetes documentation: * [API versioning](https://kubernetes.io/docs/reference/using-api/#api-versioning) * [Understanding Kubernetes Objects](https://kubernetes.io/docs/concepts/overview/working-with-objects/kubernetes-objects/) - From 956165d8a66b9a8ac3c4cbd1a8d1a8c25f637b7b Mon Sep 17 00:00:00 2001 From: Meg McRoberts Date: Thu, 6 Apr 2023 22:21:10 -0700 Subject: [PATCH 29/39] modified apiVersion in synopsis Signed-off-by: Meg McRoberts --- .../en/docs/yaml-crd-ref/taskdefinition.md | 282 ++++++++++++++++++ 1 file changed, 282 insertions(+) diff --git a/docs/content/en/docs/yaml-crd-ref/taskdefinition.md b/docs/content/en/docs/yaml-crd-ref/taskdefinition.md index 877444e902..da729c4cfc 100644 --- a/docs/content/en/docs/yaml-crd-ref/taskdefinition.md +++ b/docs/content/en/docs/yaml-crd-ref/taskdefinition.md @@ -3,3 +3,285 @@ title: KeptnTaskDefinition description: Define tasks that can be run pre- or post-deployment weight: 89 --- + +A `KeptnTaskDefinition` defines tasks +that are run by the Keptn Lifecycle Toolkit +as part of the pre- and post-deployment phases of a `KeptnApp`. + +## Yaml Synopsis + +```yaml +apiVersion: lifecycle.keptn.sh/v?alpha? +kind: KeptnTaskDefinition +metadata: + name: +spec: + function: + inline | httpRef | functionRef: + [parameters: + map: + textMessage: "This is my configuration"] + [secureParameters: + secret: slack-token] +``` + +## Fields + +* **apiVersion** -- API version being used. +` +* **kind** -- Resource type. Must be set to `KeptnTaskDefinition` + +* **name** -- Unique name of this task. + * Must be an alphanumeric string and, by convention, is all lowercase. + * Can include the special characters `_`, `-`, . + * Should not include spaces. + +* **function** -- Code to be executed, + expressed as a [Deno](https://deno.land/) script. + Refer to [function runtime](https://github.com/keptn/lifecycle-toolkit/tree/main/functions-runtime) + for more information about the runtime. + + The `function` can be defined as one of the following: + + * **inline** - Include the actual executable code to execute. + This can be written as a full-fledged Deno script + that is included in this file. + For example: + + ```yaml + function: + inline: + code: | + console.log("Deployment Task has been executed"); + ``` + + * **httpRef** - Specify a Deno script to be executed at runtime + from the remote webserver that is specified. + For example: + + ```yaml + name: hello-keptn-http + spec: + function: + httpRef: + url: "https://www.example.com/yourscript.js" + ``` + + * **functionRef** -- Execute another `KeptnTaskDefinition` that has been defined. + Populate this field with the value of the `name` field + for the `KeptnTaskDefinition` to be called. + This is commonly used to call a general function + that is used in multiple places, possibly with different parameters. + An example is: + + ```yaml + spec: + function: + functionRef: + name: slack-notification + ``` + + This can also be used to group a set of tasks + into a single `KeptnTaskDefinition`, + such as defining a `KeptnTaskDefinition` for testing. + In this case, it other, existing `KeptnTaskDefinition`s + for each type of test to be run, + specifying each by the value of the `name` field. + +* **parameters** - An optional field to supply input parameters to a function. + The Lifecycle Toolkit passes the values defined inside the `map` field + as a JSON object. + For example: + + ```yaml + spec: + parameters: + map: + textMessage: "This is my configuration" + + The JSON object can be read + through the `DATA` environment variable using `Deno.env.get("DATA");`. + + Multi-level maps are not supported at this time. + + Currently only one secret can be passed. + The secret must have a `key` called `SECURE_DATA`. + It can be accessed via the environment variable `Deno.env.get("SECURE_DATA")`. + See [Context](#context) for details. + + +* **secureParameters** -- An optional field used to pass a Kubernetes secret. + The `secret` value is the Kubernetes secret name + that is mounted into the runtime and made available to functions + using the `SECURE_DATA` environment variable. + For example: + + ```yaml + secureParameters: + secret: slack-token + + See [Create secret text](#create-secret-text) for details. + +## Usage + +A Task is responsible for executing the TaskDefinition of a workload. +The execution is done by spawning a Kubernetes Job to handle a single Task. +In its state, it tracks the current status of this Kubernetes Job. + +### Context + +A context environment variable is available via `Deno.env.get("CONTEXT")`. +It can be used like this: + +```javascript +let context = Deno.env.get("CONTEXT"); + +if (contextdata.objectType == "Application") { + let application_name = contextdata.appName; + let application_version = contextdata.appVersion; +} + +if (contextdata.objectType == "Workload") { + let application_name = contextdata.appName; + let workload_name = contextdata.workloadName; + let workload_version = contextdata.workloadVersion; +} +``` + +### Create secret text + +```yaml +# kubectl create secret generic my-secret --from-literal=SECURE_DATA=foo + +apiVersion: lifecycle.keptn.sh/v1alpha3 +kind: KeptnTaskDefinition +metadata: + name: dummy-task + namespace: "default" +spec: + function: + secureParameters: + secret: my-secret + inline: + code: | + let secret_text = Deno.env.get("SECURE_DATA"); + // secret_text = "foo" +``` + +This methodology supports multiple variables +by creating a Kubernetes secret with a JSON string: + + +```yaml +# kubectl create secret generic my-secret \ +# --from-literal=SECURE_DATA="{\"foo\": \"bar\", \"foo2\": \"bar2\"}" + +apiVersion: lifecycle.keptn.sh/v1alpha3 +kind: KeptnTaskDefinition +metadata: + name: dummy-task + namespace: "default" +spec: + function: + secureParameters: + secret: my-secret + inline: + code: | + let secret_text = Deno.env.get("SECURE_DATA"); + let secret_text_obj = JSON.parse(secret_text); + // secret_text_obj["foo"] = "bar" + // secret_text_obj["foo2"] = "bar2" +``` + +## Examples + +**Example 1** defines a full-fledged Deno script +within the `KeptnTaskDefinition` YAML file: + +```yaml +apiVersion: lifecycle.keptn.sh/v1alpha3 +kind: KeptnTaskDefinition +metadata: + name: hello-keptn-inline +spec: + function: + inline: + code: | + let text = Deno.env.get("DATA"); + let data; + let name; + data = JSON.parse(text); + + name = data.name + console.log("Hello, " + name + " new"); +``` + +**Example 2** fetches the Deno script from a remote webserver at runtime: + +```yaml +apiVersion: lifecycle.keptn.sh/v1alpha3 +kind: KeptnTaskDefinition +metadata: + name: hello-keptn-http +spec: + function: + httpRef: + url: +``` + +See the +[sample-app/version-1](https://github.com/keptn-sandbox/lifecycle-toolkit-examples/blob/main/sample-app/version-1/app-pre-deploy.yaml) +PodtatoHead example for a more complete example. + +**Example 3** calls another defined task, +illustrating how one `KeptnTaskDefinition` can build +on top of other `KeptnTaskDefinition`s. +In this case, it calls `slack-notification-dev`, +passing `parameters` and `secureParameters` to that other task: + +```yaml +apiVersion: lifecycle.keptn.sh/v1alpha3 +kind: KeptnTaskDefinition +metadata: + name: slack-notification-dev +spec: + function: + functionRef: + name: slack-notification + parameters: + map: + textMessage: "This is my configuration" + secureParameters: + secret: slack-token +``` + +**More examples** + +* See the [operator/config/samples](https://github.com/keptn/lifecycle-toolkit/tree/main/operator/config/samples) +directory for more example `KeptnTaskDefinition` YAML files. +Separate examples are provided for each API version. +For example, the `lifecycle_v1alpha3_keptntaskdefinition` file +contains examples for the `v1alpha3` version of the lifecycle API group. +## Files + +API Reference: + +* [KeptnTaskDefinition](../../api-ref/lifecycle/v1alpha3/#keptntaskdefinition) +* [KeptnTaskDefinitionList](../../api-ref/lifecycle/v1alpha3/#keptntaskdefinitionlist) +* [KeptnTaskDefinitionSpec](../../api-ref/lifecycle/v1alpha3/#keptntaskdefinitionspec) +* [FunctionReference](../../api-ref/lifecycle/v1alpha3/#functionreference) +* [FunctionSpec](../../api-ref/lifecycle/v1alpha3/#functionspec) +* [FunctionStatus](../../api-ref/lifecycle/v1alpha3/#functionstatus) +* [HttpReference](../../api-ref/lifecycle/v1alpha3/#httpreference) +* [Inline](../../api-ref/lifecycle/v1alpha3/#inline) + +## Differences between versions + +The `KeptnTaskDefinition` is the same for +all `v1alpha?` library versions. + +## See also + +* Link to "use-case" guide pages that do something interesting with this CRD +* Link to reference pages for any related CRDs + From 996dd0a5df8878c674c2831105614c190d6209d0 Mon Sep 17 00:00:00 2001 From: Meg McRoberts Date: Fri, 7 Apr 2023 00:43:42 -0700 Subject: [PATCH 30/39] fix rebase conflict Signed-off-by: Meg McRoberts --- docs/content/en/docs/crd-ref/_index.md | 1 - docs/content/en/docs/yaml-crd-ref/taskdefinition.md | 4 +--- 2 files changed, 1 insertion(+), 4 deletions(-) diff --git a/docs/content/en/docs/crd-ref/_index.md b/docs/content/en/docs/crd-ref/_index.md index cdf84aba36..8fd909997d 100644 --- a/docs/content/en/docs/crd-ref/_index.md +++ b/docs/content/en/docs/crd-ref/_index.md @@ -31,4 +31,3 @@ For more information, see the Kubernetes documentation: * [API versioning](https://kubernetes.io/docs/reference/using-api/#api-versioning) * [Understanding Kubernetes Objects](https://kubernetes.io/docs/concepts/overview/working-with-objects/kubernetes-objects/) - diff --git a/docs/content/en/docs/yaml-crd-ref/taskdefinition.md b/docs/content/en/docs/yaml-crd-ref/taskdefinition.md index da729c4cfc..a56f553fd8 100644 --- a/docs/content/en/docs/yaml-crd-ref/taskdefinition.md +++ b/docs/content/en/docs/yaml-crd-ref/taskdefinition.md @@ -109,7 +109,6 @@ spec: It can be accessed via the environment variable `Deno.env.get("SECURE_DATA")`. See [Context](#context) for details. - * **secureParameters** -- An optional field used to pass a Kubernetes secret. The `secret` value is the Kubernetes secret name that is mounted into the runtime and made available to functions @@ -171,7 +170,6 @@ spec: This methodology supports multiple variables by creating a Kubernetes secret with a JSON string: - ```yaml # kubectl create secret generic my-secret \ # --from-literal=SECURE_DATA="{\"foo\": \"bar\", \"foo2\": \"bar2\"}" @@ -262,6 +260,7 @@ directory for more example `KeptnTaskDefinition` YAML files. Separate examples are provided for each API version. For example, the `lifecycle_v1alpha3_keptntaskdefinition` file contains examples for the `v1alpha3` version of the lifecycle API group. + ## Files API Reference: @@ -284,4 +283,3 @@ all `v1alpha?` library versions. * Link to "use-case" guide pages that do something interesting with this CRD * Link to reference pages for any related CRDs - From e2845e0dcc8282f31a5a94ced28eefd5376e124f Mon Sep 17 00:00:00 2001 From: Meg McRoberts Date: Mon, 10 Apr 2023 23:10:16 -0700 Subject: [PATCH 31/39] Fix formatting, Examples Signed-off-by: Meg McRoberts --- .../en/docs/yaml-crd-ref/taskdefinition.md | 31 +++++++++++++++---- 1 file changed, 25 insertions(+), 6 deletions(-) diff --git a/docs/content/en/docs/yaml-crd-ref/taskdefinition.md b/docs/content/en/docs/yaml-crd-ref/taskdefinition.md index a56f553fd8..062f807e3c 100644 --- a/docs/content/en/docs/yaml-crd-ref/taskdefinition.md +++ b/docs/content/en/docs/yaml-crd-ref/taskdefinition.md @@ -6,7 +6,8 @@ weight: 89 A `KeptnTaskDefinition` defines tasks that are run by the Keptn Lifecycle Toolkit -as part of the pre- and post-deployment phases of a `KeptnApp`. +as part of the pre- and post-deployment phases of a +[KeptnApp](./app). ## Yaml Synopsis @@ -98,6 +99,7 @@ spec: parameters: map: textMessage: "This is my configuration" + ``` The JSON object can be read through the `DATA` environment variable using `Deno.env.get("DATA");`. @@ -123,12 +125,20 @@ spec: ## Usage -A Task is responsible for executing the TaskDefinition of a workload. -The execution is done by spawning a Kubernetes Job to handle a single Task. +A Task is responsible for executing the TaskDefinition of a +[workload](https://kubernetes.io/docs/concepts/workloads/).. +The execution is done by spawning a Kubernetes +[Job](https://kubernetes.io/docs/concepts/workloads/controllers/job/) +to handle a single Task. In its state, it tracks the current status of this Kubernetes Job. ### Context +A Kubernetes context is a set of access parameters +that contains a Kubernetes cluster, a user, and a namespace. +For more information, see +[Configure Access to Multiple Clusters](https://kubernetes.io/docs/tasks/access-application-cluster/configure-access-multiple-clusters/). + A context environment variable is available via `Deno.env.get("CONTEXT")`. It can be used like this: @@ -193,7 +203,9 @@ spec: ## Examples -**Example 1** defines a full-fledged Deno script +### Example 1: inline script + +This example defines a full-fledged Deno script within the `KeptnTaskDefinition` YAML file: ```yaml @@ -214,7 +226,9 @@ spec: console.log("Hello, " + name + " new"); ``` -**Example 2** fetches the Deno script from a remote webserver at runtime: +### Example 2: httpRef script + +This example fetches the Deno script from a remote webserver at runtime: ```yaml apiVersion: lifecycle.keptn.sh/v1alpha3 @@ -227,11 +241,16 @@ spec: url: ``` +For another example, see the +[sample-app](https://github.com/keptn-sandbox/lifecycle-toolkit-examples/blob/main/sample-app/version-1/app-pre-deploy.yaml). + See the [sample-app/version-1](https://github.com/keptn-sandbox/lifecycle-toolkit-examples/blob/main/sample-app/version-1/app-pre-deploy.yaml) PodtatoHead example for a more complete example. -**Example 3** calls another defined task, +### Example 3: functionRef + +This example calls another defined task, illustrating how one `KeptnTaskDefinition` can build on top of other `KeptnTaskDefinition`s. In this case, it calls `slack-notification-dev`, From 77d46d27e9c907c1b122f698007e51d57fac89f6 Mon Sep 17 00:00:00 2001 From: Meg McRoberts Date: Tue, 11 Apr 2023 01:33:04 -0700 Subject: [PATCH 32/39] removed concepts/tasks, moved write-tasks to integrating Signed-off-by: Meg McRoberts --- docs/content/en/docs/implementing/tasks.md | 99 +++++++++++++++ .../en/docs/tasks/write-tasks/_index.md | 115 ------------------ .../en/docs/yaml-crd-ref/taskdefinition.md | 2 +- 3 files changed, 100 insertions(+), 116 deletions(-) create mode 100644 docs/content/en/docs/implementing/tasks.md delete mode 100644 docs/content/en/docs/tasks/write-tasks/_index.md diff --git a/docs/content/en/docs/implementing/tasks.md b/docs/content/en/docs/implementing/tasks.md new file mode 100644 index 0000000000..fa1d48fc97 --- /dev/null +++ b/docs/content/en/docs/implementing/tasks.md @@ -0,0 +1,99 @@ +--- +title: Working with Keptn tasks +description: Learn how to work with Keptn tasks +icon: concepts +layout: quickstart +weight: 20 +hidechildren: false # this flag hides all sub-pages in the sidebar-multicard.html +--- + +## Keptn Task Definition + +Keptn tasks are defined in a +[KeptnTaskDefinition](../yaml-crd-ref/taskdefinition) +CRD. +A task definition includes a function +that defines the action taken by that task. +It can be configured in one of three different ways: + +- inline +- referring to an HTTP script +- referring to another `KeptnTaskDefinition` + +## Parameterized functions + +`KeptnTaskDefinition`s can use input parameters. +Consider the following example: + +```yaml +apiVersion: lifecycle.keptn.sh/v1alpha2 +kind: KeptnTaskDefinition +metadata: + name: slack-notification-dev +spec: + function: + functionRef: + name: slack-notification + parameters: + map: + textMessage: "This is my configuration" + secureParameters: + secret: slack-token +``` + +Note the following about using parameters with functions: + +- The Lifecycle Toolkit passes the values + defined inside the `map` field as a JSON object. +- Multi-level maps are not currently supported. +- The JSON object can be read through the environment variable `DATA` + using `Deno.env.get("DATA");`. + +## Passing secrets to a function + +In the previous example, you see that +Kubernetes +[secrets](https://kubernetes.io/docs/concepts/configuration/secret/) +can be passed to the function +using the `secureParameters` field. + +Here, the `secret` value is the name of the Kubernetes secret, +which contains a field with the key `SECURE_DATA`. +The value of that field is then available to the function's runtime +via an environment variable called `SECURE_DATA`. + +For example, if you have a task function that should make use of secret data, +you must first ensure that the secret containing the `SECURE_DATA` key exists +For example: + +```yaml +apiVersion: v1 +kind: Secret +metadata: + name: deno-demo-secret + namespace: default +type: Opaque +data: + SECURE_DATA: YmFyCg== # base64 encoded string, e.g. 'bar' +``` + +Then, you can make use of that secret as follows: + +```yaml +apiVersion: lifecycle.keptn.sh/v1alpha3 +kind: KeptnTaskDefinition +metadata: + name: deployment-hello + namespace: "default" +spec: + function: + secureParameters: + secret: deno-demo-secret + inline: + code: | + console.log("Deployment Hello Task has been executed"); + + let foo = Deno.env.get('SECURE_DATA'); + console.log(foo); + Deno.exit(0); +``` diff --git a/docs/content/en/docs/tasks/write-tasks/_index.md b/docs/content/en/docs/tasks/write-tasks/_index.md deleted file mode 100644 index 88bed96f31..0000000000 --- a/docs/content/en/docs/tasks/write-tasks/_index.md +++ /dev/null @@ -1,115 +0,0 @@ ---- -title: Write Keptn Tasks -description: Learn how to use the Keptn Lifecycle Toolkit and explore basic features. -icon: concepts -layout: quickstart -weight: 20 -hidechildren: true # this flag hides all sub-pages in the sidebar-multicard.html ---- - -## Keptn Task Definition - -A `KeptnTaskDefinition` is a CRD used to define tasks that can be run by the Keptn Lifecycle Toolkit -as part of pre- and post-deployment phases of a deployment. -The task definition is a [Deno](https://deno.land/) script. -In the future, we also intend to support other runtimes, especially running a container image directly. - -A task definition can be configured in three different ways: - -- inline -- referring to an HTTP script -- referring to another `KeptnTaskDefinition` - -An inline task definition looks like the following: - -```yaml -apiVersion: lifecycle.keptn.sh/v1alpha2 -kind: KeptnTaskDefinition -metadata: - name: deployment-hello -spec: - function: - inline: - code: | - console.log("Deployment Task has been executed"); -``` - -In the code section, it is possible to define a full-fletched Deno script. - -The runtime can also fetch the script on the fly from a remote webserver. -For this, the CRD should look like the -following: - -```yaml -apiVersion: lifecycle.keptn.sh/v1alpha2 -kind: KeptnTaskDefinition -metadata: - name: hello-keptn-http -spec: - function: - httpRef: - url: -``` - -Finally, `KeptnTaskDefinition` can build on top of other `KeptnTaskDefinition`s. -This is a common use case where a general function can be re-used in multiple places with different parameters. - -```yaml -apiVersion: lifecycle.keptn.sh/v1alpha2 -kind: KeptnTaskDefinition -metadata: - name: slack-notification-dev -spec: - function: - functionRef: - name: slack-notification - parameters: - map: - textMessage: "This is my configuration" - secureParameters: - secret: slack-token -``` - -As you might have noticed, Task Definitions also have the possibility to use input parameters. -The Lifecycle Toolkit passes the values defined inside the `map` field as a JSON object. -At the moment, multi-level maps are not supported. -The JSON object can be read through the environment variable `DATA` using `Deno.env.get("DATA");`. -Kubernetes secrets can also be passed to the function using the `secureParameters` field. - -Here, the `secret` value is the name of the K8s secret containing a field with the key `SECURE_DATA`. -The value of that field will then be available to the functions runtime via an environment variable called `SECURE_DATA`. - -For example, if you have a task function that should make use of secret data, you must first ensure that the secret -containing the `SECURE_DATA` key exists, as e.g.: - -```yaml -apiVersion: v1 -kind: Secret -metadata: - name: deno-demo-secret - namespace: default -type: Opaque -data: - SECURE_DATA: YmFyCg== # base64 encoded string, e.g. 'bar' -``` - -Then, you can make use of that secret as follows: - -```yaml -apiVersion: lifecycle.keptn.sh/v1alpha3 -kind: KeptnTaskDefinition -metadata: - name: deployment-hello - namespace: "default" -spec: - function: - secureParameters: - secret: deno-demo-secret - inline: - code: | - console.log("Deployment Hello Task has been executed"); - - let foo = Deno.env.get('SECURE_DATA'); - console.log(foo); - Deno.exit(0); -``` diff --git a/docs/content/en/docs/yaml-crd-ref/taskdefinition.md b/docs/content/en/docs/yaml-crd-ref/taskdefinition.md index 062f807e3c..945ba2c2ea 100644 --- a/docs/content/en/docs/yaml-crd-ref/taskdefinition.md +++ b/docs/content/en/docs/yaml-crd-ref/taskdefinition.md @@ -300,5 +300,5 @@ all `v1alpha?` library versions. ## See also -* Link to "use-case" guide pages that do something interesting with this CRD +* [Working with tasks](../implementing/tasks) * Link to reference pages for any related CRDs From f2e5f5255949dfefe56317be371bfa7d6264a5ba Mon Sep 17 00:00:00 2001 From: Meg McRoberts Date: Tue, 11 Apr 2023 01:37:46 -0700 Subject: [PATCH 33/39] markdownlint error Signed-off-by: Meg McRoberts --- docs/content/en/docs/yaml-crd-ref/taskdefinition.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/content/en/docs/yaml-crd-ref/taskdefinition.md b/docs/content/en/docs/yaml-crd-ref/taskdefinition.md index 945ba2c2ea..ac47b38d23 100644 --- a/docs/content/en/docs/yaml-crd-ref/taskdefinition.md +++ b/docs/content/en/docs/yaml-crd-ref/taskdefinition.md @@ -272,9 +272,9 @@ spec: secret: slack-token ``` -**More examples** +### More examples -* See the [operator/config/samples](https://github.com/keptn/lifecycle-toolkit/tree/main/operator/config/samples) +See the [operator/config/samples](https://github.com/keptn/lifecycle-toolkit/tree/main/operator/config/samples) directory for more example `KeptnTaskDefinition` YAML files. Separate examples are provided for each API version. For example, the `lifecycle_v1alpha3_keptntaskdefinition` file From d98bfbf24ed41a43f410e66819eb678dfe48655c Mon Sep 17 00:00:00 2001 From: Meg McRoberts Date: Tue, 11 Apr 2023 01:45:00 -0700 Subject: [PATCH 34/39] CI fixes Signed-off-by: Meg McRoberts meg.mcroberts@dynatrace.com --- docs/content/en/docs/implementing/tasks.md | 2 +- docs/content/en/docs/yaml-crd-ref/taskdefinition.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/content/en/docs/implementing/tasks.md b/docs/content/en/docs/implementing/tasks.md index fa1d48fc97..2325538e12 100644 --- a/docs/content/en/docs/implementing/tasks.md +++ b/docs/content/en/docs/implementing/tasks.md @@ -10,7 +10,7 @@ hidechildren: false # this flag hides all sub-pages in the sidebar-multicard.htm ## Keptn Task Definition Keptn tasks are defined in a -[KeptnTaskDefinition](../yaml-crd-ref/taskdefinition) +[KeptnTaskDefinition](../yaml-crd-ref/taskdefinition/) CRD. A task definition includes a function that defines the action taken by that task. diff --git a/docs/content/en/docs/yaml-crd-ref/taskdefinition.md b/docs/content/en/docs/yaml-crd-ref/taskdefinition.md index ac47b38d23..1898134ab2 100644 --- a/docs/content/en/docs/yaml-crd-ref/taskdefinition.md +++ b/docs/content/en/docs/yaml-crd-ref/taskdefinition.md @@ -34,7 +34,7 @@ spec: * **name** -- Unique name of this task. * Must be an alphanumeric string and, by convention, is all lowercase. - * Can include the special characters `_`, `-`, . + * Can include the special characters `_`, `-`, (others?) * Should not include spaces. * **function** -- Code to be executed, From 7c05e6bb39f335879b837b2bd2fbb83ddd277145 Mon Sep 17 00:00:00 2001 From: Meg McRoberts Date: Tue, 11 Apr 2023 02:22:34 -0700 Subject: [PATCH 35/39] restore crd-ref as home for autogenerated files Signed-off-by: Meg McRoberts --- .../en/docs/yaml-crd-ref/taskdefinition.md | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/docs/content/en/docs/yaml-crd-ref/taskdefinition.md b/docs/content/en/docs/yaml-crd-ref/taskdefinition.md index 1898134ab2..b7b3f4d58f 100644 --- a/docs/content/en/docs/yaml-crd-ref/taskdefinition.md +++ b/docs/content/en/docs/yaml-crd-ref/taskdefinition.md @@ -284,14 +284,14 @@ contains examples for the `v1alpha3` version of the lifecycle API group. API Reference: -* [KeptnTaskDefinition](../../api-ref/lifecycle/v1alpha3/#keptntaskdefinition) -* [KeptnTaskDefinitionList](../../api-ref/lifecycle/v1alpha3/#keptntaskdefinitionlist) -* [KeptnTaskDefinitionSpec](../../api-ref/lifecycle/v1alpha3/#keptntaskdefinitionspec) -* [FunctionReference](../../api-ref/lifecycle/v1alpha3/#functionreference) -* [FunctionSpec](../../api-ref/lifecycle/v1alpha3/#functionspec) -* [FunctionStatus](../../api-ref/lifecycle/v1alpha3/#functionstatus) -* [HttpReference](../../api-ref/lifecycle/v1alpha3/#httpreference) -* [Inline](../../api-ref/lifecycle/v1alpha3/#inline) +* [KeptnTaskDefinition](../../crd-ref/lifecycle/v1alpha3/#keptntaskdefinition) +* [KeptnTaskDefinitionList](../../crd-ref/lifecycle/v1alpha3/#keptntaskdefinitionlist) +* [KeptnTaskDefinitionSpec](../../crd-ref/lifecycle/v1alpha3/#keptntaskdefinitionspec) +* [FunctionReference](../../crd-ref/lifecycle/v1alpha3/#functionreference) +* [FunctionSpec](../../crd-ref/lifecycle/v1alpha3/#functionspec) +* [FunctionStatus](../../crd-ref/lifecycle/v1alpha3/#functionstatus) +* [HttpReference](../../crd-ref/lifecycle/v1alpha3/#httpreference) +* [Inline](../../crd-ref/lifecycle/v1alpha3/#inline) ## Differences between versions From da01ad4f694451055912b68c7ae4ed8d69472aed Mon Sep 17 00:00:00 2001 From: Simon Schrottner Date: Tue, 18 Apr 2023 11:23:07 +0200 Subject: [PATCH 36/39] fixing links Signed-off-by: Simon Schrottner --- .../en/docs/yaml-crd-ref/taskdefinition.md | 20 +++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/docs/content/en/docs/yaml-crd-ref/taskdefinition.md b/docs/content/en/docs/yaml-crd-ref/taskdefinition.md index b7b3f4d58f..68090e2bef 100644 --- a/docs/content/en/docs/yaml-crd-ref/taskdefinition.md +++ b/docs/content/en/docs/yaml-crd-ref/taskdefinition.md @@ -7,7 +7,7 @@ weight: 89 A `KeptnTaskDefinition` defines tasks that are run by the Keptn Lifecycle Toolkit as part of the pre- and post-deployment phases of a -[KeptnApp](./app). +[KeptnApp](./app.md). ## Yaml Synopsis @@ -284,14 +284,14 @@ contains examples for the `v1alpha3` version of the lifecycle API group. API Reference: -* [KeptnTaskDefinition](../../crd-ref/lifecycle/v1alpha3/#keptntaskdefinition) -* [KeptnTaskDefinitionList](../../crd-ref/lifecycle/v1alpha3/#keptntaskdefinitionlist) -* [KeptnTaskDefinitionSpec](../../crd-ref/lifecycle/v1alpha3/#keptntaskdefinitionspec) -* [FunctionReference](../../crd-ref/lifecycle/v1alpha3/#functionreference) -* [FunctionSpec](../../crd-ref/lifecycle/v1alpha3/#functionspec) -* [FunctionStatus](../../crd-ref/lifecycle/v1alpha3/#functionstatus) -* [HttpReference](../../crd-ref/lifecycle/v1alpha3/#httpreference) -* [Inline](../../crd-ref/lifecycle/v1alpha3/#inline) +* [KeptnTaskDefinition](../crd-ref/lifecycle/v1alpha3/_index.md#keptntaskdefinition) +* [KeptnTaskDefinitionList](../crd-ref/lifecycle/v1alpha3/_index.md#keptntaskdefinitionlist) +* [KeptnTaskDefinitionSpec](../crd-ref/lifecycle/v1alpha3/_index.md#keptntaskdefinitionspec) +* [FunctionReference](../crd-ref/lifecycle/v1alpha3/_index.md#functionreference) +* [FunctionSpec](../crd-ref/lifecycle/v1alpha3/_index.md#functionspec) +* [FunctionStatus](../crd-ref/lifecycle/v1alpha3/_index.md#functionstatus) +* [HttpReference](../crd-ref/lifecycle/v1alpha3/_index.md#httpreference) +* [Inline](../crd-ref/lifecycle/v1alpha3/_index.md#inline) ## Differences between versions @@ -300,5 +300,5 @@ all `v1alpha?` library versions. ## See also -* [Working with tasks](../implementing/tasks) +* [Working with tasks](../implementing/tasks.md) * Link to reference pages for any related CRDs From 3965d2c5f486c80729bad84c6979c5b68062293f Mon Sep 17 00:00:00 2001 From: Meg McRoberts Date: Sun, 23 Apr 2023 23:17:29 -0700 Subject: [PATCH 37/39] add real URL to http example Signed-off-by: Meg McRoberts --- .../content/en/docs/concepts/architecture/cert-manager.md | 1 + docs/content/en/docs/yaml-crd-ref/taskdefinition.md | 8 ++++++-- 2 files changed, 7 insertions(+), 2 deletions(-) diff --git a/docs/content/en/docs/concepts/architecture/cert-manager.md b/docs/content/en/docs/concepts/architecture/cert-manager.md index 978bc156db..d49a1940dc 100644 --- a/docs/content/en/docs/concepts/architecture/cert-manager.md +++ b/docs/content/en/docs/concepts/architecture/cert-manager.md @@ -51,6 +51,7 @@ kubectl get secrets -n keptn-lifecycle-toolkit-system ``` This returns something like: + ```shell NAME TYPE DATA AGE klt-certs Opaque 5 4d23h diff --git a/docs/content/en/docs/yaml-crd-ref/taskdefinition.md b/docs/content/en/docs/yaml-crd-ref/taskdefinition.md index 68090e2bef..bbaa30e4a6 100644 --- a/docs/content/en/docs/yaml-crd-ref/taskdefinition.md +++ b/docs/content/en/docs/yaml-crd-ref/taskdefinition.md @@ -30,7 +30,8 @@ spec: * **apiVersion** -- API version being used. ` -* **kind** -- Resource type. Must be set to `KeptnTaskDefinition` +* **kind** -- Resource type. + Must be set to `KeptnTaskDefinition` * **name** -- Unique name of this task. * Must be an alphanumeric string and, by convention, is all lowercase. @@ -68,6 +69,9 @@ spec: url: "https://www.example.com/yourscript.js" ``` + Note that the file referenced is actually JavaScript, + which is essentially the same as a Deno script. + * **functionRef** -- Execute another `KeptnTaskDefinition` that has been defined. Populate this field with the value of the `name` field for the `KeptnTaskDefinition` to be called. @@ -238,7 +242,7 @@ metadata: spec: function: httpRef: - url: + url: "https://www.example.com/yourscript.js" ``` For another example, see the From 24cf58418a782fe1cae5c3036c9782a032f783e6 Mon Sep 17 00:00:00 2001 From: Meg McRoberts Date: Wed, 26 Apr 2023 00:23:32 -0700 Subject: [PATCH 38/39] duplicated metadata for evaluationdefinition Signed-off-by: Meg McRoberts --- docs/content/en/docs/yaml-crd-ref/evaluationdefinition.md | 6 ------ 1 file changed, 6 deletions(-) diff --git a/docs/content/en/docs/yaml-crd-ref/evaluationdefinition.md b/docs/content/en/docs/yaml-crd-ref/evaluationdefinition.md index 55f94d9f4a..b442fda0af 100644 --- a/docs/content/en/docs/yaml-crd-ref/evaluationdefinition.md +++ b/docs/content/en/docs/yaml-crd-ref/evaluationdefinition.md @@ -3,12 +3,6 @@ title: KeptnEvaluationDefinition description: Define all workloads and checks associated with an application weight: 20 --- ---- - -title: KeptnEvaluationDefinition -description: Define all evaluations associated with an application -weight: 30 ---- A `KeptnEvaluationDefinition` defines evaluation tasks that can be run by the Keptn Lifecycle Toolkit From fc3bc3349a20bc0cdc41b746ff601b86ab7817c7 Mon Sep 17 00:00:00 2001 From: Meg McRoberts Date: Fri, 28 Apr 2023 17:01:14 -0700 Subject: [PATCH 39/39] remove xrefs that are failing checks Signed-off-by: Meg McRoberts --- docs/content/en/docs/yaml-crd-ref/evaluationprovider.md | 2 -- docs/content/en/docs/yaml-crd-ref/metric.md | 2 -- docs/content/en/docs/yaml-crd-ref/metricsprovider.md | 2 -- 3 files changed, 6 deletions(-) diff --git a/docs/content/en/docs/yaml-crd-ref/evaluationprovider.md b/docs/content/en/docs/yaml-crd-ref/evaluationprovider.md index 421bbba55e..5115a74f0b 100644 --- a/docs/content/en/docs/yaml-crd-ref/evaluationprovider.md +++ b/docs/content/en/docs/yaml-crd-ref/evaluationprovider.md @@ -39,8 +39,6 @@ metadata: API Reference: -* [KeptnTaskDefinition](../../crd-ref/lifecycle/v1alpha3/#keptntaskdefinition) - ## Differences between versions The `KeptnEvaluationProvider` is deprecated in the v1alpha3 API version. diff --git a/docs/content/en/docs/yaml-crd-ref/metric.md b/docs/content/en/docs/yaml-crd-ref/metric.md index a1156e2ac7..b56b581b90 100644 --- a/docs/content/en/docs/yaml-crd-ref/metric.md +++ b/docs/content/en/docs/yaml-crd-ref/metric.md @@ -83,8 +83,6 @@ spec: API Reference: -* [KeptnTaskDefinition](../../crd-ref/lifecycle/v1alpha3/#keptntaskdefinition) - ## Differences between versions Beginning with the `v1alpha3` API version, diff --git a/docs/content/en/docs/yaml-crd-ref/metricsprovider.md b/docs/content/en/docs/yaml-crd-ref/metricsprovider.md index c319c56357..873a48f7fe 100644 --- a/docs/content/en/docs/yaml-crd-ref/metricsprovider.md +++ b/docs/content/en/docs/yaml-crd-ref/metricsprovider.md @@ -83,8 +83,6 @@ spec: API Reference: -* [KeptnTaskDefinition](../../crd-ref/lifecycle/v1alpha3/#keptntaskdefinition) - ## Differences between versions For the `v1alpha1` and `v1alpha2` API versions,