diff --git a/docs/content/en/docs/implementing/tasks/_index.md b/docs/content/en/docs/implementing/tasks/_index.md index fa1f390a96..0920a47725 100644 --- a/docs/content/en/docs/implementing/tasks/_index.md +++ b/docs/content/en/docs/implementing/tasks/_index.md @@ -113,88 +113,32 @@ a sequence of executables that need to be run in order, you can put them all in one `KeptnTaskDefinition` resource, which can execute a virtually unlimited number of programs, scripts, and functions, -as long as they all need the same runner, such as Python. - -Another option is to encode all your steps in the language of your choice -and build a container that Keptn executes. -This is often the best solution if you need to execute complex sequences. - -If you use either the `deno-runtime` or `python-runtime` runner, -you can specify the actions to take by coding the actual calls -inline in the manifest, -by calling scripts from a remote webserver, -or by calling other `KeptnTaskDefinition` resources you have defined. -This provides great flexibility in -how you define your `KeptnTaskDefinition` resources, -allowing you to define the ideal mix of executables that run sequentially -and executables (or sets of executables) that run in parallel. - -As an example, let's say you need to run a set of integration tests, -a set of performance tests, and a set of regression tests. - -- You can create one `KeptnTaskDefinition` that calls all the tasks, - in order, either by putting the actual calls - in the `KeptnTaskDefinition` resource (`inline` syntax) - or by calling scripts from a remote webserver (`httpRef` syntax), or by providing a container to run. - -- You can create separate `KeptnTaskDefinition` resources - for integration tests, performance tests, and regression tests. - - - If you annotate the `KeptnApp` resource to call - each of these `KeptnTask` resources, - the three sets of tests run in parallel. - - - You can create a "parent" `KeptnTaskDefinition` resource - that uses the `functionref` syntax - to call the `KeptnTaskDefinition` resources - for integration tests, performance tests, and regression tests. - If you annotate the `KeptnApp` resource - to run this parent `KeptnTask` resource, - all tests run sequentially. - - This approach also allows you to run the test sequence if, for example, - the integration tests require the `deno-runtime` runner - but the performance and regression tests - require the `python-runtime` runner. - The parent `KeptnTaskDefinition` can run `KeptnTask` resources - that use different runners, - although the "parent" definition runtime is used - for the container that runs all the tests. - In other words, the parent `KeptnTaskDefinition` resources - is not a merge of other `KeptnTaskDefinition` resources - but rather the code/container of the parent runner. - -- If you need to test your deployment for different platforms - (such as Linux, MacOS, and Windows) - or for different software versions - (such as Java 11, Java 17, and Java 21), - parallel testing can improve performance. - In this case, you can construct different `KeptnTaskDefinition` resources - for each platform, - perhaps defining different input parameters - (such as different secrets or environment variables). - -- Define one `KeptnTaskDefinition` resource that runs - integration tests, then regression test, then performance tests - in order. - - - You could use the `functionRef` syntax - and code the calling sequences for all tests - inside your `KeptnTaskDefinition` resource - or you could use the `httpRef` syntax - to call scripts from an external webserver. - - You could code separate `KeptnTaskDefinition` resources - for integration tests, regression tests, and performance tests. - These three test sets could then run in parallel. - - You could create a `KeptnTaskDefinition` resource - that uses the `functionRef` syntax to call the - `KeptnTaskDefinition` resources for each type of testing. - Executing that resource would execute the three types of tests - in sequential order. - -If you define a `container-runtime` runner container -for your `KeptnTaskDefinition`, -you can do anything allowed by the configuration you define for that container. +as long as they are all using the same runner. +You have the following options: + +- Encode all your steps in the language of your choice + and build an image + that Keptn executes in a `container-runtime` runner. + This is often the best solution if you need to execute complex sequences + because it gives you the most flexibility.. + +- Use the `inline` syntax for one of the Keptn pre-defined runners + (either `deno-runtime` or `python-runtime`) + to code the actual calls inline in the `KeptnTaskDefinition` resource. + See + [Fields for pre-defined containers](../../yaml-crd-ref/taskdefinition.md/#fields-for-pre-defined-containers) + for more information. + +- Create a script that calls the functions, programs, and scripts + that need to execute sequentially + and install this on a remote webserver that Keptn can access. + Then use the `httpRef` syntax for one of the pre-defined runners + to call this script from your `KeptnTaskDefinition`, + which can set parameters for the script if appropriate. + +For more details about implementing these options, see the +[KeptnTaskDefinition](../../yaml-crd-ref/taskdefinition.md) +page. ## Context diff --git a/docs/content/en/docs/migrate/strategy/_index.md b/docs/content/en/docs/migrate/strategy/_index.md index 5b6d84fc90..272467137a 100644 --- a/docs/content/en/docs/migrate/strategy/_index.md +++ b/docs/content/en/docs/migrate/strategy/_index.md @@ -323,7 +323,7 @@ Most functionality coded using the Keptn v1 (Job Executor Service) facility can simply be moved into a `KeptnTaskDefinition` resource that uses the -[container-runtime runner](../../yaml-crd-ref/taskdefinition.md/#yaml-synopsis-for-container-runtime). +[container-runtime runner](../../yaml-crd-ref/taskdefinition.md/#synopsis-for-container-runtime). If the JES container code is written in JavaScript or TypeScript, you may be able to use the `deno-runtime` runner. If the JES container code is written in Python 3, diff --git a/docs/content/en/docs/yaml-crd-ref/taskdefinition.md b/docs/content/en/docs/yaml-crd-ref/taskdefinition.md index a1846a6963..a8966173d4 100644 --- a/docs/content/en/docs/yaml-crd-ref/taskdefinition.md +++ b/docs/content/en/docs/yaml-crd-ref/taskdefinition.md @@ -29,27 +29,32 @@ differentiated by the `spec` section: to define tasks using the language and facilities of your choice, although it is more complicated that using one of the pre-defined runtimes. See - [Yaml synopsis for container-runtime](#yaml-synopsis-for-container-runtime) + [Synopsis for container-runtime](#synopsis-for-container-runtime) and - [Examples for a custom-runtime container](#examples-for-a-custom-runtime-container). -* Use the pre-defined `deno-runtime` runner - to define tasks using Deno scripts, - which use a syntax similar to JavaScript and Typescript, - with a few limitations. - You can use this to specify simple actions - without having to define a container. - See - [Yaml synopsis for Deno-runtime container](#yaml-synopsis-for-deno-runtime-container) - and - [Deno-runtime examples](#examples-for-deno-runtime-runner). -* Use the pre-defined `python-runtime` runner - to define your task using Python 3. - See - [Yaml synopsis for python-runtime runner](#yaml-synopsis-for-python-runtime-runner) - and - [Examples for a python-runtime runner](#examples-for-a-python-runtime-runner). + [Examples for a container-runtime runner](#examples-for-a-container-runtime-runner). +* Pre-defined containers + + * Use the pre-defined `deno-runtime` runner + to define tasks using + [Deno](https://deno.com/) + scripts, + which use a syntax similar to JavaScript and Typescript, + with a few limitations. + You can use this to specify simple actions + without having to define a full container. + See + [Synopsis for Deno-runtime container](#deno-runtime) + and + [Deno-runtime examples](#examples-for-deno-runtime-runner). + * Use the pre-defined `python-runtime` runner + to define your task using + [Python 3](https://www.python.org/). + See + [Synopsis for python-runtime runner](#python-runtime) + and + [Examples for a python-runtime runner](#examples-for-a-python-runtime-runner). -## Yaml Synopsis for all runners +## Synopsis for all runners The `KeptnTaskDefinition` Yaml files for all runners include the same lines at the top. @@ -64,14 +69,9 @@ spec: deno | python | container ... retries: - timeout: + timeout: ``` -The API ref points to -[Kubernetes doc](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.24/#duration-v1-meta) -where I don't find a direct hit -but timeouts seem to be measured in seconds. - ### Fields used for all containers * **apiVersion** -- API version being used. @@ -95,147 +95,41 @@ but timeouts seem to be measured in seconds. and code the functionality in Deno script, which is similar to JavaScript and Typescript. See - [Yaml synopsis for deno-runtime container](#yaml-synopsis-for-deno-runtime-container). + [Synopsis for deno-runtime container](#deno-runtime) * **python** -- Use a `python-runtime` function and code the functionality in Python 3. See - [Yaml synopsis for python-runtime runner](#yaml-synopsis-for-python-runtime-runner) + [Synopsis for python-runtime runner](#python-runtime) * **container** -- Use the runner defined for the `container-runtime` container. This is a standard Kubernetes container for which you define the image, runner, runtime parameters, etc. and code the functionality to match the container you define. See - [Yaml synopsis for container-runtime container](#yaml-synopsis-for-container-runtime). - * **retries** (optional) - specifies the number of times, + [Synopsis for container-runtime container](#synopsis-for-container-runtime). + * **retries** (optional) -- specifies the number of times a job executing the `KeptnTaskDefinition` should be restarted if an attempt is unsuccessful. - * **timeout** (optional)* -- specifies the maximum time, in seconds, + * **timeout** (optional) -- specifies the maximum time to wait for the task to be completed successfully. + The value supplied should specify the unit of measurement; + for example, `5s` indicates 5 seconds and `5m` indicates 5 minutes. If the task does not complete successfully within this time frame, it is considered to be failed. -## Yaml Synopsis for deno-runtime container - -When using the `deno-runtime` runner to define a task, -the task is coded in Deno-script -(which is mostly the same as JavaScript and TypeScript) -and executed in the -[Deno](https://deno.com/manual) runner, -which is a lightweight runtime environment -that executes in your namespace. -Note that Deno has tighter restrictions -for permissions and importing data -so a script that works properly elsewhere -may not function out of the box when run in the `deno-runtime` runner. - -```yaml -apiVersion: lifecycle.keptn.sh/v?alpha? -kind: KeptnTaskDefinition -metadata: - name: -spec: - deno: - inline | httpRef | functionRef | ConfigMapRef - parameters: - map: - textMessage: "This is my configuration" - secureParameters: - secret: slack-token -``` - -### Spec fields for deno-runtime definitions - -* **spec** - * **deno** -- Specify that the task uses the `deno-runtime` - and is expressed as a [Deno](https://deno.com/) script. - Refer to [deno runtime](https://github.com/keptn/lifecycle-toolkit/tree/main/runtimes/deno-runtime) - for more information about this runner. - - The task can be defined as one of the following: +## Synopsis for container-runtime - * **inline** - Include the actual executable code to execute. - This can be written as a full-fledged Deno script - that is included in this file. - For example: +Use the `container-runtime` to specify your own +[Kubernetes container](https://kubernetes.io/docs/concepts/containers/) +and define the task you want to execute. - ```yaml - deno: - inline: - code: | - console.log("Deployment Task has been executed"); - ``` - - * **httpRef** - Specify a Deno script to be executed at runtime - from the remote webserver that is specified. - For example: - - ```yaml - name: hello-keptn-http - spec: - deno: - httpRef: - url: "https://www.example.com/yourscript.js" - ``` - - * **functionRef** -- Execute one or more `KeptnTaskDefinition` resources - that have been defined to use the `deno-runtime` runner. - Populate this field with the value(s) of the `name` field - for the `KeptnTaskDefinition`(s) to be called. - This is commonly used to call a general function - that is used in multiple places, possibly with different parameters. - An example is: - - ```yaml - spec: - deno: - functionRef: - name: slack-notification - ``` - - This can also be used to group a set of tasks - into a single `KeptnTaskDefinition`, - such as defining a `KeptnTaskDefinition` for testing. - In this case, it calls other, existing `KeptnTaskDefinition`s - for each type of test to be run, - specifying each by the value of the `name` field. - * **ConfigMapRef** - Specify the name of a - [ConfigMap](https://kubernetes.io/docs/concepts/configuration/configmap/) - resource that contains the function to be executed. +Task sequences that are not part of the lifecycle workflow and +should be handled by the pipeline engine tools being used +such as Jenkins, Argo Workflows, Flux, and Tekton. - * **parameters** - An optional field to supply input parameters to a function. - Keptn passes the values defined inside the `map` field - as a JSON object. - For example: - - ```yaml - spec: - parameters: - map: - textMessage: "This is my configuration" - ``` - - See - [Parameterized functions](../implementing/tasks/#parameterized-functions) - for more information. - - * **secureParameters** -- An optional field used to pass a Kubernetes secret. - The `secret` value is the Kubernetes secret name - that is mounted into the runtime and made available to functions - using the `SECURE_DATA` environment variable. - For example: - - ```yaml - secureParameters: - secret: slack-token - ``` - - Note that, currently, only one secret can be passed. - - See [Create secret text](../implementing/tasks/#create-secret-text) - for details. - -## Yaml Synopsis for container-runtime +If you are migrating from Keptn v1, +you can use a `container-runtime` to execute +almost anything that you implemented with JES for Keptn v1. ```yaml apiVersion: lifecycle.keptn.sh/v?alpha? @@ -251,12 +145,6 @@ spec: ### Spec used only for container-runtime -The `container-runtime` can be used to specify -your own container image and define almost task you want to do. -If you are migrating from Keptn v1, -you can use a `container-runtime` to execute -almost anything that you implemented with JES for Keptn v1. - * **spec** * **container** -- Container definition. * **name** -- Name of the container that will run, @@ -273,14 +161,46 @@ almost anything that you implemented with JES for Keptn v1. [Container](https://kubernetes.io/docs/reference/kubernetes-api/workload-resources/pod-v1/#Container) spec documentation. -## Yaml Synopsis for Python-runtime runner +## Synopsis for pre-defined containers -The `python-runtime` runner provides a way -to easily define a task using Python 3. +The pre-defined containers allow you to easily define a task +using either Deno or Python syntax. You do not need to specify the image, volumes, and so forth. -Instead, just provide a Python script +Instead, just provide either a Deno or Python script and Keptn sets up the container and runs the script as part of the task. +### deno-runtime + +When using the `deno-runtime` runner to define a task, +the executables are coded in +[Deno-script](https://deno.com/manual), +(which is mostly the same as JavaScript and TypeScript) +and executed in the +`deno-runtime` runner, +which is a lightweight runtime environment +that executes in your namespace. +Note that Deno has tighter restrictions +for permissions and importing data +so a script that works properly elsewhere +may not function out of the box when run in the `deno-runtime` runner. + +```yaml +apiVersion: lifecycle.keptn.sh/v?alpha? +kind: KeptnTaskDefinition +metadata: + name: +spec: + deno: + inline | httpRef | functionRef | ConfigMapRef + parameters: + map: + textMessage: "This is my configuration" + secureParameters: + secret: +``` + +### python-runtime + ```yaml apiVersion: lifecycle.keptn.sh/v?alpha? kind: KeptnTaskDefinition @@ -293,98 +213,122 @@ spec: map: textMessage: "This is my configuration" secureParameters: - secret: slack-token + secret: ``` -### Spec used only for the python-runtime runner - -The `python-runtime` runner is used to define tasks using Python 3 code. +### Fields for pre-defined containers -* **spec** +* **spec** -- choose either `deno` or `python` + * **deno** -- Specify that the task uses the `deno-runtime` + and is expressed as a [Deno](https://deno.com/) script. + Refer to [deno runtime](https://github.com/keptn/lifecycle-toolkit/tree/main/runtimes/deno-runtime) + for more information about this runner. * **python** -- Identifies this as a Python runner. - * **inline** -- Include the actual Python 3.1 code to execute. - For example, the following example - prints data stored in the parameters map: - - ```yaml - python: - inline: - code: | - data = os.getenv('DATA') - print(data) - ``` - - * **httpRef** - Specify a Deno script to be executed at runtime + +* **inline | httpRef | functionRef | ConfigMapRef** -- choose the syntax + used to call the executables. + Only one of these can be specified per `KeptnTaskDefinition` resource: + + * **inline** - Include the actual executable code to execute. + You can code a sequence of executables here + that need to be run in order + as long as they are executables that are part of the lifecycle workflow. + Task sequences that are not part of the lifecycle workflow + should be handled by the pipeline engine tools being used + such as Jenkins, Argo Workflows, Flux, and Tekton. + + * **deno example:** + [Example 1: inline script for a Deno script](#example-1-inline-script-for-a-deno-script) + + * **python example:** + [Example 1: inline code for a python-runtime runner](#example-1-inline-code-for-a-python-runtime-runner) + + * **httpRef** - Specify a script to be executed at runtime from the remote webserver that is specified. - For example: - - ```yaml - name: hello-keptn-http - spec: - python: - httpRef: - url: "https://www.example.com/yourscript.py" - ``` - - * **functionRef** -- Execute one or more `KeptnTaskDefinition` resources - that have been defined to use the `python-runtime` runner. + + This syntax allows you to call a general function + that is used in multiple places, + possibly with different parameters + that are provided in the calling `KeptnTaskDefinition` resource. + Another `KeptnTaskDefinition` resource could call this same script + but with different parameters. + + Only one script can be executed. + Any other scripts listed here are silently ignored. + + * **deno example:** + [Example 2: httpRef script for a Deno script](#example-2-httpref-script-for-a-deno-script) + * **python example:** + [Example 2: httpRef for a python-runtime runner](#example-2-httpref-for-a-python-runtime-runner) + + * **functionRef** -- Execute another `KeptnTaskDefinition` resources. Populate this field with the value(s) of the `metadata.name` field for each `KeptnDefinitionTask` to be called. - This is commonly used to call a general function - that is used in multiple places, - possibly with different parameters. - An example is: - - ```yaml - spec: - python: - functionRef: - name: slack-notification - ``` - - This can also be used to group a set of tasks - into a single `KeptnTaskDefinition`, - such as defining a `KeptnTaskDefinition` for testing. - In this case, it calls other, existing `KeptnTaskDefinition`s - for each type of test to be run, - specifying each by the value of the `name` field. - - * **ConfigMapRef** -- Specify the name of a + + Like the `httpRef` syntax,this is commonly used + to call a general function that is used in multiple places, + possibly with different parameters + that are set in the calling `KeptnTaskDefinition` resource. + + You must annotate the `KeptnApp` resource to run the + calling `KeptnTaskDefiniton` resource. + + The `KeptnTaskDefinition` called with `functionref` + is the `parent task` whose runner is used for the execution + even if it is not the same runner defined in the + calling `KeptnTaskDefinition`. + + Only one `KeptnTaskDefinition` resources can be listed + with the `functionRef` syntax + although that `KeptnTaskDefinition` can call multipe + executables (programs, functions, and scripts).. + Any calls to additional `KeptnTaskDefinition` resources + are silently ignored. + + * **deno example:** + [Example 3: functionRef for a Deno script](#example-3-functionref-for-a-deno-script) + * **python example:** + [Example 3: functionRef for a python-runtime runner](#example-3-functionref-for-a-python-runtime-runner) + + * **ConfigMapRef** - Specify the name of a [ConfigMap](https://kubernetes.io/docs/concepts/configuration/configmap/) resource that contains the function to be executed. + * **deno example:** + [Example 5: ConfigMap for a Deno script](#example-5-configmap-for-a-deno-script) + * **python example:** + [Example 4: ConfigMapRef for a python-runtime runner](#example-4-configmapref-for-a-python-runtime-runner) + * **parameters** - An optional field to supply input parameters to a function. Keptn passes the values defined inside the `map` field as a JSON object. - For example: - - ```yaml - spec: - parameters: - map: - textMessage: "This is my configuration" - ``` + See + [Passing secrets, environment variables, and modifying the python command](#passing-secrets-environment-variables-and-modifying-the-python-command) + and + [Parameterized functions](../implementing/tasks/#parameterized-functions) + for more information. - See - [Parameterized functions](../implementing/tasks/#parameterized-functions) - for more information. + * **deno example:** + [Example 3: functionRef for a Deno script](#example-3-functionref-for-a-deno-script) + * **python example:** + [Example 3: functionRef for a python-runner runner](#example-3-functionref-for-a-python-runtime-runner) * **secureParameters** -- An optional field used to pass a Kubernetes secret. The `secret` value is the Kubernetes secret name that is mounted into the runtime and made available to functions using the `SECURE_DATA` environment variable. - For example: - ```yaml - secureParameters: - secret: slack-token - ``` - - Note that, currently, only one secret can be passed. + Note that, currently, only one secret can be passed + per `KeptnTaskDefinition` resource. See [Create secret text](../implementing/tasks/#create-secret-text) for details. + * **deno example:** + [Example 3: functionRef for a Deno script](#example-3-functionref-for-a-deno-script) + * **python example:** + [Example 3: functionRef for a python-runner runner](#example-3-functionref-for-a-python-runtime-runner) + ## Usage A Task executes the TaskDefinition of a @@ -406,11 +350,13 @@ the size of the volume is 50% of the memory allocated for the node. A task can be executed either pre-deployment or post-deployment as specified in the pod template specs of your Workloads -[Deployments](https://kubernetes.io/docs/concepts/workloads/controllers/deployment/), +([Deployments](https://kubernetes.io/docs/concepts/workloads/controllers/deployment/), [StatefulSets](https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/), [DaemonSets](https://kubernetes.io/docs/concepts/workloads/controllers/daemonset/), and -[ReplicaSets](https://kubernetes.io/docs/concepts/workloads/controllers/replicaset/). +[ReplicaSets](https://kubernetes.io/docs/concepts/workloads/controllers/replicaset/)) +and in the +[KeptnApp](app.md) resource. See [Pre- and post-deployment tasks](../implementing/integrate/#pre--and-post-deployment-checks) for details. @@ -418,6 +364,30 @@ Note that the annotation identifies the task by `name`. This means that you can modify the `function` code in the resource definition and the revised code is picked up without additional changes. +All `KeptnTaskDefinition` resources specified to the `KeptnApp` resource +at the same stage (either pre- or post-deployment) run in parallel. +You can run multiple executables sequentially +either by using the `inline` syntax for a pre-defined container image +or by creating your own image +and running it in the Keptn `container-runtime` runner. +See +[Executing sequential tasks](../implementing/tasks/#executing-sequential-tasks) +for more information. + +## Examples for a container-runtime runner + +For an example of a `KeptnTaskDefinition` that defines a custom container. + see +[container-task.yaml](https://github.com/keptn/lifecycle-toolkit/blob/main/examples/sample-app/base/container-task.yaml). +This is a trivial example that just runs `busybox`, +then spawns a shell and runs the `sleep 30` command: + +{{< embed path="/examples/sample-app/base/container-task.yaml" >}} + +This task is then referenced in the +[app.yaml](https://github.com/keptn/lifecycle-toolkit/blob/main/examples/sample-app/version-3/app.yaml) +file. + ## Examples for deno-runtime runner ### Example 1: inline script for a Deno script @@ -542,31 +512,6 @@ data: console.log(targetDate); ``` -## Examples for a custom-runtime container - -For an example of a `KeptnTaskDefinition` that defines a custom container. - see -[container-task.yaml](. -The `spec` includes: - -```yaml -spec: - container: - name: testy-test - image: busybox:1.36.0 - command: - - 'sh' - - '-c' - - 'sleep 30' -``` - -This task is then referenced in - -[app.yaml](https://github.com/keptn/lifecycle-toolkit/blob/main/examples/sample-app/version-3/app.yaml). - -This is a trivial example that just runs `busybox`, -then spawns a shell and runs the `sleep 30` command. - ## Examples for a python-runtime runner ### Example 1: inline code for a python-runtime runner @@ -653,9 +598,12 @@ This modifies the synopsis in the following ways: ## Limitations -Only one -[runtime](https://kubernetes.io/docs/setup/production-environment/container-runtimes/) -is allowed per `KeptnTaskDefinition`. +* Only one + [runtime](https://kubernetes.io/docs/setup/production-environment/container-runtimes/) + is allowed per `KeptnTaskDefinition` resource. + +* Only one secret can be passed + per `KeptnTaskDefinition` resource. ## See also @@ -664,3 +612,4 @@ is allowed per `KeptnTaskDefinition`. * [Pre- and post-deployment tasks](../implementing/integrate/#pre--and-post-deployment-checks) * [KeptnApp and KeptnWorkload resources](../architecture/keptn-apps/). * [Orchestrate deployment checks](../intro/usecase-orchestrate.md) +* [Executing sequential tasks](../implementing/tasks/#executing-sequential-tasks)