Abstract task and nested tasks

[startxfr]

Expected Behavior

Is there a way to use a task in a task. I couldn't find any documentation or information on how to use call a task in a task.

Actual Behavior

Task are only composed of a set of container execution. Is there a way to embed a task in another task. This could be powerfull to create "abstract" task that can be use into higher level task. Workspace, volume, env and stepTemplate heritage could be hard to handle.

Steps to Reproduce the Problem

More a enhancement rather than a problem

Additional Info

Abstract task example

This task could also be used directly in a pipeline, or a taskRun. It is called abstract in this example for the demonstration purpose.

apiVersion: tekton.dev/v1alpha1
kind: Task
metadata:
  name: abstract-task
spec:
  params:
      - name: myParam
        type: string
        description: Default param from abstract task
        default: my abstract string from abstract task
      - name: myAbstractParam
        type: string
        description: Default param from abstract task
        default: my string from abstract task
      - name: myAbstract2Param
        type: string
        description: Default param from abstract2 task
        default: my string from abstract2 task
  steps:
    - name: "abstract-task"
      image: gcr.io/example-builders/build-example
      command: ["echo"]
      args: 
        - "abstract task - ",
          "myParam : ",
          "$(inputs.params.myParam)",
          "myAbstractParam : ",
          "$(inputs.params.myAbstractParam)"
          "myAbstractParam2 : ",
          "$(inputs.params.myAbstract2Param)"

Instanciable task example

apiVersion: tekton.dev/v1alpha1
kind: Task
metadata:
  name: instanciable-task
spec:
  params:
      - name: myParam
        type: string
        description: Default param from instanciable task
        default: my param string from instanciable task
      - name: myInstanciableParam
        type: string
        description: Default param from instanciable task
        default: my string from instanciable task
  steps: 
      - name: "instanciable-task"
      taskRef:
        name: abstract-task
      inputs:
         params:
           - name: myParam
             value: "$(inputs.params.myParam)"
           - name: myAbstract2Param
             value: "$(inputs.params.myInstanciableParam)"

Pipeline example

apiVersion: tekton.dev/v1alpha1
kind: Pipeline
metadata:
  name: pipeline-with-instanciable-and-abstract
spec:
  params:
    - name: myParam
      type: string
      description: Default param from pipeline task
      default: my param string from pipeline task
  tasks:
    - name: instanciable-task
      taskRef:
        name: instanciable-task
      params:
        - name: myParam
          value: "$(params.myParam)"

[chmouel]

Perhaps this is what you are looking for ? https://github.com/tektoncd/catalog/blob/master/tkn/tkn.yaml <https://github.com/tektoncd/catalog/blob/master/tkn/tkn.yaml>

[startxfr]

Thank @chmouel for you quick answer and my problem could be solved with genric task like https://github.com/tektoncd/catalog/blob/master/tkn/tkn.yaml, and there are many ways (code duplication, genreric task, container scripts,..) to solve this problem.
Can you tell me how to "embed" this task in another task ? I wanted to call task1 who rely on task2, in order to put input, image, command, arg, volume,... in task1 with minimal config and leave task2 with the full config.

I actualy resolved it by choosing code duplication and mitigating side effet by using oc apply with template generated object list to get idempotent action from the kubernetes API.

But is there a way to perform theses example using a task heritage behaviour instead of "cheating" with duplication

[ghost]

But is there a way to perform theses example using a task heritage behaviour instead of "cheating" with duplication

No, not currently.

This enhancement lines up with some other conversations currently going on around PipelineResources. We've also explored some ideas similar to your suggestions here and (slightly related) here.

Thanks for the feedback!

[startxfr]

Thank scott for you answers and your links who help me read and learn more on how tekton pipelines works and what evolutions are in process.

Theses links and discussions are working on the Pipeline level. I was more looking for a solution at the Task level witch could help us building "high level" tasks, based on "low level" tasks.
If a Task could call or inherit another task, we would have the ability to build such "high level" Task that could be used in multiples pipelines.

Let me put another example using the s2i task from the tekton catalog

apiVersion: tekton.dev/v1alpha1
kind: Task
metadata:
  name: my-corp-s2i
spec:
      - name: IMAGE_NAME
        type: string
        description: The name of the s2i builder image.
        default: nodejs
      - name: IMAGE_TAG
        type: string
        description: The s2i builder image tag to use
        default: fc31
     - name: PATH_CONTEXT
       description: The location of the path to run s2i from.
       default: .
  steps: 
      - name: "my-corp-s2i"
        taskRef:
          name: s2i <-- from tkn catalog
        inputs:
           params:
             - name: BUILDER_IMAGE
               value: "quay.io/startx/$(inputs.params.IMAGE_NAME):$(inputs.params.IMAGE_TAG)"
             - name: PATH_CONTEXT
               value: "$(inputs.params.PATH_CONTEXT)"
             - name: TLSVERIFY
               value: "false"
             - name: LOGLEVEL
               value: "2"

This example allow us to have a my-corp-s2i task based on the s2i Task comming from the tkn catalog, but restricted to images coming from one particular registry. We also force LOGLEVEL and TLSVERIFY to be always the same when using this "high level" Task.

Our pipelines will be more simple and forced to be restricted to the configuration defined in our my-corp-s2i task

---
apiVersion: tekton.dev/v1alpha1
kind: Pipeline
metadata:
  name: my-nodejs-pipeline
spec:
  params:
    - name: LANGUAGE
      type: string
      description: default builder language
      default: nodejs
  tasks:
    - name: build-nodejs
      taskRef:
        name: my-corp-s2i
      params:
        - name: IMAGE_NAME
          value: "$(params.LANGUAGE)"
---
apiVersion: tekton.dev/v1alpha1
kind: Pipeline
metadata:
  name: my-php-pipeline
spec:
  params:
    - name: LANGUAGE
      type: string
      description: default builder language
      default: php
  tasks:
    - name: build-php
      taskRef:
        name: my-corp-s2i
      params:
        - name: IMAGE_NAME
          value: "$(params.LANGUAGE)"

Without the call or inherit task ability, we would have to duplicate the BUILDER_IMAGE, TLSVERIFY and LOGLEVEL parameters in our Pipeline and couldn't "force" pipeline user to use our quay.io namespace and other configurationset.

I don't know if Is it interesting feature requested by others or if I'am the only one to think it could be usefull to architect and build task in a more flexible way ?

[vdemeester]

/kind question
/kind feature

This is a bit related to #1260 I feel, except that #1260 propose to add the notion of Step where this proposal would make Task being able to refer Task. I tend to prefer #1260 as it feels less complicated (to grasp, and to implement I guess).

[startxfr]

Hi @vdemeester,

Thanks for you answer. You're right, the solution discussed in #1260 deal with the same challenge : introducing more flexibility and reusability around the Task component.

Having a Step resource could help in sharing and reusing common task element. But step rely on PipelineResources, volumes, workspace, input and output defined at the task level.
If we have a Step resource as suggested in #1260, we can share the image, command and arg easily, but it will need more work to handle input, output volumes and PipelinesResources as they are actually on the task level.
I fear that, because the steps are unitary, this leads users to create a large number of step resources, making the structure of our Pipelines less simple to observe.

Actualy, a step is only composed of container execution.

- name: "my-classical-step"
  image: gcr.io/example-builders/build-example
  command: ["echo"]
  args: ["step 1"]
  inputs: ...

My suggestion was to introduce the ability to have a second kind of step who is a Task reference (same way we use it to call task in Pipeline resources).

- name: "my-herited-step"
  taskRef:
    name: abstract-task
  inputs: ...

I don't know what are the drawback for this solution, and if it take hours or month to implement it, but this change could allow us to embed a task in the step list of another task, leading to have kind of a task heritage.

If we perform this implementation at the Task level instead of the Step level, we could have more flexibility to build Task with single or multiples step. We could also have the hability to share input, output and all other components referenced at the task level.

I'm new to Tekton and still discover how it work. I have undoubtedly missed many questions which can be raised by this suggestion.

[dlorenc]

I think this is handled by the custom task work! tektoncd/community#143

[tekton-robot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
If this issue is safe to close now please do so with /close.

/lifecycle stale

Send feedback to tektoncd/plumbing.

[tekton-robot]

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.
If this issue is safe to close now please do so with /close.

/lifecycle rotten

Send feedback to tektoncd/plumbing.

[vdemeester]

/remove-lifecycle stale

[bobcatfish]

TEP which may be relevant to your interests @startxfr (if you are still interested in this!) tho it doesn't address this issue directly: tektoncd/community#316

[tekton-robot]

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen with a justification.
Mark the issue as fresh with /remove-lifecycle rotten with a justification.
If this issue should be exempted, mark the issue as frozen with /lifecycle frozen with a justification.

/close

Send feedback to tektoncd/plumbing.

[tekton-robot]

@tekton-robot: Closing this issue.

In response to this:

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen with a justification.
Mark the issue as fresh with /remove-lifecycle rotten with a justification.
If this issue should be exempted, mark the issue as frozen with /lifecycle frozen with a justification.

/close

Send feedback to tektoncd/plumbing.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[roelarents]

There is an experimental: pipelines-in-pipelines. I think you could achieve what you want by putting the task to extent in a pipeline and using that as the extended task.