Skip to content

Latest commit

 

History

History
907 lines (745 loc) · 31.1 KB

0059-skipping-strategies.md

File metadata and controls

907 lines (745 loc) · 31.1 KB
Raw
status title creation-date last-updated authors see-also
implemented
Skipping Strategies
2021-03-24
2021-08-23
@jerop
TEP-0007
TEP-0048
TEP-0056

TEP-0059: Skipping Strategies

Summary

This TEP addresses skipping strategies to give users the flexibility to skip a single guarded Task only and unblock execution of its dependent Tasks.

Today, WhenExpressions are specified within Tasks but they guard the Task and its dependent Tasks. To provide flexible skipping strategies, we propose changing the scope of WhenExpressions from guarding a Task and its dependent Tasks to guarding the Task only. If a user wants to guard a Task and its dependent Tasks, they can:

  1. cascade the WhenExpressions to the dependent Tasks
  2. compose the Task and its dependent Tasks as a sub-Pipeline that's guarded and executed together using Pipelines in Pipelines

Motivation

When WhenExpressions evaluate to False, the guarded Task is skipped and its dependent Tasks are skipped as well while the rest of the Pipeline executes. Users need the flexibility to skip that guarded Task only and unblock the execution of the dependent Tasks.

Pipelines are directed acyclic graphs where:

  • Nodes are Tasks
  • Edges are defined using ordering (runAfter) and resource (e.g. Results) dependencies
  • Branches are made up of Nodes that are connected by Edges

WhenExpressions are specified within Tasks, but they guard the Task and its dependent Tasks. Thus, the WhenExpressions (and Conditions) can be visualized to be along the edges between two Tasks in a Pipeline graph.

Take this example:

tasks:
  - name: previous-task
    taskRef:
      name: previous-task
  - name: current-task
    runAfter: [ previous-task ]
    when:
      - input: "foo"
        operator: in
        values: [ "bar" ]
    taskRef:
      name: current-task
  - name: next-task
    runAfter: [ current-task ]
    taskRef:
      name: next-task

The visualization/workflow of the Pipeline graph would be:

        previous-task          # executed
             |
          (guard)              # false         
             |
             v
        current-task           # skipped
             |
             v
         next-task             # skipped

This TEP aims to support WhenExpressions that are specified within Tasks to guard the Task only (not its dependent Tasks). Thus, this visualization/workflow of the Pipeline graph would be possible:

        previous-task          # executed
             |
             v
(guard) current-task           # false and skipped
             |
             v
         next-task             # executed

Goals

The main goal of this TEP is to provide the flexibility to skip a guarded Task when its WhenExpressions evaluate to False while unblocking the execution of its dependent Tasks.

Non-Goals

Providing the flexibility to skip a Task and unblock execution of its dependent Tasks when it was skipped for other reasons besides its WhenExpressions evaluating to False is out of scope for this TEP.

Today, the other reasons that a Task is skipped include:

By scoping this skipping strategy to WhenExpressions only, we can provide the flexibility safely with a minimal change. Moreover, it allows us to limit the number possible Pipeline graph execution paths and make the workflow predictable. If needed, we can explore adding this skipping strategy for the other reasons in the future.

Use Cases

A user needs to design a Pipeline with a manual approval Task that is executed when merging a pull request only. The execution of the manual approval Task is guarded using WhenExpressions. To reuse the same Pipeline when merging and not merging, the user needs the dependent Tasks to execute when the guarded manual approval Task is skipped.

          lint                     unit-tests
           |                           |
           v                           v
   report-linter-output        integration-tests
                                       |
                                       v
                                 manual-approval
                                       |
                                       v
                                  build-image
                                       |
                                       v
                                  deploy-image

If the WhenExpressions in manual-approval evaluate to True, then manual-approval is executed and:

  • if manual-approval succeeds, then build-image and deploy-image are executed
  • if manual-approval fails, then build-image and deploy-image are not executed because the Pipeline fails

Today, if the WhenExpressions in manual-approval evaluate to False, then manual-approval, build-image and deploy-image are all skipped. In this TEP, we'll provide the flexibility to execute build-image and deploy-image when manual-approval is skipped. This would allow the user to reuse the Pipeline in both scenarios (merging and not merging).

Building on the above use case, the user adds slack-msg which sends a notification to slack that it was manually approved with the name of the approver that is passed as a Result from manual-approval to slack-msg.

          lint                     unit-tests
           |                           |
           v                           v
   report-linter-output        integration-tests
                                       |
                                       v
                                 manual-approval
                                 |            |
                                 v        (approver)
                            build-image       |
                                |             v
                                v          slack-msg
                            deploy-image

If the guarded manual-approval is skipped, then build-image and deploy-image needs to be executed similarly to above. However, slack-msg should be skipped because of the missing Result reference to the approver name.

Requirements

Users should be able to specify that a guarded Task only should be skipped when its WhenExpressions evaluate to False to unblock the execution of its dependent Tasks

  • ordering-dependent Tasks, based on runAfter, should execute as expected
  • resource-dependent Tasks, based on resources such as Results, should be attempted but might be skipped if they can't resolve missing resources

Proposal

Today, WhenExpressions are specified within Tasks but they guard the Task and its dependent Tasks. To provide flexible skipping strategies, we propose changing the scope of WhenExpressions from guarding a Task and its dependent Tasks to guarding the Task only. If a user wants to guard a Task and its dependent Tasks, they can:

  1. cascade the WhenExpressions to the dependent Tasks
  2. compose the Task and its dependent Tasks as a sub-Pipeline that's guarded and executed together using Pipelines in Pipelines

Guarding a Task only

To enable guarding a Task only, we'll change the scope of WhenExpressions to guard the Task only. The migration strategy for this change is discussed in Upgrade & Migration Strategy below.

A Pipeline to solve for the use case described above would be designed as such:

tasks:
...
- name: manual-approval
  runAfter:
    - integration-tests
  when:
    - input: $(params.git-action)
      operator: in
      values:
        - merge
  taskRef:
    name: manual-approval

- name: slack-msg
  params:
    - name: approver
      value: $(tasks.manual-approval.results.approver)
  taskRef:
    name: slack-msg

- name: build-image
  runAfter:
    - manual-approval
  taskRef:
    name: build-image

- name: deploy-image
  runAfter:
    - build-image
  taskRef:
    name: deploy-image

Guarding a Task and its dependent Tasks

If user wants to guard a Task and its dependent Tasks, they have two options:

  • cascade the WhenExpressions to the specific dependent Tasks they want to guard as well
  • compose the Task and its dependent Tasks as a unit to be guarded and executed together using Pipelines in Pipelines

Cascade WhenExpressions to the dependent Tasks

Cascading WhenExpressions to specific dependent Tasks gives users more control to design their workflow. Today, we skip all dependent Tasks. With this TEP, they can pick and choose which dependent Tasks to guard as well, empowering them to solve for more complex CI/CD use cases.

A user who wants to guard manual-approval and its dependent Tasks can design the Pipeline as such:

tasks:
...
- name: manual-approval
  runAfter:
    - integration-tests
  when:
    - input: $(params.git-action)
      operator: in
      values:
        - merge
  taskRef:
    name: manual-approval

- name: slack-msg
  params:
    - name: approver
      value: $(tasks.manual-approval.results.approver)
  taskRef:
    name: slack-msg

- name: build-image
  when:
    - input: $(params.git-action)
      operator: in
      values:
        - merge
  runAfter:
    - manual-approval
  taskRef:
    name: build-image

- name: deploy-image
  when:
    - input: $(params.git-action)
      operator: in
      values:
        - merge
  runAfter:
    - build-image
  taskRef:
    name: deploy-image

Cascading is more verbose, but it provides clarity and flexibility in guarded execution by being explicit.

Composing using Pipelines in Pipelines

Composing a set of Tasks as a unit of execution using Pipelines in Pipelines will allow users to guard a Task and its dependent Tasks (as a sub-Pipeline) using WhenExpressions.

If a user wants to guard manual-approval and its dependent Tasks, they can combine them in a sub-Pipeline which we'll refer to as approve-slack-build-deploy, as such:

tasks:
  - name: manual-approval
    runAfter:
      - integration-tests
    taskRef:
      name: manual-approval

  - name: slack-msg
    params:
      - name: approver
        value: $(tasks.manual-approval.results.approver)
    taskRef:
      name: slack-msg

  - name: build-image
    runAfter:
      - manual-approval
    taskRef:
      name: build-image

  - name: deploy-image
    runAfter:
      - build-image
    taskRef:
      name: deploy-image

Pipelines in Pipelines is currently available through Custom Tasks, so it would be used in the main-Pipeline as such:

tasks:
...
- name: approve-slack-build-deploy
  runAfter:
    - integration-tests
  when:
    - input: $(params.git-action)
      operator: in
      values:
        - merge
  taskRef:
    apiVersion: tekton.dev/v1beta1
    kind: Pipeline
    name: approve-slack-build-deploy

After we promote Pipelines in Pipelines from experimental to a top-level feature, then this is a possible syntax:

tasks:
...
- name: approve-slack-build-deploy
  runAfter:
    - integration-tests
  when:
    - input: $(params.git-action)
      operator: in
      values:
        - merge
  pipelineRef:
    name: approve-slack-build-deploy

Pipelines in Pipelines would provide the flexible skipping strategies needed to solve for the use cases without verbosity.

Test Plan

Unit and integration tests for guarded execution of Tasks in a Pipeline with different skipping strategies in different combinations.

Design Evaluation

Reusability

By unblocking the execution of dependent Tasks when a guarded Task is skipped, we enable execution to continue when the guarded Task is either successful or skipped, making the Pipeline reusable for more scenarios or use cases.

By cascading WhenExpressions or composing a set of Tasks as a Pipeline in a Pipeline, we reuse existing features to provide flexible skipping strategies.

Simplicity

By scoping the skipping strategy to WhenExpressions only, we provide the flexibility safely with a minimal change. We also limit the interleaving of Pipeline graph execution paths and maintain the simplicity of the workflows.

WhenExpressions and Pipelines in Pipelines are the bare minimum features so solve for the CI/CD use cases that need skipping strategies.

Flexibility

This TEP will give users the flexibility to either guard a Task only or guard a Task and its dependent Tasks. Moreover, it gives users the flexibility to guard some dependent Tasks while executing other dependent Tasks.

Upgrade & Migration Strategy

Changing the scope of WhenExpressions to guard the Task only is backwards-incompatible, so to make the transition smooth:

  • we'll provide a feature flag, scope-when-expressions-to-task, which:
    • will default to scope-when-expressions-to-task : "false" to guard a Task and its dependent Tasks
    • can be set to scope-when-expressions-to-task : "true" to guard a Task only
  • after 9 months, per the Tekton API compatibility policy, we'll flip the feature flag and default to scope-when-expressions-to-task : true [February 2022]
  • in the next release, we'll remove the feature flag and WhenExpressions will be scoped to guard a Task only going forward [March 2022]
  • when we do v1 release (projected for early 2022), we will have when expressions guarding a Task only both in beta and v1

We will over-communicate during the migration in Slack, email and working group meetings.

Pipelines in Pipelines is available through Custom Tasks - we are iterating on it as we work towards promoting it to a top level feature. This work will be discussed separately in TEP-0056: Pipelines in Pipelines.

Alternatives

Pipelines in Pipelines with Finally Tasks

What if we don't change the scope of WhenExpressions and want to use Pipelines in Pipelines only?

In this case, we'd have to lean on Finally Tasks to execute the dependent Task in the sub-Pipelines -- which leads to convoluted Pipeline designs, such as:

tasks:
...
- name: approve-build-deploy-notify
  runAfter:
    - integration-tests
  pipelineRef:
    - name: approve-build-deploy-notify

---
# approve-build-deploy-notify (sub-pipeline)
tasks:
  - name: manual-approval
    when:
      - input: $(params.git-action)
        operator: in
        values:
          - merge
    runAfter:
      - integration-tests
    taskRef:
      - name: manual-approval

  - name: slack-msg
    params:
      - name: approver
        value: $(tasks.manual-approval.results.approver)
    taskRef:
      - name: slack-msg

finally:
  - name: build-and-deploy
    when:
      - input: $(tasks.manual-approval.status)
        operator: notin
        values:
          - Failed
    pipelineRef:
      - name: build-and-deploy

---
# build-and-deploy (sub-pipeline)
- name: build-image
  runAfter:
    - manual-approval
  taskRef:
    name: build-image

- name: deploy-image
  runAfter:
    - build-image
  taskRef:
    name: deploy-image

Scoped WhenExpressions

Today, we support specifying a list of WhenExpressions through the when field as such:

when:
  - input: 'foo'
    operator: in
    values: [ 'bar' ]

To provide the flexibility to skip a guarded Task when its WhenExpressions evaluate to False while unblocking the execution of its dependent Tasks, we could change the when field from a list to a dictionary and add scope and expressions fields under the when field.

  • The scope field would be used to specify whether the WhenExpressions guard the Task only or the whole Branch ( the Task and its dependencies). To unblock execution of subsequent Tasks, users would set scope to Task. Setting scope to Branch matches the current behavior.
  • The expressions field would be used to specify the list of WhenExpressions, each of which has input, operator and values fields, as it is currently.
when:
  scope: Task
  expressions:
    - input: 'foo'
      operator: in
      values: [ 'bar' ]
---
when:
  scope: Branch
  expressions:
    - input: 'foo'
      operator: notin
      values: [ 'bar' ]

To support both syntaxes under when, we'll detect whether it's a list or dictionary in UnmarshalJSON function that implements the json.Unmarshaller interface, using the first character. This is how similar scenarios have been handled elsewhere, including:

  • Tekton does the same thing in Parameters to detect whether the type of the value is a String or Array (code)
  • Kubernetes does the same thing in IntOrString to detect whether the type is Int or String (code)

A Pipeline to solve for the use case described above would be designed as such:

tasks:
...
- name: manual-approval
  runAfter:
    - integration-tests
  when:
    scope: Task
    expressions:
      - input: $(params.git-action)
        operator: in
        values:
          - merge
  taskRef:
    name: manual-approval

- name: slack-msg
  params:
    - name: approver
      value: $(tasks.manual-approval.results.approver)
  taskRef:
    name: slack-msg

- name: build-image
  runAfter:
    - manual-approval
  taskRef:
    name: build-image

- name: deploy-image
  runAfter:
    - build-image
  taskRef:
    name: deploy-image

If the WhenExpressions in manual-approval evaluate to False, then manual-approval would be skipped and:

  • build-image and deploy-image would be executed
  • slack-msg would be skipped due to missing resource from manual-approval

Skipping Policies

Add a field - whenSkipped - that can be set to runBranch to unblock or skipBranch to block the execution of Tasks that are dependent on the guarded Task.

type SkippingPolicy string

const (
    RunBranch  SkippingPolicy = "runBranch"
    SkipBranch SkippingPolicy = "skipBranch"
)
tasks:
  - name: task
    when:
      - input: foo
        operator: in
        values: [ bar ]
    whenSkipped: runBranch / skipBranch
    taskRef:
      - name: task

Another option would be a field - whenScope - than can be set to Task to unblock or Branch to block the execution of Tasks that are dependent on the guarded Task.

type WhenScope string

const (
    Task   WhenScope = "task"
    Branch WhenScope = "branch"
)
tasks:
  - name: task
    when:
      - input: foo
        operator: in
        values: [ bar ]
    whenScope: task / branch
    taskRef:
      - name: task

However, it won't be clear that the skipping policies are related to WhenExpressions specifically and can be confusing to reason about when they are specified separately.

Execution Policies

Add a field - executionPolicies - that takes a list of execution policies for the skipping and failure strategies for given Task. This would align well with TEP-0050: Ignore Task Failures and is easily extensible.

type ExecutionPolicy string

const (
    IgnoreFailure     ExecutionPolicy = "ignoreFailure"
    ContinueAfterSkip ExecutionPolicy = "continueAfterSkip"
    ...
)
tasks:
  - name: task
    when:
      - input: foo
        operator: in
        values: [ bar ]
    executionPolicies:
      - ignoreFailure
      - continueAfterSkip
    taskRef:
      - name: task

However, it won't be clear that the skipping policies are related to WhenExpressions specifically and can be confusing to reason about when they are specified separately.

Boolean Flag

Add a field - continueAfterSkip - that can be set to true to unblock or false to block the execution of Tasks that are dependent on the guarded Task.

tasks:
  - name: task
    when:
      - input: foo
        operator: in
        values: [ bar ]
    continueAfterSkip: true / false
    taskRef:
      - name: task

However, it won't be clear that the boolean flag is related to WhenExpressions specifically and can be confusing to reason about when they are specified separately. In addition, booleans limit future extensions.

Special runAfter

Provide a special kind of runAfter - runAfterWhenSkipped - that users can use instead of runAfter to allow for the ordering-dependent Task to execute even when the Task has been skipped. Related ideas discussed in tektoncd/pipeline#2653 as runAfterUnconditionally and tektoncd/pipeline#1684 as runOn.

tasks:
  - name: task1
    when:
      - input: foo
        operator: in
        values: [ bar ]
    taskRef:
      - name: task1
  - name: task2
    runAfterWhenSkipped:
      - task1
    taskRef:
      - name: task2

However, it won't be clear that the skipping policies are related to WhenExpressions specifically and can be confusing to reason about when they are specified separately.

References