From dbd8c70d5203c5b6c6985c25190851c42e7997a1 Mon Sep 17 00:00:00 2001 From: Christie Wilson Date: Fri, 22 Jan 2021 18:23:09 -0500 Subject: [PATCH 1/5] =?UTF-8?q?Add=20TEP-0044:=20Composing=20Tasks=20with?= =?UTF-8?q?=20Tasks=20=E2=9E=B0?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit This TEP describes some difficulty folks have run into, especially when trying to use Tekton Pipelines without PipelineResources: if you want to use the functionality of multiple Tasks, you need to put them together in a Pipeline and share data between them with volumes. It could be a useful abstraction, and more efficient, to make it possible to combine Tasks together without scheudling them on different pods. --- teps/0044-composing-tasks-with-tasks.md | 137 ++++++++++++++++++++++++ teps/README.md | 1 + 2 files changed, 138 insertions(+) create mode 100644 teps/0044-composing-tasks-with-tasks.md diff --git a/teps/0044-composing-tasks-with-tasks.md b/teps/0044-composing-tasks-with-tasks.md new file mode 100644 index 000000000..46e381557 --- /dev/null +++ b/teps/0044-composing-tasks-with-tasks.md @@ -0,0 +1,137 @@ +--- +status: proposed +title: Composing Tasks with Tasks +creation-date: '2021-01-22' +last-updated: '2021-01-22' +authors: +- '@bobcatfish' +--- + +# TEP-0044: Composing Tasks with Tasks + + +- [Summary](#summary) +- [Motivation](#motivation) + - [PipelineResources](#pipelineresources) + - [Goals](#goals) + - [Non-Goals](#non-goals) + - [Use Cases (optional)](#use-cases-optional) +- [Requirements](#requirements) +- [References (optional)](#references-optional) + +## Summary + +This TEP describes a gap in composability that isn't currently fully solved by Tasks or Pipelines +(tho PipelineResources do solve some of it!). + +Ideally by addressing the issues described here we'd make it possible to create more efficient +Pipelines and make it easier to use a Task easily via a TaskRun, and we'd address some of +[the problems PipelineResources are trying to solve](https://docs.google.com/document/d/1KpVyWi-etX00J3hIz_9HlbaNNEyuzP6S986Wjhl3ZnA/edit#) +making it more clear whether or not they are an abstraction we want to keep. + +[#1673: Pipeline Resources Redesign](https://github.com/tektoncd/pipeline/issues/1673) + +## Motivation + +Currently the only way to combine Tasks together is in a Pipeline. If you combine +Tasks in a Pipeline and they need to share data (beyond a simple +[result](https://github.com/tektoncd/pipeline/blob/master/docs/pipelines.md#using-results)), +you'll need to provision a PVC or do some other similar, cloud specific thing, +to [make a volume available](https://github.com/tektoncd/pipeline/blob/master/docs/workspaces.md#specifying-volumesources-in-workspaces) +that can be shared between them. + +Some problems with this: +* PVCs add additional overhead, both in speed and in management (somewhat + mitigated by + [letting tekton create and delete volumes for you](https://github.com/tektoncd/pipeline/blob/master/docs/workspaces.md#volumeclaimtemplate) +* Sometimes you just want to run a Task with a teeny bit more functionality, + e.g. you want to use a task to build images, but you want to clone source + with git first, and it seems like overkill to have to construct a pipeline + +### PipelineResources + +This kind of compositional functionality was being (somewhat! read on!) provided +by [PipelineResources](https://github.com/tektoncd/pipeline/blob/master/docs/resources.md). +[We did not bring these types to beta](https://github.com/tektoncd/pipeline/blob/master/docs/resources.md#why-arent-pipelineresources-in-beta). +Issues that don't let PipelineResources (as is) solve these problems are: +* PipelineResource "outputs" do not run when the steps of the Task fail (or if + a previous PipelienResource "output" fails) (see [unit test use case](#use-cases-optional)) +* PipelineResources do not use Tasks (which is something we could change), meaning + you cannot use them to compose Tasks together, you need to build PipelineResources + * Adding new PipelineResources currently needs to be done in the Tekton Pipelines controller, + tho there have been several attempts to propose ways to make it extensible: + * [PipelineResources 2 Uber Design Doc](https://docs.google.com/document/d/1euQ_gDTe_dQcVeX4oypODGIQCAkUaMYQH5h7SaeFs44/edit) + * [Tekton Pipeline Resource Extensibility](https://docs.google.com/document/d/1rcMG1cIFhhixMSmrT734MBxvH3Ghre9-4S2wbzodPiU/edit) + * All of this is even though PipelineResources are _very_ similar to Tasks in that they are + collections of steps (which [get injected before and after a Task's steps](https://github.com/tektoncd/pipeline/issues/1838) +* Tasks have to declare in advance what PipelineResources they need; you can't decide to use + a PipelineResource with a Task after it has been written (e.g. in a TaskRun or a Pipeline) + and you can't mix types of PipelineResource (e.g. if a Task declares it needs a + [git PipelineResources](https://github.com/tektoncd/pipeline/blob/master/docs/resources.md#git-resource) + it can't use a different kind of PipelineResource to provide those files (tho you can avoid + doing the clone multiple times using [from](https://github.com/tektoncd/pipeline/blob/master/docs/pipelines.md#using-the-from-parameter)) + +### Goals + +- Make it possible to combine Tasks together so that you can run multiple + Tasks as "one unit" (see [Requirements](#requirements): + - At runtime in a TaskRun + - At Pipeline authoring time +- Add some of [the features we don't have without PipelineResources](https://docs.google.com/document/d/1KpVyWi-etX00J3hIz_9HlbaNNEyuzP6S986Wjhl3ZnA/edit#) + to Tekton Pipelines (without requiring use of PipelineResources) + - If possible we can provide a migration path from PipelineResources to the solution proposed here, where applicable + +### Non-Goals + +- Composing Tasks within Tasks at [Task authoring time](https://github.com/tektoncd/community/blob/master/design-principles.md#reusability). + We could decide to include this if we have some use cases that need it; for now avoiding + this allows us to avoid many layers of nesting (i.e. Task1 uses Task2 uses Task3, etc.) + or even worse, recursion (Task 1 uses Task 2 uses Task 1...) +- Completely replacing PipelineResources: we could decide to solve this by improving PipelineResources, + or we could add a new feature via this TEP and still continue to support PipelineResources + (since they provide [more functionality than just composition](https://docs.google.com/document/d/1KpVyWi-etX00J3hIz_9HlbaNNEyuzP6S986Wjhl3ZnA/edit#)) + +### Use Cases (optional) + +- A user wants to use catalog Tasks to checkout code, run unit tests and upload results, + and does not want to incur the additional overhead (and performance impact) of creating + volume based workspaces to share data between them in a Pipeline. e.g. specifically + cloning with [git-clone](https://github.com/tektoncd/catalog/tree/master/task/git-clone/0.2), + running tests with [golang-test](https://github.com/tektoncd/catalog/tree/master/task/golang-test/0.1) + and uploading results with [gcs-upload](https://github.com/tektoncd/catalog/tree/master/task/gcs-upload/0.1). +- A user wants to use a Task from the catalog with a git repo and doesn't want the extra + overhead of using a Pipeline, they just want to make a TaskRun, + e.g. [@mattmoor's feedback on PipelineResources and the Pipeline beta](https://twitter.com/mattomata/status/1251378751515922432)) + where he wants to checkout some code and [use the kaniko task](https://github.com/tektoncd/catalog/tree/master/task/kaniko/0.1) + without having to fiddle with volumes +- An organziation does not want to use PVCs at all; for example perhaps they have decided + on uploading to and downloading from buckets in the cloud (e.g. GCS) +- An organization is willing to use PVCs to some extent but needs to put limits on their use + +## Requirements + +- Tasks can be composed together run as "one unit": + - Must be able to share data without requiring + an external volume (i.e. probably the containers that make them up are run within + the same pod) +- It should be possible to have Tasks that run even if others fail + - This is to support use cases such as uploading test results, even if the test + Task failed + +## References (optional) + +* [Tekton PipelineResources Beta Notes](https://docs.google.com/document/d/1Et10YdBXBe3o2x6lCfTindFnuBKOxuUGESLb__t11xk/edit) +* [Why aren't PipelineResources in beta?](https://github.com/tektoncd/pipeline/blob/master/docs/resources.md#why-arent-pipelineresources-in-beta) +* [@mattmoor's feedback on PipelineResources and the Pipeline beta](https://twitter.com/mattomata/status/1251378751515922432)) +* [PipelineResources 2 Uber Design Doc](https://docs.google.com/document/d/1euQ_gDTe_dQcVeX4oypODGIQCAkUaMYQH5h7SaeFs44/edit) +* [Investigate if we can run whole PipelineRun in a Pod](https://github.com/tektoncd/pipeline/issues/3638) - [TEP-0046](https://github.com/tektoncd/community/pull/318) +* Task specialization: + * [Specializing Tasks - Vision & Goals](https://docs.google.com/document/d/1G2QbpiMUHSs4LOqcNaIRswcdvoy8n7XuhTV8tXdcE7A/edit) + * [Task specialization - most appealing options?](https://docs.google.com/presentation/d/12QPKFTHBZKMFbgpOoX6o1--HyGqjjNJ7own6KqM-s68/edit#slide=id.p) +* Issues: + * [Pipeline Resources Redesign](https://github.com/tektoncd/pipeline/issues/1673) + * [#1838 Extract Pre/Post Steps from PipelineResources 2 design into Tasks](https://github.com/tektoncd/pipeline/issues/1838) + * [Abstract task and nested tasks](https://github.com/tektoncd/pipeline/issues/1796) + * Oldies but goodies: + * [Link inputs and outputs without using volumes](https://github.com/tektoncd/pipeline/issues/617) + * [Design PipelineResource extensibility](https://github.com/tektoncd/pipeline/issues/238) diff --git a/teps/README.md b/teps/README.md index e85a451f5..8409e593c 100644 --- a/teps/README.md +++ b/teps/README.md @@ -169,6 +169,7 @@ This is the complete list of Tekton teps: |[TEP-0037](0037-remove-gcs-fetcher.md) | Remove `gcs-fetcher` image | implementing | 2021-01-27 | |[TEP-0039](0039-add-variable-retries-and-retrycount.md) | Add Variable `retries` and `retry-count` | proposed | 2021-01-31 | |[TEP-0040](0040-ignore-step-errors.md) | Ignore Step Errors | proposed | 2021-02-04 | +|[TEP-0044](0044-composing-tasks-with-tasks.md) | Composing Tasks with Tasks | proposed | 2021-01-22 | |[TEP-0045](0045-whenexpressions-in-finally-tasks.md) | WhenExpressions in Finally Tasks | implementable | 2021-01-28 | |[TEP-0050](0050-ignore-task-failures.md) | Ignore Task Failures | proposed | 2021-02-19 | |[TEP-0051](0051-ppc64le-architecture-support.md) | ppc64le Support | proposed | 2021-01-28 | From 15b51f957f2ec34c97675fcd660601ff4ca03457 Mon Sep 17 00:00:00 2001 From: Christie Wilson Date: Mon, 22 Feb 2021 10:47:06 -0500 Subject: [PATCH 2/5] =?UTF-8?q?Move=20the=20TaskRun=20use=20case=20to=20no?= =?UTF-8?q?n-goals=20=E2=9A=BD?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit In the review @vdemeester pointed out that it feels like we are trying to deal with 2 different things - for me the main reason for that is b/c I have a feeling we WILL solve @mattmoor 's use case - since we'll probably have to change TaskRuns to make it happen - but I agree with @vdemeester that that's not the main thing we're targetting here. Now TEP-0044 is more aligned with TEP-0046 than ever! --- teps/0044-composing-tasks-with-tasks.md | 44 ++++++++++++------------- 1 file changed, 21 insertions(+), 23 deletions(-) diff --git a/teps/0044-composing-tasks-with-tasks.md b/teps/0044-composing-tasks-with-tasks.md index 46e381557..f3aaf714c 100644 --- a/teps/0044-composing-tasks-with-tasks.md +++ b/teps/0044-composing-tasks-with-tasks.md @@ -21,8 +21,7 @@ authors: ## Summary -This TEP describes a gap in composability that isn't currently fully solved by Tasks or Pipelines -(tho PipelineResources do solve some of it!). +This TEP describes a gap in composability that isn't currently fully solved by Tasks or Pipelines. Ideally by addressing the issues described here we'd make it possible to create more efficient Pipelines and make it easier to use a Task easily via a TaskRun, and we'd address some of @@ -36,17 +35,13 @@ making it more clear whether or not they are an abstraction we want to keep. Currently the only way to combine Tasks together is in a Pipeline. If you combine Tasks in a Pipeline and they need to share data (beyond a simple [result](https://github.com/tektoncd/pipeline/blob/master/docs/pipelines.md#using-results)), -you'll need to provision a PVC or do some other similar, cloud specific thing, +you'll need to provision a PVC or do some other similar, cloud specific storage, to [make a volume available](https://github.com/tektoncd/pipeline/blob/master/docs/workspaces.md#specifying-volumesources-in-workspaces) that can be shared between them. -Some problems with this: -* PVCs add additional overhead, both in speed and in management (somewhat - mitigated by - [letting tekton create and delete volumes for you](https://github.com/tektoncd/pipeline/blob/master/docs/workspaces.md#volumeclaimtemplate) -* Sometimes you just want to run a Task with a teeny bit more functionality, - e.g. you want to use a task to build images, but you want to clone source - with git first, and it seems like overkill to have to construct a pipeline +PVCs add additional overhead, both in speed and in management (somewhat +mitigated by [letting tekton create and delete volumes for you](https://github.com/tektoncd/pipeline/blob/master/docs/workspaces.md#volumeclaimtemplate) +but the PVCs still slow down the overall execution. ### PipelineResources @@ -74,11 +69,10 @@ Issues that don't let PipelineResources (as is) solve these problems are: ### Goals - Make it possible to combine Tasks together so that you can run multiple - Tasks as "one unit" (see [Requirements](#requirements): - - At runtime in a TaskRun - - At Pipeline authoring time + Tasks as "one unit" (see [Requirements](#requirements) at authoring time + in a way that can be reused (e.g. in a Pipeline) - Add some of [the features we don't have without PipelineResources](https://docs.google.com/document/d/1KpVyWi-etX00J3hIz_9HlbaNNEyuzP6S986Wjhl3ZnA/edit#) - to Tekton Pipelines (without requiring use of PipelineResources) + to Tekton Pipelines (without requiring use of PipelineResources), specifically **Task adapters/specialization** - If possible we can provide a migration path from PipelineResources to the solution proposed here, where applicable ### Non-Goals @@ -90,6 +84,13 @@ Issues that don't let PipelineResources (as is) solve these problems are: - Completely replacing PipelineResources: we could decide to solve this by improving PipelineResources, or we could add a new feature via this TEP and still continue to support PipelineResources (since they provide [more functionality than just composition](https://docs.google.com/document/d/1KpVyWi-etX00J3hIz_9HlbaNNEyuzP6S986Wjhl3ZnA/edit#)) +- This was previously a use case we were targetting but we've decided to descope the TEP slightly, + though if we end up solving this problem as well, that's a bonus: + * A user wants to use a Task from the catalog with a git repo and doesn't want the extra + overhead of using a Pipeline, they just want to make a TaskRun, + e.g. [@mattmoor's feedback on PipelineResources and the Pipeline beta](https://twitter.com/mattomata/status/1251378751515922432)) + where he wants to checkout some code and [use the kaniko task](https://github.com/tektoncd/catalog/tree/master/task/kaniko/0.1) + without having to fiddle with volumes ### Use Cases (optional) @@ -99,11 +100,6 @@ Issues that don't let PipelineResources (as is) solve these problems are: cloning with [git-clone](https://github.com/tektoncd/catalog/tree/master/task/git-clone/0.2), running tests with [golang-test](https://github.com/tektoncd/catalog/tree/master/task/golang-test/0.1) and uploading results with [gcs-upload](https://github.com/tektoncd/catalog/tree/master/task/gcs-upload/0.1). -- A user wants to use a Task from the catalog with a git repo and doesn't want the extra - overhead of using a Pipeline, they just want to make a TaskRun, - e.g. [@mattmoor's feedback on PipelineResources and the Pipeline beta](https://twitter.com/mattomata/status/1251378751515922432)) - where he wants to checkout some code and [use the kaniko task](https://github.com/tektoncd/catalog/tree/master/task/kaniko/0.1) - without having to fiddle with volumes - An organziation does not want to use PVCs at all; for example perhaps they have decided on uploading to and downloading from buckets in the cloud (e.g. GCS) - An organization is willing to use PVCs to some extent but needs to put limits on their use @@ -111,12 +107,14 @@ Issues that don't let PipelineResources (as is) solve these problems are: ## Requirements - Tasks can be composed together run as "one unit": - - Must be able to share data without requiring - an external volume (i.e. probably the containers that make them up are run within - the same pod) -- It should be possible to have Tasks that run even if others fail + - Must be able to share data without requiring a volume external to the pod +- It should be possible to have Tasks that run even if others fail; i.e. the Task + can be run on the same pod as another Task that fails - This is to support use cases such as uploading test results, even if the test Task failed + - This requirement is being included because we could choose a solution that doesn't + address the above use case; for example in PipelineResources, you can have a + storage "output" but if the steps fail, the "output" pipelineresource will not run ## References (optional) From 516f7761c93b15b3431cde7d2929e82836a7c52c Mon Sep 17 00:00:00 2001 From: Christie Wilson Date: Wed, 24 Feb 2021 16:35:33 -0500 Subject: [PATCH 3/5] =?UTF-8?q?Add=20comparison=20with=20TEP-0046=20?= =?UTF-8?q?=F0=9F=94=8E?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit In the most recent API working group we decided to keep this TEP and [TEP-0046](https://github.com/tektoncd/community/pull/318) separate because they are coming at a similar problem from different angles. In the WG @jerop suggested that we update the TEPs with some info on what the overlaps + differences are and that's what this TEP is adding! --- teps/0044-composing-tasks-with-tasks.md | 38 +++++++++++++++++++++++++ 1 file changed, 38 insertions(+) diff --git a/teps/0044-composing-tasks-with-tasks.md b/teps/0044-composing-tasks-with-tasks.md index f3aaf714c..e9958d919 100644 --- a/teps/0044-composing-tasks-with-tasks.md +++ b/teps/0044-composing-tasks-with-tasks.md @@ -16,6 +16,7 @@ authors: - [Goals](#goals) - [Non-Goals](#non-goals) - [Use Cases (optional)](#use-cases-optional) + - [Overlap with TEP-0046](#overlap-with-tep-0046) - [Requirements](#requirements) - [References (optional)](#references-optional) @@ -104,6 +105,43 @@ Issues that don't let PipelineResources (as is) solve these problems are: on uploading to and downloading from buckets in the cloud (e.g. GCS) - An organization is willing to use PVCs to some extent but needs to put limits on their use +### Overlap with TEP-0046 + +This TEP covers very similar ground as +[TEP-0046 (colocation of tasks and workspaces)](https://github.com/tektoncd/community/pull/318), and it may be that we +choose one solution to address both, but we have kept them separate because they are approaching the problem from +slightly different angles. + +Where they are similar: +* **Sharing data between Tasks efficiently** is the core problem at the heart of both (in this TEP, we talk about how + combining Tasks requires a PVC, in the other we talk about challenges around co-locating pods to share data and + inefficiencies of PVCs) + +Where they differ: + +* **Concurrency** (running of Tasks in parallel) is a big concern in TEP-0046 but isn't directly in the scope of this + problem (we could imagine a solution to this problem that only applies when running Tasks serially) +* **The Affinity Assistant** is another focus of TEP-0046 (i.e. revisiting it before v1) but isn't directly relevant to + this TEP (we might solve this without changing or discussing the affinity assistant at all) +* **Pipelines** - TEP-0046 is specifically concerned with Pipelines; there's a good chance the solution to this TEP + will also involve Pipelines, but there are possible solutions that could involve introducing a new type +* **Controller configuration vs. Authoring time** - This TEP assumes that we'd want to express this kind of composition + at "authoring time", i.e. make it [reusable](https://github.com/tektoncd/community/blob/main/design-principles.md#reusability) + vs configured at runtime or at the controller level., while TEP-0046 is suggesting to configure this at the controller + level. + * _Why not configure at the controller level?_ If we only allow this to be configured at the controller level, + users will not be able to control which Tasks are co-located with which: it will be all or nothing. This TEP assumes + that users will want to have at least some control, i.e. will want to take advantage of both k8s scheduling to execute + a Pipeline across multiple nodes AND might sometimes want to co-locate Tasks + * _Why not configure in the PipelineRun only? (vs the Pipeline)_ If we only allow this to be configured at runtime, + this means that: + * When looking at a Pipeline definition, or writing it, you can't predict or control how the Tasks will be + co-located (e.g. what if you want to isolate some data such that it's only available to particular Tasks) + * If you want to run a Pipeline, you'll need to make decisions at that point about what to co-locate. I can imagine + scenarios where folks want to make complex Pipelines and want to have some parts co-located and some parts not; if + we only allow for this at runtime, the Pipeline authors will only be able to provide docs or scripts (or + TriggerTemplates) to show folks how they are expected to be run. + ## Requirements - Tasks can be composed together run as "one unit": From 72d88c0b74e0c5bfa4ca1ae5c58219d4bcc462b1 Mon Sep 17 00:00:00 2001 From: Christie Wilson Date: Mon, 8 Mar 2021 14:31:08 -0500 Subject: [PATCH 4/5] =?UTF-8?q?Rename=20TEP=20to=20"Decouple=20Task=20Comp?= =?UTF-8?q?osition=20from=20Scheduling"=20=E2=9C=8D=EF=B8=8F?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit In today's API working group it was clear that I still haven't been able to articulate what problem this TEP is trying to solve in a way that's clear, so I'm trying this composition vs scheduling approach instead to see if that is more clear. Also @skaegi pointed out that the overhead required in running multiple pods is another consideration in addition to the PVC overhead so I've tried to include this as well. I'm going to follow up with a list of possible solutions so hopefully that will help folks who are confused about what this is trying to address. --- ...ouple-task-composition-from-scheduling.md} | 97 +++++++++---------- teps/README.md | 2 +- 2 files changed, 49 insertions(+), 50 deletions(-) rename teps/{0044-composing-tasks-with-tasks.md => 0044-decouple-task-composition-from-scheduling.md} (85%) diff --git a/teps/0044-composing-tasks-with-tasks.md b/teps/0044-decouple-task-composition-from-scheduling.md similarity index 85% rename from teps/0044-composing-tasks-with-tasks.md rename to teps/0044-decouple-task-composition-from-scheduling.md index e9958d919..b4accda94 100644 --- a/teps/0044-composing-tasks-with-tasks.md +++ b/teps/0044-decouple-task-composition-from-scheduling.md @@ -1,91 +1,64 @@ --- status: proposed -title: Composing Tasks with Tasks +title: Decouple Task Composition from Scheduling creation-date: '2021-01-22' -last-updated: '2021-01-22' +last-updated: '2021-02-08' authors: - '@bobcatfish' --- -# TEP-0044: Composing Tasks with Tasks +# TEP-0044: Decouple Task Composition from Scheduling - [Summary](#summary) - [Motivation](#motivation) - - [PipelineResources](#pipelineresources) - [Goals](#goals) - [Non-Goals](#non-goals) - [Use Cases (optional)](#use-cases-optional) - [Overlap with TEP-0046](#overlap-with-tep-0046) - [Requirements](#requirements) - [References (optional)](#references-optional) + - [PipelineResources](#pipelineresources) ## Summary -This TEP describes a gap in composability that isn't currently fully solved by Tasks or Pipelines. - -Ideally by addressing the issues described here we'd make it possible to create more efficient -Pipelines and make it easier to use a Task easily via a TaskRun, and we'd address some of -[the problems PipelineResources are trying to solve](https://docs.google.com/document/d/1KpVyWi-etX00J3hIz_9HlbaNNEyuzP6S986Wjhl3ZnA/edit#) -making it more clear whether or not they are an abstraction we want to keep. - -[#1673: Pipeline Resources Redesign](https://github.com/tektoncd/pipeline/issues/1673) +This TEP addresses how the current concept of a Task embodies both a reusable unit of logic but is also a unit used to +schedule execution. ## Motivation -Currently the only way to combine Tasks together is in a Pipeline. If you combine -Tasks in a Pipeline and they need to share data (beyond a simple -[result](https://github.com/tektoncd/pipeline/blob/master/docs/pipelines.md#using-results)), +* Tasks are a reusable unit combining one or more steps (container images) together and defining an interface to those + steps via params, workspaces and results +* Tasks can be combined in a Pipeline +* When a Pipeline is invoked via a PipelineRun, each Task is executed as a separate pod. + +This means that choices made around Task design (e.g. creating a Task that encapsulates a git clone and a separate Task +to run go unit tests) directly impact the performance and overhead involved in executing the Tasks. For example +if the git clone task wants to share data with the unit test task, beyond a simple +[result](https://github.com/tektoncd/pipeline/blob/master/docs/pipelines.md#using-results), it needs to you'll need to provision a PVC or do some other similar, cloud specific storage, to [make a volume available](https://github.com/tektoncd/pipeline/blob/master/docs/workspaces.md#specifying-volumesources-in-workspaces) -that can be shared between them. - -PVCs add additional overhead, both in speed and in management (somewhat -mitigated by [letting tekton create and delete volumes for you](https://github.com/tektoncd/pipeline/blob/master/docs/workspaces.md#volumeclaimtemplate) -but the PVCs still slow down the overall execution. - -### PipelineResources - -This kind of compositional functionality was being (somewhat! read on!) provided -by [PipelineResources](https://github.com/tektoncd/pipeline/blob/master/docs/resources.md). -[We did not bring these types to beta](https://github.com/tektoncd/pipeline/blob/master/docs/resources.md#why-arent-pipelineresources-in-beta). -Issues that don't let PipelineResources (as is) solve these problems are: -* PipelineResource "outputs" do not run when the steps of the Task fail (or if - a previous PipelienResource "output" fails) (see [unit test use case](#use-cases-optional)) -* PipelineResources do not use Tasks (which is something we could change), meaning - you cannot use them to compose Tasks together, you need to build PipelineResources - * Adding new PipelineResources currently needs to be done in the Tekton Pipelines controller, - tho there have been several attempts to propose ways to make it extensible: - * [PipelineResources 2 Uber Design Doc](https://docs.google.com/document/d/1euQ_gDTe_dQcVeX4oypODGIQCAkUaMYQH5h7SaeFs44/edit) - * [Tekton Pipeline Resource Extensibility](https://docs.google.com/document/d/1rcMG1cIFhhixMSmrT734MBxvH3Ghre9-4S2wbzodPiU/edit) - * All of this is even though PipelineResources are _very_ similar to Tasks in that they are - collections of steps (which [get injected before and after a Task's steps](https://github.com/tektoncd/pipeline/issues/1838) -* Tasks have to declare in advance what PipelineResources they need; you can't decide to use - a PipelineResource with a Task after it has been written (e.g. in a TaskRun or a Pipeline) - and you can't mix types of PipelineResource (e.g. if a Task declares it needs a - [git PipelineResources](https://github.com/tektoncd/pipeline/blob/master/docs/resources.md#git-resource) - it can't use a different kind of PipelineResource to provide those files (tho you can avoid - doing the clone multiple times using [from](https://github.com/tektoncd/pipeline/blob/master/docs/pipelines.md#using-the-from-parameter)) +that can be shared between them, and running the second Task will be delayed by the overhead of scheduling a second pod. ### Goals - Make it possible to combine Tasks together so that you can run multiple - Tasks as "one unit" (see [Requirements](#requirements) at authoring time + Tasks together and have control over the scheduling overhead (i.e. pods and volumes required) at authoring time in a way that can be reused (e.g. in a Pipeline) - Add some of [the features we don't have without PipelineResources](https://docs.google.com/document/d/1KpVyWi-etX00J3hIz_9HlbaNNEyuzP6S986Wjhl3ZnA/edit#) to Tekton Pipelines (without requiring use of PipelineResources), specifically **Task adapters/specialization** - - If possible we can provide a migration path from PipelineResources to the solution proposed here, where applicable ### Non-Goals -- Composing Tasks within Tasks at [Task authoring time](https://github.com/tektoncd/community/blob/master/design-principles.md#reusability). +- Updating the Task CRD to allow Tasks to reference other Tasks + at [Task authoring time](https://github.com/tektoncd/community/blob/master/design-principles.md#reusability). We could decide to include this if we have some use cases that need it; for now avoiding this allows us to avoid many layers of nesting (i.e. Task1 uses Task2 uses Task3, etc.) or even worse, recursion (Task 1 uses Task 2 uses Task 1...) - Completely replacing PipelineResources: we could decide to solve this by improving PipelineResources, or we could add a new feature via this TEP and still continue to support PipelineResources (since they provide [more functionality than just composition](https://docs.google.com/document/d/1KpVyWi-etX00J3hIz_9HlbaNNEyuzP6S986Wjhl3ZnA/edit#)) -- This was previously a use case we were targetting but we've decided to descope the TEP slightly, +- This was previously a use case we were targeting but we've decided to descope the TEP slightly, though if we end up solving this problem as well, that's a bonus: * A user wants to use a Task from the catalog with a git repo and doesn't want the extra overhead of using a Pipeline, they just want to make a TaskRun, @@ -101,9 +74,11 @@ Issues that don't let PipelineResources (as is) solve these problems are: cloning with [git-clone](https://github.com/tektoncd/catalog/tree/master/task/git-clone/0.2), running tests with [golang-test](https://github.com/tektoncd/catalog/tree/master/task/golang-test/0.1) and uploading results with [gcs-upload](https://github.com/tektoncd/catalog/tree/master/task/gcs-upload/0.1). -- An organziation does not want to use PVCs at all; for example perhaps they have decided +- An organization does not want to use PVCs at all; for example perhaps they have decided on uploading to and downloading from buckets in the cloud (e.g. GCS) - An organization is willing to use PVCs to some extent but needs to put limits on their use +- A user has decided that the overhead required in spinning up multiple pods is too much and wants to be able to have + more control over this. ### Overlap with TEP-0046 @@ -144,8 +119,9 @@ Where they differ: ## Requirements -- Tasks can be composed together run as "one unit": +- Tasks can be composed and run together: - Must be able to share data without requiring a volume external to the pod + - Must be possible to run multiple Tasks as one pod - It should be possible to have Tasks that run even if others fail; i.e. the Task can be run on the same pod as another Task that fails - This is to support use cases such as uploading test results, even if the test @@ -171,3 +147,26 @@ Where they differ: * Oldies but goodies: * [Link inputs and outputs without using volumes](https://github.com/tektoncd/pipeline/issues/617) * [Design PipelineResource extensibility](https://github.com/tektoncd/pipeline/issues/238) + +### PipelineResources + +This kind of compositional functionality was being (somewhat! read on!) provided +by [PipelineResources](https://github.com/tektoncd/pipeline/blob/master/docs/resources.md). +[We did not bring these types to beta](https://github.com/tektoncd/pipeline/blob/master/docs/resources.md#why-arent-pipelineresources-in-beta). +Issues that don't let PipelineResources (as is) solve these problems are: +* PipelineResource "outputs" do not run when the steps of the Task fail (or if + a previous PipelienResource "output" fails) (see [unit test use case](#use-cases-optional)) +* PipelineResources do not use Tasks (which is something we could change), meaning + you cannot use them to compose Tasks together, you need to build PipelineResources + * Adding new PipelineResources currently needs to be done in the Tekton Pipelines controller, + tho there have been several attempts to propose ways to make it extensible: + * [PipelineResources 2 Uber Design Doc](https://docs.google.com/document/d/1euQ_gDTe_dQcVeX4oypODGIQCAkUaMYQH5h7SaeFs44/edit) + * [Tekton Pipeline Resource Extensibility](https://docs.google.com/document/d/1rcMG1cIFhhixMSmrT734MBxvH3Ghre9-4S2wbzodPiU/edit) + * All of this is even though PipelineResources are _very_ similar to Tasks in that they are + collections of steps (which [get injected before and after a Task's steps](https://github.com/tektoncd/pipeline/issues/1838) +* Tasks have to declare in advance what PipelineResources they need; you can't decide to use + a PipelineResource with a Task after it has been written (e.g. in a TaskRun or a Pipeline) + and you can't mix types of PipelineResource (e.g. if a Task declares it needs a + [git PipelineResources](https://github.com/tektoncd/pipeline/blob/master/docs/resources.md#git-resource) + it can't use a different kind of PipelineResource to provide those files (tho you can avoid + doing the clone multiple times using [from](https://github.com/tektoncd/pipeline/blob/master/docs/pipelines.md#using-the-from-parameter)) diff --git a/teps/README.md b/teps/README.md index 8409e593c..ce56e1a02 100644 --- a/teps/README.md +++ b/teps/README.md @@ -169,7 +169,7 @@ This is the complete list of Tekton teps: |[TEP-0037](0037-remove-gcs-fetcher.md) | Remove `gcs-fetcher` image | implementing | 2021-01-27 | |[TEP-0039](0039-add-variable-retries-and-retrycount.md) | Add Variable `retries` and `retry-count` | proposed | 2021-01-31 | |[TEP-0040](0040-ignore-step-errors.md) | Ignore Step Errors | proposed | 2021-02-04 | -|[TEP-0044](0044-composing-tasks-with-tasks.md) | Composing Tasks with Tasks | proposed | 2021-01-22 | +|[TEP-0044](0044-decouple-task-composition-from-scheduling.md) | Decouple Task Composition from Scheduling | proposed | 2021-01-22 | |[TEP-0045](0045-whenexpressions-in-finally-tasks.md) | WhenExpressions in Finally Tasks | implementable | 2021-01-28 | |[TEP-0050](0050-ignore-task-failures.md) | Ignore Task Failures | proposed | 2021-02-19 | |[TEP-0051](0051-ppc64le-architecture-support.md) | ppc64le Support | proposed | 2021-01-28 | From a509831095500beb8feb673734317b5849a54540 Mon Sep 17 00:00:00 2001 From: Christie Wilson Date: Mon, 8 Mar 2021 16:54:59 -0500 Subject: [PATCH 5/5] =?UTF-8?q?Start=20brainstorming=20options=20?= =?UTF-8?q?=F0=9F=A7=A0?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit This commit adds possible solutions to the problem described in TEP-0044, including references to solutions in other TEPS (46 & 54). I was hoping to merge the problem statement before starting to talk about solutions but it seems like the problem statement is too abstract to get enough traction, and meanwhile folks have been opening more TEPs with related proposals in the meantime, so hopefully starting to list the options here will help us move the discussion forward. I'm hoping we can merge the problem + possible options without needing to decide on which one(s) we want to pursue. --- ...couple-task-composition-from-scheduling.md | 834 +++++++++++++++++- teps/README.md | 2 +- 2 files changed, 832 insertions(+), 4 deletions(-) diff --git a/teps/0044-decouple-task-composition-from-scheduling.md b/teps/0044-decouple-task-composition-from-scheduling.md index b4accda94..66b7b8575 100644 --- a/teps/0044-decouple-task-composition-from-scheduling.md +++ b/teps/0044-decouple-task-composition-from-scheduling.md @@ -2,7 +2,7 @@ status: proposed title: Decouple Task Composition from Scheduling creation-date: '2021-01-22' -last-updated: '2021-02-08' +last-updated: '2021-03-10' authors: - '@bobcatfish' --- @@ -19,6 +19,8 @@ authors: - [Requirements](#requirements) - [References (optional)](#references-optional) - [PipelineResources](#pipelineresources) +- [Design Details](#design-details) +- [Alternatives](#alternatives) ## Summary @@ -35,7 +37,7 @@ schedule execution. This means that choices made around Task design (e.g. creating a Task that encapsulates a git clone and a separate Task to run go unit tests) directly impact the performance and overhead involved in executing the Tasks. For example if the git clone task wants to share data with the unit test task, beyond a simple -[result](https://github.com/tektoncd/pipeline/blob/master/docs/pipelines.md#using-results), it needs to +[result](https://github.com/tektoncd/pipeline/blob/master/docs/pipelines.md#using-results), you'll need to provision a PVC or do some other similar, cloud specific storage, to [make a volume available](https://github.com/tektoncd/pipeline/blob/master/docs/workspaces.md#specifying-volumesources-in-workspaces) that can be shared between them, and running the second Task will be delayed by the overhead of scheduling a second pod. @@ -46,7 +48,9 @@ that can be shared between them, and running the second Task will be delayed by Tasks together and have control over the scheduling overhead (i.e. pods and volumes required) at authoring time in a way that can be reused (e.g. in a Pipeline) - Add some of [the features we don't have without PipelineResources](https://docs.google.com/document/d/1KpVyWi-etX00J3hIz_9HlbaNNEyuzP6S986Wjhl3ZnA/edit#) - to Tekton Pipelines (without requiring use of PipelineResources), specifically **Task adapters/specialization** + to Tekton Pipelines (without requiring use of PipelineResources), specifically the first feature listed in + [the doc](https://docs.google.com/document/d/1KpVyWi-etX00J3hIz_9HlbaNNEyuzP6S986Wjhl3ZnA/edit#heading=h.gi1d1dikb39u): + **Task adapters/specialization** ### Non-Goals @@ -130,6 +134,830 @@ Where they differ: address the above use case; for example in PipelineResources, you can have a storage "output" but if the steps fail, the "output" pipelineresource will not run +## Design details + +TBD - currently focusing on enumerating and examining alternatives before selecting one or more ways forward. + +## Alternatives + +Most of these options are not mutually exclusive: + +* [Task composition in Pipeline Tasks](#task-composition-in-pipeline-tasks) +* [Update PipelineResources to use Tasks](#update-pipelineresources-to-use-tasks) +* [Automatically combine Tasks based on workspace use](#automagically-combine-tasks-based-on-workspace-use) +* [Introduce scheduling rules to Pipeline](#introduce-scheduling-rules-to-pipeline) +* [PipelineRun: emptyDir](#pipelinerun-emptydir) +* [Controller configuration](#controller-level) +* [Within the Task](#within-the-task) +* [Remove distinction between Tasks and Pipelines](#remove-distinction-between-tasks-and-pipelines) +* [Custom Pipeline](#custom-pipeline) +* [Create a new Grouping CRD](#create-a-new-grouping-crd) +* [Rely on the Affinity Assistant](#rely-on-the-affinity-assistant) +* [Custom scheduler](#custom-scheduler) +* [Support other ways to share data (e.g. buckets)](#support-other-ways-to-share-data-eg-buckets) +* [Focus on workspaces](#focus-on-workspaces) + +Most of the solutions above involve allowing more than one Task to be run in the same pod, and those proposals all share +the following pros & cons. + +Pros: +* Making it possible to execute a Pipeline in a pod will also pave the way to be able to support use cases such as + [local execution](https://github.com/tektoncd/pipeline/issues/235) + +Cons: + +* Increased complexity around requesting the correct amount of resources when scheduling (having to look at the + requirements of all containers in all Tasks, esp. when they run in parallel) +* Requires re-architecting and/or duplicating logic that currently is handled outside the pods in the controller + (e.g. passing results between Tasks and other variable interpolation) + + +### Task composition in Pipeline Tasks + +In this option we make it possible to express Tasks which can be combined together to run sequentially as one pod. + +In the following example, 3 Tasks will be combined and run in one pod sequentially: + +1. `git-clone` +2. `just-unit-test` +3. `gcs-upload` + +```yaml +apiVersion: tekton.dev/v1beta1 +kind: Pipeline +metadata: + name: build-test-deploy +spec: + params: + - name: url + value: https://github.com/tektoncd/pipeline.git + - name: revision + value: v0.11.3 + workspaces: + - name: source-code + - name: test-results + tasks: + - name: run-unit-tests + taskRef: + name: just-unit-tests + workspaces: + - name: source-code + - name: test-results + init/before: + - taskRef: git-clone + params: + - name: url + value: $(params.url) + - name: revision + value: $(params.revision) + workspaces: + - name: source-code + workspace: source-code + finally/after: + - taskRef: gcs-upload + params: + - name: location + value: gs://my-test-results-bucket/testrun-$(taskRun.name) + workspaces: + - name: data + workspace: test-results +``` + +The `finally/after` Task(s) would run even if the previous steps fail. + +Pros: +* Is an optional addition to the existing types (doesn't require massive re-architecting) +* We have some initial indication (via PipelineResources) that this should be possible to do +* Maintains a line between when to use a complex DAG and when to use this functionality since this is only sequential + (but the line is fuzzy) + +Cons: +* Only helps us with some scheduling problems (e.g. doesn't help with parallel tasks or finally task execution) +* What if you _don't_ want the last Tasks to run if the previous tasks fail? + * Not clear how we would support more sophisticated use cases, e.g. if folks wanted to start mixing `when` expressions + into the `before/init` and/or `finally/after` Tasks +* If you want some other Task to run after these, you'll still need a workspace/volume + separate pod +* What if you want more flexibility than just before and after? (e.g. you want to completely control the ordering) + * Should still be possible, can put as many Tasks as you want into before and after + +Related: +* [Task Specialization: most appealing options?](https://docs.google.com/presentation/d/12QPKFTHBZKMFbgpOoX6o1--HyGqjjNJ7own6KqM-s68) +* [TEP-0054](https://github.com/tektoncd/community/pull/369) suggests something similar to this but: + * Uses "steps" as the unit + * Wants to combine these in the embedded Task spec vs in the Pipeline Task + +### Update PipelineResources to use Tasks + +In this option, we directly tackle problems with PipelineResources by updating them to refer to Tasks (e.g. catalog +Tasks). + +In the example below a PipelineResource type can refer to Tasks: + +```yaml +kind: PipelineResourceType +apiVersion: v1beta1 +metadata: + name: GCS +spec: + description: | + GCS PipelineResources download files onto a + Workspace from GCS when used as an input and uploads + files to GCS from a Workspace when used as an output. + input: + taskRef: + name: gcs-download # From catalog + output: + taskRef: + name: gcs-upload # From catalog +``` + +```yaml +kind: PipelineResourceType +apiVersion: v1beta1 +metadata: + name: GIT +spec: + description: | + GIT PipelineResources clone files from a Git + repo onto a Workspace when used as an input. It has + no output behaviour. + input: + taskRef: + name: git-clone # From catalog +``` + +```yaml +apiVersion: tekton.dev/v1beta1 +kind: Pipeline +metadata: + name: build-test-deploy +spec: + params: + - name: url + value: https://github.com/tektoncd/pipeline.git + - name: revision + value: v0.11.3 + workspaces: + - name: source-code + - name: test-results + tasks: + - name: run-unit-tests + taskRef: + name: just-unit-tests + workspaces: + - name: source-code + - name: test-results + resources: + inputs: + - resourceRef: GIT # the pipelineresource defined above + params: + - name: url + value: $(params.url) + - name: revision + value: $(params.revision) + workspaces: + - name: source-code + workspace: source-code + outputs: + - resourceRef: GCS # the pipelineresource defined above + params: + - name: location + value: gs://my-test-results-bucket/testrun-$(taskRun.name) + workspaces: + - name: data + workspace: test-results +``` + +(Credit to @sbwsg for this proposal and example!) + +If we pursue this we can make some choices around whether this works similar to today's PipelineResources where Tasks +need to declare that they expect them, or we could make it so that PipelineResources can be used with a Task regardless +of what it declares (the most flexible). + +Pros: +* "fixes" PipelineResources +* Uses concepts we already have in Tekton but upgrades them + +Cons: +* Not clear what the idea of a PipelineResource is really giving us if it's just a wrapper for Tasks +* If you want to use 2 Tasks together, you'll have to make a PipelineResource type for at least one of them +* Only helps us with some scheduling problems (e.g. doesn't help with parallel tasks or finally task execution) + +Related: +* [Specializing Tasks: Visions and Goals](https://docs.google.com/document/d/1G2QbpiMUHSs4LOqcNaIRswcdvoy8n7XuhTV8tXdcE7A/edit) +* [Specializing Tasks: Possible Designs](https://docs.google.com/document/d/1p8zq_wkAcwr1l5BpNQDyNjgWngOtnEhCYEpcNKMHvG4/edit) + +### Automagically combine Tasks based on workspace use + +In this option we could leave Pipelines as they are, but at runtime instead of mapping a Task to a pod, we could decide +what belongs in what pod based on workspace usage. + +In the example below, `get-source`, `run-unit-tests` and `upload-results` are all at least one of the two workspaces +so they will be executed as one pod, while `update-slack` would be run as a separate pod: + +```yaml +apiVersion: tekton.dev/v1beta1 +kind: Pipeline +metadata: + name: build-test-deploy +spec: + params: + - name: url + value: https://github.com/tektoncd/pipeline.git + - name: revision + value: v0.11.3 + workspaces: + - name: source-code + - name: test-results + tasks: + - name: get-source + workspaces: + - name: source-code + workspace: source-code + taskRef: + name: git-clone + params: + - name: url + value: $(params.url) + - name: revision + value: $(params.revision) + - name: run-unit-tests + runAfter: get-source + taskRef: + name: just-unit-tests + workspaces: + - name: source-code + workspcae: source-code + - name: test-results + workspace: test-results + - name: upload-results + runAfter: run-unit-tests + taskRef: + name: gcs-upload + params: + - name: location + value: gs://my-test-results-bucket/testrun-$(taskRun.name) + workspaces: + - name: data + workspace: test-results +finally: +- name: update-slack + params: + - name: message + value: "Tests completed with $(tasks.run-unit-tests.status) status" +``` + +Possible tweaks: +* We could do this scheduling only when + [a Task requires a workspace `from` another Task](https://github.com/tektoncd/pipeline/issues/3109). +* We could combine this with other options but have this be the default behavior + + +Pros: +* Doesn't require any changes for Pipeline or Task authors + +Cons: +* Will need to update our entrypoint logic to allow for steps running in parallel +* Doesn't give as much flexibility as being explicit + * This functionality might not even be desirable for folks who want to make use of multiple nodes + * We could mitigate this by adding more configuration, e.g. opt in or out at a Pipeline level, but could get + complicated if people want more control (e.g. opting in for one workspace but not another) + +### Introduce scheduling rules to pipeline + +In these options, we add some syntax that allows Pipeline authors to express how they want Tasks to be executed. + +#### Add "grouping" to tasks in a pipeline + +In this option we add some notion of "groups" into a Pipeline; any Tasks in a group will be scheduled together. + +In this example, everything in the `fetch-test-upload` group would be executed as one pod. The `update-slack` Task would +be a separate pod. + +```yaml +kind: Pipeline +metadata: + name: build-test-deploy +spec: + params: + - name: url + value: https://github.com/tektoncd/pipeline.git + - name: revision + value: v0.11.3 + workspaces: + - name: source-code + - name: test-results + tasks: + - name: get-source + group: fetch-test-upload # our new group syntax + workspaces: + - name: source-code + workspace: source-code + taskRef: + name: git-clone + params: + - name: url + value: $(params.url) + - name: revision + value: $(params.revision) + - name: run-unit-tests + group: fetch-test-upload # our new group syntax + runAfter: get-source + taskRef: + name: just-unit-tests + workspaces: + - name: source-code + workspcae: source-code + - name: test-results + workspace: test-results + - name: upload-results + group: fetch-test-upload # our new group syntax + runAfter: run-unit-tests + taskRef: + name: gcs-upload + params: + - name: location + value: gs://my-test-results-bucket/testrun-$(taskRun.name) + workspaces: + - name: data + workspace: test-results +finally: +- name: update-slack + params: + - name: message + value: "Tests completed with $(tasks.run-unit-tests.status) status" +``` + +Or we could have a group syntax that exists as a root element in the Pipeline, for example for the above: + +```yaml +groups: +- [get-source, run-unit-tests, upload-results] +``` + +Pros: +* Minimal changes for Pipeline authors + +Cons: +* Will need to update our entrypoint logic to allow for steps running in parallel + * We could (at least initially) only support sequential groups +* Might be hard to reason about what is executed together +* Might be hard to reason about what which Tasks can be combined in a group and which can't + +#### some other directive, e.g. labels, to indicate what should be scheduled together? + +This option is the same as the previous `groups` proposal but maybe we decide on some other ways to indicating grouping, +e.g. labels. + +### Runtime instead of authoring time + +These options pursue a solution that only works at runtime; this means Pipeline authors would not have any control +over the scheduling. + +#### PipelineRun: emptyDir + +In this solution we use the values provided at runtime for workspaces to determine what to run. Specifically, we allow +[`emptyDir`](https://github.com/tektoncd/pipeline/blob/a7ad683af52e3745887e6f9ed58750f682b4f07d/docs/workspaces.md#emptydir) +to be provided as a workspace at the Pipeline level even when that workspace is used by multiple Tasks, and when that +happens, we take that as the cue to schedule those Tasks together. + +For example given this Pipeline: + +```yaml +kind: Pipeline +metadata: + name: build-test-deploy +spec: + workspaces: + - name: source-code + - name: test-results + tasks: + - name: get-source + workspaces: + - name: source-code + workspace: source-code + taskRef: + name: git-clone + params: + - name: url + value: $(params.url) + - name: revision + value: $(params.revision) + - name: run-unit-tests + runAfter: get-source + taskRef: + name: just-unit-tests + workspaces: + - name: source-code + workspcae: source-code + - name: test-results + workspace: test-results + - name: upload-results + runAfter: run-unit-tests + taskRef: + name: gcs-upload + params: + - name: location + value: gs://my-test-results-bucket/testrun-$(taskRun.name) + workspaces: + - name: data + workspace: test-results +``` + +Running with this PipelineRun would cause `get-source` and `run-unit-tests` to be run in one pod, with `upload-results` +in another: + +```yaml +apiVersion: tekton.dev/v1beta1 +kind: PipelineRun +metadata: + name: run +spec: + pipelineRef: + name: build-test-deply + workspaces: + - name: source-code + emptyDir: {} + - name: test-results + persistentVolumeClaim: + claimName: mypvc +``` + +Running with this PipelineRun would cause all of the Tasks to be run in one pod: + +```yaml +apiVersion: tekton.dev/v1beta1 +kind: PipelineRun +metadata: + name: run +spec: + pipelineRef: + name: build-test-deply + workspaces: + - name: source-code + emptyDir: {} + - name: test-results + emptyDir: {} +``` + +Running with this PipelineRun would cause all of the Tasks to be run in separate pods: + +```yaml +apiVersion: tekton.dev/v1beta1 +kind: PipelineRun +metadata: + name: run +spec: + pipelineRef: + name: build-test-deply + workspaces: + - name: source-code + persistentVolumeClaim: + claimName: otherpvc + - name: test-results + persistentVolumeClaim: + claimName: mypvc +``` + +Pros: +* Allows runtime decisions about scheduling without changing the Pod + +Cons: +* If it's important for a Pipeline to be executed in a certain way, that information will have to be encoded somewhere + other than the Pipeline +* For very large Pipelines, this default behavior may cause problems (e.g. if the Pipeline is too large to be scheduled + into one pod) +* A bit strange and confusing to overload the meaning of `emptyDir`, might be simpler and clearer to have a field instead + +#### PipelineRun: field + +This is similar to the `emptyDir` based solution but instead of adding extra meaning to `emptyDir` we add a field to the +runtime workspace information or to the entire PipelineRun (maybe when this field is set workspaces do not need to be +provided.) + +A field could also be added as part of the Pipeline definition if desired (vs at runtime via a PipelineRun). + +#### Controller level + +This option is [TEP-0046](https://github.com/tektoncd/community/pull/318). In this option, the Tekton controller can +be configured to always execute Pipelines inside one pod. + +Pros: +* Authors of PipelineRuns and Pipelines don't have to think about how the Pipeline will be executed +* Pipelines can be used without updates + +Cons: +* Only cluster administrators will be able to control this scheduling, there will be no runtime or authoring time + flexibility +* Executing a pipeline in a pod will require significantly re-architecting our graph logic so it can execute outside + the controller and has a lot of gotchas we'll need to iron out (see + [https://hackmd.io/@vdemeester/SkPFtAQXd](https://hackmd.io/@vdemeester/SkPFtAQXd) for some more brainstorming) + +### Within the Task + +In this option we ignore the "not at Task authoring time" requirement and we allow for Tasks to contain other Tasks. + +This is similar to [TEP-0054](https://github.com/tektoncd/community/pull/369) which proposes this via the Task spec +in a Pipeline, but does not (yet) propose it for Tasks outside of Pipelines. + +For example: + +```yaml +apiVersion: tekton.dev/v1beta1 +kind: Task +metadata: + name: build-test-upload +spec: + workspaces: + - name: source + mountPath: /workspace/source/go/src/github.com/GoogleContainerTools/skaffold + steps: + - name: get-source + uses: git-clone + params: + url: $(params.url) + workspaces: + - name: source + workspace: source + - name: run-tests + image: golang + workingDir: $(workspaces.source.path) + script: | + go test + - name: upload-results + uses: gcs-upload +``` + +Pros: +* Doesn't require many new concepts + +Cons: +* Can create confusing chains of nested Tasks (Task A can contain Task B which can Contain Task C...) +* Requires creating new Tasks to leverage the reuse (maybe embedded specs negate this?) +* Doesn't help with parallel use cases + +### Remove distinction between Tasks and Pipelines + +In this version, we try to combine Tasks and Pipelines into one thing; e.g. by getting rid of Pipelines and adding all +the features they have to Tasks, and by giving Tasks the features that Pipelines have which they do not have. + +Things Tasks can do that Pipelines can't: +* Sidecars +* Refer to images (including args to images like script, command, args, env....) + +Things Pipelines can do that Tasks can't: +* Create DAGs, including running in parallel +* Finally +* When expressions + +For example, say our new thing is called a Foobar: + +```yaml +kind: Foobar +metadata: + name: git-clone +spec: + workspaces: + - name: source-code + foobars: + - name: get-source + steps: # or maybe each FooBar can only have 1 step and we need to use runAfter / dependencies to indicate ordering? + - name: clone + image: gcr.io/tekton-releases/github.com/tektoncd/pipeline/cmd/git-init:v0.21.0 + script: