PipelineRun with embedded pipelineSpec and taskSpec

samples/e2e-mnist/mnist.ipynb

@@ -347,7 +347,7 @@
    "outputs": [],
    "source": [
     "from kfp_tekton.compiler import TektonCompiler\n",
-    "TektonCompiler().compile(mnist_pipeline, 'mnist.yaml', generate_pipelinerun=True)"
+    "TektonCompiler().compile(mnist_pipeline, 'mnist.yaml')"
    ]
   },
   {

samples/lightweight-component/lightweight_component.ipynb

@@ -235,7 +235,7 @@
    "outputs": [],
    "source": [
     "from kfp_tekton.compiler import TektonCompiler\n",
-    "TektonCompiler().compile(calc_pipeline, 'calc.yaml', generate_pipelinerun=True)"
+    "TektonCompiler().compile(calc_pipeline, 'calc.yaml')"
    ]
   },
   {

sdk/FEATURES.md

@@ -77,10 +77,12 @@ Tekton task name is the same as the containerOp name whereas the step name is al
 
 ### Affinity, Node Selector, and Tolerations
 
-Affinity, Node Selector, and Tolerations are Kubernetes spec for selecting which node should run the component based on user-defined constraints. They are implemented with Tekton's [PipelineRunTaskSpec](https://github.com/tektoncd/pipeline/blob/master/docs/pipelineruns.md#specifying-task-run-specs) features under Tekton PipelineRun.
+Affinity, Node Selector, and Tolerations are Kubernetes spec for selecting which node should run the component based on
+user-defined constraints. They are implemented with Tekton's [PipelineRunTaskSpec](https://github.com/tektoncd/pipeline/blob/master/docs/pipelineruns.md#specifying-task-run-specs)
+features under Tekton PipelineRun.
 The [affinity](/sdk/python/tests/compiler/testdata/affinity.py),
 [node_selector](/sdk/python/tests/compiler/testdata/node_selector.py), and
-[tolerations](/sdk/python/tests/compiler/testdata/tolerations.py) python tests are examples of how to use these features.
+[tolerations](/sdk/python/tests/compiler/testdata/tolerations.py) Python tests are examples of how to use these features.
 
 This feature is recently implemented in Tekton and is available on Tekton v0.13.0 onwards.
 
@@ -94,6 +96,7 @@ feature.
 This feature is recently implemented in Tekton and is available on Tekton v0.13.0 onwards.
 
 # Pipeline DSL features with custom Tekton implementation
+
 ## Features with same behavior as Argo
 Below are the features that don't have one to one mapping to Tekton's native implementation, but the same behaviors can be replicated with
 extra custom processing code or workaround within the compiler.

sdk/README.md

@@ -19,8 +19,7 @@ for [Tekton](https://github.com/tektoncd/pipeline).
     - [Running the compiled pipeline on a Tekton cluster](#running-the-compiled-pipeline-on-a-tekton-cluster)
   - [Build Tekton from Master](#build-tekton-from-master)
   - [Additional Features](#additional-features)
-    - [1. Compile Kubeflow Pipelines as a Tekton PipelineRun](#1-compile-kubeflow-pipelines-as-a-tekton-pipelinerun)
-    - [2. Compile Kubeflow Pipelines with Artifacts Enabled](#2-compile-kubeflow-pipelines-with-artifacts-enabled)
+    - [Compile Kubeflow Pipelines with Artifacts Enabled](#compile-kubeflow-pipelines-with-artifacts-enabled)
   - [List of Available Features](#list-of-available-features)
   - [Troubleshooting](#troubleshooting)
 
@@ -111,19 +110,13 @@ If you cloned the `kfp-tekton` project, you can find example pipelines in the
 
 After compiling the `sdk/python/tests/compiler/testdata/parallel_join.py` DSL script
 in the step above, we need to deploy the generated Tekton YAML to our Kubernetes
-cluster with `kubectl` and start a pipeline run with `tkn`:
+cluster with `kubectl`. The Tekton server will automatically start a pipeline run
+for which we can follow the logs using the `tkn` CLI:
 
     kubectl apply -f pipeline.yaml
-    tkn pipeline start parallel-pipeline --showlog
-
-A prompt should be asking for the pipeline arguments. Press `enter` and
-accept the defaults:
-
-    ? Value for param `url1` of type `string`? (Default is `gs://ml-pipeline-playground/shakespeare1.txt`) gs://ml-pipeline-playground/shakespeare1.txt
-    ? Value for param `url2` of type `string`? (Default is `gs://ml-pipeline-playground/shakespeare2.txt`) gs://ml-pipeline-playground/shakespeare2.txt
 
-    Pipelinerun started: parallel-pipeline-run-th4x6
-   
+    tkn pipelinerun logs --last
+
 Once the Tekton Pipeline is running, the logs should start streaming:
 
     Waiting for logs to be available...
@@ -147,38 +140,9 @@ will be listed below.
 
 ## Additional Features
 
-### 1. Compile Kubeflow Pipelines as a Tekton PipelineRun
-
-By default, a Tekton [`PipelineRun`](https://github.com/tektoncd/pipeline/blob/master/docs/pipelineruns.md#overview)
-is generated by the `tkn` CLI so that users can interactively change their pipeline
-parameters during each execution. 
-However, `tkn` CLI is lagging several important features when generating a `PipelineRun`.
-Therefore, we added support for generating pipelineRun using `dsl-compile-tekton`
-with all the latest `kfp-tekton` compiler features. The comparison between Tekton
-pipeline and Argo workflow is described in our 
-[design docs](https://docs.google.com/document/d/1oXOdiItI4GbEe_qzyBmMAqfLBjfYX1nM94WHY3EPa94/edit#heading=h.f38y0bqkxo87).
-
-Compiling Kubeflow Pipelines into a Tekton `PipelineRun` is currently in the experimental
-stage. [Here](https://github.com/tektoncd/pipeline/blob/master/docs/pipelineruns.md) is
-the list of supported features in `PipelineRun`.
-
-As of today, the below `PipelineRun` features are available within `dsl-compile-tekton`:
- - Affinity
- - Node Selector
- - Tolerations
-
-To compile Kubeflow Pipelines as Tekton pipelineRun, add the `--generate-pipelinerun`
-parameter to the `dsl-compile-tekton` command:
-
-    dsl-compile-tekton \
-        --py sdk/python/tests/compiler/testdata/tolerations.py \
-        --output pipeline.yaml \
-        --generate-pipelinerun
-
-
-### 2. Compile Kubeflow Pipelines with Artifacts Enabled
+### Compile Kubeflow Pipelines with Artifacts Enabled
 
-**Prerequisites**: Install [Kubeflow Pipelines](https://www.kubeflow.org/docs/pipelines/installation/).
+**Prerequisite**: Install [Kubeflow Pipelines](https://www.kubeflow.org/docs/pipelines/installation/).
 
 By default, _artifacts_ are disabled because they are dependent on Kubeflow Pipeline's
 [Minio](https://docs.minio.io/) storage. When artifacts are enabled, all the output
@@ -187,10 +151,10 @@ Enabling artifacts also allows files to be downloaded or stored as artifact inpu
 Since artifacts are dependent on the Kubeflow Pipeline's deployment, the generated
 Tekton pipeline must be deployed to the same namespace as Kubeflow Pipelines.
 
-To compile Kubeflow Pipelines as a Tekton `PipelineRun`, add the `--enable-artifacts`
-argument to your `dsl-compile-tekton` commands. Then, run the pipeline in the same
-namespace that is used by Kubeflow Pipelines (typically `kubeflow`) by using the
-`-n` flag. e.g.:
+To compile Kubeflow Pipelines DSL script into a Tekton `PipelineRun`, add the
+`--enable-artifacts` argument to your `dsl-compile-tekton` commands. Then, run the
+pipeline in the same namespace that is used by Kubeflow Pipelines (typically `kubeflow`)
+by specifying the `-n` flag:
 
     dsl-compile-tekton \
         --py sdk/python/tests/compiler/testdata/parallel_join.py \
@@ -199,17 +163,12 @@ namespace that is used by Kubeflow Pipelines (typically `kubeflow`) by using the
 
     kubectl apply -f pipeline.yaml -n kubeflow
 
-    tkn pipeline start parallel-pipeline --showlog -n kubeflow
+    tkn pipelinerun logs --last -n kubeflow
 
+You should see log messages saying the artifacts were stored in the object storage:
 
-You should see log messages saying the artifacts were stored in the object storage you specified:
-
-    ? Value for param `url1` of type `string`? (Default is `gs://ml-pipeline-playground/shakespeare1.txt`) gs://ml-pipeline-playground/shakespeare1.txt
-    ? Value for param `url2` of type `string`? (Default is `gs://ml-pipeline-playground/shakespeare2.txt`) gs://ml-pipeline-playground/shakespeare2.txt
-
-    Pipelinerun started: parallel-pipeline-run-g87bs
-
     Waiting for logs to be available...
+
     [gcs-download : main] With which he yoketh your rebellious necks Razeth your cities and subverts your towns And in a moment makes them desolate
 
     [gcs-download : copy-artifacts] Added `storage` successfully.
@@ -243,6 +202,6 @@ the [FEATURES](FEATURES.md) doc.
 - When you encounter permission issues related to ServiceAccount, refer to
   [Servince Account and RBAC doc](sa-and-rbac.md)
 
-- If you run into `bad interpreter: No such file or director` when trying to use
+- If you run into the error `bad interpreter: No such file or director` when trying to use
   python's venv, remove the current virtual environment in the `.venv` directory and
   create a new one using `virtualenv .venv`