[feature proposal] Alternate ways of running triggers

[afrittoli]

Expected Behavior

Triggers are an excellent way to provide a partially configured set of runtime resources, and isolated a limited number of parameters that are relevant for specific use cases.

Triggers receive a payload that includes the information needed to complete the runtime setup of resources. Today that payload can only be received over HTTP by.a service associated to the event listener.

After using triggers for some time in the dogfooding work, I feel that the ability to associate a set of runtime resources to a well defined set of parameters is very powerful, and it would be beneficial being able to use that in different ways.

One very specific use case that I have in mind is cronjobs, that we extensively for continuous delivery. Right now the setup we use is a k8 CronJob that curls and payload to the event listener service. It would be nice to have a way to define that in a more Tekton native way, with less boiler plate code. One syntax idea here could be to introduce different types of trigger, e.g.

  triggers:
    - name: pull-request-trigger
      type: http
      interceptors:
        - github:
            secretRef:
              secretName: ci-webhook
              secretKey: secret
            eventTypes:
              - pull_request
        - cel:
            filter: >-
              body.action == 'opened' || body.action == 'synchronize'
            overlays:
            - key: extensions.git_clone_depth
              expression: "string(body.pull_request.commits + 1.0)"
      bindings:
        - name: tekton-ci-webhook-pull-request
      template:
        name: tekton-ci-pipeline
    - name: periodic-tests-trigger
      type: cron
      schedule: "0 1 * * * "
      bindings:
        - name: tekton-ci-periodic-job
      template:
        name: tekton-ci-pipeline

In the example above, the tekton-ci-periodic-job binding would define static values that define the periodic job settings like git branch and so. This approach would allow us to use an existing CRD (the binding) to store information that today we need to embed in the k8s cronjob in the form of a JSON payload.

Another important use case is DSLs on top of Tekton.
A DSL normally will expose a higher level API that will generate tasks and pipelines. However the DSL may want / need to specify some runtime aspects in the generated resources. We have this case for instance with Kubeflow pipelines, were pipeline editors can attach a PVC to their pipeline, or configure affinity settings, to ensure their pipeline runs on nodes with GPUs.

I think the following would provide a great user experience:

dsl compile -> tasks, pipelines, trigger template, event listener (new trigger type)
tkn run trigger -p p1=v1 -p p2-v2

I'm not sure how that would work exactly, it might be a new type of CRD, e.g. a TriggerTemplateRun.

[vdemeester]

/kind feature

[afrittoli]

@bobcatfish @dibyom @vdemeester @ckadner @Tomcli

[vdemeester]

This could be a way to do #480 (but the scope of this feature request is broader 👼 )
I really like the idea 👼

It is also a tiny bit related to #482 for the attaching some runtime aspects.

[ncskier]

Really interesting idea for extending Triggers functionality using a trigger type 👍I'd be interested in seeing a list of use-cases for different ways that people want to trigger a Trigger – I know there are a couple other issues floating around like #478.

One quick note:

In the example above, the tekton-ci-periodic-job binding would define static values that define the periodic job settings like git branch and so. This approach would allow us to use an existing CRD (the binding) to store information that today we need to embed in the k8s cronjob in the form of a JSON payload.

Unless I'm misunderstanding something, there's nothing stopping you from doing this right now 👼You don't need to embed the TriggerBinding information in the k8s cronjob as a JSON. You can have a cronjob with an empty payload/no payload and use a static TriggerBinding like in your example.

(I know this doesn't completely address your use-case, but just thought I would point this out.)

[afrittoli]

Really interesting idea for extending Triggers functionality using a trigger type 👍I'd be interested in seeing a list of use-cases for different ways that people want to trigger a Trigger – I know there are a couple other issues floating around like #478.

One quick note:

In the example above, the tekton-ci-periodic-job binding would define static values that define the periodic job settings like git branch and so. This approach would allow us to use an existing CRD (the binding) to store information that today we need to embed in the k8s cronjob in the form of a JSON payload.

Unless I'm misunderstanding something, there's nothing stopping you from doing this right now 👼You don't need to embed the TriggerBinding information in the k8s cronjob as a JSON. You can have a cronjob with an empty payload/no payload and use a static TriggerBinding like in your example.

That's a very good point, thank you.
To have a completely empty body though I would need to have to have a dedicated event listener for each cronjob. I could have a body that only includes a cronjob name, and use a CEL filter to direct that to a trigger with the right binding.

(I know this doesn't completely address your use-case, but just thought I would point this out.)

[ncskier]

To have a completely empty body though I would need to have to have a dedicated event listener for each cronjob. I could have a body that only includes a cronjob name, and use a CEL filter to direct that to a trigger with the right binding.

Ah, good point; for multiple cron jobs & one EventListener you would need something to filter on 👍

[dibyom]

Right now the setup we use is a k8s CronJob that curls and payload to the event listener service.

Maybe we can do something like create the CronJob resource implicitly instead of requiring the user to create it

To have a completely empty body though I would need to have to have a dedicated event listener for each cronjob. I could have a body that only includes a cronjob name, and use a CEL filter to direct that to a trigger with the right binding.

I wonder if each trigger having a dedicated path within the Listener might help here.
Also, since each Trigger now includes interceptors, bindings, name, template...maybe it makes sense for it to be its own CRD.

I'm not sure how that would work exactly, it might be a new type of CRD, e.g. a TriggerTemplateRun.

has been proposed before as well. Perhaps time to look at it again: #200

[wlynch]

Maybe we can do something like create the CronJob resource implicitly instead of requiring the user to create it

I've also been thinking about this! :D

One idea I've been thinking about is "resource bundles" (I know the bundles term is being used elsewhere in Tekton, but I don't have a better name atm) - allowing users to compose resources together to refer to them as a single entity.

For example, we currently define a "Cron Trigger" as a CronJob + Tekton Trigger. But what if the "Cron Trigger" was its own CRD that a user can reference as its own object that is composed of subcomponents (maybe with some preconfigured templates for TriggerBindings, Interceptors, or TriggerTemplates to make it easier)? This would allow users to define their own, and Tekton can host a few predefined ones for ease of use.

As long as there is a Trigger somewhere in there, an EventListener should be able to hook in - I'm thinking something analogous to Knative Addressables (Triggerable? 😅 ).

[dibyom]

One idea I've been thinking about is "resource bundles" (I know the bundles term is being used elsewhere in Tekton, but I don't have a better name atm) - allowing users to compose resources together to refer to them as a single entity.

Neat! I can think of this being either a "overarching" CRD or something like the Catalog bundles which (I think) can contain multiple things (Tasks, Pipelines, Resources) all packaged into a single OCI image.

Besides Cron, it would be interesting to see if we can add other non-Trigger types here...I'm thinking some of the Knative EventSources could be interesting here.

[EliZucker]

Was procrastinating on my midterms and decided to check on Triggers... It's awesome to see how far this project has come! I hope everyone is doing well.

This seems like a cool proposal, and I was thinking that this discussion might tie back into earlier design discussions; Feel free to ignore my comments, but I'll write them here in case they are helpful:
 

For example, we currently define a "Cron Trigger" as a CronJob + Tekton Trigger. But what if the "Cron Trigger" was its own CRD that a user can reference as its own object that is composed of subcomponents (maybe with some preconfigured templates for TriggerBindings, Interceptors, or TriggerTemplates to make it easier)? This would allow users to define their own, and Tekton can host a few predefined ones for ease of use.

As long as there is a Trigger somewhere in there, an EventListener should be able to hook in - I'm thinking something analogous to Knative Addressables

I remember having similar thoughts when I was researching related eventing projects. If you look at projects like Argo Events or Knative Eventing, these projects eventually evolved to have their own libraries of event-adapter code. This is because many events, like Kafka Events, GCloud pub/sub, or even Cron events, do not naturally emit messages over HTTP. It's probably useful to think ahead about what Tekton Triggers will look like as more resource types are added.

It seems wasteful for the Triggers project to eventually reimplement the exact same adaptive logic in its own proprietary library. To avoid doing this, I personally suggested that a generic event-to-http adapter library, probably separated from the Triggers project, be worked on. This would align well with Tekton's mission, IMO. Also note that @iancoffey had this same idea here. There are more notes in the document I linked above as well.
 

Neat! I can think of this being either a "overarching" CRD or something like the Catalog bundles which (I think) can contain multiple things (Tasks, Pipelines, Resources) all packaged into a single OCI image.

Besides Cron, it would be interesting to see if we can add other non-Trigger types here...I'm thinking some of the Knative EventSources could be interesting here.

Linking a previous discussion we've had, where a Knative person said that modifying their intra-project adapters to be standalone would be "doable work"—so definitely something to consider.

[tekton-robot]

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.
If this issue is safe to close now please do so with /close.

/lifecycle rotten

Send feedback to tektoncd/plumbing.

[tekton-robot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
If this issue is safe to close now please do so with /close.

/lifecycle stale

Send feedback to tektoncd/plumbing.

[tekton-robot]

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

/close

Send feedback to tektoncd/plumbing.

[tekton-robot]

@tekton-robot: Closing this issue.

In response to this:

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

/close

Send feedback to tektoncd/plumbing.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[afrittoli]

/remove-lifecycle stale
/remove-lifecycle rotten

[afrittoli]

I think this is still valid - I'm not sure if the solution should be in Triggers but I would like to keep the discussion going on this one.

[dibyom]

/lifecycle frozen

[wstrange]

Was just looking at this today. k8s Cron tasks are really unwieldy. It would be great to have a tekton native way of running timed tasks.

[iancoffey]

I'm not sure if the solution should be in Triggers but I would like to keep the discussion going on this one.

I agree Triggers may not be the right place for this, but maybe there is room for an experimental "sources" project to flesh out what a better k8s cron task might look like. Or we could recommend folks just use knative cron event sources.

[afrittoli]

This feature would be really valuable for tracking purposes, at least when the source of the event is within the cluster.
An event sent over HTTP does not leave any trace in etcd. A TriggerRun object would be stored in the cluster, and it would make it easier to have a cohesive view of asynchronous workflows, keep track of events in a result store and it would be a great way for a notification controller to trigger a notification.

[dibyom]

Was adding TriggerRun also part of this proposal? (maybe #200 instead? 👼)
Definitely agree that we need a good way to track an event but I'm not sure if we want to do that using etcd as that would make the API server a direct dependency in the path of processing each Trigger. Maybe the results store can be an alternative way to keeping track or we could use something like Channels in Knative Events (with an in memory default)

[ibexmonj]

I believe this will be very valuable and the justification I have here is Cronjob wont be GA until K8s v1.20 . The fact that Cronjob is still beta and cronjob triggering is best-effort at best leaves a lot to be desired. We do see cronjobs failing to schedule any jobs from time to time and this would be a huge limitation for us specially when we are trying to leverage Tekton to onboard hundreds of thousands of job in our cluster.

[lbernick]

Opened tektoncd/community#904 to propose a possible implementation for cron based triggering. Happy to receive feedback!

[lbernick]

TEP-128 for scheduled triggers is now implementable!

[JustinGuese]

why not just something easy like:

apiVersion: tekton.dev/v1beta1
kind: CronTrigger
metadata:
  name: breakout-bot-v1-cron
spec:
  schedule: "0 9 * * *"
  timezone: "Europe/New_York"
  params:
  - name: taskRef
    value:
      name: breakout-bot-v1
      kind: Task

Shouldn't be too hard to implement and I'm surprised it's not integrated into tekton?

[lbernick]

Thanks for the feedback @JustinGuese, that's pretty similar to the accepted design proposal in TEP-128; if you'd like to see any changes to that design feel free to open a PR proposing them! We do not have anyone working on implementing this proposal right now but any contributions are welcome.

[fische]

Instead of making a CronJob for each ScheduledTrigger, what about using the RequeueAfter feature on reconcile? The operator could basically compute the duration until the next trigger and tell Kubernetes to wait that long before re-reconciling. When that happens, the operator can send the request to the given Trigger. That has the advantage of not creating additional resources on the cluster and keeps all the logic in the operator.

[willzhang]

Why not use a CronTask, it will more simple, and not need a triggers，triggers are used to listen to external events, and cronjob has no external events to listen to. It can automatically execute based on time

Argo-workflows have a cron-workflows, it's very easy to use: https://argoproj.github.io/argo-workflows/cron-workflows/

apiVersion: tekton.dev/v1beta1
kind: CronTask
metadata:
  name: hello
spec:
  schedule: "* * * * *"
  steps:
    - name: echo
      image: alpine
      script: |
        #!/bin/sh
        echo "Hello World"

For use tekton cronjob ,i must create so many yamls, this is too troublesome.

apiVersion: triggers.tekton.dev/v1beta1
kind: TriggerTemplate
metadata:
  name: hello-template
spec:
  params:
  - name: username
    default: "Kubernetes"
  resourcetemplates:
  - apiVersion: tekton.dev/v1beta1
    kind: PipelineRun
    metadata:
      generateName: matrixed-pr-with-retries-
    spec:
      workspaces:
        - name: myworkspace
          volumeClaimTemplate:
            spec:
              accessModes:
                - ReadWriteOnce
              resources:
                requests:
                  storage: 1Gi
      pipelineSpec:
        tasks:
          - name: clone-repo
            matrix:
              params:
                - name: repo
                  value:
                    - linux
                    - mac
                    - windows
            retries: 1
            taskSpec:
              params:
                - name: repo
              steps:
                - name: run-clone
                  image: docker.io/alpine/git:2.40.1
                  workingDir: /workspace
                  env:
                    - name: "DOCKER_CONFIG"
                      value: "/tekton/home/.docker/"
                  script: |
                    echo $(params.repo) >> $(workspaces.myworkspace.path)/recipe.txt
          - name: backup-repo
            taskSpec:
              steps:
                - name: run-backup
                  image: docker.io/tofran/restic-rclone:latest_latest
                  script: |
                    cat $(workspaces.myworkspace.path)/recipe.txt
            runAfter:
              - clone-repo
---
apiVersion: triggers.tekton.dev/v1beta1
kind: TriggerBinding
metadata:
  name: hello-binding
spec: 
  params:
  - name: username
    value: $(body.username)
---
apiVersion: triggers.tekton.dev/v1beta1
kind: EventListener
metadata:
  name: hello-listener
spec:
  serviceAccountName: tekton-robot
  triggers:
    - name: hello-trigger 
      bindings:
      - ref: hello-binding
      template:
        ref: hello-template
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: tekton-robot
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: triggers-example-eventlistener-binding
subjects:
- kind: ServiceAccount
  name: tekton-robot
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: tekton-triggers-eventlistener-roles
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: triggers-example-eventlistener-clusterbinding
subjects:
- kind: ServiceAccount
  name: tekton-robot
  namespace: default
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: tekton-triggers-eventlistener-clusterroles