| status | title | creation-date | last-updated | authors | collaborators | ||||
|---|---|---|---|---|---|---|---|---|---|
implementable |
Scheduled Runs |
2022-12-20 |
2023-05-17 |
|
- Summary
- Motivation
- Proposal
- Design Evaluation
- Alternatives
- Implementation Plan
- Implementation Pull Requests
- References
This TEP proposes allowing Task/Pipeline users to schedule Task/Pipeline runs on a recurring basis.
Currently, to get a run to occur on a regular basis, a user must write a cron job that either creates the run itself (which duplicates much of triggers' functionality) or pings an EventListener. If the EventListener is exposed outside of the cluster, the user must also sign the cron job event body and verify it in a custom interceptor (although it's unlikely a user would choose to expose an ingress for an EventListener used only for cron jobs).
- Can create new PipelineRuns, TaskRuns, or CustomRuns on a recurring basis without creating a webhook or writing a CronJob
- Polling a repository (or other external system) for changes; this is covered in TEP-0083.
- Release Pipeline: User would like to create releases of their project on a nightly basis.
Create ScheduledTemplate CRD with resourcetemplates and schedule field
This solution doesn't use TriggerTemplate or TriggerBindings. Instead, it's spec has resourcetemplates which can have one or more pipelineruns, taskruns or other allowed Kubernetes and Tekton resources.
apiVersion: triggers.tekton.dev/v1alpha1
kind: ScheduledTemplate
metadata:
name: nightly-release
spec:
cloudEventSink: http://eventlistener.free.beeceptor.com
resourcetemplates:
- apiVersion: tekton.dev/v1beta1
kind: PipelineRun
metadata:
generateName: simple-pipeline-run-
spec:
pipelineRef:
name: simple-pipeline
params:
- name: message
value: "Hi"
- name: git-url
value: "https://github.com/tektoncd/community.git"
workspaces:
- name: git-source
emptyDir: {}
schedule: "0 0 * * *"ScheduledTemplate reconciler which creates resources will be part of the Tekton Triggers.
As part of future work for this proposal, we can add a CLI command to manually trigger execution of a ScheduledTrigger
similar to the kubectl functionality kubectl create job --from=cronjob/my-cronjob, for example:
tkn crontrigger start <scheduled_template_cr_name>This functionality is tracked in tektoncd/cli#1833.
This feature is essentially syntactic sugar for a CronJob creating a Tekton resource. It uses the functionality of TriggerTemplate but does not depends upon an EventListener. The schedule is a runtime concern, similar to other fields specified in EventListeners.
The user experience with this proposal is much simpler with this proposal (a single additional line of configuration) than without it. For an example of what the user experience looks like without this proposal, see the nightly release CronJob used in the plumbing repo.
- No new dependencies needed for this proposal
- We are not coupling projects together but we add a new controller to Triggers.
- This proposal doesn't require the user to understand how the API is implemented, or introduce new Kubernetes concepts into the API.
Creating a CronJob to simply create a new Tekton resource may not be very performant; however, performance of creating scheduled runs is not as important as (for example) performance of creating a run used for CI. Using CronJobs is a reasonable way to start, especially if it helps us avoid reimplementing cron syntax.
This proposal adds a new controller to the triggers with new permissions to create, update, and delete CronJobs.
-
We might want to avoid setting a precedent of building functionality into Triggers to trigger on many different types of events. This proposal couples an event (a time occurring) with a Tekton resource creation, when we may prefer to keep events and resource creation separate. However, cron jobs are an extremely common simple use case that likely makes sense to address specifically.
-
We might want to build this functionality into a higher level API like Workflows or a larger feature like polling runs. This proposal doesn't prevent that, and ScheduledTriggers could be leveraged to implement similar functionality in a higher level system.
Create a new ScheduledTrigger CRD that fires on a specified schedule. For example:
apiVersion: triggers.tekton.dev/v1alpha1
kind: ScheduledTrigger
metadata:
name: nightly-release
spec:
schedule: "0 0 * * *"
triggers:
- triggerRef: experimental-release-pipeline
serviceAccountName: default
cloudEventSink: http://eventlistener.free.beeceptor.comThe schedule field uses the same syntax as Kubernetes CronJobs.
Like EventListeners, ScheduledTriggers can define Triggers or TriggerGroups inline, or contain references to Triggers.
serviceAccountName defaults to "default", and cloudEventSink is optional.
A ScheduledTrigger sends an event with an empty body.
Consider a use case where a company has a release Pipeline they'd like to use for both nightly releases and event-driven releases. They can reuse a TriggerTemplate with this Pipeline in two different Triggers; one that is used in an EventListener and one that's used in a ScheduledTrigger. For example:
kind: EventListener
spec:
triggers:
- name: release
interceptors:
- ref:
name: github
params:
- name: "eventTypes"
value: ["push"]
bindings:
- name: git-sha
value: $(body.head_commit.id)
template:
ref: release-pipeline-templatekind: ScheduledTrigger
spec:
schedule: "0 0 * * *"
triggers:
- bindings:
- name: git-sha
value: main
template:
ref: release-pipeline-templateCompared to the alternative solution of adding a schedule to the Trigger CRD, this solution has two benefits:
- Users can specify cloudEvent sinks for Triggers fired on a scheduled basis.
- Users can fire multiple Triggers on a scheduled basis with the same service account, using TriggerGroups. (For example, Tekton releases all of its experimental projects on a nightly basis with the same Pipeline.)
As part of future work for this proposal, we can add two new variable replacements in bindings:
- $(context.date): date when the run was created, in RFC3339 format
- $(context.datetime): timestamp when the run was created, in RFC3339 format
Like EventListeners, ScheduledTriggers should generate an event ID (a UUID) and support variable replacement for $(context.eventID) in bindings.
In this case, we add a controller in the experimental repo which can later be added to pipelines. Or it can be part of the Pipelinerun controller. We will add field for scheduling:
apiVersion: tekton.dev/v1
kind: Pipelinerun
metadata:
generateName: nightly-release
spec:
schedule: "0 0 * * *"
pipelineRef:
name: "release"
taskRunTemplate:
serviceAccountName: ...Or an annotation:
apiVersion: tekton.dev/v1
kind: Pipelinerun
metadata:
generateName: nightly-release
annotations:
tekton.dev/schedule: "0 0 * * *"
spec:
pipelineRef:
name: "release"
taskRunTemplate:
serviceAccountName: ...The controller will watch Pipelinerun and generate jobs based on annotation or field. Controller would need to be modified to skip the pipelinerun with `scheduled` field or annotation.
Pros:
- No dependency on Triggers. Enduser doesn't need to know about Trigger CRDs to use scheduling.
Cons:
- An additional controller in pipelines.
- Feature doesn't seem to be part of pipelines repo but because we are adding a new field to Pipelinerun, we can only keep it there. If we proceed with annotations, then this can be part of other components also.
- Pipeline controllers would have to understand to skip PipelineRuns with a
schedulefield or annotation set. - This breaks the assumption that a Pipelinerun is associated with a single instance of Pipeline.
- Only one pipelinerun can be created.
This solution is the same as the proposed solution, with one additional feature: the ability to specify a fixed event body for a ScheduledTrigger. For example:
apiVersion: triggers.tekton.dev/v1alpha1
kind: ScheduledTrigger
metadata:
name: nightly-release
spec:
triggers:
- triggerRef: release-pipeline
body: "{'head_commit':{'id':12345...}}"The body field is a JSON-serialized string.
Users could use this to reuse existing Triggers (which may depend on variable values from event bodies) in their ScheduledTrigger.
However, it's unclear whether it's realistic to expect Triggers to be reused (as opposed to TriggerTemplates).
Fixed event bodies might not make sense for existing bindings that expect variable events.
For example, if an existing binding relies on a push event,
$(body.head_commit) makes more sense when used in a fixed payload than $(body.before) or $(body.commits).
Fixed payloads probably wouldn't make sense at all for CI, although this proposal doesn't target CI use cases.
For example, if a binding uses $(body.pull_request.head.sha), the ScheduledTrigger body must be "{'pull_request':{'head':{'sha':12345...}}}".
Lastly, it may be especially difficult to reuse Triggers between EventListeners and ScheduledTriggers if the Triggers use interceptors to validate event payloads. For example, if you want to reuse a Trigger with the GitHub interceptor in a ScheduledTrigger, we would need to allow specifying headers in a ScheduledTrigger payload, and the user would have to hard-code a header like "{'X-Hub-Signature': {'sha1':'ba0cdc263b3492a74b601d240c27efe81c4720cb'}}".
Reusing TriggerTemplates instead of Triggers is simpler, and users can use params instead of having to JSON-serialize a string to mimic an event body.
Add a schedule field to the Trigger CRD following cron syntax, for example:
kind: Trigger
spec:
schedule: "0 0 * * *"
bindings:
- name: project-name
value: wait-task
- name: triggers-uuid
value: $(context.eventID)
- name: datetime
value: $(context.datetime)
template:
ref: experimental-release-pipeline-templateIf a company has a release Pipeline defined in a TriggerTemplate, and wants to use that TriggerTemplate for both nightly releases (from the main branch) and event-driven releases (based on a Git SHA from a push event body), they could define the following CRDs:
kind: EventListener
spec:
triggers:
- name: release
interceptors:
- ref:
name: github
params:
- name: "eventTypes"
value: ["push"]
bindings:
- name: git-sha
value: $(body.head_commit.id)
template:
ref: release-pipeline-templatekind: Trigger
spec:
schedule: "0 0 * * *"
bindings:
- name: git-sha
value: main
template:
ref: release-pipeline-templateTriggers with a schedule can be referenced in an EventListener;
however, Triggers with both a schedule and interceptors would result in a validation error.
Inline TriggerBindings using $(body.foo) and $(header.foo) syntax would also result in a validation error.
Resources created from a schedule always use the Trigger's serviceAccountName, or the default service account
if none is specified.
Pros:
- Can reuse existing TriggerTemplates
Cons:
- Triggers with
schedulesare unlikely to be used in an EventListener, since they can't depend on an event body - Can't specify the same schedule and service account for a group of Triggers, or specify a cloud events sink for scheduled Triggers
Add a schedule field following Kubernetes CronJob syntax to the EventListener CRD, for example:
kind: EventListener
spec:
schedule: "0 0 * * *"
triggers:
- bindings:
- name: project-name
value: wait-task
- name: triggers-uuid
value: $(context.eventID)
- name: datetime
value: $(context.datetime)
template:
ref: experimental-release-pipeline-templatekind: EventListener
spec:
schedule: "0 0 * * *"
triggerGroups:
- triggerSelector:
labelSelector:
matchLabels:
type: nightly-releaseAn EventListener with a schedule would send an empty event body at the specified times.
This means that any bindings containing $(body.foo) or $(header.bar) would fail, as well as any interceptors
responsible for processing the event.
An EventListener with a schedule would still create a service for receiving events.
Pros:
- Reuses existing functionality
Cons:
- Users might want to have different schedules for different Triggers in the EventListener
- The use of the name "EventListener" for a CRD that doesn't need to "listen" for incoming events may be confusing
This CRD would be extremely similar to the Knative Eventing PingSource, without requiring installation of Knative Eventing.
For example:
apiVersion: triggers.tekton.dev/v1alpha1
kind: PingSource
metadata:
name: nightly-release
spec:
schedule: "0 0 * * *"
sink:
ref:
name: my-eventlistenerThis would send an empty event body to the EventListener, and we could later add support for a fixed payload if desired.
Pros:
- Avoids coupling eventing with resource creation
- Adding support for fixed payloads allows Triggers that do process event bodies to be reused on a cron schedule
Cons
- Reimplements same functionality as Knative
- More verbose than proposed solution
- Exposes more implementation details than is necessary
Creating a ScheduledTrigger should result in the creation of a CronJob which processes any referenced Triggers to create Tekton resources.