Automate Pull Request Checklist - Check Tasks Completed

[jerop]

Changes

As described in #910, we have a submitter checklists in pull requests that
enforce with our standards.

Currently, maintainers and reviewers have to manually check that the
submitter checklist has been checked off and verify each item in the
list. If the submitter checklist has not been checked off, they need
to follow up with the user and request them to follow the standards
and check them off.

In this change, we add automation that checks that the checklist has
been checked off. If there's an incomplete item, the check reports that
there are incomplete tasks but does not fail yet. In a future change, we
will make the check fail if the submitter checklist is not checked off.
We hope that enforcing checking off the submitter checklist will encourage
contributors to look into our standards and apply them to their
contributions.

The implementation is modeled after the check-kind-label job.

Submitter Checklist

These are the criteria that every PR should meet, please check them off as you
review them:

• Includes docs (if user facing)
• Commit messages follow commit message best practices

See the contribution guide for more details.

[dibyom]

Overall this looks great! I have a couple of general comments (mostly about the layout of the resources) which do not have to be addressed in this PR:

1. I think we will be creating two pipelineruns with conditions for each event - and then in each condition we filter out to see if the command name from the PR matches the name of the check. I wonder if we can do this filtering either using a CEL expression in Triggers or in pipelines using a when expression so we can reduce the overhead of the extra pod being created.

2. We are adding a single new binding -- looks like we are using it in a bunch of places. Wonder if there is a way to reduce this boilerplate

3. This will suffer from the same race condition as the the existing check-labels run though I think we should tackle those outside this PR as part of Improve reliability of the check-pr-has-kind-label job  #888 or How to run TaskRuns in non-parallel mode pipeline#3989 (comment)

Would be nice to get @afrittoli 's comments on these as well!

/lgtm

[bobcatfish]

I wonder if we can do this filtering either using a CEL expression in Triggers

I think this same approach is being used for all the triggering - and it seems like the motivation is to be able to have one TriggerTemplate per repo being triggered (e.g. https://github.com/tektoncd/plumbing/blob/main/tekton/ci/templates/plumbing-template.yaml is the template for the plumbing repo) - the CEL expressions filter by repo but then leave it up to the pipelines to decide if they should continue running.

or in pipelines using a when expression so we can reduce the overhead of the extra pod being created.

At the moment the entire comment left on github is being passed to the pipeline so it needs to extract the "command" part of that which atm would still require a pod (or a CELRun :D). Maybe the trigger configuration could do the extraction and pass only the "check name" down to the pipelines - then all we'd need is a when expression!

[afrittoli]

Thanks for this!
I agree with @bobcatfish - I would prefer this to be in the ci-workspace folder.
Apart from that it looks good. As long as it does not fail and no comment is published, it won't really have an impact on users, but it's fine to add this incrementally.

[jerop]

@afrittoli @bobcatfish @dibyom @vdemeester thank you for the initial reviews on this PR - it is ready for another look

[ghost]

Thanks for creating the env var issue!

/lgtm

[tekton-robot]

@sbwsg: changing LGTM is restricted to collaborators

In response to this:

Thanks for creating the env var issue!

/lgtm

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[ghost]

Let's try this again:

/lgtm

[lbernick]

lgtm!

[bobcatfish]

Thanks for putting this together @jerop ! Very excited to see it in action :D

/approve

[tekton-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: bobcatfish

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [bobcatfish]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment