Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Start emitting CloudEvents for TaskRuns #2837

Merged

Conversation

afrittoli
Copy link
Member

@afrittoli afrittoli commented Jun 18, 2020

Changes

Add a new method 'SendCloudEventWithRetries' to the cloud events
controller. It that allows emitting cloud events asynchronously
(in a go routine), taking benefit of the retries capabilities of
the cloudevents go sdk.

Rework the fake client to allow for unit testing of
'SendCloudEventWithRetries'. The new implementation is similar
to that of the fake recorder from client-go, which allows to
unit test k8s events and cloud events in a similar fashion.

Add a new config option default-cloud-events-sink in the
defaults config map. This options allows setting a default sink
for cloud events. If the default sink is setup, cloud events are
sent, else they're disabled.

Invoke 'SendCloudEventWithRetries' from the events modules so
that we start sending cloud events in the same places we were
send k8s events (except for errors).

Most of the plumbing is in place to start emitting cloud events
for pipeline runs too, however there is a small refactor for
the config maps required before we can add that, so this commit
only enabled cloud events for task runs for now.

Partially addresses #2082

Submitter Checklist

These are the criteria that every PR should meet, please check them off as you
review them:

  • Includes tests (if functionality changed/added)
  • Includes docs (if user facing)
  • Commit messages follow commit message best practices
  • Release notes block has been filled in or deleted (only if no user facing changes)

See the contribution guide for more details.

Double check this list of stuff that's easy to miss:

Reviewer Notes

If API changes are included, additive changes must be approved by at least two OWNERS and backwards incompatible changes must be approved by more than 50% of the OWNERS, and they must first be added in a backwards compatible way.

Release Notes

Add a new configuration option `default-cloud-events-sink` in the defaults config map.
This options allows setting a default sink for cloud events. 
If the default sink is setup, cloud events are sent, else they're disabled.

Send cloud events (when enabled) from the task run and controller, for all non-error
events that are already notified via k8s events.

/cc @n3wscott
/cc @bobcatfish

/kind feature

@tekton-robot tekton-robot requested review from bobcatfish and a user June 18, 2020 21:40
@tekton-robot tekton-robot added the size/L Denotes a PR that changes 100-499 lines, ignoring generated files. label Jun 18, 2020
@afrittoli afrittoli changed the title Start emitting CloudEvents for TaskRuns WIP Start emitting CloudEvents for TaskRuns Jun 18, 2020
@tekton-robot
Copy link
Collaborator

This PR cannot be merged: expecting exactly one kind/ label

Available kind/ labels are:

kind/bug: Categorizes issue or PR as related to a bug.
kind/flake: Categorizes issue or PR as related to a flakey test
kind/cleanup: Categorizes issue or PR as related to cleaning up code, process, or technical debt.
kind/design: Categorizes issue or PR as related to design.
kind/documentation: Categorizes issue or PR as related to documentation.
kind/feature: Categorizes issue or PR as related to a new feature.
kind/misc: Categorizes issue or PR as a miscellaneuous one.

@tekton-robot tekton-robot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jun 18, 2020
@tekton-robot
Copy link
Collaborator

This PR cannot be merged: expecting exactly one kind/ label

Available kind/ labels are:

kind/bug: Categorizes issue or PR as related to a bug.
kind/flake: Categorizes issue or PR as related to a flakey test
kind/cleanup: Categorizes issue or PR as related to cleaning up code, process, or technical debt.
kind/design: Categorizes issue or PR as related to design.
kind/documentation: Categorizes issue or PR as related to documentation.
kind/feature: Categorizes issue or PR as related to a new feature.
kind/misc: Categorizes issue or PR as a miscellaneuous one.

1 similar comment
@tekton-robot
Copy link
Collaborator

This PR cannot be merged: expecting exactly one kind/ label

Available kind/ labels are:

kind/bug: Categorizes issue or PR as related to a bug.
kind/flake: Categorizes issue or PR as related to a flakey test
kind/cleanup: Categorizes issue or PR as related to cleaning up code, process, or technical debt.
kind/design: Categorizes issue or PR as related to design.
kind/documentation: Categorizes issue or PR as related to documentation.
kind/feature: Categorizes issue or PR as related to a new feature.
kind/misc: Categorizes issue or PR as a miscellaneuous one.

@tekton-robot
Copy link
Collaborator

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/reconciler/events/cloudevent/cloud_event_controller.go 94.6% 92.3% -2.3
pkg/reconciler/events/cloudevent/cloudevent.go 88.2% 90.6% 2.3
pkg/reconciler/events/cloudevent/cloudeventclient.go 7.1% 21.1% 13.9
pkg/reconciler/events/cloudevent/cloudeventsfakeclient.go 50.0% 54.5% 4.5
pkg/reconciler/taskrun/taskrun.go 77.6% 77.4% -0.3

@tekton-robot tekton-robot added the kind/feature Categorizes issue or PR as related to a new feature. label Jun 18, 2020
@afrittoli
Copy link
Member Author

WIP because tests are missing for the config map part of the PR - I need to test that the config option is pulled in correctly in the context. It may be worth adding at least one reconciler test.
Adding an E2E test would require a custom config of Tekton for that.

@afrittoli afrittoli force-pushed the issues/2082-add-cloud-events branch from 37e9ed5 to 127f563 Compare June 18, 2020 22:01
@afrittoli afrittoli requested a review from n3wscott June 18, 2020 22:02
@afrittoli afrittoli force-pushed the issues/2082-add-cloud-events branch from 127f563 to 7675af8 Compare June 19, 2020 10:16
@tekton-robot
Copy link
Collaborator

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/reconciler/events/cloudevent/cloud_event_controller.go 94.6% 92.3% -2.3
pkg/reconciler/events/cloudevent/cloudevent.go 88.2% 90.6% 2.3
pkg/reconciler/events/cloudevent/cloudeventclient.go 7.1% 26.7% 19.5
pkg/reconciler/events/cloudevent/cloudeventsfakeclient.go 50.0% 54.5% 4.5

@afrittoli afrittoli force-pushed the issues/2082-add-cloud-events branch from 7675af8 to 16c5ad0 Compare June 19, 2020 10:19
@tekton-robot
Copy link
Collaborator

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/reconciler/events/cloudevent/cloud_event_controller.go 94.6% 92.3% -2.3
pkg/reconciler/events/cloudevent/cloudevent.go 88.2% 90.6% 2.3
pkg/reconciler/events/cloudevent/cloudeventclient.go 7.1% 26.7% 19.5
pkg/reconciler/events/cloudevent/cloudeventsfakeclient.go 50.0% 54.5% 4.5

@afrittoli afrittoli force-pushed the issues/2082-add-cloud-events branch from 16c5ad0 to d7c6327 Compare June 19, 2020 10:22
@tekton-robot
Copy link
Collaborator

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/reconciler/events/cloudevent/cloud_event_controller.go 94.6% 92.3% -2.3
pkg/reconciler/events/cloudevent/cloudevent.go 88.2% 90.6% 2.3
pkg/reconciler/events/cloudevent/cloudeventclient.go 7.1% 26.7% 19.5
pkg/reconciler/events/cloudevent/cloudeventsfakeclient.go 50.0% 54.5% 4.5
pkg/reconciler/taskrun/taskrun.go 77.6% 75.9% -1.7

@afrittoli afrittoli force-pushed the issues/2082-add-cloud-events branch from d7c6327 to 817a82e Compare June 19, 2020 10:34
@tekton-robot
Copy link
Collaborator

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/reconciler/events/cloudevent/cloud_event_controller.go 94.6% 92.3% -2.3
pkg/reconciler/events/cloudevent/cloudevent.go 88.2% 90.6% 2.3
pkg/reconciler/events/cloudevent/cloudeventclient.go 7.1% 26.7% 19.5
pkg/reconciler/events/cloudevent/cloudeventsfakeclient.go 50.0% 54.5% 4.5
pkg/reconciler/taskrun/taskrun.go 77.6% 75.9% -1.7

@afrittoli afrittoli force-pushed the issues/2082-add-cloud-events branch from 817a82e to b08f7c7 Compare June 19, 2020 12:30
@tekton-robot
Copy link
Collaborator

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/reconciler/events/cloudevent/cloud_event_controller.go 94.6% 92.3% -2.3
pkg/reconciler/events/cloudevent/cloudevent.go 88.2% 90.6% 2.3
pkg/reconciler/events/cloudevent/cloudeventclient.go 7.1% 26.7% 19.5
pkg/reconciler/events/cloudevent/cloudeventsfakeclient.go 50.0% 54.5% 4.5
pkg/reconciler/taskrun/taskrun.go 77.6% 75.9% -1.7

@tekton-robot
Copy link
Collaborator

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/reconciler/events/cloudevent/cloud_event_controller.go 94.6% 92.3% -2.3
pkg/reconciler/events/cloudevent/cloudevent.go 88.2% 90.6% 2.3
pkg/reconciler/events/cloudevent/cloudeventclient.go 7.1% 26.7% 19.5
pkg/reconciler/events/cloudevent/cloudeventsfakeclient.go 50.0% 54.5% 4.5
pkg/reconciler/taskrun/taskrun.go 77.6% 75.8% -1.7

@afrittoli afrittoli force-pushed the issues/2082-add-cloud-events branch from 1b5cadd to 83beab3 Compare June 25, 2020 16:25
@afrittoli afrittoli changed the title WIP Start emitting CloudEvents for TaskRuns Start emitting CloudEvents for TaskRuns Jun 25, 2020
@tekton-robot tekton-robot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jun 25, 2020
@afrittoli afrittoli force-pushed the issues/2082-add-cloud-events branch from 59da6a3 to df55797 Compare July 1, 2020 17:16
@tekton-robot tekton-robot added size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. and removed size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Jul 1, 2020
@tekton-robot
Copy link
Collaborator

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/reconciler/events/cloudevent/cloud_event_controller.go 94.6% 92.7% -1.9
pkg/reconciler/events/cloudevent/cloudevent.go 88.2% 90.6% 2.3
pkg/reconciler/events/cloudevent/cloudeventclient.go 7.1% 37.5% 30.4
pkg/reconciler/events/cloudevent/cloudeventsfakeclient.go 50.0% 54.5% 4.5
pkg/reconciler/events/event.go 81.8% 40.9% -40.9

@afrittoli afrittoli force-pushed the issues/2082-add-cloud-events branch from df55797 to b4235c0 Compare July 1, 2020 20:21
@afrittoli afrittoli changed the title Start emitting CloudEvents for TaskRuns Start emitting CloudEvents for TaskRuns and PipelineRuns Jul 1, 2020
@tekton-robot tekton-robot added the do-not-merge/release-note-label-needed Indicates that a PR should not merge because it's missing one of the release note labels. label Jul 1, 2020
@tekton-robot
Copy link
Collaborator

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/reconciler/events/cloudevent/cloud_event_controller.go 94.6% 92.7% -1.9
pkg/reconciler/events/cloudevent/cloudevent.go 88.2% 90.6% 2.3
pkg/reconciler/events/cloudevent/cloudeventclient.go 7.1% 37.5% 30.4
pkg/reconciler/events/cloudevent/cloudeventsfakeclient.go 50.0% 54.5% 4.5
pkg/reconciler/events/event.go 81.8% 40.9% -40.9
pkg/reconciler/pipelinerun/pipelinerun.go 85.9% 85.9% -0.0
pkg/reconciler/taskrun/taskrun.go 76.5% 76.2% -0.3

@tekton-robot tekton-robot added release-note Denotes a PR that will be considered when it comes time to generate release notes. and removed do-not-merge/release-note-label-needed Indicates that a PR should not merge because it's missing one of the release note labels. labels Jul 1, 2020
@afrittoli afrittoli force-pushed the issues/2082-add-cloud-events branch from b4235c0 to c47c926 Compare July 1, 2020 20:42
@tekton-robot
Copy link
Collaborator

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/reconciler/events/cloudevent/cloud_event_controller.go 94.6% 92.7% -1.9
pkg/reconciler/events/cloudevent/cloudevent.go 88.2% 90.6% 2.3
pkg/reconciler/events/cloudevent/cloudeventclient.go 7.1% 37.5% 30.4
pkg/reconciler/events/cloudevent/cloudeventsfakeclient.go 50.0% 54.5% 4.5
pkg/reconciler/events/event.go 81.8% 40.9% -40.9
pkg/reconciler/pipelinerun/pipelinerun.go 86.0% 85.9% -0.0
pkg/reconciler/taskrun/taskrun.go 76.6% 76.3% -0.3

@afrittoli afrittoli removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jul 1, 2020
@afrittoli afrittoli force-pushed the issues/2082-add-cloud-events branch from c47c926 to a90bc08 Compare July 1, 2020 21:01
@tekton-robot
Copy link
Collaborator

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/reconciler/events/cloudevent/cloud_event_controller.go 94.6% 92.7% -1.9
pkg/reconciler/events/cloudevent/cloudevent.go 88.2% 90.6% 2.3
pkg/reconciler/events/cloudevent/cloudeventclient.go 7.1% 37.5% 30.4
pkg/reconciler/events/cloudevent/cloudeventsfakeclient.go 50.0% 54.5% 4.5
pkg/reconciler/events/event.go 81.8% 40.9% -40.9
pkg/reconciler/pipelinerun/pipelinerun.go 86.0% 85.9% -0.0
pkg/reconciler/taskrun/taskrun.go 76.6% 76.3% -0.3

@afrittoli
Copy link
Member Author

Should add unit tests to events to restore a good coverage. Also need to add cloud event testing to pipelinerun_test

@afrittoli afrittoli force-pushed the issues/2082-add-cloud-events branch from a90bc08 to d4bbe49 Compare July 2, 2020 09:39
@tekton-robot
Copy link
Collaborator

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/reconciler/events/cloudevent/cloud_event_controller.go 94.6% 92.7% -1.9
pkg/reconciler/events/cloudevent/cloudevent.go 88.2% 90.6% 2.3
pkg/reconciler/events/cloudevent/cloudeventclient.go 7.1% 37.5% 30.4
pkg/reconciler/events/cloudevent/cloudeventsfakeclient.go 50.0% 54.5% 4.5
pkg/reconciler/events/event.go 81.8% 86.4% 4.5
pkg/reconciler/pipelinerun/pipelinerun.go 86.0% 85.9% -0.0
pkg/reconciler/taskrun/taskrun.go 76.6% 76.3% -0.3

Copy link
Member

@vdemeester vdemeester left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/meow

@tekton-robot
Copy link
Collaborator

@vdemeester: cat image

In response to this:

/meow

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@tekton-robot
Copy link
Collaborator

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: vdemeester

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@tekton-robot tekton-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jul 2, 2020
Add a new method 'SendCloudEventWithRetries' to the cloud events
controller. It that allows emitting cloud events asynchronously
(in a go routine), taking benefit of the retries capabilities of
the cloudevents go sdk.

Rework the fake client to allow for unit testing of
'SendCloudEventWithRetries'. The new implementation is similar
to that of the fake recorder from client-go, which allows to
unit test k8s events and cloud events in a similar fashion.

Add a new config option default-cloud-events-sink in the
defaults config map. This options allows setting a default sink
for cloud events. If the default sink is setup, cloud events are
sent, else they're disabled.

Invoke 'SendCloudEventWithRetries' from the events modules so
that we start sending cloud events in the same places we were
send k8s events (except for errors).

Most of the plumbing is in place to start emitting cloud events
for pipeline runs too, however there is a small refactor for
the config maps required before we can add that, so this commit
only enabled cloud events for task runs for now.
@afrittoli afrittoli force-pushed the issues/2082-add-cloud-events branch from d4bbe49 to 84a60bd Compare July 2, 2020 14:42
@afrittoli afrittoli changed the title Start emitting CloudEvents for TaskRuns and PipelineRuns Start emitting CloudEvents for TaskRuns Jul 2, 2020
@tekton-robot
Copy link
Collaborator

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/reconciler/events/cloudevent/cloud_event_controller.go 94.6% 92.7% -1.9
pkg/reconciler/events/cloudevent/cloudevent.go 88.2% 90.6% 2.3
pkg/reconciler/events/cloudevent/cloudeventclient.go 7.1% 37.5% 30.4
pkg/reconciler/events/cloudevent/cloudeventsfakeclient.go 50.0% 54.5% 4.5
pkg/reconciler/events/event.go 81.8% 86.4% 4.5
pkg/reconciler/pipelinerun/pipelinerun.go 86.0% 85.9% -0.0
pkg/reconciler/taskrun/taskrun.go 76.6% 76.3% -0.3

Copy link
Contributor

@khrm khrm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

@tekton-robot tekton-robot added the lgtm Indicates that a PR is ready to be merged. label Jul 2, 2020
@tekton-robot tekton-robot merged commit 3866374 into tektoncd:master Jul 2, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. kind/feature Categorizes issue or PR as related to a new feature. lgtm Indicates that a PR is ready to be merged. release-note Denotes a PR that will be considered when it comes time to generate release notes. size/XL Denotes a PR that changes 500-999 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants