Add Tekton-owned Step, StepTemplate, and Sidecar #4778

lbernick · 2022-04-19T19:49:10Z

Changes

This commit copies the Container struct from Kubernetes into Tekton into the
existing structs Task.Step.Container, Task.Sidecar.Container, and Task.StepTemplate.
This change breaks the Tekton Go libraries, but does not require any changes to the yaml
that users provide. This should not change anything for users who do not use the Go libraries.

The reason for this change is to have more control over our Task API. For example,
we may want to modify the Container fields we support (e.g. #4737), or change fields to string types
to allow them to be parameterized (e.g. #4080). This commit is the first step towards being able to make changes
to parts of the Task API embedded in Container.

Part 1 of #4737

/kind misc

Submitter Checklist

As the author of this PR, please check off the items in this checklist:

n/a Docs included if any changes are user facing
Tests included if any functionality added or changed
Follows the commit message standard
Meets the Tekton contributor standards (including
functionality, content, code)
Release notes block below has been filled in
(if there are no user facing changes, use release note "NONE")

Release Notes

[Change to Go libraries]: Task.Step, Task.StepTemplate, and Task.Sidecar use Container fields directly instead of embedding the Container struct

tekton-robot · 2022-04-19T19:51:42Z

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/apis/pipeline/v1beta1/container_types.go	Do not exist	0.0%

tekton-robot · 2022-04-19T20:36:58Z

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/apis/pipeline/v1beta1/container_types.go	Do not exist	0.0%

tekton-robot · 2022-04-19T21:12:57Z

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/apis/pipeline/v1beta1/container_types.go	Do not exist	0.0%

tekton-robot · 2022-04-19T21:13:45Z

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/apis/pipeline/v1beta1/container_types.go	Do not exist	0.0%

tekton-robot · 2022-04-19T22:05:23Z

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/apis/pipeline/v1beta1/container_types.go	Do not exist	0.0%

tekton-robot · 2022-04-19T22:27:29Z

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/apis/pipeline/v1beta1/container_types.go	Do not exist	0.0%

vdemeester

Really quick review but few questions:

Would it make sense to have two different struct for sidecars and steps ? (sidecars container would be step container + some things like readinessprobe, *probe …)
I see v1alpha1.Container, any reason for changing code in v1alpha1 ? (I guess it might be because we "share" the Step struct)

lbernick · 2022-04-20T12:13:05Z

Would it make sense to have two different struct for sidecars and steps ? (sidecars container would be step container + some things like readinessprobe, *probe …)

Yes that's definitely where I see this going but I didn't want to remove any fields of the Step container yet (might need a TEP). But I can create two identical structs for sidecars and steps that we can later change, what do you think?

I see v1alpha1.Container, any reason for changing code in v1alpha1 ? (I guess it might be because we "share" the Step struct)

Yeah pretty much because we share the Step struct. I could update the v1alpha1 Step to just copy over all the fields of the v1beta1 step but then we have to keep them in sync. Open to doing that though.

tekton-robot · 2022-04-20T13:42:33Z

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/apis/pipeline/v1beta1/container_types.go	Do not exist	0.0%

abayer · 2022-04-22T14:07:02Z

fwiw, I'd lean towards copying over to v1alpha1 normally, but I'm not sure if it's worth it given that TEP-0105 is starting to come together for actually removing v1alpha1. Given that, the additional work involved in copying over and keeping in sync for the time being may not be worth it.

tekton-robot · 2022-04-27T17:08:13Z

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/apis/pipeline/v1beta1/container_types.go	Do not exist	0.0%

imjasonh · 2022-04-28T04:32:15Z

I didn't want to remove any fields of the Step container yet (might need a TEP)

Happy to discuss if folks disagree, but in general I don't think deprecating a "feature" like, e.g., readinessProbes on a step container, should require a TEP. We should signal deprecations of these fields, wait 1+ release, and remove them. Especially so for things like terminationMessage which just silently don't do anything -- there's no way to use that correctly today.

If we think eventually we'd like to get to a spot where the Step and Sidecar types just have all the fields they reasonably could need and don't embed a Container type (even if it's owned by us), then I think we should just go straight to that and not incur the churn on ourselves and Go-API-consumers in having to account for two breaking type changes.

tekton-robot · 2022-04-28T18:53:09Z

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/apis/pipeline/v1beta1/container_types.go	Do not exist	100.0%

lbernick · 2022-04-28T18:53:45Z

fwiw, I'd lean towards copying over to v1alpha1 normally, but I'm not sure if it's worth it given that TEP-0105 is starting to come together for actually removing v1alpha1. Given that, the additional work involved in copying over and keeping in sync for the time being may not be worth it.

That's fair-- I wasn't sure if it would be more work to continue to alias v1alpha1 types to v1beta1 types, or to copy the old v1beta1 struct over to v1alpha1 so I wouldn't have to modify the v1alpha1 tests. I already did most of the grunt work though for pipelines -- when you're talking about the work to support this are you anticipating that this will be more difficult for the CLI team?

I didn't want to remove any fields of the Step container yet (might need a TEP)

Happy to discuss if folks disagree, but in general I don't think deprecating a "feature" like, e.g., readinessProbes on a step container, should require a TEP. We should signal deprecations of these fields, wait 1+ release, and remove them. Especially so for things like terminationMessage which just silently don't do anything -- there's no way to use that correctly today.

If we think eventually we'd like to get to a spot where the Step and Sidecar types just have all the fields they reasonably could need and don't embed a Container type (even if it's owned by us), then I think we should just go straight to that and not incur the churn on ourselves and Go-API-consumers in having to account for two breaking type changes.

Point taken! I'd like to keep the scope of this PR to just swapping over the type definitions, and not deprecating anything. However, I created separate structs for Step and Sidecar containers to hopefully minimize churn. There's still a lot of copypasta so LMK if you'd prefer to see it done differently.

lbernick · 2022-04-28T19:35:25Z

tests pass! 💪

/assign @vdemeester
/assign @imjasonh

lbernick · 2022-04-29T15:03:34Z

sounds good-- I think this PR should be good to go then!

tekton-robot · 2022-04-29T15:06:47Z

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/apis/pipeline/v1beta1/container_types.go	Do not exist	100.0%

imjasonh · 2022-04-29T15:11:28Z

sounds good-- I think this PR should be good to go then!

I think with this latest change we should just drop the StepContainer and SidecarContainer structs, and move the fields directly into Step and Sidecar. WDYT?

lbernick · 2022-04-29T15:18:51Z

sounds good-- I think this PR should be good to go then!

I think with this latest change we should just drop the StepContainer and SidecarContainer structs, and move the fields directly into Step and Sidecar. WDYT?

Oh my bad, I understand now what you were suggesting. Will do, I'll comment here when I've made that change.

lbernick · 2022-04-29T15:27:48Z

I'm thinking of keeping StepContainer so it can be reused in StepTemplate, but copying the container fields into sidecar as well, thoughts?

imjasonh · 2022-04-29T15:41:25Z

I'm thinking of keeping StepContainer so it can be reused in StepTemplate, but copying the container fields into sidecar as well, thoughts?

Good question. Are there fields that make sense in a Step but not a StepTemplate, or vice versa? Can the type of StepTemplate just be Step?

vdemeester · 2022-04-29T16:16:16Z

pkg/apis/pipeline/v1beta1/task_types.go

@@ -139,7 +139,7 @@ type TaskResult struct {
 // Step embeds the Container type, which allows it to include fields not
 // provided by Container.
 type Step struct {
-	corev1.Container `json:",inline"`
+	StepContainer `json:",inline"`


The idea was to move all that is to not have this inlined/embedded struct but have the field directly in. So no need for StepContainer.

vdemeester · 2022-04-29T16:16:26Z

pkg/apis/pipeline/v1beta1/task_types.go

@@ -172,7 +172,7 @@ type Step struct {

 // Sidecar has nearly the same data structure as Step, consisting of a Container and an optional Script, but does not have the ability to timeout.
 type Sidecar struct {
-	corev1.Container `json:",inline"`
+	SidecarContainer `json:",inline"`


The idea was to move all that is to not have this inlined/embedded struct but have the field directly in. So no need for SidecarContainer.

lbernick · 2022-04-29T17:46:07Z

I'm thinking of keeping StepContainer so it can be reused in StepTemplate, but copying the container fields into sidecar as well, thoughts?

Good question. Are there fields that make sense in a Step but not a StepTemplate, or vice versa? Can the type of StepTemplate just be Step?

I think "Name" wouldn't make sense in StepTemplate since steps can't have the same name. In addition, "timeout" is not currently present in StepTemplate, but it could make sense. so maybe these fields should be directly copied into stepTemplate as well, thoughts?

imjasonh · 2022-04-29T18:04:36Z

I think "Name" wouldn't make sense in StepTemplate since steps can't have the same name. In addition, "timeout" is not currently present in StepTemplate, but it could make sense. so maybe these fields should be directly copied into stepTemplate as well, thoughts?

Thanks for looking into it. It sounds like we should just copy these between the types that need them. I'm not sure we should add timeout to StepTemplate in this PR, but it's something worth keeping in mind for a future change.

lbernick · 2022-04-29T18:16:07Z

I think "Name" wouldn't make sense in StepTemplate since steps can't have the same name. In addition, "timeout" is not currently present in StepTemplate, but it could make sense. so maybe these fields should be directly copied into stepTemplate as well, thoughts?

Thanks for looking into it. It sounds like we should just copy these between the types that need them. I'm not sure we should add timeout to StepTemplate in this PR, but it's something worth keeping in mind for a future change.

Agreed! I don't want to change the user facing API at all with this PR-- just mentioning timeout as something that could be added in the future.

tekton-robot · 2022-04-29T20:25:02Z

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/apis/pipeline/v1beta1/container_types.go	Do not exist	100.0%
pkg/apis/pipeline/v1beta1/merge.go	81.8%	82.4%	0.5
pkg/reconciler/taskrun/resources/apply.go	99.2%	99.2%	0.0

tekton-robot · 2022-04-29T20:51:35Z

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/apis/pipeline/v1beta1/container_types.go	Do not exist	100.0%
pkg/apis/pipeline/v1beta1/merge.go	81.8%	82.4%	0.5
pkg/reconciler/taskrun/resources/apply.go	99.2%	99.2%	0.0

lbernick · 2022-04-29T20:52:30Z

@vdemeester @imjasonh I've addressed your feedback

tekton-robot · 2022-04-29T21:27:38Z

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/apis/pipeline/v1beta1/container_types.go	Do not exist	100.0%
pkg/apis/pipeline/v1beta1/merge.go	81.8%	82.4%	0.5
pkg/reconciler/taskrun/resources/apply.go	99.2%	99.2%	0.0

This commit copies the Container struct from Kubernetes into Tekton into the existing structs Task.Step.Container, Task.Sidecar.Container, and Task.StepTemplate. This change breaks the Tekton Go libraries, but does not require any changes to the yaml that users provide. This should not change anything for users who do not use the Go libraries. The reason for this change is to have more control over our Task API. For example, we may want to modify the Container fields we support, or change fields to string types to allow them to be parameterized. This commit is the first step towards being able to make changes to parts of the Task API embedded in Container.

tekton-robot · 2022-04-29T22:19:35Z

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/apis/pipeline/v1beta1/container_types.go	Do not exist	100.0%
pkg/apis/pipeline/v1beta1/merge.go	81.8%	82.4%	0.5
pkg/reconciler/taskrun/resources/apply.go	99.2%	99.2%	0.0

imjasonh

/lgtm

Thanks for this!

lbernick · 2022-05-02T12:43:07Z

Thanks for this!

Happy to help!

/test pull-tekton-pipeline-alpha-integration-tests
/test pull-tekton-pipeline-integration-tests

vdemeester

/meow

tekton-robot · 2022-05-03T04:51:09Z

@vdemeester:

In response to this:

/meow

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

tekton-robot · 2022-05-03T04:51:14Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: vdemeester

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [vdemeester]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

tekton-robot added release-note Denotes a PR that will be considered when it comes time to generate release notes. kind/misc Categorizes issue or PR as a miscellaneuous one. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. labels Apr 19, 2022

tekton-robot requested review from dlorenc and imjasonh April 19, 2022 19:49

lbernick force-pushed the container branch from 651e88a to 559c2b8 Compare April 19, 2022 20:34

lbernick force-pushed the container branch 2 times, most recently from 595e908 to a0e12f2 Compare April 19, 2022 21:11

lbernick force-pushed the container branch from a0e12f2 to 844ee09 Compare April 19, 2022 22:02

lbernick force-pushed the container branch from 844ee09 to 2f055ae Compare April 19, 2022 22:24

vdemeester reviewed Apr 20, 2022

View reviewed changes

lbernick force-pushed the container branch from 2f055ae to 148c66c Compare April 20, 2022 13:39

tekton-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Apr 25, 2022

lbernick force-pushed the container branch from 148c66c to ba62b18 Compare April 27, 2022 17:05

tekton-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Apr 27, 2022

lbernick force-pushed the container branch from ba62b18 to 1739621 Compare April 28, 2022 18:50

vdemeester requested changes Apr 29, 2022

View reviewed changes

lbernick force-pushed the container branch from d4f9a6b to 385a43e Compare April 29, 2022 20:23

lbernick force-pushed the container branch from 385a43e to 2a67927 Compare April 29, 2022 20:49

lbernick changed the title ~~Add Tekton-owned Container struct~~ Add Tekton-owned Step, StepTemplate, and Sidecar Apr 29, 2022

lbernick force-pushed the container branch from 2a67927 to 7ba591b Compare April 29, 2022 21:25

lbernick force-pushed the container branch from 7ba591b to 824108b Compare April 29, 2022 22:17

imjasonh reviewed Apr 30, 2022

View reviewed changes

tekton-robot added the lgtm Indicates that a PR is ready to be merged. label Apr 30, 2022

vdemeester approved these changes May 3, 2022

View reviewed changes

tekton-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label May 3, 2022

tekton-robot merged commit 7de70f1 into tektoncd:main May 3, 2022

lbernick deleted the container branch August 15, 2022 13:33

Add Tekton-owned Step, StepTemplate, and Sidecar #4778

Add Tekton-owned Step, StepTemplate, and Sidecar #4778

Conversation

lbernick commented Apr 19, 2022 • edited Loading

Changes

Submitter Checklist

Release Notes

tekton-robot commented Apr 19, 2022

tekton-robot commented Apr 19, 2022

tekton-robot commented Apr 19, 2022

tekton-robot commented Apr 19, 2022

tekton-robot commented Apr 19, 2022

tekton-robot commented Apr 19, 2022

vdemeester left a comment

Choose a reason for hiding this comment

lbernick commented Apr 20, 2022

tekton-robot commented Apr 20, 2022

abayer commented Apr 22, 2022

tekton-robot commented Apr 27, 2022

imjasonh commented Apr 28, 2022

tekton-robot commented Apr 28, 2022

lbernick commented Apr 28, 2022

lbernick commented Apr 28, 2022

lbernick commented Apr 29, 2022

tekton-robot commented Apr 29, 2022

imjasonh commented Apr 29, 2022

lbernick commented Apr 29, 2022

lbernick commented Apr 29, 2022

imjasonh commented Apr 29, 2022

vdemeester Apr 29, 2022

Choose a reason for hiding this comment

vdemeester Apr 29, 2022

Choose a reason for hiding this comment

lbernick commented Apr 29, 2022

imjasonh commented Apr 29, 2022

lbernick commented Apr 29, 2022

tekton-robot commented Apr 29, 2022

tekton-robot commented Apr 29, 2022

lbernick commented Apr 29, 2022

tekton-robot commented Apr 29, 2022

tekton-robot commented Apr 29, 2022

imjasonh left a comment

Choose a reason for hiding this comment

lbernick commented May 2, 2022

vdemeester left a comment

Choose a reason for hiding this comment

tekton-robot commented May 3, 2022

tekton-robot commented May 3, 2022

lbernick commented Apr 19, 2022 •

edited

Loading