v1alpha1 e2e tests not being updated along with v1beta1 tests

[bobcatfish]

Expected Behavior

• When I look at the way an end to end test exercising the v1beta1 API compares to the same test for the same functionality against the v1alpha1 API, the test functionality should be the same
• When problems are found and fixed in end to end tests, they same fixes should be propogated to v1beta1 and v1alpha1 tests
• This should be made sustainable such that if we add another api version (e.g. v1beta2 or v1) we don't compound this issue

Actual Behavior

The tests have diverged and updates are happening to v1beta1 and not v1alpha1.

For example compare v1beta1 timeout_test.go and v1alpha1 timeout_test.go and recent fixes such as fb613dc

Additional Info

I wonder if we could set up these tests such that the same tests (when appropriate) can be run for both (all) api versions by passing the types in or creating some kind of agnostic builders.

[tekton-robot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale with a justification.
Stale issues rot after an additional 30d of inactivity and eventually close.
If this issue is safe to close now please do so with /close with a justification.
If this issue should be exempted, mark the issue as frozen with /lifecycle frozen with a justification.

/lifecycle stale

Send feedback to tektoncd/plumbing.

[bobcatfish]

/remove-lifecycle stale

i think we should find a way (if possible) to write our e2e tests such that the same test can be run for multiple API versions instead of having to duplicate the tests for each API

[vdemeester]

@bobcatfish I am not sure to entirely follow. v1alpha1 is "frozen" in that, we don't update it anymore. I don't really see any easy way to write e2e tests that can run multiple API versions as there is no guarantee (especially with v1alpha1) that a type can be convert from on to another.

The "framework" of test can make it as simple as possible to write test (aka removing any noise) — #3550 is a first step attempt at doing that — but other than "we should update tests in both v1beta1 and v1alpha1 if the fix on tests applies to both", I don't see what else to do.

[tekton-robot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale with a justification.
Stale issues rot after an additional 30d of inactivity and eventually close.
If this issue is safe to close now please do so with /close with a justification.
If this issue should be exempted, mark the issue as frozen with /lifecycle frozen with a justification.

/lifecycle stale

Send feedback to tektoncd/plumbing.

[tekton-robot]

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten with a justification.
Rotten issues close after an additional 30d of inactivity.
If this issue is safe to close now please do so with /close with a justification.
If this issue should be exempted, mark the issue as frozen with /lifecycle frozen with a justification.

/lifecycle rotten

Send feedback to tektoncd/plumbing.

[tekton-robot]

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen with a justification.
Mark the issue as fresh with /remove-lifecycle rotten with a justification.
If this issue should be exempted, mark the issue as frozen with /lifecycle frozen with a justification.

/close

Send feedback to tektoncd/plumbing.

[tekton-robot]

@tekton-robot: Closing this issue.

In response to this:

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen with a justification.
Mark the issue as fresh with /remove-lifecycle rotten with a justification.
If this issue should be exempted, mark the issue as frozen with /lifecycle frozen with a justification.

/close

Send feedback to tektoncd/plumbing.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[bobcatfish]

ah whoops, let this get rotten and close XD

v1alpha1 is "frozen" in that, we don't update it anymore

the API is frozen, but the logic in the tests may still need updates as we maintain them over time

/reopen
/remove-lifecycle rotten

[tekton-robot]

@bobcatfish: Reopened this issue.

In response to this:

ah whoops, let this get rotten and close XD

v1alpha1 is "frozen" in that, we don't update it anymore

the API is frozen, but the logic in the tests may still need updates as we maintain them over time

/reopen
/remove-lifecycle rotten

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[tekton-robot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale with a justification.
Stale issues rot after an additional 30d of inactivity and eventually close.
If this issue is safe to close now please do so with /close with a justification.
If this issue should be exempted, mark the issue as frozen with /lifecycle frozen with a justification.

/lifecycle stale

Send feedback to tektoncd/plumbing.

[tekton-robot]

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten with a justification.
Rotten issues close after an additional 30d of inactivity.
If this issue is safe to close now please do so with /close with a justification.
If this issue should be exempted, mark the issue as frozen with /lifecycle frozen with a justification.

/lifecycle rotten

Send feedback to tektoncd/plumbing.

[tekton-robot]

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen with a justification.
Mark the issue as fresh with /remove-lifecycle rotten with a justification.
If this issue should be exempted, mark the issue as frozen with /lifecycle frozen with a justification.

/close

Send feedback to tektoncd/plumbing.

[tekton-robot]

@tekton-robot: Closing this issue.

In response to this:

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen with a justification.
Mark the issue as fresh with /remove-lifecycle rotten with a justification.
If this issue should be exempted, mark the issue as frozen with /lifecycle frozen with a justification.

/close

Send feedback to tektoncd/plumbing.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.