Reset all AdmissionChecks on Eviction to Pending

[PBundyra]

What type of PR is this?

/kind bug

What this PR does / why we need it:

Currently, when AdmissionCheck is in Retry state, the Workloads gets evicted and there's no actual retry. The changes improves that flow, changing all AdmissionChecks to Pending if one of them is in Retry state (unless there's a Rejected AdmissionCheck).

Which issue(s) this PR fixes:

Fixes #3322

Special notes for your reviewer:

Does this PR introduce a user-facing change?

Workload is requeued with all AdmissionChecks set to Pending if there was an AdmissionCheck in Retry state.

[PBundyra]

/assign @mimowo

[netlify]

✅ Deploy Preview for kubernetes-sigs-kueue canceled.

Name	Link
🔨 Latest commit	1451b4e
🔍 Latest deploy log	https://app.netlify.com/sites/kubernetes-sigs-kueue/deploys/671f97c1d83c5100085ca0aa

[mimowo]

I think we actually need to reset the checks regardless of the reason of eviction. For example, if the checks passed, but the workload is evicted due to WaitForPodsReady?

[mimowo]

When deactivating (evicting with deactivation reason) a workload we reset the requeueState so that it is "clean" or re-activation:

kueue/pkg/controller/core/workload_controller.go

Lines 203 to 206 in 42c7b47

	if wl.Status.RequeueState != nil {
	wl.Status.RequeueState = nil
	updated = true
	}

I think for consistency we should do the same with checks, but we can re-visit this as a follow up.

/lgtm
/approve
/hold cancel

[k8s-ci-robot]

LGTM label has been added.

Git tree hash: 2c3dd1a32282d6a662b4c6c8192d1f63d446f7e9

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: mimowo, PBundyra

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [mimowo]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[tenzen-y]

It seems that this is obviously bug.
/cherry-pick release-0.8

[k8s-infra-cherrypick-robot]

@tenzen-y: #3323 failed to apply on top of branch "release-0.8":

Applying: Make AdmissionChecks Pending after one is in Retry state
Applying: Reduce number of patch requests
Using index info to reconstruct a base tree...
M	pkg/controller/core/workload_controller.go
M	pkg/scheduler/scheduler.go
M	pkg/workload/admissionchecks.go
M	pkg/workload/workload.go
Falling back to patching base and 3-way merge...
Auto-merging pkg/workload/workload.go
CONFLICT (content): Merge conflict in pkg/workload/workload.go
Auto-merging pkg/workload/admissionchecks.go
Auto-merging pkg/scheduler/scheduler.go
CONFLICT (content): Merge conflict in pkg/scheduler/scheduler.go
Auto-merging pkg/controller/core/workload_controller.go
error: Failed to merge in the changes.
hint: Use 'git am --show-current-patch=diff' to see the failed patch
hint: When you have resolved this problem, run "git am --continue".
hint: If you prefer to skip this patch, run "git am --skip" instead.
hint: To restore the original branch and stop patching, run "git am --abort".
hint: Disable this message with "git config advice.mergeConflict false"
Patch failed at 0002 Reduce number of patch requests

In response to this:

It seems that this is obviously bug.
/cherry-pick release-0.8

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[mimowo]

/cherry-pick release-0.8

I guess we could cherry-pick but then we also need to cherry-pick the forced-ownership PR which might be more that we want.