Start JobFramework controller and webhook on new CRDs availability

[ChristianZaccaria]

What type of PR is this?

/kind feature

What this PR does / why we need it:

Add function to wait for API availability using the RESTMapper in controller manager.

This PR checks for availability of JobFramework integrations in the Kubernetes cluster before executing the action().

The function utilises the RESTMapper from the controller-runtime manager to verify the existence of the CRD by its GroupVersionKind (GVK). If one of the CRDs become available, the respective controller and webhook will start.

The user/dev will no longer be required to restart the Kueue pod to allow for integration on new CRDs available.

No restarts on the Kueue pod is performed.

Which issue(s) this PR fixes:

Fixes #2414

Special notes for your reviewer:

Does this PR introduce a user-facing change?

Detect and enable support for job CRDs installed after Kueue starts.

[k8s-ci-robot]

Hi @ChristianZaccaria. Thanks for your PR.

I'm waiting for a kubernetes-sigs member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[netlify]

✅ Deploy Preview for kubernetes-sigs-kueue ready!

Name	Link
🔨 Latest commit	86e480d
🔍 Latest deploy log	https://app.netlify.com/sites/kubernetes-sigs-kueue/deploys/66c8e26c405f47000744dd52
😎 Deploy Preview	https://deploy-preview-2574--kubernetes-sigs-kueue.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

---

To edit notification comments on pull requests, go to your Netlify site configuration.

[ChristianZaccaria]

cc: @trasc thank you for your previous input on this.

[mbobrovskyi]

/ok-to-test

[KPostOffice]

Nice, I tested this out on a cluster and it worked as expected for me

[ChristianZaccaria]

Rebased

[Ygnas]

Works as expected.

[ChristianZaccaria]

Hi @trasc, wondering what are the next steps for this PR to be successful and merged? Thank you.

[ChristianZaccaria]

Rebased

[alculquicondor]

/lgtm
/approve

[alculquicondor]

nit: this line is not needed

[k8s-ci-robot]

LGTM label has been added.

Git tree hash: a08aaa26c555df954eb00e2fbe84214100690f0a

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: alculquicondor, ChristianZaccaria, Ygnas

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• cmd/OWNERS [alculquicondor]
• pkg/controller/OWNERS [alculquicondor]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[mimowo]

@tenzen-y @alculquicondor as we are about to prepare the 0.8.1 release - how do you find cherry-picking this PR?

I feel this issue is on a boundary between a feature and a bug. It could be considered a feature, because we didn't plan for it up front. OTOH, I participated in some debugging sessions for users, and I think the users' perception was always that this is a bug.

Finally, the change does not introduce any API changes, so I would suggest considering it for cherry-picking.

[tenzen-y]

OTOH, I participated in some debugging sessions for users, and I think the users' perception was always that this is a bug.

Thank you for sharing this experience. In that case, I agree with cherry-picking.

[mimowo]

/cherry-pick release-0.8

[k8s-infra-cherrypick-robot]

@mimowo: #2574 failed to apply on top of branch "release-0.8":

Applying: Start Controller and Webhook on new CRDs availability
Applying: Log errors from JobFramework controller and webhook
Applying: Add test case for delayed JobFramework API becoming available
Applying: Wait for API integration to be enabled
Applying: Implement synchronization for safe concurrent access
Using index info to reconstruct a base tree...
M	pkg/controller/jobframework/integrationmanager.go
M	pkg/controller/jobframework/setup.go
M	pkg/controller/jobframework/setup_test.go
Falling back to patching base and 3-way merge...
Auto-merging pkg/controller/jobframework/setup_test.go
Auto-merging pkg/controller/jobframework/setup.go
Auto-merging pkg/controller/jobframework/integrationmanager.go
Applying: Implement exponential backoff for waitForAPI() and add function type to streamline REST mapping checks
Using index info to reconstruct a base tree...
M	pkg/controller/jobframework/setup.go
M	pkg/controller/jobframework/setup_test.go
Falling back to patching base and 3-way merge...
Auto-merging pkg/controller/jobframework/setup_test.go
Auto-merging pkg/controller/jobframework/setup.go
Applying: Add Job Framework API to mapper directly
Using index info to reconstruct a base tree...
M	pkg/controller/jobframework/integrationmanager.go
M	pkg/controller/jobframework/integrationmanager_test.go
M	pkg/controller/jobframework/setup.go
M	pkg/controller/jobframework/setup_test.go
Falling back to patching base and 3-way merge...
Auto-merging pkg/controller/jobframework/setup_test.go
Auto-merging pkg/controller/jobframework/setup.go
Auto-merging pkg/controller/jobframework/integrationmanager_test.go
CONFLICT (content): Merge conflict in pkg/controller/jobframework/integrationmanager_test.go
Auto-merging pkg/controller/jobframework/integrationmanager.go
CONFLICT (content): Merge conflict in pkg/controller/jobframework/integrationmanager.go
error: Failed to merge in the changes.
hint: Use 'git am --show-current-patch=diff' to see the failed patch
Patch failed at 0007 Add Job Framework API to mapper directly
When you have resolved this problem, run "git am --continue".
If you prefer to skip this patch, run "git am --skip" instead.
To restore the original branch and stop patching, run "git am --abort".

In response to this:

/cherry-pick release-0.8

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[mimowo]

There is a conflict in integrationmanager. @ChristianZaccaria would you like to prepare the cherry-pick manually? You can use the https://github.com/kubernetes-sigs/kueue/blob/main/hack/cherry_pick_pull.sh script

[ChristianZaccaria]

Thank you, on it now.

[ChristianZaccaria]

For reference, here is the generated PR: #2991 Thanks!

[mimowo]

To x-reference the follow up issues:

• [Discussion] Enhance/optimise current implementation of processing newly added Job Frameworks. #2837
• Add integration tests for detecting CRD installed after Kueue started #3015