[MultiKueue] Promote MultiKueue API and feature gate to Beta

[mimowo]

What would you like to be added:

Promote the MultiKueue API to Beta.

Why is this needed:

For the ease of use.

The feature was first introduced in 0.6, and since then a bunch of issues have been fixed.

Completion requirements:

Issues I would like to get addressed before graduating (except from other graduation criteria mentioned in KEP):

• [ClusterQueue] Provide more details about the mis-configuration in the message for Active=False #3099
• [MultiKueue] AdmissionCheck conditions are confusing #3094

This enhancement requires the following artifacts:

• Design doc
• API change
• Docs update

The artifacts should be linked in subsequent comments.

[mimowo]

/cc @mwielgus @tenzen-y @alculquicondor @trasc

I think we should resolve #3094 before the gradution, but discussion is welcome.

[trasc]

/assign

[tenzen-y]

@mimowo @trasc Could we finalize #2458 before we graduate MK?
I am suspecting that the new functional requirements will be found by #2458.

[mimowo]

Thanks for the comment, I was thinking about it, but I think we don't need to block on it, here is the couple of thoughts I have:

• it is very unlikely we will be able to finalize #2458 before end of November, given the load of other workstreams. The KEP proved to be challenging, and we need a proper design for it, which I think includes decoupling Job handling and ACs.
• supporting external Jobs may will likely require new APIs, but I believe we can grow it later step by step, nothing in the current API seems would be blocking the support
• in the past we supported built-in Jobs in Kueue (initially just batch Job) as Beta API, and only later we added Job framwork and expanded and support external Jobs. I believe we can repeat the pattern here, while already in Beta.
• we are already getting user feedback that the current API is useful for JobSet, but the setup still requires many steps

EDIT: also, the support for external Jobs isn't mentioned in the KEP, so I believe we can de-couple it.

[tenzen-y]

I discussed this graduation with @mimowo offline.
During the discussion, we agreed that external Job support would take a long time since #2458 has not been finalized and is pending status.

Additionally, I guess that this graduation would be useful for collecting feedback from users, and we can reflect the feedback to the external Job support. Based on user feedback, we will prepare the second beta version to support additional features and external Jobs.

Anyway, I agree with this beta graduation, but for the GA graduation, I would like to wait for external Job support.

[mimowo]

/close
The remaining work is tracked in separate issue #3094

[k8s-ci-robot]

@mimowo: Closing this issue.

In response to this:

/close
The remaining work is tracked in separate issue #3094

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.