Per-plugin callback functions for accurate requeueing in kube-scheduler

[sanposhiho]

Enhancement Description

• One-line enhancement description (can be used as a release note):
• Kubernetes Enhancement Proposal: https://github.com/kubernetes/enhancements/blob/master/keps/sig-scheduling/4247-queueinghint/README.md
• Discussion Link:
  • introduce QueueingHint for wise-enqueueing kubernetes#114297
  • https://kubernetes.slack.com/archives/C5P3FE08M/p1695639140018139?thread_ts=1694167948.846139&cid=C5P3FE08M
• Primary contact (assignee): @sanposhiho @macsko
• Responsible SIGs: SIG-Scheduling
• Enhancement target (which target equals to which milestone):
  • Alpha release target (x.y): v1.26 (This KEP stems from KEP-3063)
  • Beta release target (x.y): v1.28　(We've actually had this feature already by default in v1.28)
  • Stable release target (x.y): v1.34
• Alpha: This KEP stems from KEP-3063.
• Beta (v1.28)
  • KEP (k/enhancements) update PR(s): add(KEP-4247): Per-plugin callback functions for efficient enqueueing in the scheduling queue #4256
  • Code (k/k) update PR(s): feature(scheduler): implement ClusterEventWithHint to filter out useless events kubernetes#118551
  • Docs (k/website) update(s): Modify SchedulerQueueingHints website#43427
• Beta (v1.32)
  • KEP (k/enhancements) update PR(s): KEP-4247: Update criteria targeting to enable the QueueingHints by default #4877
  • Code (k/k) requirement: Beta requirement: QueueingHint kubernetes#122597
  • Code (k/k) update PR(s):
  • Docs (k/website) update(s): Promote SchedulerQueueingHints to beta in 1.32 website#48406
• Stable
  • KEP (k/enhancements) update PR(s):
  • Code (k/k) update PR(s):
  • Docs (k/website) update(s):

Please keep this description up to date. This will help the Enhancement Team to track the evolution of the enhancement efficiently.

[sanposhiho]

/sig scheduling

It was suggested we have a small KEP for QueueingHint. It's kind of a special case though, we can assume DRA is the parent KEP and this KEP stems from it. And I set the alpha version v1.26 which is the same as DRA KEP (or maybe we can just leave it as n/a), and the beta version v1.28 which we actually implemented it and enable it via the beta feature flag (enabled by default).

you are essentially splitting one KEP into two. So there is no grade-skipping as such as the grades were part of the original KEP. So we do know when the feature was alpha and when it went into beta from alpha etc. So please go ahead with a smaller KEP for SchedulerQueueingHints and use the dates from before.
https://kubernetes.slack.com/archives/C5P3FE08M/p1695639140018139?thread_ts=1694167948.846139&cid=C5P3FE08M

@kubernetes/sig-scheduling-leads Can anyone give this PR lead-opted-in?

[alculquicondor]

Do you have a PR for this already?

[sanposhiho]

Not yet. Will be ready probably within this weekend.

[sanposhiho]

Here it is: #4256

[Huang-Wei]

/label lead-opted-in

[rayandas]

Hello @sanposhiho 👋, v1.29 Enhancements team here.

Just checking in as we approach enhancements freeze on 01:00 UTC, Friday, 6th October, 2023.

This enhancement is targeting for stage beta for v1.29 (correct me, if otherwise)

Here's where this enhancement currently stands:

• KEP readme using the latest template has been merged into the k/enhancements repo.
• KEP status is marked as implementable for latest-milestone: 1.29.
• KEP readme has a updated detailed test plan section filled out
• KEP readme has up to date graduation criteria
• KEP has a production readiness review that has been completed and merged into k/enhancements.

For this KEP, we would just need to update the following:

• Update and merge PR add(KEP-4247): Per-plugin callback functions for efficient enqueueing in the scheduling queue #4256 should be good.

The status of this enhancement is marked as at risk for enhancement freeze. Please keep the issue description up-to-date with appropriate stages as well. Thank you!

[npolshakova]

Hello 👋, 1.29 Enhancements Lead here.
Unfortunately, this enhancement did not meet requirements for v1.29 enhancements freeze.
Feel free to file an exception to add this back to the release tracking process. Thanks!

/milestone clear

[npolshakova]

Hey again 👋
As #4256 was merged by the additional time approved in the exception request, I am adding this back to v1.29 milestone and changing the status of this enhancement to tracked for enhancement freeze 🚀

/milestone v1.29

[alculquicondor]

/remove-lifecycle stale

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue as fresh with /remove-lifecycle stale
• Close this issue with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[sanposhiho]

/remove-lifecycle stale

[alculquicondor]

/label lead-opted-in

We are going for another beta iteration, re-enabling the feature gate to true by default

[tjons]

Hello @macsko @alculquicondor @sanposhiho 👋, Enhancements team here.

Just checking in as we approach enhancements freeze on 02:00 UTC Friday 11th October 2024 / 19:00 PDT Thursday 10th October 2024.

This enhancement is targeting for stage beta for v1.32 (correct me, if otherwise).

Here's where this enhancement currently stands:

• KEP readme using the latest template has been merged into the k/enhancements repo.
• KEP status is marked as implementable for latest-milestone: v1.32.
• KEP readme has up-to-date graduation criteria
• KEP has a production readiness review that has been completed and merged into k/enhancements. (For more information on the PRR process, check here). If your production readiness review is not completed yet, please make sure to fill the production readiness questionnaire in your KEP by the PRR Freeze deadline on Thursday, October 3rd, 2024 so that the PRR team has enough time to review your KEP.

For this KEP, we would just need to update the following:

• KEP status is marked as implementable for latest-milestone: v1.32. This should be done when KEP-4247: Update criteria targeting to enable the QueueingHints by default #4877 merges.

The status of this enhancement is marked as at risk for enhancement freeze. Please keep the issue description up-to-date with appropriate stages as well.

If you anticipate missing enhancements freeze, you can file an exception request in advance. Thank you!

[alculquicondor]

KEP status is marked as implementable for latest-milestone: v1.32

#4877 already satisfies this criteria

[tjons]

With all the requirements met, this enhancement is now tracked for enhancements freeze! 🚀

[AnshumanTripathi]

Hello @sanposhiho 👋, 1.32 Docs Shadow here.
Does this enhancement work planned for 1.32 require any new docs or modification to existing docs?
If so, please follows the steps here to open a PR against dev-1.32 branch in the k/website repo. This PR can be just a placeholder at this time and must be created before Thursday October 24th 2024 18:00 PDT.
Also, take a look at Documenting for a release to get yourself familiarize with the docs requirement for the release.
Thank you!

[sanposhiho]

Created: kubernetes/website#48406

[sanposhiho]

Also, it'd be great to have a blog post for this.
Actually, we have a previous draft that we didn't publish at a previous release kubernetes/website#43686

[wrkode]

👋 Hi there, William here from v1.32 Comms
We'd love for you to consider writing a feature blog about your enhancement! Some reasons why you might want to write a blog for this feature include (but are not limited to) if this introduces breaking changes, is important to our users, or has been in progress for a long time and is graduating.

To opt-in, let us know and open a Feature Blog placeholder PR against the website repository by 30th Oct 2024. For more information about writing a blog see the blog contribution guidelines.

Note: In your placeholder PR, use XX characters for the blog date in the front matter and file name. We will work with you on updating the PR with the publication date once we have a final number of feature blogs for this release.

[sanposhiho]

@wrkode

Created: kubernetes/website#48541

[wrkode]

@sanposhiho

Thanks a million

[tjons]

Hello @sanposhiho 👋, Enhancements team here.

Are there any code changes planned for this KEP in 1.32? I am marking this enhancement as at risk for code freeze until I hear back from you, if no changes are planned. If changes are planned, please add them to the issue description. Thanks!

[sanposhiho]

@tjons Here's our all necessary requirements for the beta graduation. Basically, the remaining part is only testing.
I just put this to the issue description too.
kubernetes/kubernetes#122597

[tjons]

@sanposhiho ok, great! As we approach code freeze at 02:00 UTC Friday 8th November 2024 / 19:00 PDT Thursday 7th November 2024, here's where this enhancement currently stands:

• All PRs to the Kubernetes repo that are related to your enhancement are linked in the above issue description (for tracking purposes).
• All PR/s are ready to be merged (they have approved and lgtm labels applied) by the code freeze deadline. This includes tests.

For this enhancement, it looks like the following PRs are open and need to be merged before code freeze:

• Beta requirement: QueueingHint kubernetes#122597

Additionally, please let me know if there are any other PRs in k/k not listed in the description or not linked with this GitHub issue that we should track for this KEP, so that we can maintain accurate status.

The status of this enhancement is marked as at risk for code freeze.

If you anticipate missing code freeze, you can file an exception request in advance. Thank you!

[tjons]

Hello @sanposhiho 👋, Enhancements team here

With all the implementation (code related) PRs merged as per the issue description:

• Beta requirement: QueueingHint kubernetes#122597

This enhancement is now marked as tracked for code freeze for the v1.32 Code Freeze!

Please note that KEPs targeting stable need to have the status field marked as implemented in the kep.yaml file after code PRs are merged and the feature gates are removed.