Kubelet Resource Metrics Endpoint

[dashpole]

Enhancement Description

• One-line enhancement description: Introduce a new kubelet endpoint for serving metrics about first-class resources.

• Primary contact: @dashpole

• Responsible SIGs: Sig-Node, Sig-Instrumentation

• KEP: https://github.com/kubernetes/enhancements/tree/master/keps/sig-node/727-resource-metrics-endpoint

• Link to e2e and/or unit tests: Coming soon

• Reviewer(s) - (for LGTM) recommend having 2+ reviewers (at least one from code-area OWNERS file) agreed to review. Reviewers from multiple companies preferred: @DirectXMan12

• Approver: @dchen1107 @brancz

• Enhancement target (which target equals to which milestone):

  • Alpha release target 1.14
  • Beta release target 1.27
  • Stable release target 1.29

• v1alpha1 endpoint

  • KEP (k/enhancements) update PR(s): [KEP] Kubelet Resource Metrics Endpoint #726
  • Code (k/k) update PR(s): Add kubelet resource metrics v1alpha1 endpoint kubernetes#73946
  • Docs (k/website) update(s): Documentation for Kubelet Resource Metrics Endpoint website#12934

• metric-server

  • metrics server v0.6 Implement using /metrics/resource Kubelet endpoint kubernetes-sigs/metrics-server#777 ; Bump metrics-server to v0.6.3 kubernetes#117120 bump v0.6.3 (v1.28)

• Stable

  • KEP (k/enhancements) update PR(s):
    • Update KEP for Kubelet Resource Metrics endpoint GA release #4037
  • Code (k/k) update PR(s): Graduate kubelet resource metrics to GA kubernetes#116897
  • Docs (k/website) update(s):

[dashpole]

/sig node

[claurence]

@dashpole I've added this enhancement to the 1.14 tracking doc since it looks like y'all are targeting alpha here.

Are there any open PRs for this that should be tracked as well for it to be added to 1.14 as alpha?

[dashpole]

@claurence I am going to wait until the KEP is approved to open PRs. Ill reference this feature issue from the PRs if/when that happens

[WanLinghao]

/cc

[claurence]

@dashpole since the KEP for this issue isn't merged yet we will be removing it from the 1.14 milestone. To have it added back in please file an exception - information on the exception process can be found here: https://github.com/kubernetes/sig-release/blob/master/releases/EXCEPTIONS.md

[dashpole]

@claurence where is it specified that KEPs must be merged by feature freeze?
In previous releases the only requirement was to have this issue open.

[dashpole]

@claurence
https://github.com/kubernetes/sig-release/tree/master/releases/release-1.14#enhancements-freeze states:

"By Tuesday January 29, 2019 all enhancements landing in v1.14.0 must ... have an associated KEP merged or in progress in kubernetes/enhancements, linked to in the description of the tracking issue"

[claurence]

Hey @dashpole apologies as our docs have been changing a bit today - given feedback we've heard we are going to be lenient with KEPs though we want them merged by Feb 4th.

[claurence]

Added back to 1.14 per exception request submitted and approved

[npentrel]

Hey @dashpole 👋! I'm Naomi, a v1.14 docs release shadow.

Does this enhancement require any new docs (or modifications)?

Just a friendly reminder we're looking for a PR against k/website (branch dev-1.14) due by Friday, March 1. It would be great if it's the start of the full documentation, but even a placeholder PR is acceptable. Let me know if you have any questions!

[kacole2]

Hello @dashpole , I'm the Enhancement Lead for 1.15. Is this feature going to be graduating alpha/beta/stable stages in 1.15? Please let me know so it can be tracked properly and added to the spreadsheet.

Once coding begins, please list all relevant k/k PRs in this issue so they can be tracked properly.

/milestone clear

[dashpole]

@kacole2 I don't have any plans to graduate this in 1.15

[k8s-ci-robot]

@richabanker: You must be a member of the kubernetes/milestone-maintainers GitHub team to set the milestone. If you believe you should be able to issue the /milestone command, please contact your Milestone Maintainers Team and have them propose you as an additional delegate for this responsibility.

In response to this:

/milestone v1.29

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[npolshakova]

Hi @richabanker, 1.29 enhancement team here! Is this enhancement targeting 1.29? If it is, can you reopen the issue and follow the instructions here to opt in the enhancement and make sure the lead-opted-in label is set so it can get added to the tracking board? Thanks!

[wojtek-t]

/milestone v1.29
/label lead-opted-in

[npolshakova]

Hi @richabanker 👋, 1.29 Enhancements team here!

Just checking in as we approach enhancements freeze on 01:00 UTC, Friday, 6th October, 2023.

This enhancement is targeting for stage stable for 1.29 (correct me, if otherwise)

Here's where this enhancement currently stands:

• KEP readme using the latest template has been merged into the k/enhancements repo.
• KEP status is marked as implementable for latest-milestone: 1.29. KEPs targeting stable will need to be marked as implemented after code PRs are merged and the feature gates are removed.
• KEP readme has up-to-date graduation criteria
• KEP has a production readiness review that has been completed and merged into k/enhancements. (For more information on the PRR process, check here).

It looks like #4037 will address most of these issues. One thing to note- the KEP should only be marked after the all code PRs are merged and feature gates are removed. Since there are no feature gate involved for this and kubernetes/kubernetes#116897 has merged, this is fine to mark the KEP as implemented. Please let me know if that's not the case!

The status of this enhancement is marked as at risk for enhancement freeze. Please keep the issue description up-to-date with appropriate stages as well. Thank you!

[npolshakova]

Hi @richabanker, just checking in once more as we approach the 1.29 enhancement freeze deadline this week on 01:00 UTC, Friday, 6th October, 2023. With #4037 merged, the status of this enhancement is marked as tracked for enhancement freeze! 🚀

[katcosgrove]

Hey there @dashpole and @richabanker! 👋, v1.29 Docs Lead here.
Does this enhancement work planned for v1.29 require any new docs or modification to existing docs?
If so, please follows the steps here to open a PR against dev-1.29 branch in the k/website repo. This PR can be just a placeholder at this time and must be created before Thursday, 19 October 2023.
Also, take a look at Documenting for a release to get yourself familiarize with the docs requirement for the release.
Thank you!

[richabanker]

@dashpole do you think we need any updates in the existing docs for kubelet resource metrics?

[dashpole]

@richabanker I don't think so.

[kcmartin]

Hi @dashpole @richabanker! 👋 from the v1.29 Release Team-Communications! We would like to check if you have any plans to publish a blog for this KEP regarding new features, removals, and deprecations for this release.

If so, you need to open a PR placeholder in the website repository.
The deadline will be on Tuesday 14th November 2023 (after the Docs deadline PR ready for review)

Here's the 1.29 Calendar

[logicalhan]

@richabanker I don't think so.

I actually think we should: https://kubernetes.io/docs/reference/instrumentation/node-metrics/ doesn't seem to include the resource metrics endpoint.

[logicalhan]

Also the existing documentation: https://kubernetes.io/docs/tasks/debug/debug-cluster/resource-metrics-pipeline/ is weirdly located and doesn't have much information about kubelet resource metrics.

[richabanker]

There's a brief mention of the endpoint itself
but lacks details yeah.

[npolshakova]

Hey again @richabanker 👋 1.29 Enhancements team here,

Just checking in as we approach code freeze at 01:00 UTC Wednesday 1st November 2023: .

Here's where this enhancement currently stands:

• All PRs to the Kubernetes repo that are related to your enhancement are linked in the above issue description (for tracking purposes).
• All PR/s are ready to be merged (they have approved and lgtm labels applied) by the code freeze deadline. This includes tests.

For this enhancement, it looks like the following PRs has merged and update in the Issue description:

• Graduate kubelet resource metrics to GA kubernetes#116897

With this pr merged, if there are no additional code PRs, please update the status as implemented since there are no feature gates for this feature.

The status of this KEP is tracked for code freeze. 🚀

Also, please let me know if there are other PRs in k/k we should be tracking for this KEP.
As always, we are here to help if any questions come up. Thanks!

[salehsedghpour]

Hi @richabanker , 👋 1.30 Enhancements lead here
As it has graduated to Stable and the KEP status is still implementable, may you please update the status to implemented and close this issue once the update PR has merged?

[richabanker]

Hi @richabanker , 👋 1.30 Enhancements lead here As it has graduated to Stable and the KEP status is still implementable, may you please update the status to implemented and close this issue once the update PR has merged?

Ack'd @salehsedghpour , created a PR #4388, and will close the issue once this is merged. Thanks!

[salehsedghpour]

Awesome and Thanks!

/remove-label lead-opted-in

[salehsedghpour]

I'm closing milestone 1.29 now,

/milestone clear

[richabanker]

/close
since #4388 is now merged.

[k8s-ci-robot]

@richabanker: Closing this issue.

In response to this:

/close
since #4388 is now merged.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.