Make critical jobs Guaranteed Pod QOS: pull-kubernetes-e2e-kind

[spiffxp]

What should be cleaned up or changed:

This is part of #18530

The following jobs should be Guaranteed Pod QOS, meaning they should have CPU and memory resource limits, and matching resource requests:

• pull-kubernetes-e2e-kind

These jobs run on (google.com only) k8s-prow-build, so @spiffxp has provided the following guess:

• suggest 4 cpu, slightly above 8 Gi (9?) mem

General steps to follow:

• update the job definitions in /config/jobs to have matching CPU and memory limits and requests
  • e.g. (ci-kubernetes-e2e-gci-gce has a resources: field with matching entries)
• open a pull request, include a link to this issue and cc @kubernetes/ci-signal in the description
• keep an eye on the jobs for the next few days, and if they start failing more than usual, open followup pull requests to raise resources
  • can look at recent prow.k8s.io runs (e.g. https://prow.k8s.io/?job=ci-kubernetes-e2e-gci-gce)
  • can look at job history (e.g. https://prow.k8s.io/job-history/gs/kubernetes-jenkins/logs/ci-kubernetes-e2e-gci-gce)
  • can look at testgrid dashboards (e.g. https://testgrid.k8s.io/sig-release-master-blocking#gce-cos-master-default)
• after the job has remained stable for a few days, declare victory

/sig testing
/sig release
/area jobs
/area release-eng

[ameukam]

/assign

[spiffxp]

/remove-help
since @ameukam has it

[spiffxp]

https://prow.k8s.io/?job=pull-kubernetes-e2e-kind&state=error

Seeing jobs in error state this morning. Most timing out after ~5min (which I'm assuming means they couldn't get the resources they wanted).

Two timing out at 15-17min

[ameukam]

With #18817 merged, things we can watch :

• Memory utilization for 6H
• CPU utilization for 6H

[ameukam]

I'm not sure if it's accurate (or relevant) but we got a better success rate in 48h:

Taken at ~ 12:20 CEST August 14 :

Taken at ~7:30 CEST August 12 :

[spiffxp]

/close
Yeah I think this is looking much more stable

[k8s-ci-robot]

@spiffxp: Closing this issue.

In response to this:

/close
Yeah I think this is looking much more stable

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.