kubeadm: add jobs to test the kubelet n-1 and n-2 skew

[neolit123]

see discussion here:
https://groups.google.com/d/msgid/kubernetes-sig-architecture/CAH1uJ6U4qftRnmWthjYKtBKAzRFFnX7XWLMkSbyVWHUsX8%3DBBg%40mail.gmail.com?utm_medium=email&utm_source=footer

and explanation here:
kubernetes/kubeadm#2396

kubernetes/kubeadm#2396 is required before this merges.

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: neolit123

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• config/jobs/kubernetes/sig-cluster-lifecycle/OWNERS [neolit123]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[neolit123]

/sig cluster-lifecycle
/sig node
/cc @fabriziopandini
/hold

[kfox1111]

I think there used to be a check for kubeadm upgrade that would prevent an upgrade of the control plane with n-2 kubelets. Is that still there? Should it be disabled once these checks are enabled?

[neolit123]

I think there used to be a check for kubeadm upgrade that would prevent an upgrade of the control plane with n-2 kubelets. Is that still there? Should it be disabled once these checks are enabled?

these test jobs don't include kubeadm upgrade, as this was not discussed.
kubeadm upgrade does have a check for unsupported current kubelet version on the node which can be skipped with -f.

[kfox1111]

Should we file a separate issue for kubeadm to update the check once this is in?

[neolit123]

Should we file a separate issue for kubeadm to update the check once this is in?

the kubeadm kubelet skew will remain limited to N - 1 of the kubeadm version. these tests are making an exception.

[SergeyKanzhelev]

/assign @cynepco3hahue

[neolit123]

this is ready for review / LGTM as the k/kubeadm PR that has the test job configuration is merging:
kubernetes/kubeadm#2396

[cynepco3hahue]

I do not see any problems with jobs, the single question who will be responsible to analyze failures(in the case when we will have any)

[cynepco3hahue]

/lgtm

[neolit123]

I do not see any problems with jobs, the single question who will be responsible to analyze failures(in the case when we will have any)

my proposal:

• if the failures are deployer related (kubeadm) - the kubeadm team will triage and fix.
• if the kubeadm team sees potential problems in the kubelet skew we will ping sig-node, but sig-node will see the failures on their mailing list too.
• if kubelet skew failures are not resolved by the sig-node team after a long period of time, the kubeadm team will pause / remove affected jobs. in such a case we must claim a particular skew as unsupported.

/hold cancel

[cynepco3hahue]

@neolit123 Thanks, works for me.

[neolit123]

/retest
(job stuck on status?)

[neolit123]

/test pull-test-infra-integration

[k8s-ci-robot]

@neolit123: Updated the job-config configmap in namespace default at cluster test-infra-trusted using the following files:

• key kubeadm-kinder-kubelet-x-on-y.yaml using file config/jobs/kubernetes/sig-cluster-lifecycle/kubeadm-kinder-kubelet-x-on-y.yaml

In response to this:

see discussion here:
https://groups.google.com/d/msgid/kubernetes-sig-architecture/CAH1uJ6U4qftRnmWthjYKtBKAzRFFnX7XWLMkSbyVWHUsX8%3DBBg%40mail.gmail.com?utm_medium=email&utm_source=footer

and explanation here:
kubernetes/kubeadm#2396

kubernetes/kubeadm#2396 is required before this merges.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.