🌱 test/e2e: pin cgroupDriver to cgroupfs #4614
Conversation
k8s-ci-robot
added
the
cncf-cla: yes
k8s-ci-robot
added
the
size/M
|
/test pull-cluster-api-e2e-full-main |
|
Can we add a comment inline to the cgroup driver and link to the issue? |
Yes, which one of those? :) I think the most comprehensive summary of the issue is probably this PR. Or do you mean something like the following? # We have to pin the cgroupDriver to cgroupfs as kubeadm >=1.21 defaults to systemd
# kind will implement systemd support in: https://github.com/kubernetes-sigs/kind/issues/1726" |
There was a problem hiding this comment.
this generally looks good to me.
i realize this is out of scope for this PR, but i am curious about the notion that these flags are being deprecated[0], do we have an issue around changing our kubeadm implementation to create the config file (or does it do that already)?
[0] https://kubernetes.io/docs/reference/command-line-tools-reference/kubelet/#options
|
@sbueringer Yes, something inlined with the yaml so we don't forget later |
|
/lgtm |
k8s-ci-robot
added
the
lgtm
sbueringer
force-pushed
the
pr-capd-pin-cgroupfs
branch
from
3456a4b to
42f93d8
Compare
|
/test pull-cluster-api-e2e-full-main |
|
@vincepri I can additionally create an issue which tracks the upstream kind issue to change the cgroupDriver back to systemd. WDYT? |
k8s-ci-robot
removed
the
lgtm
|
/test pull-cluster-api-e2e-full-main |
|
I'll look into the failed test later today |
Same issue on the main branch: https://testgrid.k8s.io/sig-cluster-lifecycle-cluster-api#capi-e2e-main-1-21-latest I'm currently debugging it locally and depending on how easy the fix is either open an issue or open another PR. But i think there's no reason to block the current PR because of that. |
|
Should we merge this and tackle that separately? |
|
@vincepri I would definitely tackle it separately, but let's wait for @fabriziopandini 's review (which was probably your plan anyway) |
|
@vincepri @fabriziopandini more details in: #4619 |
There was a problem hiding this comment.
@sbueringer changes looks ok to me
Could you kindly change also templates under /test/infrastructure/docker/examples ?
Starting with kind v0.11, kind images will have cgroup-root=/kubelet hard-coded to fix an issue with nested cgroup hierarchies. The hierarchy below /kubelet is not compatible with systemd cgroupfs (in contrast to /). Starting with kubeadm v1.21, kubeadm will default to cgroupDriver systemd. Thus, the combination of images from kind >=v0.11 and kubeadm >=v1.21 are not compatible. To solve this problem we will pin cgroupDriver to cgroupfs until kind supports the systemd hierarchy with the /kubelet cgroup-root.
sbueringer
force-pushed
the
pr-capd-pin-cgroupfs
branch
from
42f93d8 to
972c45c
Compare
|
@fabriziopandini of course, done. I now have 47 occurrences of both eviction-hard and cgroup-driver below test/* (locally, including rendered templates). We have to also ensure that we don't miss this on the ipv6 PR |
|
@sbueringer: GitHub didn't allow me to request PR reviews from the following users: mcwumbly. Note that only kubernetes-sigs members and repo collaborators can review this PR, and authors cannot review their own PRs. In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
|
@sbueringer: The following test failed, say
Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR. Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here. |
|
/retest |
|
/approve
Might be kustomize can help here, but this could be tackled in a follow up PR. |
k8s-ci-robot
added
the
lgtm
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: fabriziopandini The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
k8s-ci-robot
added
the
approved
@fabriziopandini that comment was mostly about that I think I know found all of the relevant occurrences :) |
Starting with kind v0.11, kind images will have cgroup-root=/kubelet
hard-coded to fix an issue with nested cgroup hierarchies. The hierarchy
below /kubelet is not compatible with systemd cgroupfs (in contrast to
/).
Starting with kubeadm v1.21, kubeadm will default to cgroupDriver
systemd.
Thus, the combination of images from kind >=v0.11 and kubeadm >=v1.21 are
not compatible. To solve this problem we will pin cgroupDriver to cgroupfs
until kind supports the systemd hierarchy with the /kubelet cgroup-root.
What this PR does / why we need it:
Preparation to be compatible with the upcoming newly published kind images. Somewhat related to: #4469
xref:
Not sure if it's 🌱 or⚠️ because the breaking change is comes from kind and not from this PR.
I hope the description is enough context. I tried to summarize it as compact as possible. Let me know if I should provide more context as Slack and the Google Doc are not really persistent documentation.