WSL2 ERROR: failed to create cluster #2323
Comments
|
Hi, since this is WSL2 have you followed https://kind.sigs.k8s.io/docs/user/using-wsl2/ ? |
BenTheElder
changed the title
I seem to recall maybe we need to detect extfs and mount devicemapper in #2149 also:
rootless comes with some issues to be aware of still, officially Kubernetes does not support rootless yes, but kind does anyhow, with some limitations and workarounds. |
BenTheElder
changed the title
|
Creating cluster "kind" ... couldn't initialize a Kubernetes cluster |
|
Hi, can you please answer my previous questions? I can see that kubelet is timing out reaching API-server, that does not tell me:
If not for those, please do: |
|
I'm confused. The issue title seems about rootless, but |
|
I tagged it rootless due to
but in general we don't have enough info to go on for this issue yet, and the phrasing is not unambiguous |
BenTheElder
changed the title
|
Experiencing the same issue, and yes I followed the instructions on the WSL2 page.
|
|
Using WSL2 in Alpine on my Windows 10 box. I had this same issue when I upgraded to kind-node |
|
Same problem, also following the steps on the WSL2 page Kind used to work in the past on WSL2 :'( |
NOTE: This is not, in general, guaranteed to be supported. For pre-built images, please see the release notes for your release. I just had a chance to look at the logs in #2323 (comment) Do you all have:
? ( If so, try #1945 (comment) If that resolves it, we probably do still need to follow up around mounting this when we detect this backing filesystem. I've not yet found a good reference for which filesystem actually use this (and it is not always present at all), so we've added specific cases for known ones previously (zfs, btrfs). From the serial.log for the node: This may or may not be a problem. We are attempting to recreate a more natural hierarchy from the node POV, but it can function without this typically. |
|
Yep I have extfs too It's not something I've ever even thought about until now This is the config I'm using with config.yaml kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
nodes:
- role: control-plane
extraPortMappings:
- containerPort: 30000
hostPort: 30000
protocol: TCP
extraMounts:
- hostPath: /dev/mapper
containerPath: /dev/mapperstill getting the issue on WSL2 :( |
|
looking around: it seems distros (ubuntu) under WSL2 are not using systemd even if they do otherwise? (alpine typically doesn't period) we've had trouble with this before, Kubernetes, and most of our developers on linux etc. are largely supporting / using systemd (which relates to cgroup setup), #2156 (comment) d777456 was meant to resolve some of this, but that's in v0.11+ images |
|
In WSL2 I'm running Ubuntu 20.04 Running 1.17.17 image finally worked for me |
|
without pinning the digest it's possible this was an older cached iteration of that image. for v01.11. the digests are here https://github.com/kubernetes-sigs/kind/releases/tag/v0.11.1 |
Also works for me thanks! |
|
I'm seeing the same behaviour on a Pop-OS (Ubuntu) laptop. Stops at same point as OP, cluster fails to come up. This was working until I installed the latest batch of updates for my OS. Will try workaround EDIT: No dice. |
|
I think I may have found a problem. Starting control-plane fails because kubelet on worker nodes fails to start which is because Since this is baked into an image, the best workaround I've found is to create a file without and modify kind.yaml to mount this over the baked-in file: Seems to work but I'm not sure why this flag was added and what are the consequences of removing it. |
|
Great investigation, the cgroup-root=/kubelet is needed to run kind inside of kubernetes The logs attached clearly confirm this is the problem
The serial.log files has these entries is not working the bind mounting on wsl? |
|
it does work with bind mounts on wsl. just wanted to make sure it doesn't break anything unexpectedly. based on the referenced PR, the workaround should work fine. |
but .... why the /kubelet cgroup doesn't exist? |
|
I'm not sure why. I couldn't find where it's supposed to be created. I can try a dig deeper, need to have a look at that PR you referenced |
|
ok, so I dug a bit deeper and it looks like it could be a bug in kind's entrypoint. Normally, with cgroupv2, |
|
what I use to do to debug this problems is to log into a runing kind node (despite it fails you can use --retain to keep it alive) and execute the kind/images/base/files/usr/local/bin/entrypoint Lines 220 to 251 in 9e6e911 |
|
this is exactly what I'd done. I've carved out the entire fix_cgroup() function together with Here's the output
root@k8s-guide-control-plane:/# ./test.sh
+ set -e
+ fix_cgroup
+ [[ -f /sys/fs/cgroup/cgroup.controllers ]]
+ echo 'INFO: detected cgroup v1'
INFO: detected cgroup v1
+ echo 'INFO: fix cgroup mounts for all subsystems'
INFO: fix cgroup mounts for all subsystems
+ local current_cgroup
++ grep -E '^[^:]*:([^:]*,)?cpu(,[^,:]*)?:.*' /proc/self/cgroup
++ cut -d: -f3
+ current_cgroup=/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
+ local cgroup_subsystems
++ findmnt -lun -o source,target -t cgroup
++ grep /docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
++ awk '{print $2}'
+ cgroup_subsystems='/sys/fs/cgroup/cpuset
/sys/fs/cgroup/cpu
/sys/fs/cgroup/cpuacct
/sys/fs/cgroup/blkio
/sys/fs/cgroup/memory
/sys/fs/cgroup/devices
/sys/fs/cgroup/freezer
/sys/fs/cgroup/net_cls
/sys/fs/cgroup/perf_event
/sys/fs/cgroup/net_prio
/sys/fs/cgroup/hugetlb
/sys/fs/cgroup/pids
/sys/fs/cgroup/cpuset/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
/sys/fs/cgroup/cpu/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
/sys/fs/cgroup/cpuacct/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
/sys/fs/cgroup/blkio/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
/sys/fs/cgroup/memory/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
/sys/fs/cgroup/devices/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
/sys/fs/cgroup/freezer/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
/sys/fs/cgroup/net_cls/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
/sys/fs/cgroup/perf_event/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
/sys/fs/cgroup/net_prio/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
/sys/fs/cgroup/hugetlb/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
/sys/fs/cgroup/pids/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
/sys/fs/cgroup/cpuset/kubelet
/sys/fs/cgroup/cpu/kubelet
/sys/fs/cgroup/cpuacct/kubelet
/sys/fs/cgroup/blkio/kubelet
/sys/fs/cgroup/memory/kubelet
/sys/fs/cgroup/devices/kubelet
/sys/fs/cgroup/freezer/kubelet
/sys/fs/cgroup/net_cls/kubelet
/sys/fs/cgroup/perf_event/kubelet
/sys/fs/cgroup/net_prio/kubelet
/sys/fs/cgroup/hugetlb/kubelet
/sys/fs/cgroup/pids/kubelet'
+ local cgroup_mounts
++ grep -E -o '/[[:alnum:]].* /sys/fs/cgroup.*.*cgroup' /proc/self/mountinfo
+ cgroup_mounts='/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c /sys/fs/cgroup/cpuset rw,nosuid,nodev,noexec,relatime - cgroup cgroup
/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c /sys/fs/cgroup/cpu rw,nosuid,nodev,noexec,relatime - cgroup cgroup
/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c /sys/fs/cgroup/cpuacct rw,nosuid,nodev,noexec,relatime - cgroup cgroup
/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c /sys/fs/cgroup/blkio rw,nosuid,nodev,noexec,relatime - cgroup cgroup
/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c /sys/fs/cgroup/memory rw,nosuid,nodev,noexec,relatime - cgroup cgroup
/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c /sys/fs/cgroup/devices rw,nosuid,nodev,noexec,relatime - cgroup cgroup
/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c /sys/fs/cgroup/freezer rw,nosuid,nodev,noexec,relatime - cgroup cgroup
/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c /sys/fs/cgroup/net_cls rw,nosuid,nodev,noexec,relatime - cgroup cgroup
/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c /sys/fs/cgroup/perf_event rw,nosuid,nodev,noexec,relatime - cgroup cgroup
/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c /sys/fs/cgroup/net_prio rw,nosuid,nodev,noexec,relatime - cgroup cgroup
/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c /sys/fs/cgroup/hugetlb rw,nosuid,nodev,noexec,relatime - cgroup cgroup
/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c /sys/fs/cgroup/pids rw,nosuid,nodev,noexec,relatime - cgroup cgroup
/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c /sys/fs/cgroup/cpuset/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c rw,nosuid,nodev,noexec,relatime - cgroup cgroup
/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c /sys/fs/cgroup/cpu/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c rw,nosuid,nodev,noexec,relatime - cgroup cgroup
/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c /sys/fs/cgroup/cpuacct/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c rw,nosuid,nodev,noexec,relatime - cgroup cgroup
/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c /sys/fs/cgroup/blkio/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c rw,nosuid,nodev,noexec,relatime - cgroup cgroup
/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c /sys/fs/cgroup/memory/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c rw,nosuid,nodev,noexec,relatime - cgroup cgroup
/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c /sys/fs/cgroup/devices/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c rw,nosuid,nodev,noexec,relatime - cgroup cgroup
/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c /sys/fs/cgroup/freezer/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c rw,nosuid,nodev,noexec,relatime - cgroup cgroup
/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c /sys/fs/cgroup/net_cls/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c rw,nosuid,nodev,noexec,relatime - cgroup cgroup
/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c /sys/fs/cgroup/perf_event/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c rw,nosuid,nodev,noexec,relatime - cgroup cgroup
/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c /sys/fs/cgroup/net_prio/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c rw,nosuid,nodev,noexec,relatime - cgroup cgroup
/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c /sys/fs/cgroup/hugetlb/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c rw,nosuid,nodev,noexec,relatime - cgroup cgroup
/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c /sys/fs/cgroup/pids/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c rw,nosuid,nodev,noexec,relatime - cgroup cgroup
/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c/kubelet /sys/fs/cgroup/cpuset/kubelet rw,nosuid,nodev,noexec,relatime - cgroup cgroup
/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c/kubelet /sys/fs/cgroup/cpu/kubelet rw,nosuid,nodev,noexec,relatime - cgroup cgroup
/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c/kubelet /sys/fs/cgroup/cpuacct/kubelet rw,nosuid,nodev,noexec,relatime - cgroup cgroup
/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c/kubelet /sys/fs/cgroup/blkio/kubelet rw,nosuid,nodev,noexec,relatime - cgroup cgroup
/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c/kubelet /sys/fs/cgroup/memory/kubelet rw,nosuid,nodev,noexec,relatime - cgroup cgroup
/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c/kubelet /sys/fs/cgroup/devices/kubelet rw,nosuid,nodev,noexec,relatime - cgroup cgroup
/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c/kubelet /sys/fs/cgroup/freezer/kubelet rw,nosuid,nodev,noexec,relatime - cgroup cgroup
/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c/kubelet /sys/fs/cgroup/net_cls/kubelet rw,nosuid,nodev,noexec,relatime - cgroup cgroup
/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c/kubelet /sys/fs/cgroup/perf_event/kubelet rw,nosuid,nodev,noexec,relatime - cgroup cgroup
/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c/kubelet /sys/fs/cgroup/net_prio/kubelet rw,nosuid,nodev,noexec,relatime - cgroup cgroup
/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c/kubelet /sys/fs/cgroup/hugetlb/kubelet rw,nosuid,nodev,noexec,relatime - cgroup cgroup
/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c/kubelet /sys/fs/cgroup/pids/kubelet rw,nosuid,nodev,noexec,relatime - cgroup cgroup'
+ [[ -n /docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c /sys/fs/cgroup/cpuset rw,nosuid,nodev,noexec,relatime - cgroup cgroup
/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c /sys/fs/cgroup/cpu rw,nosuid,nodev,noexec,relatime - cgroup cgroup
/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c /sys/fs/cgroup/cpuacct rw,nosuid,nodev,noexec,relatime - cgroup cgroup
/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c /sys/fs/cgroup/blkio rw,nosuid,nodev,noexec,relatime - cgroup cgroup
/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c /sys/fs/cgroup/memory rw,nosuid,nodev,noexec,relatime - cgroup cgroup
/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c /sys/fs/cgroup/devices rw,nosuid,nodev,noexec,relatime - cgroup cgroup
/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c /sys/fs/cgroup/freezer rw,nosuid,nodev,noexec,relatime - cgroup cgroup
/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c /sys/fs/cgroup/net_cls rw,nosuid,nodev,noexec,relatime - cgroup cgroup
/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c /sys/fs/cgroup/perf_event rw,nosuid,nodev,noexec,relatime - cgroup cgroup
/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c /sys/fs/cgroup/net_prio rw,nosuid,nodev,noexec,relatime - cgroup cgroup
/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c /sys/fs/cgroup/hugetlb rw,nosuid,nodev,noexec,relatime - cgroup cgroup
/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c /sys/fs/cgroup/pids rw,nosuid,nodev,noexec,relatime - cgroup cgroup
/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c /sys/fs/cgroup/cpuset/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c rw,nosuid,nodev,noexec,relatime - cgroup cgroup
/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c /sys/fs/cgroup/cpu/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c rw,nosuid,nodev,noexec,relatime - cgroup cgroup
/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c /sys/fs/cgroup/cpuacct/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c rw,nosuid,nodev,noexec,relatime - cgroup cgroup
/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c /sys/fs/cgroup/blkio/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c rw,nosuid,nodev,noexec,relatime - cgroup cgroup
/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c /sys/fs/cgroup/memory/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c rw,nosuid,nodev,noexec,relatime - cgroup cgroup
/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c /sys/fs/cgroup/devices/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c rw,nosuid,nodev,noexec,relatime - cgroup cgroup
/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c /sys/fs/cgroup/freezer/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c rw,nosuid,nodev,noexec,relatime - cgroup cgroup
/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c /sys/fs/cgroup/net_cls/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c rw,nosuid,nodev,noexec,relatime - cgroup cgroup
/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c /sys/fs/cgroup/perf_event/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c rw,nosuid,nodev,noexec,relatime - cgroup cgroup
/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c /sys/fs/cgroup/net_prio/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c rw,nosuid,nodev,noexec,relatime - cgroup cgroup
/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c /sys/fs/cgroup/hugetlb/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c rw,nosuid,nodev,noexec,relatime - cgroup cgroup
/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c /sys/fs/cgroup/pids/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c rw,nosuid,nodev,noexec,relatime - cgroup cgroup
/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c/kubelet /sys/fs/cgroup/cpuset/kubelet rw,nosuid,nodev,noexec,relatime - cgroup cgroup
/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c/kubelet /sys/fs/cgroup/cpu/kubelet rw,nosuid,nodev,noexec,relatime - cgroup cgroup
/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c/kubelet /sys/fs/cgroup/cpuacct/kubelet rw,nosuid,nodev,noexec,relatime - cgroup cgroup
/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c/kubelet /sys/fs/cgroup/blkio/kubelet rw,nosuid,nodev,noexec,relatime - cgroup cgroup
/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c/kubelet /sys/fs/cgroup/memory/kubelet rw,nosuid,nodev,noexec,relatime - cgroup cgroup
/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c/kubelet /sys/fs/cgroup/devices/kubelet rw,nosuid,nodev,noexec,relatime - cgroup cgroup
/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c/kubelet /sys/fs/cgroup/freezer/kubelet rw,nosuid,nodev,noexec,relatime - cgroup cgroup
/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c/kubelet /sys/fs/cgroup/net_cls/kubelet rw,nosuid,nodev,noexec,relatime - cgroup cgroup
/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c/kubelet /sys/fs/cgroup/perf_event/kubelet rw,nosuid,nodev,noexec,relatime - cgroup cgroup
/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c/kubelet /sys/fs/cgroup/net_prio/kubelet rw,nosuid,nodev,noexec,relatime - cgroup cgroup
/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c/kubelet /sys/fs/cgroup/hugetlb/kubelet rw,nosuid,nodev,noexec,relatime - cgroup cgroup
/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c/kubelet /sys/fs/cgroup/pids/kubelet rw,nosuid,nodev,noexec,relatime - cgroup cgroup ]]
+ local mount_root
++ head -n 1
++ cut '-d ' -f1
+ mount_root=/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
++ echo '/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c /sys/fs/cgroup/cpuset rw,nosuid,nodev,noexec,relatime - cgroup cgroup
/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c /sys/fs/cgroup/cpu rw,nosuid,nodev,noexec,relatime - cgroup cgroup
/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c /sys/fs/cgroup/cpuacct rw,nosuid,nodev,noexec,relatime - cgroup cgroup
/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c /sys/fs/cgroup/blkio rw,nosuid,nodev,noexec,relatime - cgroup cgroup
/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c /sys/fs/cgroup/memory rw,nosuid,nodev,noexec,relatime - cgroup cgroup
/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c /sys/fs/cgroup/devices rw,nosuid,nodev,noexec,relatime - cgroup cgroup
/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c /sys/fs/cgroup/freezer rw,nosuid,nodev,noexec,relatime - cgroup cgroup
/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c /sys/fs/cgroup/net_cls rw,nosuid,nodev,noexec,relatime - cgroup cgroup
/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c /sys/fs/cgroup/perf_event rw,nosuid,nodev,noexec,relatime - cgroup cgroup
/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c /sys/fs/cgroup/net_prio rw,nosuid,nodev,noexec,relatime - cgroup cgroup
/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c /sys/fs/cgroup/hugetlb rw,nosuid,nodev,noexec,relatime - cgroup cgroup
/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c /sys/fs/cgroup/pids rw,nosuid,nodev,noexec,relatime - cgroup cgroup
/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c /sys/fs/cgroup/cpuset/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c rw,nosuid,nodev,noexec,relatime - cgroup cgroup
/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c /sys/fs/cgroup/cpu/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c rw,nosuid,nodev,noexec,relatime - cgroup cgroup
/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c /sys/fs/cgroup/cpuacct/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c rw,nosuid,nodev,noexec,relatime - cgroup cgroup
/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c /sys/fs/cgroup/blkio/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c rw,nosuid,nodev,noexec,relatime - cgroup cgroup
/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c /sys/fs/cgroup/memory/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c rw,nosuid,nodev,noexec,relatime - cgroup cgroup
/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c /sys/fs/cgroup/devices/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c rw,nosuid,nodev,noexec,relatime - cgroup cgroup
/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c /sys/fs/cgroup/freezer/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c rw,nosuid,nodev,noexec,relatime - cgroup cgroup
/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c /sys/fs/cgroup/net_cls/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c rw,nosuid,nodev,noexec,relatime - cgroup cgroup
/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c /sys/fs/cgroup/perf_event/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c rw,nosuid,nodev,noexec,relatime - cgroup cgroup
/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c /sys/fs/cgroup/net_prio/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c rw,nosuid,nodev,noexec,relatime - cgroup cgroup
/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c /sys/fs/cgroup/hugetlb/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c rw,nosuid,nodev,noexec,relatime - cgroup cgroup
/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c /sys/fs/cgroup/pids/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c rw,nosuid,nodev,noexec,relatime - cgroup cgroup
/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c/kubelet /sys/fs/cgroup/cpuset/kubelet rw,nosuid,nodev,noexec,relatime - cgroup cgroup
/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c/kubelet /sys/fs/cgroup/cpu/kubelet rw,nosuid,nodev,noexec,relatime - cgroup cgroup
/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c/kubelet /sys/fs/cgroup/cpuacct/kubelet rw,nosuid,nodev,noexec,relatime - cgroup cgroup
/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c/kubelet /sys/fs/cgroup/blkio/kubelet rw,nosuid,nodev,noexec,relatime - cgroup cgroup
/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c/kubelet /sys/fs/cgroup/memory/kubelet rw,nosuid,nodev,noexec,relatime - cgroup cgroup
/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c/kubelet /sys/fs/cgroup/devices/kubelet rw,nosuid,nodev,noexec,relatime - cgroup cgroup
/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c/kubelet /sys/fs/cgroup/freezer/kubelet rw,nosuid,nodev,noexec,relatime - cgroup cgroup
/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c/kubelet /sys/fs/cgroup/net_cls/kubelet rw,nosuid,nodev,noexec,relatime - cgroup cgroup
/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c/kubelet /sys/fs/cgroup/perf_event/kubelet rw,nosuid,nodev,noexec,relatime - cgroup cgroup
/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c/kubelet /sys/fs/cgroup/net_prio/kubelet rw,nosuid,nodev,noexec,relatime - cgroup cgroup
/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c/kubelet /sys/fs/cgroup/hugetlb/kubelet rw,nosuid,nodev,noexec,relatime - cgroup cgroup
/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c/kubelet /sys/fs/cgroup/pids/kubelet rw,nosuid,nodev,noexec,relatime - cgroup cgroup'
++ cut '-d ' -f 2
+ for mount_point in $(echo "${cgroup_mounts}" | cut -d' ' -f 2)
+ local target=/sys/fs/cgroup/cpuset/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
+ findmnt /sys/fs/cgroup/cpuset/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
TARGET SOURCE FSTYPE OPTIONS
/sys/fs/cgroup/cpuset/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c cgroup[/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c] cgroup rw,nosuid,nodev,noexec,relatime,cpuset
+ for mount_point in $(echo "${cgroup_mounts}" | cut -d' ' -f 2)
+ local target=/sys/fs/cgroup/cpu/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
+ findmnt /sys/fs/cgroup/cpu/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
TARGET SOURCE FSTYPE OPTIONS
/sys/fs/cgroup/cpu/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c cgroup[/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c] cgroup rw,nosuid,nodev,noexec,relatime,cpu
+ for mount_point in $(echo "${cgroup_mounts}" | cut -d' ' -f 2)
+ local target=/sys/fs/cgroup/cpuacct/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
+ findmnt /sys/fs/cgroup/cpuacct/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
TARGET SOURCE FSTYPE OPTIONS
/sys/fs/cgroup/cpuacct/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c cgroup[/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c] cgroup rw,nosuid,nodev,noexec,relatime,cpuacct
+ for mount_point in $(echo "${cgroup_mounts}" | cut -d' ' -f 2)
+ local target=/sys/fs/cgroup/blkio/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
+ findmnt /sys/fs/cgroup/blkio/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
TARGET SOURCE FSTYPE OPTIONS
/sys/fs/cgroup/blkio/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c cgroup[/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c] cgroup rw,nosuid,nodev,noexec,relatime,blkio
+ for mount_point in $(echo "${cgroup_mounts}" | cut -d' ' -f 2)
+ local target=/sys/fs/cgroup/memory/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
+ findmnt /sys/fs/cgroup/memory/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
TARGET SOURCE FSTYPE OPTIONS
/sys/fs/cgroup/memory/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c cgroup[/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c] cgroup rw,nosuid,nodev,noexec,relatime,memory
+ for mount_point in $(echo "${cgroup_mounts}" | cut -d' ' -f 2)
+ local target=/sys/fs/cgroup/devices/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
+ findmnt /sys/fs/cgroup/devices/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
TARGET SOURCE FSTYPE OPTIONS
/sys/fs/cgroup/devices/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c cgroup[/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c] cgroup rw,nosuid,nodev,noexec,relatime,devices
+ for mount_point in $(echo "${cgroup_mounts}" | cut -d' ' -f 2)
+ local target=/sys/fs/cgroup/freezer/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
+ findmnt /sys/fs/cgroup/freezer/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
TARGET SOURCE FSTYPE OPTIONS
/sys/fs/cgroup/freezer/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c cgroup[/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c] cgroup rw,nosuid,nodev,noexec,relatime,freezer
+ for mount_point in $(echo "${cgroup_mounts}" | cut -d' ' -f 2)
+ local target=/sys/fs/cgroup/net_cls/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
+ findmnt /sys/fs/cgroup/net_cls/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
TARGET SOURCE FSTYPE OPTIONS
/sys/fs/cgroup/net_cls/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c cgroup[/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c] cgroup rw,nosuid,nodev,noexec,relatime,net_cls
+ for mount_point in $(echo "${cgroup_mounts}" | cut -d' ' -f 2)
+ local target=/sys/fs/cgroup/perf_event/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
+ findmnt /sys/fs/cgroup/perf_event/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
TARGET SOURCE FSTYPE OPTIONS
/sys/fs/cgroup/perf_event/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c cgroup[/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c] cgroup rw,nosuid,nodev,noexec,relatime,perf_event
+ for mount_point in $(echo "${cgroup_mounts}" | cut -d' ' -f 2)
+ local target=/sys/fs/cgroup/net_prio/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
+ findmnt /sys/fs/cgroup/net_prio/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
TARGET SOURCE FSTYPE OPTIONS
/sys/fs/cgroup/net_prio/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c cgroup[/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c] cgroup rw,nosuid,nodev,noexec,relatime,net_prio
+ for mount_point in $(echo "${cgroup_mounts}" | cut -d' ' -f 2)
+ local target=/sys/fs/cgroup/hugetlb/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
+ findmnt /sys/fs/cgroup/hugetlb/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
TARGET SOURCE FSTYPE OPTIONS
/sys/fs/cgroup/hugetlb/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c cgroup[/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c] cgroup rw,nosuid,nodev,noexec,relatime,hugetlb
+ for mount_point in $(echo "${cgroup_mounts}" | cut -d' ' -f 2)
+ local target=/sys/fs/cgroup/pids/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
+ findmnt /sys/fs/cgroup/pids/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
TARGET SOURCE FSTYPE OPTIONS
/sys/fs/cgroup/pids/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c cgroup[/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c] cgroup rw,nosuid,nodev,noexec,relatime,pids
+ for mount_point in $(echo "${cgroup_mounts}" | cut -d' ' -f 2)
+ local target=/sys/fs/cgroup/cpuset/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
+ findmnt /sys/fs/cgroup/cpuset/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
+ mkdir -p /sys/fs/cgroup/cpuset/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
+ mount --bind /sys/fs/cgroup/cpuset/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c /sys/fs/cgroup/cpuset/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
+ for mount_point in $(echo "${cgroup_mounts}" | cut -d' ' -f 2)
+ local target=/sys/fs/cgroup/cpu/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
+ findmnt /sys/fs/cgroup/cpu/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
+ mkdir -p /sys/fs/cgroup/cpu/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
+ mount --bind /sys/fs/cgroup/cpu/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c /sys/fs/cgroup/cpu/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
+ for mount_point in $(echo "${cgroup_mounts}" | cut -d' ' -f 2)
+ local target=/sys/fs/cgroup/cpuacct/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
+ findmnt /sys/fs/cgroup/cpuacct/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
+ mkdir -p /sys/fs/cgroup/cpuacct/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
+ mount --bind /sys/fs/cgroup/cpuacct/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c /sys/fs/cgroup/cpuacct/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
+ for mount_point in $(echo "${cgroup_mounts}" | cut -d' ' -f 2)
+ local target=/sys/fs/cgroup/blkio/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
+ findmnt /sys/fs/cgroup/blkio/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
+ mkdir -p /sys/fs/cgroup/blkio/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
+ mount --bind /sys/fs/cgroup/blkio/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c /sys/fs/cgroup/blkio/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
+ for mount_point in $(echo "${cgroup_mounts}" | cut -d' ' -f 2)
+ local target=/sys/fs/cgroup/memory/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
+ findmnt /sys/fs/cgroup/memory/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
+ mkdir -p /sys/fs/cgroup/memory/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
+ mount --bind /sys/fs/cgroup/memory/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c /sys/fs/cgroup/memory/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
+ for mount_point in $(echo "${cgroup_mounts}" | cut -d' ' -f 2)
+ local target=/sys/fs/cgroup/devices/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
+ findmnt /sys/fs/cgroup/devices/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
+ mkdir -p /sys/fs/cgroup/devices/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
+ mount --bind /sys/fs/cgroup/devices/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c /sys/fs/cgroup/devices/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
+ for mount_point in $(echo "${cgroup_mounts}" | cut -d' ' -f 2)
+ local target=/sys/fs/cgroup/freezer/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
+ findmnt /sys/fs/cgroup/freezer/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
+ mkdir -p /sys/fs/cgroup/freezer/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
+ mount --bind /sys/fs/cgroup/freezer/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c /sys/fs/cgroup/freezer/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
+ for mount_point in $(echo "${cgroup_mounts}" | cut -d' ' -f 2)
+ local target=/sys/fs/cgroup/net_cls/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
+ findmnt /sys/fs/cgroup/net_cls/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
+ mkdir -p /sys/fs/cgroup/net_cls/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
+ mount --bind /sys/fs/cgroup/net_cls/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c /sys/fs/cgroup/net_cls/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
+ for mount_point in $(echo "${cgroup_mounts}" | cut -d' ' -f 2)
+ local target=/sys/fs/cgroup/perf_event/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
+ findmnt /sys/fs/cgroup/perf_event/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
+ mkdir -p /sys/fs/cgroup/perf_event/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
+ mount --bind /sys/fs/cgroup/perf_event/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c /sys/fs/cgroup/perf_event/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
+ for mount_point in $(echo "${cgroup_mounts}" | cut -d' ' -f 2)
+ local target=/sys/fs/cgroup/net_prio/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
+ findmnt /sys/fs/cgroup/net_prio/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
+ mkdir -p /sys/fs/cgroup/net_prio/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
+ mount --bind /sys/fs/cgroup/net_prio/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c /sys/fs/cgroup/net_prio/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
+ for mount_point in $(echo "${cgroup_mounts}" | cut -d' ' -f 2)
+ local target=/sys/fs/cgroup/hugetlb/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
+ findmnt /sys/fs/cgroup/hugetlb/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
+ mkdir -p /sys/fs/cgroup/hugetlb/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
+ mount --bind /sys/fs/cgroup/hugetlb/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c /sys/fs/cgroup/hugetlb/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
+ for mount_point in $(echo "${cgroup_mounts}" | cut -d' ' -f 2)
+ local target=/sys/fs/cgroup/pids/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
+ findmnt /sys/fs/cgroup/pids/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
+ mkdir -p /sys/fs/cgroup/pids/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
+ mount --bind /sys/fs/cgroup/pids/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c /sys/fs/cgroup/pids/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c+ for mount_point in $(echo "${cgroup_mounts}" | cut -d' ' -f 2)
+ local target=/sys/fs/cgroup/cpuset/kubelet/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
+ findmnt /sys/fs/cgroup/cpuset/kubelet/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
+ mkdir -p /sys/fs/cgroup/cpuset/kubelet/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
+ mount --bind /sys/fs/cgroup/cpuset/kubelet /sys/fs/cgroup/cpuset/kubelet/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
+ for mount_point in $(echo "${cgroup_mounts}" | cut -d' ' -f 2)
+ local target=/sys/fs/cgroup/cpu/kubelet/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
+ findmnt /sys/fs/cgroup/cpu/kubelet/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
+ mkdir -p /sys/fs/cgroup/cpu/kubelet/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
+ mount --bind /sys/fs/cgroup/cpu/kubelet /sys/fs/cgroup/cpu/kubelet/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
+ for mount_point in $(echo "${cgroup_mounts}" | cut -d' ' -f 2)
+ local target=/sys/fs/cgroup/cpuacct/kubelet/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
+ findmnt /sys/fs/cgroup/cpuacct/kubelet/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
+ mkdir -p /sys/fs/cgroup/cpuacct/kubelet/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
+ mount --bind /sys/fs/cgroup/cpuacct/kubelet /sys/fs/cgroup/cpuacct/kubelet/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
+ for mount_point in $(echo "${cgroup_mounts}" | cut -d' ' -f 2)
+ local target=/sys/fs/cgroup/blkio/kubelet/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
+ findmnt /sys/fs/cgroup/blkio/kubelet/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
+ mkdir -p /sys/fs/cgroup/blkio/kubelet/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
+ mount --bind /sys/fs/cgroup/blkio/kubelet /sys/fs/cgroup/blkio/kubelet/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
+ for mount_point in $(echo "${cgroup_mounts}" | cut -d' ' -f 2)
+ local target=/sys/fs/cgroup/memory/kubelet/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
+ findmnt /sys/fs/cgroup/memory/kubelet/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
+ mkdir -p /sys/fs/cgroup/memory/kubelet/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
+ mount --bind /sys/fs/cgroup/memory/kubelet /sys/fs/cgroup/memory/kubelet/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
+ for mount_point in $(echo "${cgroup_mounts}" | cut -d' ' -f 2)
+ local target=/sys/fs/cgroup/devices/kubelet/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
+ findmnt /sys/fs/cgroup/devices/kubelet/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
+ mkdir -p /sys/fs/cgroup/devices/kubelet/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
+ mount --bind /sys/fs/cgroup/devices/kubelet /sys/fs/cgroup/devices/kubelet/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
+ for mount_point in $(echo "${cgroup_mounts}" | cut -d' ' -f 2)
+ local target=/sys/fs/cgroup/freezer/kubelet/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
+ findmnt /sys/fs/cgroup/freezer/kubelet/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
+ mkdir -p /sys/fs/cgroup/freezer/kubelet/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
+ mount --bind /sys/fs/cgroup/freezer/kubelet /sys/fs/cgroup/freezer/kubelet/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
+ for mount_point in $(echo "${cgroup_mounts}" | cut -d' ' -f 2)
+ local target=/sys/fs/cgroup/net_cls/kubelet/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
+ findmnt /sys/fs/cgroup/net_cls/kubelet/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
+ mkdir -p /sys/fs/cgroup/net_cls/kubelet/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
+ mount --bind /sys/fs/cgroup/net_cls/kubelet /sys/fs/cgroup/net_cls/kubelet/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
+ for mount_point in $(echo "${cgroup_mounts}" | cut -d' ' -f 2)
+ local target=/sys/fs/cgroup/perf_event/kubelet/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
+ findmnt /sys/fs/cgroup/perf_event/kubelet/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
+ mkdir -p /sys/fs/cgroup/perf_event/kubelet/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
+ mount --bind /sys/fs/cgroup/perf_event/kubelet /sys/fs/cgroup/perf_event/kubelet/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
+ for mount_point in $(echo "${cgroup_mounts}" | cut -d' ' -f 2)
+ local target=/sys/fs/cgroup/net_prio/kubelet/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
+ findmnt /sys/fs/cgroup/net_prio/kubelet/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
+ mkdir -p /sys/fs/cgroup/net_prio/kubelet/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
+ mount --bind /sys/fs/cgroup/net_prio/kubelet /sys/fs/cgroup/net_prio/kubelet/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
+ for mount_point in $(echo "${cgroup_mounts}" | cut -d' ' -f 2)
+ local target=/sys/fs/cgroup/hugetlb/kubelet/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
+ findmnt /sys/fs/cgroup/hugetlb/kubelet/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
+ mkdir -p /sys/fs/cgroup/hugetlb/kubelet/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
+ mount --bind /sys/fs/cgroup/hugetlb/kubelet /sys/fs/cgroup/hugetlb/kubelet/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
+ for mount_point in $(echo "${cgroup_mounts}" | cut -d' ' -f 2)
+ local target=/sys/fs/cgroup/pids/kubelet/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
+ findmnt /sys/fs/cgroup/pids/kubelet/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
+ mkdir -p /sys/fs/cgroup/pids/kubelet/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
+ mount --bind /sys/fs/cgroup/pids/kubelet /sys/fs/cgroup/pids/kubelet/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
+ mount --make-rprivate /sys/fs/cgroup
+ echo '/sys/fs/cgroup/cpuset
/sys/fs/cgroup/cpu
/sys/fs/cgroup/cpuacct
/sys/fs/cgroup/blkio
/sys/fs/cgroup/memory
/sys/fs/cgroup/devices
/sys/fs/cgroup/freezer
/sys/fs/cgroup/net_cls
/sys/fs/cgroup/perf_event
/sys/fs/cgroup/net_prio
/sys/fs/cgroup/hugetlb
/sys/fs/cgroup/pids
/sys/fs/cgroup/cpuset/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
/sys/fs/cgroup/cpu/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
/sys/fs/cgroup/cpuacct/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
/sys/fs/cgroup/blkio/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
/sys/fs/cgroup/memory/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
/sys/fs/cgroup/devices/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
/sys/fs/cgroup/freezer/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
/sys/fs/cgroup/net_cls/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
/sys/fs/cgroup/perf_event/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
/sys/fs/cgroup/net_prio/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
/sys/fs/cgroup/hugetlb/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
/sys/fs/cgroup/pids/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
/sys/fs/cgroup/cpuset/kubelet
/sys/fs/cgroup/cpu/kubelet
/sys/fs/cgroup/cpuacct/kubelet
/sys/fs/cgroup/blkio/kubelet
/sys/fs/cgroup/memory/kubelet
/sys/fs/cgroup/devices/kubelet
/sys/fs/cgroup/freezer/kubelet
/sys/fs/cgroup/net_cls/kubelet
/sys/fs/cgroup/perf_event/kubelet
/sys/fs/cgroup/net_prio/kubelet
/sys/fs/cgroup/hugetlb/kubelet
/sys/fs/cgroup/pids/kubelet'
+ IFS=
+ read -r subsystem
+ mount_kubelet_cgroup_root /kubelet /sys/fs/cgroup/cpuset
+ local cgroup_root=/kubelet
+ local subsystem=/sys/fs/cgroup/cpuset
+ '[' -z /kubelet ']'
+ mkdir -p /sys/fs/cgroup/cpuset//kubelet
+ '[' /sys/fs/cgroup/cpuset == /sys/fs/cgroup/cpuset ']'
+ cat /sys/fs/cgroup/cpuset/cpuset.cpus
+ cat /sys/fs/cgroup/cpuset/cpuset.mems
+ mount --bind /sys/fs/cgroup/cpuset//kubelet /sys/fs/cgroup/cpuset//kubelet
+ IFS=
+ read -r subsystem
+ mount_kubelet_cgroup_root /kubelet /sys/fs/cgroup/cpu
+ local cgroup_root=/kubelet
+ local subsystem=/sys/fs/cgroup/cpu
+ '[' -z /kubelet ']'
+ mkdir -p /sys/fs/cgroup/cpu//kubelet
+ '[' /sys/fs/cgroup/cpu == /sys/fs/cgroup/cpuset ']'
+ mount --bind /sys/fs/cgroup/cpu//kubelet /sys/fs/cgroup/cpu//kubelet
+ IFS=
+ read -r subsystem
+ mount_kubelet_cgroup_root /kubelet /sys/fs/cgroup/cpuacct
+ local cgroup_root=/kubelet
+ local subsystem=/sys/fs/cgroup/cpuacct
+ '[' -z /kubelet ']'
+ mkdir -p /sys/fs/cgroup/cpuacct//kubelet
+ '[' /sys/fs/cgroup/cpuacct == /sys/fs/cgroup/cpuset ']'
+ mount --bind /sys/fs/cgroup/cpuacct//kubelet /sys/fs/cgroup/cpuacct//kubelet
+ IFS=
+ read -r subsystem
+ mount_kubelet_cgroup_root /kubelet /sys/fs/cgroup/blkio
+ local cgroup_root=/kubelet
+ local subsystem=/sys/fs/cgroup/blkio
+ '[' -z /kubelet ']'
+ mkdir -p /sys/fs/cgroup/blkio//kubelet
+ '[' /sys/fs/cgroup/blkio == /sys/fs/cgroup/cpuset ']'
+ mount --bind /sys/fs/cgroup/blkio//kubelet /sys/fs/cgroup/blkio//kubelet
+ IFS=
+ read -r subsystem
+ mount_kubelet_cgroup_root /kubelet /sys/fs/cgroup/memory
+ local cgroup_root=/kubelet
+ local subsystem=/sys/fs/cgroup/memory
+ '[' -z /kubelet ']'
+ mkdir -p /sys/fs/cgroup/memory//kubelet
+ '[' /sys/fs/cgroup/memory == /sys/fs/cgroup/cpuset ']'
+ mount --bind /sys/fs/cgroup/memory//kubelet /sys/fs/cgroup/memory//kubelet
+ IFS=
+ read -r subsystem
+ mount_kubelet_cgroup_root /kubelet /sys/fs/cgroup/devices
+ local cgroup_root=/kubelet
+ local subsystem=/sys/fs/cgroup/devices
+ '[' -z /kubelet ']'
+ mkdir -p /sys/fs/cgroup/devices//kubelet
+ '[' /sys/fs/cgroup/devices == /sys/fs/cgroup/cpuset ']'
+ mount --bind /sys/fs/cgroup/devices//kubelet /sys/fs/cgroup/devices//kubelet
+ IFS=
+ read -r subsystem
+ mount_kubelet_cgroup_root /kubelet /sys/fs/cgroup/freezer
+ local cgroup_root=/kubelet
+ local subsystem=/sys/fs/cgroup/freezer
+ '[' -z /kubelet ']'
+ mkdir -p /sys/fs/cgroup/freezer//kubelet
+ '[' /sys/fs/cgroup/freezer == /sys/fs/cgroup/cpuset ']'
+ mount --bind /sys/fs/cgroup/freezer//kubelet /sys/fs/cgroup/freezer//kubelet
+ IFS=
+ read -r subsystem
+ mount_kubelet_cgroup_root /kubelet /sys/fs/cgroup/net_cls
+ local cgroup_root=/kubelet
+ local subsystem=/sys/fs/cgroup/net_cls
+ '[' -z /kubelet ']'
+ mkdir -p /sys/fs/cgroup/net_cls//kubelet
+ '[' /sys/fs/cgroup/net_cls == /sys/fs/cgroup/cpuset ']'
+ mount --bind /sys/fs/cgroup/net_cls//kubelet /sys/fs/cgroup/net_cls//kubelet
+ IFS=
+ read -r subsystem
+ mount_kubelet_cgroup_root /kubelet /sys/fs/cgroup/perf_event
+ local cgroup_root=/kubelet
+ local subsystem=/sys/fs/cgroup/perf_event
+ '[' -z /kubelet ']'
+ mkdir -p /sys/fs/cgroup/perf_event//kubelet
+ '[' /sys/fs/cgroup/perf_event == /sys/fs/cgroup/cpuset ']'
+ mount --bind /sys/fs/cgroup/perf_event//kubelet /sys/fs/cgroup/perf_event//kubelet
+ IFS=
+ read -r subsystem
+ mount_kubelet_cgroup_root /kubelet /sys/fs/cgroup/net_prio
+ local cgroup_root=/kubelet
+ local subsystem=/sys/fs/cgroup/net_prio
+ '[' -z /kubelet ']'
+ mkdir -p /sys/fs/cgroup/net_prio//kubelet
+ '[' /sys/fs/cgroup/net_prio == /sys/fs/cgroup/cpuset ']'
+ mount --bind /sys/fs/cgroup/net_prio//kubelet /sys/fs/cgroup/net_prio//kubelet
+ IFS=
+ read -r subsystem
+ mount_kubelet_cgroup_root /kubelet /sys/fs/cgroup/hugetlb
+ local cgroup_root=/kubelet
+ local subsystem=/sys/fs/cgroup/hugetlb
+ '[' -z /kubelet ']'
+ mkdir -p /sys/fs/cgroup/hugetlb//kubelet
+ '[' /sys/fs/cgroup/hugetlb == /sys/fs/cgroup/cpuset ']'
+ mount --bind /sys/fs/cgroup/hugetlb//kubelet /sys/fs/cgroup/hugetlb//kubelet
+ IFS=
+ read -r subsystem
+ mount_kubelet_cgroup_root /kubelet /sys/fs/cgroup/pids
+ local cgroup_root=/kubelet
+ local subsystem=/sys/fs/cgroup/pids
+ '[' -z /kubelet ']'
+ mkdir -p /sys/fs/cgroup/pids//kubelet
+ '[' /sys/fs/cgroup/pids == /sys/fs/cgroup/cpuset ']'
+ mount --bind /sys/fs/cgroup/pids//kubelet /sys/fs/cgroup/pids//kubelet
+ IFS=
+ read -r subsystem
+ mount_kubelet_cgroup_root /kubelet /sys/fs/cgroup/cpuset/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
+ local cgroup_root=/kubelet
+ local subsystem=/sys/fs/cgroup/cpuset/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
+ '[' -z /kubelet ']'
+ mkdir -p /sys/fs/cgroup/cpuset/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c//kubelet
+ '[' /sys/fs/cgroup/cpuset/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c == /sys/fs/cgroup/cpuset ']'
+ mount --bind /sys/fs/cgroup/cpuset/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c//kubelet /sys/fs/cgroup/cpuset/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c//kubelet
+ IFS=
+ read -r subsystem
+ mount_kubelet_cgroup_root /kubelet /sys/fs/cgroup/cpu/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
+ local cgroup_root=/kubelet
+ local subsystem=/sys/fs/cgroup/cpu/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
+ '[' -z /kubelet ']'
+ mkdir -p /sys/fs/cgroup/cpu/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c//kubelet
+ '[' /sys/fs/cgroup/cpu/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c == /sys/fs/cgroup/cpuset ']'
+ mount --bind /sys/fs/cgroup/cpu/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c//kubelet /sys/fs/cgroup/cpu/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c//kubelet
+ IFS=
+ read -r subsystem
+ mount_kubelet_cgroup_root /kubelet /sys/fs/cgroup/cpuacct/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
+ local cgroup_root=/kubelet
+ local subsystem=/sys/fs/cgroup/cpuacct/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
+ '[' -z /kubelet ']'
+ mkdir -p /sys/fs/cgroup/cpuacct/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c//kubelet
+ '[' /sys/fs/cgroup/cpuacct/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c == /sys/fs/cgroup/cpuset ']'
+ mount --bind /sys/fs/cgroup/cpuacct/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c//kubelet /sys/fs/cgroup/cpuacct/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c//kubelet
+ IFS=
+ read -r subsystem
+ mount_kubelet_cgroup_root /kubelet /sys/fs/cgroup/blkio/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
+ local cgroup_root=/kubelet
+ local subsystem=/sys/fs/cgroup/blkio/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
+ '[' -z /kubelet ']'
+ mkdir -p /sys/fs/cgroup/blkio/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c//kubelet
+ '[' /sys/fs/cgroup/blkio/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c == /sys/fs/cgroup/cpuset ']'
+ mount --bind /sys/fs/cgroup/blkio/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c//kubelet /sys/fs/cgroup/blkio/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c//kubelet
+ IFS=
+ read -r subsystem
+ mount_kubelet_cgroup_root /kubelet /sys/fs/cgroup/memory/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
+ local cgroup_root=/kubelet
+ local subsystem=/sys/fs/cgroup/memory/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
+ '[' -z /kubelet ']'
+ mkdir -p /sys/fs/cgroup/memory/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c//kubelet
+ '[' /sys/fs/cgroup/memory/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c == /sys/fs/cgroup/cpuset ']'
+ mount --bind /sys/fs/cgroup/memory/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c//kubelet /sys/fs/cgroup/memory/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c//kubelet
+ IFS=
+ read -r subsystem
+ mount_kubelet_cgroup_root /kubelet /sys/fs/cgroup/devices/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
+ local cgroup_root=/kubelet
+ local subsystem=/sys/fs/cgroup/devices/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
+ '[' -z /kubelet ']'
+ mkdir -p /sys/fs/cgroup/devices/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c//kubelet
+ '[' /sys/fs/cgroup/devices/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c == /sys/fs/cgroup/cpuset ']'
+ mount --bind /sys/fs/cgroup/devices/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c//kubelet /sys/fs/cgroup/devices/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c//kubelet
+ IFS=
+ read -r subsystem
+ mount_kubelet_cgroup_root /kubelet /sys/fs/cgroup/freezer/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
+ local cgroup_root=/kubelet
+ local subsystem=/sys/fs/cgroup/freezer/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
+ '[' -z /kubelet ']'
+ mkdir -p /sys/fs/cgroup/freezer/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c//kubelet
+ '[' /sys/fs/cgroup/freezer/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c == /sys/fs/cgroup/cpuset ']'
+ mount --bind /sys/fs/cgroup/freezer/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c//kubelet /sys/fs/cgroup/freezer/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c//kubelet
+ IFS=
+ read -r subsystem
+ mount_kubelet_cgroup_root /kubelet /sys/fs/cgroup/net_cls/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
+ local cgroup_root=/kubelet
+ local subsystem=/sys/fs/cgroup/net_cls/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
+ '[' -z /kubelet ']'
+ mkdir -p /sys/fs/cgroup/net_cls/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c//kubelet
+ '[' /sys/fs/cgroup/net_cls/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c == /sys/fs/cgroup/cpuset ']'
+ mount --bind /sys/fs/cgroup/net_cls/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c//kubelet /sys/fs/cgroup/net_cls/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c//kubelet
+ IFS=
+ read -r subsystem
+ mount_kubelet_cgroup_root /kubelet /sys/fs/cgroup/perf_event/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
+ local cgroup_root=/kubelet
+ local subsystem=/sys/fs/cgroup/perf_event/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
+ '[' -z /kubelet ']'
+ mkdir -p /sys/fs/cgroup/perf_event/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c//kubelet
+ '[' /sys/fs/cgroup/perf_event/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c == /sys/fs/cgroup/cpuset ']'
+ mount --bind /sys/fs/cgroup/perf_event/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c//kubelet /sys/fs/cgroup/perf_event/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c//kubelet
+ IFS=
+ read -r subsystem
+ mount_kubelet_cgroup_root /kubelet /sys/fs/cgroup/net_prio/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
+ local cgroup_root=/kubelet
+ local subsystem=/sys/fs/cgroup/net_prio/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
+ '[' -z /kubelet ']'
+ mkdir -p /sys/fs/cgroup/net_prio/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c//kubelet
+ '[' /sys/fs/cgroup/net_prio/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c == /sys/fs/cgroup/cpuset ']'
+ mount --bind /sys/fs/cgroup/net_prio/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c//kubelet /sys/fs/cgroup/net_prio/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c//kubelet
+ IFS=
+ read -r subsystem
+ mount_kubelet_cgroup_root /kubelet /sys/fs/cgroup/hugetlb/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
+ local cgroup_root=/kubelet
+ local subsystem=/sys/fs/cgroup/hugetlb/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
+ '[' -z /kubelet ']'
+ mkdir -p /sys/fs/cgroup/hugetlb/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c//kubelet
+ '[' /sys/fs/cgroup/hugetlb/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c == /sys/fs/cgroup/cpuset ']'
+ mount --bind /sys/fs/cgroup/hugetlb/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c//kubelet /sys/fs/cgroup/hugetlb/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c//kubelet
+ IFS=
+ read -r subsystem
+ mount_kubelet_cgroup_root /kubelet /sys/fs/cgroup/pids/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
+ local cgroup_root=/kubelet
+ local subsystem=/sys/fs/cgroup/pids/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c
+ '[' -z /kubelet ']'
+ mkdir -p /sys/fs/cgroup/pids/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c//kubelet
+ '[' /sys/fs/cgroup/pids/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c == /sys/fs/cgroup/cpuset ']'
+ mount --bind /sys/fs/cgroup/pids/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c//kubelet /sys/fs/cgroup/pids/docker/b54f7ec25f7cfcd9edea8523c936e9c8926cb44362a10af3f1a7d102ecb7638c//kubelet
+ IFS=
+ read -r subsystem
+ mount_kubelet_cgroup_root /kubelet /sys/fs/cgroup/cpuset/kubelet
+ local cgroup_root=/kubelet
+ local subsystem=/sys/fs/cgroup/cpuset/kubelet
+ '[' -z /kubelet ']'
+ mkdir -p /sys/fs/cgroup/cpuset/kubelet//kubelet
+ '[' /sys/fs/cgroup/cpuset/kubelet == /sys/fs/cgroup/cpuset ']'
+ mount --bind /sys/fs/cgroup/cpuset/kubelet//kubelet /sys/fs/cgroup/cpuset/kubelet//kubelet
+ IFS=
+ read -r subsystem
+ mount_kubelet_cgroup_root /kubelet /sys/fs/cgroup/cpu/kubelet
+ local cgroup_root=/kubelet
+ local subsystem=/sys/fs/cgroup/cpu/kubelet
+ '[' -z /kubelet ']'
+ mkdir -p /sys/fs/cgroup/cpu/kubelet//kubelet
+ '[' /sys/fs/cgroup/cpu/kubelet == /sys/fs/cgroup/cpuset ']'
+ mount --bind /sys/fs/cgroup/cpu/kubelet//kubelet /sys/fs/cgroup/cpu/kubelet//kubelet
+ IFS=
+ read -r subsystem
+ mount_kubelet_cgroup_root /kubelet /sys/fs/cgroup/cpuacct/kubelet
+ local cgroup_root=/kubelet
+ local subsystem=/sys/fs/cgroup/cpuacct/kubelet
+ '[' -z /kubelet ']'
+ mkdir -p /sys/fs/cgroup/cpuacct/kubelet//kubelet
+ '[' /sys/fs/cgroup/cpuacct/kubelet == /sys/fs/cgroup/cpuset ']'
+ mount --bind /sys/fs/cgroup/cpuacct/kubelet//kubelet /sys/fs/cgroup/cpuacct/kubelet//kubelet
+ IFS=
+ read -r subsystem
+ mount_kubelet_cgroup_root /kubelet /sys/fs/cgroup/blkio/kubelet
+ local cgroup_root=/kubelet
+ local subsystem=/sys/fs/cgroup/blkio/kubelet
+ '[' -z /kubelet ']'
+ mkdir -p /sys/fs/cgroup/blkio/kubelet//kubelet
+ '[' /sys/fs/cgroup/blkio/kubelet == /sys/fs/cgroup/cpuset ']'
+ mount --bind /sys/fs/cgroup/blkio/kubelet//kubelet /sys/fs/cgroup/blkio/kubelet//kubelet
+ IFS=
+ read -r subsystem
+ mount_kubelet_cgroup_root /kubelet /sys/fs/cgroup/memory/kubelet
+ local cgroup_root=/kubelet
+ local subsystem=/sys/fs/cgroup/memory/kubelet
+ '[' -z /kubelet ']'
+ mkdir -p /sys/fs/cgroup/memory/kubelet//kubelet
+ '[' /sys/fs/cgroup/memory/kubelet == /sys/fs/cgroup/cpuset ']'
+ mount --bind /sys/fs/cgroup/memory/kubelet//kubelet /sys/fs/cgroup/memory/kubelet//kubelet
+ IFS=
+ read -r subsystem
+ mount_kubelet_cgroup_root /kubelet /sys/fs/cgroup/devices/kubelet
+ local cgroup_root=/kubelet
+ local subsystem=/sys/fs/cgroup/devices/kubelet
+ '[' -z /kubelet ']'
+ mkdir -p /sys/fs/cgroup/devices/kubelet//kubelet
+ '[' /sys/fs/cgroup/devices/kubelet == /sys/fs/cgroup/cpuset ']'
+ mount --bind /sys/fs/cgroup/devices/kubelet//kubelet /sys/fs/cgroup/devices/kubelet//kubelet
+ IFS=
+ read -r subsystem
+ mount_kubelet_cgroup_root /kubelet /sys/fs/cgroup/freezer/kubelet
+ local cgroup_root=/kubelet
+ local subsystem=/sys/fs/cgroup/freezer/kubelet
+ '[' -z /kubelet ']'
+ mkdir -p /sys/fs/cgroup/freezer/kubelet//kubelet
+ '[' /sys/fs/cgroup/freezer/kubelet == /sys/fs/cgroup/cpuset ']'
+ mount --bind /sys/fs/cgroup/freezer/kubelet//kubelet /sys/fs/cgroup/freezer/kubelet//kubelet
+ IFS=
+ read -r subsystem
+ mount_kubelet_cgroup_root /kubelet /sys/fs/cgroup/net_cls/kubelet
+ local cgroup_root=/kubelet
+ local subsystem=/sys/fs/cgroup/net_cls/kubelet
+ '[' -z /kubelet ']'
+ mkdir -p /sys/fs/cgroup/net_cls/kubelet//kubelet
+ '[' /sys/fs/cgroup/net_cls/kubelet == /sys/fs/cgroup/cpuset ']'
+ mount --bind /sys/fs/cgroup/net_cls/kubelet//kubelet /sys/fs/cgroup/net_cls/kubelet//kubelet
+ IFS=
+ read -r subsystem
+ mount_kubelet_cgroup_root /kubelet /sys/fs/cgroup/perf_event/kubelet
+ local cgroup_root=/kubelet
+ local subsystem=/sys/fs/cgroup/perf_event/kubelet
+ '[' -z /kubelet ']'
+ mkdir -p /sys/fs/cgroup/perf_event/kubelet//kubelet
+ '[' /sys/fs/cgroup/perf_event/kubelet == /sys/fs/cgroup/cpuset ']'
+ mount --bind /sys/fs/cgroup/perf_event/kubelet//kubelet /sys/fs/cgroup/perf_event/kubelet//kubelet
+ IFS=
+ read -r subsystem
+ mount_kubelet_cgroup_root /kubelet /sys/fs/cgroup/net_prio/kubelet
+ local cgroup_root=/kubelet
+ local subsystem=/sys/fs/cgroup/net_prio/kubelet
+ '[' -z /kubelet ']'
+ mkdir -p /sys/fs/cgroup/net_prio/kubelet//kubelet
+ '[' /sys/fs/cgroup/net_prio/kubelet == /sys/fs/cgroup/cpuset ']'
+ mount --bind /sys/fs/cgroup/net_prio/kubelet//kubelet /sys/fs/cgroup/net_prio/kubelet//kubelet
+ IFS=
+ read -r subsystem
+ mount_kubelet_cgroup_root /kubelet /sys/fs/cgroup/hugetlb/kubelet
+ local cgroup_root=/kubelet
+ local subsystem=/sys/fs/cgroup/hugetlb/kubelet
+ '[' -z /kubelet ']'
+ mkdir -p /sys/fs/cgroup/hugetlb/kubelet//kubelet
+ '[' /sys/fs/cgroup/hugetlb/kubelet == /sys/fs/cgroup/cpuset ']'
+ mount --bind /sys/fs/cgroup/hugetlb/kubelet//kubelet /sys/fs/cgroup/hugetlb/kubelet//kubelet
+ IFS=
+ read -r subsystem
+ mount_kubelet_cgroup_root /kubelet /sys/fs/cgroup/pids/kubelet
+ local cgroup_root=/kubelet
+ local subsystem=/sys/fs/cgroup/pids/kubelet
+ '[' -z /kubelet ']'
+ mkdir -p /sys/fs/cgroup/pids/kubelet//kubelet
+ '[' /sys/fs/cgroup/pids/kubelet == /sys/fs/cgroup/cpuset ']'
+ mount --bind /sys/fs/cgroup/pids/kubelet//kubelet /sys/fs/cgroup/pids/kubelet//kubelet
+ IFS=
+ read -r subsystem
root@k8s-guide-control-plane:/# echo $?
0 |
|
hmm, these are how it looks like in my working system (Linux), maybe we can spot some differences? |
|
you don't have |
|
yep, I can confirm that if I I think it fails because could it be because WSL doesn't run systemd itself? |
yep, and this issue on wls confirms it microsoft/WSL#4189 (comment) @networkop can you submit a patch to add to the entrypoint script kind/images/base/files/usr/local/bin/entrypoint Lines 242 to 245 in 9e6e911 please add a comment explaining the reason referencing this issue, |
kind is expected to work on systems without systemd (see e.g. #2091) but for practical reasons kubernetes, containerd, kind, etc. .. the ecosystem is tested on systemd. |
|
I think the issue here may be the interaction with the containerized systemd + kubelet running on a host that does not itself run systemd. Containerized systemd expects to either do all of the mounting itself or have the hierarchy already fully mounted IIRC, so it wouldn't mount this. We should probably ensure that path and ensure it is configured like a systemd host would (see comment on the PR).
We're nearly always running without cgroupv2 (though the kind project does have a CI job to test cgroupv2 specifically), this may change in the future but currently it also applies to Kubernetes core. Running without cgroupv2 should be fine. We do however need the cgroup-root workaround, as we often run CI inside of Kubernetes because that is how the Kubernetes project largely operates currently. |
For details see kubernetes-sigs/kind#2323 and kubernetes-sigs/kind#2390
|
@BenTheElder; I was wondering if you would mind letting me know which release of kind might include this fix? I'm on 0.11.1 on WSL2 w/out Docker Desktop installed and I'm getting the dreaded kubelet cgroup error during kubelet bootstrap. |
|
next version 0.12.0, or you can use kind from HEAd |
|
@aojea; first, thanks for your prompt response! Second, do you have any rough idea when 0.12.0 might be released? |
|
we want to do it soon, but I can't promise anything, unfortunately we are being very busy this days |
|
I got to install 'kind' because some articles recommended to use it instead of minikube on wsl. Which kubernetes implementation does not require systemd and runs well inside wsl ubuntu, so that I do not need to use docker-desktop for windows? |
|
|
This is a regression in v0.11.X, it is fixed in the latest sources, and it is not present in previous releases. You can use an older release or build from the latest sources (clone the repo in WSL/linux/macOS, run All Kubernetes distros I'm aware of use systemd. Here we had a bug triggered when systemd was not used on the host machine. After the regression fix systemd is not required in the host environment but it is and will continue to be used within the Kubernetes node containers by KIND and within the Kubernetes nodes by every other major Kubernetes distro because it is free software that performs the init / PID1 task well. Nearly all major linux distros use systemd and Kubernetes upstream is developed exclusively with systemd, so tricky system interactions like this can go uncaught. Testing every possible combination of system software is prohibitive and Kubernetes's CI and GitHub actions lack support for Windows (nested) virtualization so WSL2 CI is not available. #1529 If you're interested in solving this. |
|
for those still experiencing this issue on |
* update scoretrak+server helm chart * update scoretrak+server helm chart * upgrade versions * upgrade versions * upgrade versions * upgrade versions * upgrade versions * upgrade versions * upgrade versions * upgrade versions * upgrade versions * upgrade versions * update chart for testing * change bank condition and fqdn * change bank sub charts conditions * provide container version tag value * change payment user cmd * use newer image * update versions * fix kubernetes-sigs/kind#2323 (comment) for wsl2. * update versions Co-authored-by: Ibrahim Diallo <[email protected]>
|
I've hit this bug again since some months ago. (Yes I know I could clone and build HEAD, but that's unnecessary hassle, when you have a build CI) Forgive me, I don't doubt that you are busy, but it seems a bit silly to be busy with development if that development does not get released (since 9 months now) ? Or maybe I am wrong and there is not CI to make a release simple ? |
|
2 outstanding issues https://github.com/kubernetes-sigs/kind/milestone/15 for release :( |
@aojea and I are the two active maintainers right now. There are only a few recurring contributors and maintainers. Of those, none of us work on KIND exclusively / full-time at this point. KIND is not commercial software. I myself also chair + tech lead Kubernetes SIG Testing, co- or solo- maintain many other parts of the Kubernetes project, and I have other work at my job. Unfortunately, throughout the Kubernetes project you will find that the bugs / issues and work required outpace the consistently available developer time. I've had to pick up the slack more in other critical areas, and I'm working on finding people to take over some of those.
This is not the issue, there is build tooling - though we do more than that and organize and write up the changes, emphasizing the most notable ones to make upgrades easier, that takes maybe an afternoon. In this repo we track releases in milestones, like much of the rest of the Kubernetes organization. There are outstanding regressions that will notably affect users, so we have not released. Incoming changes that "fix" things have caused more regressions. It is not so simple to cover supporting all these things, as mentioned last time I commented in this thread: for Windows we do not have CI for and nobody has helped resolve that #1529. As a result regressions to e.g. Windows quirks are difficult to catch (or in this case rootless which we did add CI for, but now linux without /dev/fuse is hit due to the rootless fixes ... ). As Antonio linked above, the v0.12.0 milestone shows what remains before we're ready for a general v0.12.0 release. If you're happy with the current state and won't be affected by these, it is trivial to obtain a binary, but it will not be a supported release on our part due to these outstanding issues. I have only been accepting fix PRs and documentation improvements, but we're not there yet, and there is no ETA. If folks are interested in helping work on these problems, we have a detailed contributor guide. |
|
I hope you are feeling well again, sorry to hear that you and your family had some sadness 😞 . Personally I wouldn't even begin to know where to start with #1529. |
Works for me (wsl2，ubuntu20.04, kind 0.11) |
That was perfect. You have saved my life man! 👍 |
ERROR: failed to create cluster:
ERROR: failed to create cluster: failed to init node with kubeadm: command "docker exec --privileged kind-control-plane kubeadm init --skip-phases=preflight --config=/kind/kubeadm.conf --skip-token-print --v=6" failed with error: exit status 1
Command Output: I0622 15:16:13.468494 216 initconfiguration.go:246] loading configuration from "/kind/kubeadm.conf"
What is Expected:
cluster should be created without any error.
How to reproduce it:
run below command to reproduce it :
$ kind create cluster
Anything else we need to know?:
I have recently install ubuntu as virtual machine on windows 10 as wsl 2.
running ubuntu on windows terminal as admin user, also installed docker and set it as non root user.
below I am providing environment related information.
ENVIRONMENT:
Ubuntu
command used
$lsb_release -aNo LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 20.04.2 LTS
Release: 20.04
Codename: focal
Kubectl Installation
https://kubernetes.io/docs/tasks/tools/install-kubectl-linux/#install-using-native-package-management
Kubectl Version
command used
kubectl version --clientClient Version:
version.Info{
Major:"1", Minor:"21",
GitVersion:"v1.21.2",
GitCommit:"092fbfbf53427de67cac1e9fa54aaa09a28371d7",
GitTreeState:"clean",
BuildDate:"2021-06-16T12:59:11Z",
GoVersion:"go1.16.5",
Compiler:"gc",
Platform:"linux/amd64"
}
kind Installtion :
curl -Lo ./kind https://kind.sigs.k8s.io/dl/v0.11.1/kind-linux-amd64
chmod +x ./kind
sudo mv ./kind /usr/local/bin/kind
kind version: :
command used
$kind versionkind v0.11.1 go1.16.4 linux/amd64
docker info
command used
$docker infoClient:
Context: default
Debug Mode: false
Plugins:
app: Docker App (Docker Inc., v0.9.1-beta3)
buildx: Build with BuildKit (Docker Inc., v0.5.1-docker)
scan: Docker Scan (Docker Inc., v0.8.0)
Server:
Containers: 1
Running: 0
Paused: 0
Stopped: 1
Images: 2
Server Version: 20.10.7
Storage Driver: overlay2
Backing Filesystem: extfs
Supports d_type: true
Native Overlay Diff: true
userxattr: false
Logging Driver: json-file
Cgroup Driver: cgroupfs
Cgroup Version: 1
Plugins:
Volume: local
Network: bridge host ipvlan macvlan null overlay
Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
Swarm: inactive
Runtimes: runc io.containerd.runc.v2 io.containerd.runtime.v1.linux
Default Runtime: runc
Init Binary: docker-init
containerd version: d71fcd7d8303cbf684402823e425e9dd2e99285d
runc version: b9ee9c6314599f1b4a7f497e1f1f856fe433d3b7
init version: de40ad0
Security Options:
seccomp
Profile: default
Kernel Version: 5.4.72-microsoft-standard-WSL2
Operating System: Ubuntu 20.04.2 LTS
OSType: linux
Architecture: x86_64
CPUs: 12
Total Memory: 6.133GiB
Name: LAPTOP-TN6NO0LS
ID: JDCK:NRQ2:ML5P:EUMK:OBYG:76PM:5SXD:FMYK:KHCX:NDTB:IQ4R:KIBJ
Docker Root Dir: /var/lib/docker
Debug Mode: false
Registry: https://index.docker.io/v1/
Labels:
Experimental: false
Insecure Registries:
127.0.0.0/8
Live Restore Enabled: false
WARNING: No blkio throttle.read_bps_device support
WARNING: No blkio throttle.write_bps_device support
WARNING: No blkio throttle.read_iops_device support
WARNING: No blkio throttle.write_iops_device support
The text was updated successfully, but these errors were encountered: