Cannot run kind from HEAD #1634
Comments
|
Re: cgroups ... cgroupsv2 is a kernel boot parameter. It's possible your setup was updated to use it, if cgroupsv2 is enabled kubernetes does not work (or runc or ...) If previous versions of kind are working right now then you're not in cgroupsv2. |
|
Previous versions work fine |
|
also to confirm: do you have kubernetes checked out to HEAD? currently updating everything on glinux before attempting repro.. |
|
I was using |
Possibly the difference in these two images? |
|
I built from k8s HEAD with the kind HEAD binary, as well as the new |
|
/priority critical-urgent |
k8s-ci-robot
added
the
priority/critical-urgent
|
i have glinux fully updated, bisecting kind. |
|
I'm not 100% sure but I think it broke from the containerd PR linked in the
description
…On Fri, May 29, 2020, 6:14 PM Benjamin Elder ***@***.***> wrote:
i have glinux fully updated, bisecting kind.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
<#1634 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAEYGXIRDLCHRVPXOPTVR33RUBMXZANCNFSM4NOLJLJA>
.
|
|
fce9435 definitely seems to be the culprit, now to identify which change ... |
|
tentatively just rolling back containerd is enough to fix diff --git a/images/base/Dockerfile b/images/base/Dockerfile
index 0f1a2e2f..067b8147 100644
--- a/images/base/Dockerfile
+++ b/images/base/Dockerfile
@@ -23,7 +23,7 @@ FROM ubuntu:20.04
# Configure containerd and runc binaries from kind-ci/containerd-nightlies repository
# The repository contains latest stable releases and nightlies built for multiple architectures
-ARG CONTAINERD_VERSION="v1.4.0-beta.0-2-g6312b52d"
+ARG CONTAINERD_VERSION="v1.3.3-14-g449e9269"
# Configure CNI binaries from upstream
ARG CNI_VERSION="v0.8.6"
# Configure crictl binary from upstream |
... that's no good. |
|
I'll file a revert and follow up more later tonight. |
|
|
|
working on a PR to switch to the latest 1.3.X, will need to follow up with identifying / fixing the breakage upstream and re-opening #1560 in the meantime |
k8s-ci-robot
added
the
lifecycle/active
|
tested #1635 on my glinux box, still need to identify the root cause of the cgroups issue in newer builds |
|
@howardjohn please let me know if you see issues after this #1635 I'm going to focus on some CI issues for a bit before figuring out how to roll containerd forward. |
|
tracking follow-up in #1637 |
|
Narrowed it down to a handful of containerd commits, I suspect some CRI changes around hugetlb related to Kubernetes support for it (inadvertently?) made that cgroup required, which glinux does not ship currently (though docker for mac appears to). poked some upstream maintainers. |
What happened:
I ran
go build ./cmd/kind && ./kind create clusterand it did not startWhat you expected to happen:
Cluster is created
How to reproduce it (as minimally and precisely as possible):
I would imagine running on gLinux will reproduce, but not 100% sure
Anything else we need to know?:
This started from #1599. I would have just commented there but don't won't to get buried in an old PR.
In the PR "Kubernetes doesn't currently support cgroupsv2, when it does I'm not sure how quickly we'll be able to.". I am a bit confused by this. My system works entirely fine on previous versions of kind, so it seems this is an issue with
kindnot supporting rather than Kubernetes.Full logs:
Environment:
kind version): from 94031cdkubectl version): 1.18.2docker info): 19.03.5/etc/os-release):5.2.17-1rodete3-amd64 #1 SMP Debian 5.2.17-1rodete3The text was updated successfully, but these errors were encountered: