minikube couldn't initialize a Kubernetes cluster #14813
Comments
|
same problem, confusing me —>kubelet service is not enabled |
I got the same problem. |
|
Hi @bitpeng – for additional assistance, please consider reaching out to the minikube community: https://minikube.sigs.k8s.io/community/ We also offer support through Slack, Groups, and office hours. /triage needs-information |
k8s-ci-robot
added
triage/needs-information
|
The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs. This bot triages issues and PRs according to the following rules:
You can:
Please send feedback to sig-contributor-experience at kubernetes/community. /lifecycle stale |
k8s-ci-robot
added
the
lifecycle/stale
|
Hi @bitpeng, the cause is the docker container-runtime is trying to pull something from k8s.gcr.io but because you're in China it's blocked and it's causing your minikube to fail starting. We should investigate why it's trying to access it. |
|
/remove-lifecycle stale |
k8s-ci-robot
removed
the
lifecycle/stale
|
same problem |
|
I thiink there is a workaround: Fall back to kubernetes v1.23.8 or lower Use To be honest, I don't know why this is happening at all...(I have explained what I found to @spowelljr) |
|
The Kubernetes project currently lacks enough contributors to adequately respond to all issues. This bot triages un-triaged issues according to the following rules:
You can:
Please send feedback to sig-contributor-experience at kubernetes/community. /lifecycle stale |
k8s-ci-robot
added
the
lifecycle/stale
|
The Kubernetes project currently lacks enough active contributors to adequately respond to all issues. This bot triages un-triaged issues according to the following rules:
You can:
Please send feedback to sig-contributor-experience at kubernetes/community. /lifecycle rotten |
k8s-ci-robot
added
lifecycle/rotten
|
The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs. This bot triages issues according to the following rules:
You can:
Please send feedback to sig-contributor-experience at kubernetes/community. /close not-planned |
|
@k8s-triage-robot: Closing this issue, marking it as "Not Planned". In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
What Happened?
$$ lsb_release
LSB Version: :core-4.1-amd64:core-4.1-noarch:cxx-4.1-amd64:cxx-4.1-noarch:desktop-4.1-amd64:desktop-4.1-noarch:languages-4.1-amd64:languages-4.1-noarch:printing-4.1-amd64:printing-4.1-noarch
[chensq20@dev-nmg-huhehaote4-devtest-214 ~]$ -a
-bash: -a: command not found
[chensq20@dev-nmg-huhehaote4-devtest-214 ~]$ lsb_release -a
LSB Version: :core-4.1-amd64:core-4.1-noarch:cxx-4.1-amd64:cxx-4.1-noarch:desktop-4.1-amd64:desktop-4.1-noarch:languages-4.1-amd64:languages-4.1-noarch:printing-4.1-amd64:printing-4.1-noarch
Distributor ID: CentOS
Description: CentOS Linux release 7.9.2009 (Core)
Release: 7.9.2009
Codename: Core
$$ docker version
Client: Docker Engine - Community
Version: 20.10.17
API version: 1.41
Go version: go1.17.11
Git commit: 100c701
Built: Mon Jun 6 23:05:12 2022
OS/Arch: linux/amd64
Context: default
Experimental: true
Server: Docker Engine - Community
Engine:
Version: 20.10.17
API version: 1.41 (minimum version 1.12)
Go version: go1.17.11
Git commit: a89b842
Built: Mon Jun 6 23:03:33 2022
OS/Arch: linux/amd64
Experimental: false
containerd:
Version: 1.6.7
GitCommit: 0197261a30bf81f1ee8e6a4dd2dea0ef95d67ccb
runc:
Version: 1.1.3
GitCommit: v1.1.3-0-g6724737
docker-init:
Version: 0.19.0
GitCommit: de40ad0
$$ minikube start --image-repository=registry.cn-hangzhou.aliyuncs.com/google_containers
! initialization failed, will try again: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.24.3:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,Mem,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.24.3
[preflight] Running pre-flight checks
[preflight] The system verification failed. Printing the output from the verification:
KERNEL_VERSION: 3.10.0-1160.36.2.el7.x86_64
OS: Linux
CGROUPS_CPU: enabled
CGROUPS_CPUACCT: enabled
CGROUPS_CPUSET: enabled
CGROUPS_DEVICES: enabled
CGROUPS_FREEZER: enabled
CGROUPS_MEMORY: enabled
CGROUPS_PIDS: enabled
CGROUPS_HUGETLB: enabled
CGROUPS_BLKIO: enabled
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Generating "apiserver-kubelet-client" certificate and key
[certs] Generating "front-proxy-ca" certificate and key
[certs] Generating "front-proxy-client" certificate and key
[certs] Generating "etcd/ca" certificate and key
[certs] Generating "etcd/server" certificate and key
[certs] etcd/server serving cert is signed for DNS names [localhost minikube] and IPs [192.168.49.2 127.0.0.1 ::1]
[certs] Generating "etcd/peer" certificate and key
[certs] etcd/peer serving cert is signed for DNS names [localhost minikube] and IPs [192.168.49.2 127.0.0.1 ::1]
[certs] Generating "etcd/healthcheck-client" certificate and key
[certs] Generating "apiserver-etcd-client" certificate and key
[certs] Generating "sa" key and public key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI.
Here is one example how you may list all running Kubernetes containers by using crictl:
- 'crictl --runtime-endpoint unix:///var/run/cri-dockerd.sock ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'crictl --runtime-endpoint unix:///var/run/cri-dockerd.sock logs CONTAINERID'
stderr:
W0818 13:32:48.291504 1717 initconfiguration.go:120] Usage of CRI endpoints without URL scheme is deprecated and can cause kubelet errors in the future. Automatically prepending scheme "unix" to the "criSocket" with value "/var/run/cri-dockerd.sock". Please update your configuration!
[WARNING FileContent--proc-sys-net-bridge-bridge-nf-call-iptables]: /proc/sys/net/bridge/bridge-nf-call-iptables does not exist
[WARNING SystemVerification]: failed to parse kernel config: unable to load kernel module: "configs", output: "modprobe: FATAL: Module configs not found in directory /lib/modules/3.10.0-1160.36.2.el7.x86_64\n", err: exit status 1
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
Attach the log file
`
[Service]
ExecStart=
ExecStart=/var/lib/minikube/binaries/v1.24.3/kubelet --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --config=/var/lib/kubelet/config.yaml --container-runtime=remote --container-runtime-endpoint=/var/run/cri-dockerd.sock --hostname-override=minikube --image-service-endpoint=/var/run/cri-dockerd.sock --kubeconfig=/etc/kubernetes/kubelet.conf --node-ip=192.168.49.2 --pod-infra-container-image=registry.cn-hangzhou.aliyuncs.com/google_containers/pause:3.7 --runtime-request-timeout=15m
[Install]
config:
{KubernetesVersion:v1.24.3 ClusterName:minikube Namespace:default APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin: FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository:registry.cn-hangzhou.aliyuncs.com/google_containers LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:}
I0818 21:32:38.376831 313848 ssh_runner.go:195] Run: sudo ls /var/lib/minikube/binaries/v1.24.3
I0818 21:32:38.387205 313848 binaries.go:47] Didn't find k8s binaries: sudo ls /var/lib/minikube/binaries/v1.24.3: Process exited with status 2
stdout:
stderr:
ls: cannot access '/var/lib/minikube/binaries/v1.24.3': No such file or directory
Initiating transfer...
I0818 21:32:38.387259 313848 ssh_runner.go:195] Run: sudo mkdir -p /var/lib/minikube/binaries/v1.24.3
I0818 21:32:38.397259 313848 download.go:101] Downloading: https://kubernetes.oss-cn-hangzhou.aliyuncs.com/kubernetes-release/release/v1.24.3/bin/linux/amd64/kubectl?checksum=file:https://kubernetes.oss-cn-hangzhou.aliyuncs.com/kubernetes-release/release/v1.24.3/bin/linux/amd64/kubectl.sha256 -> /home/chensq20/.minikube/cache/linux/amd64/v1.24.3/kubectl
I0818 21:32:38.397266 313848 download.go:101] Downloading: https://kubernetes.oss-cn-hangzhou.aliyuncs.com/kubernetes-release/release/v1.24.3/bin/linux/amd64/kubelet?checksum=file:https://kubernetes.oss-cn-hangzhou.aliyuncs.com/kubernetes-release/release/v1.24.3/bin/linux/amd64/kubelet.sha256 -> /home/chensq20/.minikube/cache/linux/amd64/v1.24.3/kubelet
I0818 21:32:38.397605 313848 download.go:101] Downloading: https://kubernetes.oss-cn-hangzhou.aliyuncs.com/kubernetes-release/release/v1.24.3/bin/linux/amd64/kubeadm?checksum=file:https://kubernetes.oss-cn-hangzhou.aliyuncs.com/kubernetes-release/release/v1.24.3/bin/linux/amd64/kubeadm.sha256 -> /home/chensq20/.minikube/cache/linux/amd64/v1.24.3/kubeadm
I0818 21:32:41.323246 313848 ssh_runner.go:195] Run: stat -c "%!s(MISSING) %!y(MISSING)" /var/lib/minikube/binaries/v1.24.3/kubeadm
I0818 21:32:41.328471 313848 ssh_runner.go:352] existence check for /var/lib/minikube/binaries/v1.24.3/kubeadm: stat -c "%!s(MISSING) %!y(MISSING)" /var/lib/minikube/binaries/v1.24.3/kubeadm: Process exited with status 1
stdout:
stderr:
stat: cannot stat '/var/lib/minikube/binaries/v1.24.3/kubeadm': No such file or directory
I0818 21:32:41.328492 313848 ssh_runner.go:362] scp /home/chensq20/.minikube/cache/linux/amd64/v1.24.3/kubeadm --> /var/lib/minikube/binaries/v1.24.3/kubeadm (44376064 bytes)
I0818 21:32:41.394168 313848 ssh_runner.go:195] Run: stat -c "%!s(MISSING) %!y(MISSING)" /var/lib/minikube/binaries/v1.24.3/kubectl
I0818 21:32:41.404296 313848 ssh_runner.go:352] existence check for /var/lib/minikube/binaries/v1.24.3/kubectl: stat -c "%!s(MISSING) %!y(MISSING)" /var/lib/minikube/binaries/v1.24.3/kubectl: Process exited with status 1
stdout:
stderr:
stat: cannot stat '/var/lib/minikube/binaries/v1.24.3/kubectl': No such file or directory
I0818 21:32:41.404316 313848 ssh_runner.go:362] scp /home/chensq20/.minikube/cache/linux/amd64/v1.24.3/kubectl --> /var/lib/minikube/binaries/v1.24.3/kubectl (45711360 bytes)
I0818 21:32:44.450092 313848 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service kubelet
I0818 21:32:45.024595 313848 ssh_runner.go:195] Run: stat -c "%!s(MISSING) %!y(MISSING)" /var/lib/minikube/binaries/v1.24.3/kubelet
I0818 21:32:45.028503 313848 ssh_runner.go:352] existence check for /var/lib/minikube/binaries/v1.24.3/kubelet: stat -c "%!s(MISSING) %!y(MISSING)" /var/lib/minikube/binaries/v1.24.3/kubelet: Process exited with status 1
stdout:
stderr:$'\tcontrol-plane.minikube.internal$ ' "/etc/hosts"; echo "192.168.49.2 control-plane.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""
stat: cannot stat '/var/lib/minikube/binaries/v1.24.3/kubelet': No such file or directory
I0818 21:32:45.028521 313848 ssh_runner.go:362] scp /home/chensq20/.minikube/cache/linux/amd64/v1.24.3/kubelet --> /var/lib/minikube/binaries/v1.24.3/kubelet (116013432 bytes)
I0818 21:32:46.329002 313848 ssh_runner.go:195] Run: sudo mkdir -p /etc/systemd/system/kubelet.service.d /lib/systemd/system /var/tmp/minikube
I0818 21:32:46.338096 313848 ssh_runner.go:362] scp memory --> /etc/systemd/system/kubelet.service.d/10-kubeadm.conf (560 bytes)
I0818 21:32:46.705365 313848 ssh_runner.go:362] scp memory --> /lib/systemd/system/kubelet.service (352 bytes)
I0818 21:32:46.723082 313848 ssh_runner.go:362] scp memory --> /var/tmp/minikube/kubeadm.yaml.new (2100 bytes)
I0818 21:32:46.739762 313848 ssh_runner.go:195] Run: grep 192.168.49.2 control-plane.minikube.internal$ /etc/hosts
I0818 21:32:46.743794 313848 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v
I0818 21:32:46.755130 313848 certs.go:54] Setting up /home/chensq20/.minikube/profiles/minikube for IP: 192.168.49.2
I0818 21:32:46.755168 313848 certs.go:187] generating minikubeCA CA: /home/chensq20/.minikube/ca.key
I0818 21:32:47.078945 313848 crypto.go:156] Writing cert to /home/chensq20/.minikube/ca.crt ...
I0818 21:32:47.078960 313848 lock.go:35] WriteFile acquiring /home/chensq20/.minikube/ca.crt: {Name:mk437a7d469ad05e27f99ab5c18a9079bee12624 Clock:{} Delay:500ms Timeout:1m0s Cancel:}
I0818 21:32:47.079162 313848 crypto.go:164] Writing key to /home/chensq20/.minikube/ca.key ...
I0818 21:32:47.079168 313848 lock.go:35] WriteFile acquiring /home/chensq20/.minikube/ca.key: {Name:mk532552802f8673f18b1d9880401206b0e5d950 Clock:{} Delay:500ms Timeout:1m0s Cancel:}
I0818 21:32:47.079273 313848 certs.go:187] generating proxyClientCA CA: /home/chensq20/.minikube/proxy-client-ca.key
I0818 21:32:47.253641 313848 crypto.go:156] Writing cert to /home/chensq20/.minikube/proxy-client-ca.crt ...
I0818 21:32:47.253656 313848 lock.go:35] WriteFile acquiring /home/chensq20/.minikube/proxy-client-ca.crt: {Name:mk1be8b8cb5952a31abc60d1bc8c8f33f356d3e0 Clock:{} Delay:500ms Timeout:1m0s Cancel:}
I0818 21:32:47.253863 313848 crypto.go:164] Writing key to /home/chensq20/.minikube/proxy-client-ca.key ...
I0818 21:32:47.253869 313848 lock.go:35] WriteFile acquiring /home/chensq20/.minikube/proxy-client-ca.key: {Name:mk6118c07eba2de0814a7a581ed6d3317172217f Clock:{} Delay:500ms Timeout:1m0s Cancel:}
I0818 21:32:47.254030 313848 certs.go:302] generating minikube-user signed cert: /home/chensq20/.minikube/profiles/minikube/client.key
I0818 21:32:47.254047 313848 crypto.go:68] Generating cert /home/chensq20/.minikube/profiles/minikube/client.crt with IP's: []
I0818 21:32:47.446200 313848 crypto.go:156] Writing cert to /home/chensq20/.minikube/profiles/minikube/client.crt ...
I0818 21:32:47.446215 313848 lock.go:35] WriteFile acquiring /home/chensq20/.minikube/profiles/minikube/client.crt: {Name:mk74f5f9c6060b1b85ad4d2d5fe39f5e591dcb7a Clock:{} Delay:500ms Timeout:1m0s Cancel:}
I0818 21:32:47.446423 313848 crypto.go:164] Writing key to /home/chensq20/.minikube/profiles/minikube/client.key ...
I0818 21:32:47.446429 313848 lock.go:35] WriteFile acquiring /home/chensq20/.minikube/profiles/minikube/client.key: {Name:mkf4ba52783165daa15018125df61c54d91c7426 Clock:{} Delay:500ms Timeout:1m0s Cancel:}
I0818 21:32:47.446516 313848 certs.go:302] generating minikube signed cert: /home/chensq20/.minikube/profiles/minikube/apiserver.key.dd3b5fb2
I0818 21:32:47.446529 313848 crypto.go:68] Generating cert /home/chensq20/.minikube/profiles/minikube/apiserver.crt.dd3b5fb2 with IP's: [192.168.49.2 10.96.0.1 127.0.0.1 10.0.0.1]
I0818 21:32:47.578832 313848 crypto.go:156] Writing cert to /home/chensq20/.minikube/profiles/minikube/apiserver.crt.dd3b5fb2 ...
I0818 21:32:47.578846 313848 lock.go:35] WriteFile acquiring /home/chensq20/.minikube/profiles/minikube/apiserver.crt.dd3b5fb2: {Name:mkc9b0ce79bceaa349f61fda59dab7759147db5c Clock:{} Delay:500ms Timeout:1m0s Cancel:}
I0818 21:32:47.579001 313848 crypto.go:164] Writing key to /home/chensq20/.minikube/profiles/minikube/apiserver.key.dd3b5fb2 ...
I0818 21:32:47.579007 313848 lock.go:35] WriteFile acquiring /home/chensq20/.minikube/profiles/minikube/apiserver.key.dd3b5fb2: {Name:mk2a6de824f789d01a818139a449e2676463b24e Clock:{} Delay:500ms Timeout:1m0s Cancel:}
I0818 21:32:47.579091 313848 certs.go:320] copying /home/chensq20/.minikube/profiles/minikube/apiserver.crt.dd3b5fb2 -> /home/chensq20/.minikube/profiles/minikube/apiserver.crt
I0818 21:32:47.579188 313848 certs.go:324] copying /home/chensq20/.minikube/profiles/minikube/apiserver.key.dd3b5fb2 -> /home/chensq20/.minikube/profiles/minikube/apiserver.key
I0818 21:32:47.579247 313848 certs.go:302] generating aggregator signed cert: /home/chensq20/.minikube/profiles/minikube/proxy-client.key
I0818 21:32:47.579261 313848 crypto.go:68] Generating cert /home/chensq20/.minikube/profiles/minikube/proxy-client.crt with IP's: []
I0818 21:32:47.903200 313848 crypto.go:156] Writing cert to /home/chensq20/.minikube/profiles/minikube/proxy-client.crt ...
I0818 21:32:47.903218 313848 lock.go:35] WriteFile acquiring /home/chensq20/.minikube/profiles/minikube/proxy-client.crt: {Name:mk2c5a4e3166834fad7e89b6fbd6e3a5882e7f4d Clock:{} Delay:500ms Timeout:1m0s Cancel:}
I0818 21:32:47.903444 313848 crypto.go:164] Writing key to /home/chensq20/.minikube/profiles/minikube/proxy-client.key ...
I0818 21:32:47.903450 313848 lock.go:35] WriteFile acquiring /home/chensq20/.minikube/profiles/minikube/proxy-client.key: {Name:mkcea6bd810f17ff98a91ec5b2819dc83789ae3d Clock:{} Delay:500ms Timeout:1m0s Cancel:}
I0818 21:32:47.903645 313848 certs.go:388] found cert: /home/chensq20/.minikube/certs/home/chensq20/.minikube/certs/ca-key.pem (1679 bytes)
I0818 21:32:47.903679 313848 certs.go:388] found cert: /home/chensq20/.minikube/certs/home/chensq20/.minikube/certs/ca.pem (1082 bytes)
I0818 21:32:47.903750 313848 certs.go:388] found cert: /home/chensq20/.minikube/certs/home/chensq20/.minikube/certs/cert.pem (1127 bytes)
I0818 21:32:47.903779 313848 certs.go:388] found cert: /home/chensq20/.minikube/certs/home/chensq20/.minikube/certs/key.pem (1679 bytes)
I0818 21:32:47.904566 313848 ssh_runner.go:362] scp /home/chensq20/.minikube/profiles/minikube/apiserver.crt --> /var/lib/minikube/certs/apiserver.crt (1399 bytes)
I0818 21:32:47.928309 313848 ssh_runner.go:362] scp /home/chensq20/.minikube/profiles/minikube/apiserver.key --> /var/lib/minikube/certs/apiserver.key (1679 bytes)
I0818 21:32:47.951258 313848 ssh_runner.go:362] scp /home/chensq20/.minikube/profiles/minikube/proxy-client.crt --> /var/lib/minikube/certs/proxy-client.crt (1147 bytes)
I0818 21:32:47.975071 313848 ssh_runner.go:362] scp /home/chensq20/.minikube/profiles/minikube/proxy-client.key --> /var/lib/minikube/certs/proxy-client.key (1675 bytes)
I0818 21:32:47.999716 313848 ssh_runner.go:362] scp /home/chensq20/.minikube/ca.crt --> /var/lib/minikube/certs/ca.crt (1111 bytes)
I0818 21:32:48.023451 313848 ssh_runner.go:362] scp /home/chensq20/.minikube/ca.key --> /var/lib/minikube/certs/ca.key (1679 bytes)
I0818 21:32:48.046518 313848 ssh_runner.go:362] scp /home/chensq20/.minikube/proxy-client-ca.crt --> /var/lib/minikube/certs/proxy-client-ca.crt (1119 bytes)
I0818 21:32:48.069805 313848 ssh_runner.go:362] scp /home/chensq20/.minikube/proxy-client-ca.key --> /var/lib/minikube/certs/proxy-client-ca.key (1679 bytes)
I0818 21:32:48.092393 313848 ssh_runner.go:362] scp /home/chensq20/.minikube/ca.crt --> /usr/share/ca-certificates/minikubeCA.pem (1111 bytes)
I0818 21:32:48.122498 313848 ssh_runner.go:362] scp memory --> /var/lib/minikube/kubeconfig (738 bytes)
I0818 21:32:48.141232 313848 ssh_runner.go:195] Run: openssl version
I0818 21:32:48.147371 313848 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/minikubeCA.pem && ln -fs /usr/share/ca-certificates/minikubeCA.pem /etc/ssl/certs/minikubeCA.pem"
I0818 21:32:48.157766 313848 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/minikubeCA.pem
I0818 21:32:48.162015 313848 certs.go:431] hashing: -rw-r--r-- 1 root root 1111 Aug 18 13:32 /usr/share/ca-certificates/minikubeCA.pem
I0818 21:32:48.162043 313848 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/minikubeCA.pem
I0818 21:32:48.169983 313848 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/b5213941.0 || ln -fs /etc/ssl/certs/minikubeCA.pem /etc/ssl/certs/b5213941.0"
I0818 21:32:48.182946 313848 kubeadm.go:395] StartCluster: {Name:minikube KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:registry.cn-hangzhou.aliyuncs.com/google_containers/kicbase:v0.0.33@sha256:73b259e144d926189cf169ae5b46bbec4e08e4e2f2bd87296054c3244f70feb8 Memory:2200 CPUs:2 DiskSize:20000 VMDriver: Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:0 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.24.3 ClusterName:minikube Namespace:default APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin: FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository:registry.cn-hangzhou.aliyuncs.com/google_containers LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:} Nodes:[{Name: IP:192.168.49.2 Port:8443 KubernetesVersion:v1.24.3 ContainerRuntime:docker ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true system_pods:true] StartHostTimeout:6m0s ScheduledStop: ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/chensq20:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath:}
I0818 21:32:48.183044 313848 ssh_runner.go:195] Run: docker ps --filter status=paused --filter=name=k8s_.*(kube-system) --format={{.ID}}
I0818 21:32:48.223878 313848 ssh_runner.go:195] Run: sudo ls /var/lib/kubelet/kubeadm-flags.env /var/lib/kubelet/config.yaml /var/lib/minikube/etcd
I0818 21:32:48.232734 313848 ssh_runner.go:195] Run: sudo cp /var/tmp/minikube/kubeadm.yaml.new /var/tmp/minikube/kubeadm.yaml
I0818 21:32:48.240947 313848 kubeadm.go:221] ignoring SystemVerification for kubeadm because of docker driver
I0818 21:32:48.240976 313848 ssh_runner.go:195] Run: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf
I0818 21:32:48.248818 313848 kubeadm.go:152] config check failed, skipping stale config cleanup: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf: Process exited with status 2
stdout:
stderr:
ls: cannot access '/etc/kubernetes/admin.conf': No such file or directory
ls: cannot access '/etc/kubernetes/kubelet.conf': No such file or directory
ls: cannot access '/etc/kubernetes/controller-manager.conf': No such file or directory
ls: cannot access '/etc/kubernetes/scheduler.conf': No such file or directory
I0818 21:32:48.248849 313848 ssh_runner.go:286] Start: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.24.3:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,Mem,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables"
I0818 21:32:48.634566 313848 out.go:204] - Generating certificates and keys ...
I0818 21:32:51.435003 313848 out.go:204] - Booting up control plane ...
W0818 21:36:51.443611 313848 out.go:239] ! initialization failed, will try again: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.24.3:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,Mem,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.24.3
[preflight] Running pre-flight checks
[preflight] The system verification failed. Printing the output from the verification:
�[0;37mKERNEL_VERSION�[0m: �[0;32m3.10.0-1160.36.2.el7.x86_64�[0m
�[0;37mOS�[0m: �[0;32mLinux�[0m
�[0;37mCGROUPS_CPU�[0m: �[0;32menabled�[0m
�[0;37mCGROUPS_CPUACCT�[0m: �[0;32menabled�[0m
�[0;37mCGROUPS_CPUSET�[0m: �[0;32menabled�[0m
�[0;37mCGROUPS_DEVICES�[0m: �[0;32menabled�[0m
�[0;37mCGROUPS_FREEZER�[0m: �[0;32menabled�[0m
�[0;37mCGROUPS_MEMORY�[0m: �[0;32menabled�[0m
�[0;37mCGROUPS_PIDS�[0m: �[0;32menabled�[0m
�[0;37mCGROUPS_HUGETLB�[0m: �[0;32menabled�[0m
�[0;37mCGROUPS_BLKIO�[0m: �[0;32menabled�[0m
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Generating "apiserver-kubelet-client" certificate and key
[certs] Generating "front-proxy-ca" certificate and key
[certs] Generating "front-proxy-client" certificate and key
[certs] Generating "etcd/ca" certificate and key
[certs] Generating "etcd/server" certificate and key
[certs] etcd/server serving cert is signed for DNS names [localhost minikube] and IPs [192.168.49.2 127.0.0.1 ::1]
[certs] Generating "etcd/peer" certificate and key
[certs] etcd/peer serving cert is signed for DNS names [localhost minikube] and IPs [192.168.49.2 127.0.0.1 ::1]
[certs] Generating "etcd/healthcheck-client" certificate and key
[certs] Generating "apiserver-etcd-client" certificate and key
[certs] Generating "sa" key and public key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI.
Here is one example how you may list all running Kubernetes containers by using crictl:
- 'crictl --runtime-endpoint unix:///var/run/cri-dockerd.sock ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'crictl --runtime-endpoint unix:///var/run/cri-dockerd.sock logs CONTAINERID'
stderr:
W0818 13:32:48.291504 1717 initconfiguration.go:120] Usage of CRI endpoints without URL scheme is deprecated and can cause kubelet errors in the future. Automatically prepending scheme "unix" to the "criSocket" with value "/var/run/cri-dockerd.sock". Please update your configuration!
[WARNING FileContent--proc-sys-net-bridge-bridge-nf-call-iptables]: /proc/sys/net/bridge/bridge-nf-call-iptables does not exist
[WARNING SystemVerification]: failed to parse kernel config: unable to load kernel module: "configs", output: "modprobe: FATAL: Module configs not found in directory /lib/modules/3.10.0-1160.36.2.el7.x86_64\n", err: exit status 1
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
I0818 21:36:51.443814 313848 ssh_runner.go:195] Run: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.24.3:$PATH" kubeadm reset --cri-socket /var/run/cri-dockerd.sock --force"
I0818 21:36:52.390344 313848 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service kubelet
I0818 21:36:52.401653 313848 kubeadm.go:221] ignoring SystemVerification for kubeadm because of docker driver
I0818 21:36:52.401685 313848 ssh_runner.go:195] Run: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf
I0818 21:36:52.409771 313848 kubeadm.go:152] config check failed, skipping stale config cleanup: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf: Process exited with status 2
stdout:
stderr:
ls: cannot access '/etc/kubernetes/admin.conf': No such file or directory
ls: cannot access '/etc/kubernetes/kubelet.conf': No such file or directory
ls: cannot access '/etc/kubernetes/controller-manager.conf': No such file or directory
ls: cannot access '/etc/kubernetes/scheduler.conf': No such file or directory
I0818 21:36:52.409800 313848 ssh_runner.go:286] Start: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.24.3:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,Mem,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables"
I0818 21:36:53.293323 313848 out.go:204] - Generating certificates and keys ...
I0818 21:36:53.731987 313848 out.go:204] - Booting up control plane ...
==> Docker <==
-- Logs begin at Thu 2022-08-18 13:32:18 UTC, end at Thu 2022-08-18 13:39:48 UTC. --
Aug 18 13:37:39 minikube dockerd[685]: time="2022-08-18T13:37:39.261218322Z" level=warning msg="Error getting v2 registry: Get "https://k8s.gcr.io/v2/\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Aug 18 13:37:39 minikube dockerd[685]: time="2022-08-18T13:37:39.261258339Z" level=info msg="Attempting next endpoint for pull after error: Get "https://k8s.gcr.io/v2/\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Aug 18 13:37:39 minikube dockerd[685]: time="2022-08-18T13:37:39.394256785Z" level=warning msg="Error getting v2 registry: Get "https://k8s.gcr.io/v2/\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Aug 18 13:37:39 minikube dockerd[685]: time="2022-08-18T13:37:39.394288975Z" level=info msg="Attempting next endpoint for pull after error: Get "https://k8s.gcr.io/v2/\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Aug 18 13:37:39 minikube dockerd[685]: time="2022-08-18T13:37:39.541405997Z" level=error msg="Handler for POST /v1.40/images/create returned error: Get "https://k8s.gcr.io/v2/\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Aug 18 13:37:39 minikube dockerd[685]: time="2022-08-18T13:37:39.545829165Z" level=error msg="Handler for POST /v1.40/images/create returned error: Get "https://k8s.gcr.io/v2/\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Aug 18 13:37:41 minikube dockerd[685]: time="2022-08-18T13:37:41.061617939Z" level=warning msg="Error getting v2 registry: Get "https://k8s.gcr.io/v2/\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Aug 18 13:37:41 minikube dockerd[685]: time="2022-08-18T13:37:41.061645642Z" level=info msg="Attempting next endpoint for pull after error: Get "https://k8s.gcr.io/v2/\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Aug 18 13:37:41 minikube dockerd[685]: time="2022-08-18T13:37:41.066500345Z" level=error msg="Handler for POST /v1.40/images/create returned error: Get "https://k8s.gcr.io/v2/\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Aug 18 13:37:41 minikube dockerd[685]: time="2022-08-18T13:37:41.197921726Z" level=warning msg="Error getting v2 registry: Get "https://k8s.gcr.io/v2/\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Aug 18 13:37:41 minikube dockerd[685]: time="2022-08-18T13:37:41.197947323Z" level=info msg="Attempting next endpoint for pull after error: Get "https://k8s.gcr.io/v2/\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Aug 18 13:37:41 minikube dockerd[685]: time="2022-08-18T13:37:41.203032950Z" level=error msg="Handler for POST /v1.40/images/create returned error: Get "https://k8s.gcr.io/v2/\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Aug 18 13:38:06 minikube dockerd[685]: time="2022-08-18T13:38:06.062513819Z" level=warning msg="Error getting v2 registry: Get "https://k8s.gcr.io/v2/\": dial tcp 74.125.203.82:443: i/o timeout"
Aug 18 13:38:06 minikube dockerd[685]: time="2022-08-18T13:38:06.062541702Z" level=info msg="Attempting next endpoint for pull after error: Get "https://k8s.gcr.io/v2/\": dial tcp 74.125.203.82:443: i/o timeout"
Aug 18 13:38:06 minikube dockerd[685]: time="2022-08-18T13:38:06.067791269Z" level=error msg="Handler for POST /v1.40/images/create returned error: Get "https://k8s.gcr.io/v2/\": dial tcp 74.125.203.82:443: i/o timeout"
Aug 18 13:38:09 minikube dockerd[685]: time="2022-08-18T13:38:09.096251824Z" level=warning msg="Error getting v2 registry: Get "https://k8s.gcr.io/v2/\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Aug 18 13:38:09 minikube dockerd[685]: time="2022-08-18T13:38:09.096297282Z" level=info msg="Attempting next endpoint for pull after error: Get "https://k8s.gcr.io/v2/\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Aug 18 13:38:09 minikube dockerd[685]: time="2022-08-18T13:38:09.101540138Z" level=error msg="Handler for POST /v1.40/images/create returned error: Get "https://k8s.gcr.io/v2/\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Aug 18 13:38:10 minikube dockerd[685]: time="2022-08-18T13:38:10.058634671Z" level=warning msg="Error getting v2 registry: Get "https://k8s.gcr.io/v2/\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Aug 18 13:38:10 minikube dockerd[685]: time="2022-08-18T13:38:10.058671374Z" level=info msg="Attempting next endpoint for pull after error: Get "https://k8s.gcr.io/v2/\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Aug 18 13:38:10 minikube dockerd[685]: time="2022-08-18T13:38:10.064015611Z" level=error msg="Handler for POST /v1.40/images/create returned error: Get "https://k8s.gcr.io/v2/\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Aug 18 13:38:11 minikube dockerd[685]: time="2022-08-18T13:38:11.736772587Z" level=warning msg="Error getting v2 registry: Get "https://k8s.gcr.io/v2/\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Aug 18 13:38:11 minikube dockerd[685]: time="2022-08-18T13:38:11.736879069Z" level=info msg="Attempting next endpoint for pull after error: Get "https://k8s.gcr.io/v2/\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Aug 18 13:38:11 minikube dockerd[685]: time="2022-08-18T13:38:11.742201845Z" level=error msg="Handler for POST /v1.40/images/create returned error: Get "https://k8s.gcr.io/v2/\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Aug 18 13:38:36 minikube dockerd[685]: time="2022-08-18T13:38:36.094707613Z" level=warning msg="Error getting v2 registry: Get "https://k8s.gcr.io/v2/\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Aug 18 13:38:36 minikube dockerd[685]: time="2022-08-18T13:38:36.094740932Z" level=info msg="Attempting next endpoint for pull after error: Get "https://k8s.gcr.io/v2/\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Aug 18 13:38:36 minikube dockerd[685]: time="2022-08-18T13:38:36.101894108Z" level=error msg="Handler for POST /v1.40/images/create returned error: Get "https://k8s.gcr.io/v2/\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Aug 18 13:38:38 minikube dockerd[685]: time="2022-08-18T13:38:38.065442403Z" level=warning msg="Error getting v2 registry: Get "https://k8s.gcr.io/v2/\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Aug 18 13:38:38 minikube dockerd[685]: time="2022-08-18T13:38:38.065477917Z" level=info msg="Attempting next endpoint for pull after error: Get "https://k8s.gcr.io/v2/\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Aug 18 13:38:38 minikube dockerd[685]: time="2022-08-18T13:38:38.070379155Z" level=error msg="Handler for POST /v1.40/images/create returned error: Get "https://k8s.gcr.io/v2/\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Aug 18 13:38:39 minikube dockerd[685]: time="2022-08-18T13:38:39.059255971Z" level=warning msg="Error getting v2 registry: Get "https://k8s.gcr.io/v2/\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Aug 18 13:38:39 minikube dockerd[685]: time="2022-08-18T13:38:39.059378879Z" level=info msg="Attempting next endpoint for pull after error: Get "https://k8s.gcr.io/v2/\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Aug 18 13:38:39 minikube dockerd[685]: time="2022-08-18T13:38:39.064870703Z" level=error msg="Handler for POST /v1.40/images/create returned error: Get "https://k8s.gcr.io/v2/\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Aug 18 13:38:39 minikube dockerd[685]: time="2022-08-18T13:38:39.195031426Z" level=warning msg="Error getting v2 registry: Get "https://k8s.gcr.io/v2/\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Aug 18 13:38:39 minikube dockerd[685]: time="2022-08-18T13:38:39.195059401Z" level=info msg="Attempting next endpoint for pull after error: Get "https://k8s.gcr.io/v2/\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Aug 18 13:38:39 minikube dockerd[685]: time="2022-08-18T13:38:39.200027904Z" level=error msg="Handler for POST /v1.40/images/create returned error: Get "https://k8s.gcr.io/v2/\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Aug 18 13:39:05 minikube dockerd[685]: time="2022-08-18T13:39:05.055301617Z" level=warning msg="Error getting v2 registry: Get "https://k8s.gcr.io/v2/\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Aug 18 13:39:05 minikube dockerd[685]: time="2022-08-18T13:39:05.055350989Z" level=info msg="Attempting next endpoint for pull after error: Get "https://k8s.gcr.io/v2/\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Aug 18 13:39:05 minikube dockerd[685]: time="2022-08-18T13:39:05.060067569Z" level=error msg="Handler for POST /v1.40/images/create returned error: Get "https://k8s.gcr.io/v2/\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Aug 18 13:39:05 minikube dockerd[685]: time="2022-08-18T13:39:05.231361371Z" level=warning msg="Error getting v2 registry: Get "https://k8s.gcr.io/v2/\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Aug 18 13:39:05 minikube dockerd[685]: time="2022-08-18T13:39:05.231393342Z" level=info msg="Attempting next endpoint for pull after error: Get "https://k8s.gcr.io/v2/\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Aug 18 13:39:05 minikube dockerd[685]: time="2022-08-18T13:39:05.236482699Z" level=error msg="Handler for POST /v1.40/images/create returned error: Get "https://k8s.gcr.io/v2/\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Aug 18 13:39:05 minikube dockerd[685]: time="2022-08-18T13:39:05.320848086Z" level=warning msg="Error getting v2 registry: Get "https://k8s.gcr.io/v2/\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Aug 18 13:39:05 minikube dockerd[685]: time="2022-08-18T13:39:05.320873544Z" level=info msg="Attempting next endpoint for pull after error: Get "https://k8s.gcr.io/v2/\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Aug 18 13:39:05 minikube dockerd[685]: time="2022-08-18T13:39:05.325592576Z" level=error msg="Handler for POST /v1.40/images/create returned error: Get "https://k8s.gcr.io/v2/\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Aug 18 13:39:08 minikube dockerd[685]: time="2022-08-18T13:39:08.063267783Z" level=warning msg="Error getting v2 registry: Get "https://k8s.gcr.io/v2/\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Aug 18 13:39:08 minikube dockerd[685]: time="2022-08-18T13:39:08.063311562Z" level=info msg="Attempting next endpoint for pull after error: Get "https://k8s.gcr.io/v2/\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Aug 18 13:39:08 minikube dockerd[685]: time="2022-08-18T13:39:08.068314342Z" level=error msg="Handler for POST /v1.40/images/create returned error: Get "https://k8s.gcr.io/v2/\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Aug 18 13:39:32 minikube dockerd[685]: time="2022-08-18T13:39:32.056052361Z" level=warning msg="Error getting v2 registry: Get "https://k8s.gcr.io/v2/\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Aug 18 13:39:32 minikube dockerd[685]: time="2022-08-18T13:39:32.056086703Z" level=info msg="Attempting next endpoint for pull after error: Get "https://k8s.gcr.io/v2/\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Aug 18 13:39:32 minikube dockerd[685]: time="2022-08-18T13:39:32.061201933Z" level=error msg="Handler for POST /v1.40/images/create returned error: Get "https://k8s.gcr.io/v2/\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Aug 18 13:39:34 minikube dockerd[685]: time="2022-08-18T13:39:34.216996220Z" level=warning msg="Error getting v2 registry: Get "https://k8s.gcr.io/v2/\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Aug 18 13:39:34 minikube dockerd[685]: time="2022-08-18T13:39:34.217035372Z" level=info msg="Attempting next endpoint for pull after error: Get "https://k8s.gcr.io/v2/\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Aug 18 13:39:34 minikube dockerd[685]: time="2022-08-18T13:39:34.332984933Z" level=error msg="Handler for POST /v1.40/images/create returned error: Get "https://k8s.gcr.io/v2/\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Aug 18 13:39:35 minikube dockerd[685]: time="2022-08-18T13:39:35.057065693Z" level=warning msg="Error getting v2 registry: Get "https://k8s.gcr.io/v2/\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Aug 18 13:39:35 minikube dockerd[685]: time="2022-08-18T13:39:35.057106300Z" level=info msg="Attempting next endpoint for pull after error: Get "https://k8s.gcr.io/v2/\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Aug 18 13:39:35 minikube dockerd[685]: time="2022-08-18T13:39:35.062141002Z" level=error msg="Handler for POST /v1.40/images/create returned error: Get "https://k8s.gcr.io/v2/\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Aug 18 13:39:36 minikube dockerd[685]: time="2022-08-18T13:39:36.058515358Z" level=warning msg="Error getting v2 registry: Get "https://k8s.gcr.io/v2/\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Aug 18 13:39:36 minikube dockerd[685]: time="2022-08-18T13:39:36.058593918Z" level=info msg="Attempting next endpoint for pull after error: Get "https://k8s.gcr.io/v2/\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Aug 18 13:39:36 minikube dockerd[685]: time="2022-08-18T13:39:36.730007770Z" level=error msg="Handler for POST /v1.40/images/create returned error: Get "https://k8s.gcr.io/v2/\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
==> container status <==
CONTAINER IMAGE CREATED STATE NAME ATTEMPT POD ID
==> describe nodes <==
==> dmesg <==
[Aug17 09:45] ACPI: RSDP 00000000000f6210 00014 (v00 BOCHS )
[ +0.000000] ACPI: RSDT 00000000bffe2510 00030 (v01 BOCHS BXPCRSDT 00000001 BXPC 00000001)
[ +0.000000] ACPI: FACP 00000000bffe21dc 00074 (v01 BOCHS BXPCFACP 00000001 BXPC 00000001)
[ +0.000000] ACPI: DSDT 00000000bffdfd00 024DC (v01 BOCHS BXPCDSDT 00000001 BXPC 00000001)
[ +0.000000] ACPI: FACS 00000000bffdfcc0 00040
[ +0.000000] ACPI: APIC 00000000bffe2250 000F0 (v01 BOCHS BXPCAPIC 00000001 BXPC 00000001)
[ +0.000000] ACPI: SRAT 00000000bffe2340 001D0 (v01 BOCHS BXPCSRAT 00000001 BXPC 00000001)
[ +0.000000] Zone ranges:
[ +0.000000] DMA [mem 0x00001000-0x00ffffff]
[ +0.000000] DMA32 [mem 0x01000000-0xffffffff]
[ +0.000000] Normal [mem 0x100000000-0x23fffffff]
[ +0.000000] Movable zone start for each node
[ +0.000000] Early memory node ranges
[ +0.000000] node 0: [mem 0x00001000-0x0009efff]
[ +0.000000] node 0: [mem 0x00100000-0xbffdbfff]
[ +0.000000] node 0: [mem 0x100000000-0x23fffffff]
[ +0.000000] Built 1 zonelists in Zone order, mobility grouping on. Total pages: 2064229
[ +0.000000] Policy zone: Normal
[ +0.000000] ACPI: All ACPI Tables successfully acquired
[ +0.167757] ACPI: Enabled 3 GPEs in block 00 to 0F
[ +0.005463] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge.
[ +0.107571] ACPI: PCI Interrupt Link [LNKD] enabled at IRQ 11
[ +0.311251] Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
[ +0.281407] ACPI: PCI Interrupt Link [LNKC] enabled at IRQ 10
[ +0.055266] ACPI: PCI Interrupt Link [LNKA] enabled at IRQ 10
[ +0.028149] ACPI: PCI Interrupt Link [LNKB] enabled at IRQ 11
[ +0.152939] sd 2:0:0:0: Power-on or device reset occurred
[Aug18 05:41] TECH PREVIEW: Overlay filesystem may not be fully supported.
Please review provided documentation for limitations.
==> kernel <==
13:39:51 up 1 day, 3:53, 0 users, load average: 0.12, 0.22, 0.22
Linux minikube 3.10.0-1160.36.2.el7.x86_64 Need a reliable and low latency local cluster setup for Kubernetes #1 SMP Wed Jul 21 11:57:15 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
PRETTY_NAME="Ubuntu 20.04.4 LTS"
==> kubelet <==
-- Logs begin at Thu 2022-08-18 13:32:18 UTC, end at Thu 2022-08-18 13:39:51 UTC. --
Aug 18 13:39:45 minikube kubelet[3586]: E0818 13:39:45.925776 3586 kubelet.go:2424] "Error getting node" err="node "minikube" not found"
Aug 18 13:39:46 minikube kubelet[3586]: E0818 13:39:46.026602 3586 kubelet.go:2424] "Error getting node" err="node "minikube" not found"
Aug 18 13:39:46 minikube kubelet[3586]: E0818 13:39:46.127362 3586 kubelet.go:2424] "Error getting node" err="node "minikube" not found"
Aug 18 13:39:46 minikube kubelet[3586]: E0818 13:39:46.227934 3586 kubelet.go:2424] "Error getting node" err="node "minikube" not found"
Aug 18 13:39:46 minikube kubelet[3586]: E0818 13:39:46.328831 3586 kubelet.go:2424] "Error getting node" err="node "minikube" not found"
Aug 18 13:39:46 minikube kubelet[3586]: E0818 13:39:46.429621 3586 kubelet.go:2424] "Error getting node" err="node "minikube" not found"
Aug 18 13:39:46 minikube kubelet[3586]: E0818 13:39:46.530370 3586 kubelet.go:2424] "Error getting node" err="node "minikube" not found"
Aug 18 13:39:46 minikube kubelet[3586]: E0818 13:39:46.630950 3586 kubelet.go:2424] "Error getting node" err="node "minikube" not found"
Aug 18 13:39:46 minikube kubelet[3586]: E0818 13:39:46.731537 3586 kubelet.go:2424] "Error getting node" err="node "minikube" not found"
Aug 18 13:39:46 minikube kubelet[3586]: E0818 13:39:46.832351 3586 kubelet.go:2424] "Error getting node" err="node "minikube" not found"
Aug 18 13:39:46 minikube kubelet[3586]: E0818 13:39:46.933003 3586 kubelet.go:2424] "Error getting node" err="node "minikube" not found"
Aug 18 13:39:47 minikube kubelet[3586]: E0818 13:39:47.034082 3586 kubelet.go:2424] "Error getting node" err="node "minikube" not found"
Aug 18 13:39:47 minikube kubelet[3586]: E0818 13:39:47.134833 3586 kubelet.go:2424] "Error getting node" err="node "minikube" not found"
Aug 18 13:39:47 minikube kubelet[3586]: W0818 13:39:47.160628 3586 reflector.go:324] vendor/k8s.io/client-go/informers/factory.go:134: failed to list *v1.Node: Get "https://control-plane.minikube.internal:8443/api/v1/nodes?fieldSelector=metadata.name%!D(MISSING)minikube&limit=500&resourceVersion=0": dial tcp 192.168.49.2:8443: connect: connection refused
Aug 18 13:39:47 minikube kubelet[3586]: E0818 13:39:47.160693 3586 reflector.go:138] vendor/k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.Node: failed to list *v1.Node: Get "https://control-plane.minikube.internal:8443/api/v1/nodes?fieldSelector=metadata.name%!D(MISSING)minikube&limit=500&resourceVersion=0": dial tcp 192.168.49.2:8443: connect: connection refused
Aug 18 13:39:47 minikube kubelet[3586]: E0818 13:39:47.234920 3586 kubelet.go:2424] "Error getting node" err="node "minikube" not found"
Aug 18 13:39:47 minikube kubelet[3586]: E0818 13:39:47.335942 3586 kubelet.go:2424] "Error getting node" err="node "minikube" not found"
Aug 18 13:39:47 minikube kubelet[3586]: E0818 13:39:47.436556 3586 kubelet.go:2424] "Error getting node" err="node "minikube" not found"
Aug 18 13:39:47 minikube kubelet[3586]: E0818 13:39:47.481322 3586 controller.go:144] failed to ensure lease exists, will retry in 7s, error: Get "https://control-plane.minikube.internal:8443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/minikube?timeout=10s": dial tcp 192.168.49.2:8443: connect: connection refused
Aug 18 13:39:47 minikube kubelet[3586]: E0818 13:39:47.537600 3586 kubelet.go:2424] "Error getting node" err="node "minikube" not found"
Aug 18 13:39:47 minikube kubelet[3586]: E0818 13:39:47.638155 3586 kubelet.go:2424] "Error getting node" err="node "minikube" not found"
Aug 18 13:39:47 minikube kubelet[3586]: E0818 13:39:47.738786 3586 kubelet.go:2424] "Error getting node" err="node "minikube" not found"
Aug 18 13:39:47 minikube kubelet[3586]: E0818 13:39:47.839516 3586 kubelet.go:2424] "Error getting node" err="node "minikube" not found"
Aug 18 13:39:47 minikube kubelet[3586]: I0818 13:39:47.870430 3586 kubelet_node_status.go:70] "Attempting to register node" node="minikube"
Aug 18 13:39:47 minikube kubelet[3586]: E0818 13:39:47.870799 3586 kubelet_node_status.go:92] "Unable to register node with API server" err="Post "https://control-plane.minikube.internal:8443/api/v1/nodes\": dial tcp 192.168.49.2:8443: connect: connection refused" node="minikube"
Aug 18 13:39:47 minikube kubelet[3586]: E0818 13:39:47.940281 3586 kubelet.go:2424] "Error getting node" err="node "minikube" not found"
Aug 18 13:39:48 minikube kubelet[3586]: E0818 13:39:48.041435 3586 kubelet.go:2424] "Error getting node" err="node "minikube" not found"
Aug 18 13:39:48 minikube kubelet[3586]: E0818 13:39:48.142046 3586 kubelet.go:2424] "Error getting node" err="node "minikube" not found"
Aug 18 13:39:48 minikube kubelet[3586]: E0818 13:39:48.242671 3586 kubelet.go:2424] "Error getting node" err="node "minikube" not found"
Aug 18 13:39:48 minikube kubelet[3586]: E0818 13:39:48.343766 3586 kubelet.go:2424] "Error getting node" err="node "minikube" not found"
Aug 18 13:39:48 minikube kubelet[3586]: E0818 13:39:48.444788 3586 kubelet.go:2424] "Error getting node" err="node "minikube" not found"
Aug 18 13:39:48 minikube kubelet[3586]: E0818 13:39:48.545324 3586 kubelet.go:2424] "Error getting node" err="node "minikube" not found"
Aug 18 13:39:48 minikube kubelet[3586]: E0818 13:39:48.646338 3586 kubelet.go:2424] "Error getting node" err="node "minikube" not found"
Aug 18 13:39:48 minikube kubelet[3586]: E0818 13:39:48.746948 3586 kubelet.go:2424] "Error getting node" err="node "minikube" not found"
Aug 18 13:39:48 minikube kubelet[3586]: E0818 13:39:48.847799 3586 kubelet.go:2424] "Error getting node" err="node "minikube" not found"
Aug 18 13:39:48 minikube kubelet[3586]: E0818 13:39:48.948546 3586 kubelet.go:2424] "Error getting node" err="node "minikube" not found"
Aug 18 13:39:49 minikube kubelet[3586]: E0818 13:39:49.049301 3586 kubelet.go:2424] "Error getting node" err="node "minikube" not found"
Aug 18 13:39:49 minikube kubelet[3586]: E0818 13:39:49.149873 3586 kubelet.go:2424] "Error getting node" err="node "minikube" not found"
Aug 18 13:39:49 minikube kubelet[3586]: E0818 13:39:49.250135 3586 kubelet.go:2424] "Error getting node" err="node "minikube" not found"
Aug 18 13:39:49 minikube kubelet[3586]: E0818 13:39:49.350771 3586 kubelet.go:2424] "Error getting node" err="node "minikube" not found"
Aug 18 13:39:49 minikube kubelet[3586]: E0818 13:39:49.451433 3586 kubelet.go:2424] "Error getting node" err="node "minikube" not found"
Aug 18 13:39:49 minikube kubelet[3586]: E0818 13:39:49.552067 3586 kubelet.go:2424] "Error getting node" err="node "minikube" not found"
Aug 18 13:39:49 minikube kubelet[3586]: E0818 13:39:49.652634 3586 kubelet.go:2424] "Error getting node" err="node "minikube" not found"
Aug 18 13:39:49 minikube kubelet[3586]: E0818 13:39:49.753372 3586 kubelet.go:2424] "Error getting node" err="node "minikube" not found"
Aug 18 13:39:49 minikube kubelet[3586]: E0818 13:39:49.853456 3586 kubelet.go:2424] "Error getting node" err="node "minikube" not found"
Aug 18 13:39:49 minikube kubelet[3586]: E0818 13:39:49.954425 3586 kubelet.go:2424] "Error getting node" err="node "minikube" not found"
Aug 18 13:39:50 minikube kubelet[3586]: E0818 13:39:50.054986 3586 kubelet.go:2424] "Error getting node" err="node "minikube" not found"
Aug 18 13:39:50 minikube kubelet[3586]: E0818 13:39:50.145725 3586 event.go:276] Unable to write event: '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"minikube.170c73ea9506bd88", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ZZZ_DeprecatedClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"minikube", UID:"minikube", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasSufficientMemory", Message:"Node minikube status is now: NodeHasSufficientMemory", Source:v1.EventSource{Component:"kubelet", Host:"minikube"}, FirstTimestamp:time.Date(2022, time.August, 18, 13, 36, 53, 952658824, time.Local), LastTimestamp:time.Date(2022, time.August, 18, 13, 36, 53, 952658824, time.Local), Count:1, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"", ReportingInstance:""}': 'Post "https://control-plane.minikube.internal:8443/api/v1/namespaces/default/events": dial tcp 192.168.49.2:8443: connect: connection refused'(may retry after sleeping)
Aug 18 13:39:50 minikube kubelet[3586]: E0818 13:39:50.155137 3586 kubelet.go:2424] "Error getting node" err="node "minikube" not found"
Aug 18 13:39:50 minikube kubelet[3586]: E0818 13:39:50.255447 3586 kubelet.go:2424] "Error getting node" err="node "minikube" not found"
Aug 18 13:39:50 minikube kubelet[3586]: E0818 13:39:50.356155 3586 kubelet.go:2424] "Error getting node" err="node "minikube" not found"
Aug 18 13:39:50 minikube kubelet[3586]: E0818 13:39:50.456880 3586 kubelet.go:2424] "Error getting node" err="node "minikube" not found"
Aug 18 13:39:50 minikube kubelet[3586]: E0818 13:39:50.557419 3586 kubelet.go:2424] "Error getting node" err="node "minikube" not found"
Aug 18 13:39:50 minikube kubelet[3586]: E0818 13:39:50.657939 3586 kubelet.go:2424] "Error getting node" err="node "minikube" not found"
Aug 18 13:39:50 minikube kubelet[3586]: E0818 13:39:50.758482 3586 kubelet.go:2424] "Error getting node" err="node "minikube" not found"
Aug 18 13:39:50 minikube kubelet[3586]: E0818 13:39:50.858941 3586 kubelet.go:2424] "Error getting node" err="node "minikube" not found"
Aug 18 13:39:50 minikube kubelet[3586]: E0818 13:39:50.960005 3586 kubelet.go:2424] "Error getting node" err="node "minikube" not found"
Aug 18 13:39:51 minikube kubelet[3586]: E0818 13:39:51.060736 3586 kubelet.go:2424] "Error getting node" err="node "minikube" not found"
Aug 18 13:39:51 minikube kubelet[3586]: E0818 13:39:51.161384 3586 kubelet.go:2424] "Error getting node" err="node "minikube" not found"
Aug 18 13:39:51 minikube kubelet[3586]: E0818 13:39:51.261789 3586 kubelet.go:2424] "Error getting node" err="node "minikube" not found"`
Operating System
No response
Driver
No response
The text was updated successfully, but these errors were encountered: