Error when run kubeadm init

[tpha26644]

**Hello,

I am having a problem when run 'kubeadm init' in a server without internet connection. I have already downloaded the necessary packages and images. But when I run 'kubeadm init', I had the problem like this:**

[root@vtn-cds-bigdata-10211202021 k8s-packages]# kubeadm init --pod-network-cidr=10.244.0.0/16
W0822 15:07:08.537453   26250 version.go:104] could not fetch a Kubernetes version from the internet: unable to get URL "https://dl.k8s.io/release/stable-1.txt": Get "https://dl.k8s.io/release/stable-1.txt": dial tcp: lookup dl.k8s.io on 10.211.202.199:53: write udp 10.211.202.21:63219->10.211.202.199:53: write: operation not permitted
W0822 15:07:08.537514   26250 version.go:105] falling back to the local client version: v1.30.2
[init] Using Kubernetes version: v1.30.2
[preflight] Running pre-flight checks
        [WARNING Hostname]: hostname "vtn-cds-bigdata-10211202021" could not be reached
        [WARNING Hostname]: hostname "vtn-cds-bigdata-10211202021": lookup vtn-cds-bigdata-10211202021 on 10.211.202.199:53: write udp 10.211.202.21:50797->10.211.202.199:53: write: operation not permitted
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
W0822 15:07:08.690819   26250 checks.go:844] detected that the sandbox image "registry.k8s.io/pause:3.6" of the container runtime is inconsistent with that used by kubeadm.It is recommended to use "registry.k8s.io/pause:3.9" as the CRI sandbox image.
[certs] Using certificateDir folder "/etc/kubernetes/pki"
[certs] Generating "ca" certificate and key
[certs] Generating "apiserver" certificate and key
[certs] apiserver serving cert is signed for DNS names [kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local vtn-cds-bigdata-10211202021] and IPs [10.96.0.1 10.211.202.21]
[certs] Generating "apiserver-kubelet-client" certificate and key
[certs] Generating "front-proxy-ca" certificate and key
[certs] Generating "front-proxy-client" certificate and key
[certs] Generating "etcd/ca" certificate and key
[certs] Generating "etcd/server" certificate and key
[certs] etcd/server serving cert is signed for DNS names [localhost vtn-cds-bigdata-10211202021] and IPs [10.211.202.21 127.0.0.1 ::1]
[certs] Generating "etcd/peer" certificate and key
[certs] etcd/peer serving cert is signed for DNS names [localhost vtn-cds-bigdata-10211202021] and IPs [10.211.202.21 127.0.0.1 ::1]
[certs] Generating "etcd/healthcheck-client" certificate and key
[certs] Generating "apiserver-etcd-client" certificate and key
[certs] Generating "sa" key and public key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "super-admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests"
[kubelet-check] Waiting for a healthy kubelet. This can take up to 4m0s
[kubelet-check] The kubelet is healthy after 501.342598ms
[api-check] Waiting for a healthy API server. This can take up to 4m0s
[api-check] The API server is not healthy after 4m0.000247807s

Unfortunately, an error has occurred:
        context deadline exceeded

This error is likely caused by:
        - The kubelet is not running
        - The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)

If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
        - 'systemctl status kubelet'
        - 'journalctl -xeu kubelet'

Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI.
Here is one example how you may list all running Kubernetes containers by using crictl:
        - 'crictl --runtime-endpoint unix:///var/run/containerd/containerd.sock ps -a | grep kube | grep -v pause'
        Once you have found the failing container, you can inspect its logs with:
        - 'crictl --runtime-endpoint unix:///var/run/containerd/containerd.sock logs CONTAINERID'
error execution phase wait-control-plane: could not initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher

And i check the kubelet then I have this problem:

[root@vtn-cds-bigdata-10211202021 k8s-packages]# journalctl -xeu kubelet
Aug 22 15:11:46 vtn-cds-bigdata-10211202021 kubelet[26382]: E0822 15:11:46.607246   26382 pod_workers.go:1298] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"kube-apiserver-vtn-cds-bigdata-10211202021_kube
Aug 22 15:11:47 vtn-cds-bigdata-10211202021 kubelet[26382]: E0822 15:11:47.550397   26382 event.go:368] "Unable to write event (may retry after sleeping)" err="Post \"[https://10.211.202.21:6443/api/v1/namespaces/default/events\](https://10.211.202.21:6443/api/v1/namespaces/default/events/)":
Aug 22 15:11:47 vtn-cds-bigdata-10211202021 kubelet[26382]: E0822 15:11:47.606831   26382 remote_runtime.go:193] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to get sandbox image \"regis
Aug 22 15:11:47 vtn-cds-bigdata-10211202021 kubelet[26382]: E0822 15:11:47.606872   26382 kuberuntime_sandbox.go:72] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to get sandbox image \"registry.k
Aug 22 15:11:47 vtn-cds-bigdata-10211202021 kubelet[26382]: E0822 15:11:47.606888   26382 kuberuntime_manager.go:1166] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to get sandbox image \"registry.
Aug 22 15:11:47 vtn-cds-bigdata-10211202021 kubelet[26382]: E0822 15:11:47.606920   26382 pod_workers.go:1298] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"kube-scheduler-vtn-cds-bigdata-10211202021_kube
Aug 22 15:11:48 vtn-cds-bigdata-10211202021 kubelet[26382]: E0822 15:11:48.607731   26382 remote_runtime.go:193] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to get sandbox image \"regis
Aug 22 15:11:48 vtn-cds-bigdata-10211202021 kubelet[26382]: E0822 15:11:48.607770   26382 kuberuntime_sandbox.go:72] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to get sandbox image \"registry.k
Aug 22 15:11:48 vtn-cds-bigdata-10211202021 kubelet[26382]: E0822 15:11:48.607787   26382 kuberuntime_manager.go:1166] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to get sandbox image \"registry.
Aug 22 15:11:48 vtn-cds-bigdata-10211202021 kubelet[26382]: E0822 15:11:48.607816   26382 pod_workers.go:1298] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"kube-controller-manager-vtn-cds-bigdata-1021120
Aug 22 15:11:50 vtn-cds-bigdata-10211202021 kubelet[26382]: E0822 15:11:50.231016   26382 controller.go:145] "Failed to ensure lease exists, will retry" err="Get \"https://10.211.202.21:6443/apis/coordination.k8s.io/v1/namespaces/
Aug 22 15:11:50 vtn-cds-bigdata-10211202021 kubelet[26382]: I0822 15:11:50.383091   26382 kubelet_node_status.go:73] "Attempting to register node" node="vtn-cds-bigdata-10211202021"
Aug 22 15:11:50 vtn-cds-bigdata-10211202021 kubelet[26382]: E0822 15:11:50.383308   26382 kubelet_node_status.go:96] "Unable to register node with API server" err="Post \"[https://10.211.202.21:6443/api/v1/nodes\](https://10.211.202.21:6443/api/v1/nodes/)": dial tcp 10.211.
Aug 22 15:11:51 vtn-cds-bigdata-10211202021 kubelet[26382]: E0822 15:11:51.669813   26382 eviction_manager.go:282] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"vtn-cds-bigdata-10211202021\"
Aug 22 15:11:53 vtn-cds-bigdata-10211202021 kubelet[26382]: E0822 15:11:53.608933   26382 remote_runtime.go:193] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to get sandbox image \"regis
Aug 22 15:11:53 vtn-cds-bigdata-10211202021 kubelet[26382]: E0822 15:11:53.608971   26382 kuberuntime_sandbox.go:72] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to get sandbox image \"registry.k
Aug 22 15:11:53 vtn-cds-bigdata-10211202021 kubelet[26382]: E0822 15:11:53.608988   26382 kuberuntime_manager.go:1166] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to get sandbox image \"registry.
Aug 22 15:11:53 vtn-cds-bigdata-10211202021 kubelet[26382]: E0822 15:11:53.609016   26382 pod_workers.go:1298] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"etcd-vtn-cds-bigdata-10211202021_kube-system(5a
Aug 22 15:11:57 vtn-cds-bigdata-10211202021 kubelet[26382]: E0822 15:11:57.232202   26382 controller.go:145] "Failed to ensure lease exists, will retry" err="Get \"https://10.211.202.21:6443/apis/coordination.k8s.io/v1/namespaces/
Aug 22 15:11:57 vtn-cds-bigdata-10211202021 kubelet[26382]: I0822 15:11:57.385120   26382 kubelet_node_status.go:73] "Attempting to register node" node="vtn-cds-bigdata-10211202021"
Aug 22 15:11:57 vtn-cds-bigdata-10211202021 kubelet[26382]: E0822 15:11:57.385539   26382 kubelet_node_status.go:96] "Unable to register node with API server" err="Post \"[https://10.211.202.21:6443/api/v1/nodes\](https://10.211.202.21:6443/api/v1/nodes/)": dial tcp 10.211.
Aug 22 15:11:57 vtn-cds-bigdata-10211202021 kubelet[26382]: E0822 15:11:57.551310   26382 event.go:368] "Unable to write event (may retry after sleeping)" err="Post \"[https://10.211.202.21:6443/api/v1/namespaces/default/events\](https://10.211.202.21:6443/api/v1/namespaces/default/events/)":
Aug 22 15:11:57 vtn-cds-bigdata-10211202021 kubelet[26382]: E0822 15:11:57.608678   26382 remote_runtime.go:193] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to get sandbox image \"regis
Aug 22 15:11:57 vtn-cds-bigdata-10211202021 kubelet[26382]: E0822 15:11:57.608713   26382 kuberuntime_sandbox.go:72] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to get sandbox image \"registry.k
Aug 22 15:11:57 vtn-cds-bigdata-10211202021 kubelet[26382]: E0822 15:11:57.608729   26382 kuberuntime_manager.go:1166] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to get sandbox image \"registry.
Aug 22 15:11:57 vtn-cds-bigdata-10211202021 kubelet[26382]: E0822 15:11:57.608768   26382 pod_workers.go:1298] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"kube-apiserver-vtn-cds-bigdata-10211202021_kube
Aug 22 15:11:58 vtn-cds-bigdata-10211202021 kubelet[26382]: E0822 15:11:58.786848   26382 certificate_manager.go:562] kubernetes.io/kube-apiserver-client-kubelet: Failed while requesting a signed certificate from the control plane
Aug 22 15:12:01 vtn-cds-bigdata-10211202021 kubelet[26382]: E0822 15:12:01.610303   26382 remote_runtime.go:193] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to get sandbox image \"regis
Aug 22 15:12:01 vtn-cds-bigdata-10211202021 kubelet[26382]: E0822 15:12:01.610344   26382 kuberuntime_sandbox.go:72] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to get sandbox image \"registry.k
Aug 22 15:12:01 vtn-cds-bigdata-10211202021 kubelet[26382]: E0822 15:12:01.610360   26382 kuberuntime_manager.go:1166] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to get sandbox image \"registry.
Aug 22 15:12:01 vtn-cds-bigdata-10211202021 kubelet[26382]: E0822 15:12:01.610394   26382 pod_workers.go:1298] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"kube-scheduler-vtn-cds-bigdata-10211202021_kube
Aug 22 15:12:01 vtn-cds-bigdata-10211202021 kubelet[26382]: E0822 15:12:01.613359   26382 remote_runtime.go:193] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to get sandbox image \"regis
Aug 22 15:12:01 vtn-cds-bigdata-10211202021 kubelet[26382]: E0822 15:12:01.613387   26382 kuberuntime_sandbox.go:72] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to get sandbox image \"registry.k
Aug 22 15:12:01 vtn-cds-bigdata-10211202021 kubelet[26382]: E0822 15:12:01.613402   26382 kuberuntime_manager.go:1166] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to get sandbox image \"registry.
Aug 22 15:12:01 vtn-cds-bigdata-10211202021 kubelet[26382]: E0822 15:12:01.613429   26382 pod_workers.go:1298] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"kube-controller-manager-vtn-cds-bigdata-1021120
Aug 22 15:12:01 vtn-cds-bigdata-10211202021 kubelet[26382]: E0822 15:12:01.670874   26382 eviction_manager.go:282] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"vtn-cds-bigdata-10211202021\"
Aug 22 15:12:04 vtn-cds-bigdata-10211202021 kubelet[26382]: E0822 15:12:04.233579   26382 controller.go:145] "Failed to ensure lease exists, will retry" err="Get \"https://10.211.202.21:6443/apis/coordination.k8s.io/v1/namespaces/
Aug 22 15:12:04 vtn-cds-bigdata-10211202021 kubelet[26382]: I0822 15:12:04.387015   26382 kubelet_node_status.go:73] "Attempting to register node" node="vtn-cds-bigdata-10211202021"
Aug 22 15:12:04 vtn-cds-bigdata-10211202021 kubelet[26382]: E0822 15:12:04.387315   26382 kubelet_node_status.go:96] "Unable to register node with API server" err="Post \"[https://10.211.202.21:6443/api/v1/nodes\](https://10.211.202.21:6443/api/v1/nodes/)": dial tcp 10.211.
Aug 22 15:12:04 vtn-cds-bigdata-10211202021 kubelet[26382]: E0822 15:12:04.607292   26382 remote_runtime.go:193] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to get sandbox image \"regis
Aug 22 15:12:04 vtn-cds-bigdata-10211202021 kubelet[26382]: E0822 15:12:04.607338   26382 kuberuntime_sandbox.go:72] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to get sandbox image \"registry.k
Aug 22 15:12:04 vtn-cds-bigdata-10211202021 kubelet[26382]: E0822 15:12:04.607353   26382 kuberuntime_manager.go:1166] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to get sandbox image \"registry.
Aug 22 15:12:04 vtn-cds-bigdata-10211202021 kubelet[26382]: E0822 15:12:04.607383   26382 pod_workers.go:1298] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"etcd-vtn-cds-bigdata-10211202021_kube-system(5a
Aug 22 15:12:07 vtn-cds-bigdata-10211202021 kubelet[26382]: E0822 15:12:07.552716   26382 event.go:368] "Unable to write event (may retry after sleeping)" err="Post \"[https://10.211.202.21:6443/api/v1/namespaces/default/events\](https://10.211.202.21:6443/api/v1/namespaces/default/events/)":
Aug 22 15:12:08 vtn-cds-bigdata-10211202021 kubelet[26382]: W0822 15:12:08.999272   26382 reflector.go:547] k8s.io/client-go/informers/factory.go:160: failed to list *v1.RuntimeClass: Get "https://10.211.202.21:6443/apis/node.k8s.
Aug 22 15:12:08 vtn-cds-bigdata-10211202021 kubelet[26382]: E0822 15:12:08.999334   26382 reflector.go:150] k8s.io/client-go/informers/factory.go:160: Failed to watch *v1.RuntimeClass: failed to list *v1.RuntimeClass: Get "https:/
Aug 22 15:12:10 vtn-cds-bigdata-10211202021 kubelet[26382]: E0822 15:12:10.606228   26382 remote_runtime.go:193] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to get sandbox image \"regis
Aug 22 15:12:10 vtn-cds-bigdata-10211202021 kubelet[26382]: E0822 15:12:10.606271   26382 kuberuntime_sandbox.go:72] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to get sandbox image \"registry.k
Aug 22 15:12:10 vtn-cds-bigdata-10211202021 kubelet[26382]: E0822 15:12:10.606288   26382 kuberuntime_manager.go:1166] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to get sandbox image \"registry.
Aug 22 15:12:10 vtn-cds-bigdata-10211202021 kubelet[26382]: E0822 15:12:10.606318   26382 pod_workers.go:1298] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"kube-apiserver-vtn-cds-bigdata-10211202021_kube
Aug 22 15:12:11 vtn-cds-bigdata-10211202021 kubelet[26382]: E0822 15:12:11.235187   26382 controller.go:145] "Failed to ensure lease exists, will retry" err="Get \"https://10.211.202.21:6443/apis/coordination.k8s.io/v1/namespaces/
Aug 22 15:12:11 vtn-cds-bigdata-10211202021 kubelet[26382]: I0822 15:12:11.389266   26382 kubelet_node_status.go:73] "Attempting to register node" node="vtn-cds-bigdata-10211202021"
Aug 22 15:12:11 vtn-cds-bigdata-10211202021 kubelet[26382]: E0822 15:12:11.389465   26382 kubelet_node_status.go:96] "Unable to register node with API server" err="Post \"[https://10.211.202.21:6443/api/v1/nodes\](https://10.211.202.21:6443/api/v1/nodes/)": dial tcp 10.211.
Aug 22 15:12:11 vtn-cds-bigdata-10211202021 kubelet[26382]: E0822 15:12:11.671247   26382 eviction_manager.go:282] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"vtn-cds-bigdata-10211202021\"
Aug 22 15:12:14 vtn-cds-bigdata-10211202021 kubelet[26382]: E0822 15:12:14.609579   26382 remote_runtime.go:193] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to get sandbox image \"regis
Aug 22 15:12:14 vtn-cds-bigdata-10211202021 kubelet[26382]: E0822 15:12:14.609681   26382 kuberuntime_sandbox.go:72] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to get sandbox image \"registry.k
Aug 22 15:12:14 vtn-cds-bigdata-10211202021 kubelet[26382]: E0822 15:12:14.609736   26382 kuberuntime_manager.go:1166] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to get sandbox image \"registry.
Aug 22 15:12:14 vtn-cds-bigdata-10211202021 kubelet[26382]: E0822 15:12:14.609840   26382 pod_workers.go:1298] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"kube-scheduler-vtn-cds-bigdata-10211202021_kube

How can I fix this problem? Thanks

[neolit123]

https://10.211.202.21:6443/api/v1/nodes/)": dial tcp 10.211.

seems like the api server is not starting.

but please use the support channels. check info from the bot below:
/support

[github-actions]

Hello, @tpha26644 🤖 👋

You seem to have troubles using Kubernetes and kubeadm.
Note that our issue trackers should not be used for providing support to users.
There are special channels for that purpose.

Please see:

• https://github.com/kubernetes/kubeadm#support