failed to pull image when images already exists in local #2936
Labels
kind/support
Categorizes issue or PR as a support question.
Comments
|
this reference resolve seems like a setup problem and not a kubeadm bug. kubeadm just shells out to crictl. try asking for help on support channels. /support |
|
Hello, @chicbyte 🤖 👋 You seem to have troubles using Kubernetes and kubeadm. Please see: |
github-actions
bot
added
the
kind/support
I pushed the images to local docker hub, but Note: pause3.6 is already exists in local and local docker hub I found the same question on stackoverflow : |
|
i don't see a kubeadm bug but something is not ok with the kubelet. kubeadm passes --pod-infra-container-image but the kubelet still uses a different image (the hardcoded one)? @chicbyte make sure your container runtime config uses the same pause image. if it's already the same, please log a kubernetes/kubernetes ticket, tag it with "/sig node" and show your kubelet logs. |
It was still looking for registry.k8s.io/pause:3.6 , what should I do to change this config? |
|
|
As Lubomir pointed out, this is not related to kubeadm anymore.
|
|
@neolit123 @pacoxu Thanks for your help. |
how to solve the problem, i met the same problem as you, i've got all the images to init the k8s cluster, but it still pull images from remote. [root@master ~]# docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
mudnewbie/test 7.0 e55c0812ee41 27 hours ago 5.4GB
registry.k8s.io/kube-apiserver v1.28.2 cdcab12b2dd1 5 months ago 126MB
registry.k8s.io/kube-controller-manager v1.28.2 55f13c92defb 5 months ago 122MB
registry.k8s.io/kube-scheduler v1.28.2 7a5d9d67a13f 5 months ago 60.1MB
registry.k8s.io/kube-proxy v1.28.2 c120fed2beb8 5 months ago 73.1MB
registry.k8s.io/etcd 3.5.9-0 73deb9a3f702 9 months ago 294MB
registry.k8s.io/coredns v1.10.1 ead0a4a53df8 13 months ago 53.6MB
registry.k8s.io/pause 3.9 e6f181688397 17 months ago 744kB
this is the init logs. [init] Using Kubernetes version: v1.28.2
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
W0307 20:37:32.201678 29726 checks.go:835] detected that the sandbox image "registry.cn-hangzhou.aliyuncs.com/google_containers/pause:3.9" of the container runtime is inconsistent with that used by kubeadm. It is recommended that using "registry.k8s.io/pause:3.9" as the CRI sandbox image.
error execution phase preflight: [preflight] Some fatal errors occurred:
[ERROR ImagePull]: failed to pull image registry.k8s.io/kube-apiserver:v1.28.2: output: E0307 20:29:45.970404 29783 remote_image.go:171] "PullImage from image service failed" err="rpc error: code = DeadlineExceeded desc = failed to pull and unpack image \"registry.k8s.io/kube-apiserver:v1.28.2\": failed to resolve reference \"registry.k8s.io/kube-apiserver:v1.28.2\": failed to do request: Head \"https://us-west2-docker.pkg.dev/v2/k8s-artifacts-prod/images/kube-apiserver/manifests/v1.28.2\": dial tcp 173.194.174.82:443: i/o timeout" image="registry.k8s.io/kube-apiserver:v1.28.2"
time="2024-03-07T20:29:45+08:00" level=fatal msg="pulling image: rpc error: code = DeadlineExceeded desc = failed to pull and unpack image \"registry.k8s.io/kube-apiserver:v1.28.2\": failed to resolve reference \"registry.k8s.io/kube-apiserver:v1.28.2\": failed to do request: Head \"https://us-west2-docker.pkg.dev/v2/k8s-artifacts-prod/images/kube-apiserver/manifests/v1.28.2\": dial tcp 173.194.174.82:443: i/o timeout"
, error: exit status 1
[ERROR ImagePull]: failed to pull image registry.k8s.io/kube-controller-manager:v1.28.2: output: E0307 20:32:22.769445 29824 remote_image.go:171] "PullImage from image service failed" err="rpc error: code = DeadlineExceeded desc = failed to pull and unpack image \"registry.k8s.io/kube-controller-manager:v1.28.2\": failed to resolve reference \"registry.k8s.io/kube-controller-manager:v1.28.2\": failed to do request: Head \"https://us-west2-docker.pkg.dev/v2/k8s-artifacts-prod/images/kube-controller-manager/manifests/v1.28.2\": dial tcp 142.251.170.82:443: i/o timeout" image="registry.k8s.io/kube-controller-manager:v1.28.2"
time="2024-03-07T20:32:22+08:00" level=fatal msg="pulling image: rpc error: code = DeadlineExceeded desc = failed to pull and unpack image \"registry.k8s.io/kube-controller-manager:v1.28.2\": failed to resolve reference \"registry.k8s.io/kube-controller-manager:v1.28.2\": failed to do request: Head \"https://us-west2-docker.pkg.dev/v2/k8s-artifacts-prod/images/kube-controller-manager/manifests/v1.28.2\": dial tcp 142.251.170.82:443: i/o timeout"
, error: exit status 1
[ERROR ImagePull]: failed to pull image registry.k8s.io/kube-scheduler:v1.28.2: output: E0307 20:34:57.365710 29860 remote_image.go:171] "PullImage from image service failed" err="rpc error: code = DeadlineExceeded desc = failed to pull and unpack image \"registry.k8s.io/kube-scheduler:v1.28.2\": failed to resolve reference \"registry.k8s.io/kube-scheduler:v1.28.2\": failed to do request: Head \"https://us-west2-docker.pkg.dev/v2/k8s-artifacts-prod/images/kube-scheduler/manifests/v1.28.2\": dial tcp 108.177.125.82:443: i/o timeout" image="registry.k8s.io/kube-scheduler:v1.28.2"
time="2024-03-07T20:34:57+08:00" level=fatal msg="pulling image: rpc error: code = DeadlineExceeded desc = failed to pull and unpack image \"registry.k8s.io/kube-scheduler:v1.28.2\": failed to resolve reference \"registry.k8s.io/kube-scheduler:v1.28.2\": failed to do request: Head \"https://us-west2-docker.pkg.dev/v2/k8s-artifacts-prod/images/kube-scheduler/manifests/v1.28.2\": dial tcp 108.177.125.82:443: i/o timeout"
, error: exit status 1
[ERROR ImagePull]: failed to pull image registry.k8s.io/kube-proxy:v1.28.2: output: E0307 20:37:32.171182 29896 remote_image.go:171] "PullImage from image service failed" err="rpc error: code = DeadlineExceeded desc = failed to pull and unpack image \"registry.k8s.io/kube-proxy:v1.28.2\": failed to resolve reference \"registry.k8s.io/kube-proxy:v1.28.2\": failed to do request: Head \"https://us-west2-docker.pkg.dev/v2/k8s-artifacts-prod/images/kube-proxy/manifests/v1.28.2\": dial tcp 108.177.125.82:443: i/o timeout" image="registry.k8s.io/kube-proxy:v1.28.2"
time="2024-03-07T20:37:32+08:00" level=fatal msg="pulling image: rpc error: code = DeadlineExceeded desc = failed to pull and unpack image \"registry.k8s.io/kube-proxy:v1.28.2\": failed to resolve reference \"registry.k8s.io/kube-proxy:v1.28.2\": failed to do request: Head \"https://us-west2-docker.pkg.dev/v2/k8s-artifacts-prod/images/kube-proxy/manifests/v1.28.2\": dial tcp 108.177.125.82:443: i/o timeout"
, error: exit status 1
[ERROR ImagePull]: failed to pull image registry.k8s.io/pause:3.9: output: E0307 20:40:09.220777 29939 remote_image.go:171] "PullImage from image service failed" err="rpc error: code = DeadlineExceeded desc = failed to pull and unpack image \"registry.k8s.io/pause:3.9\": failed to resolve reference \"registry.k8s.io/pause:3.9\": failed to do request: Head \"https://us-west2-docker.pkg.dev/v2/k8s-artifacts-prod/images/pause/manifests/3.9\": dial tcp 142.250.157.82:443: i/o timeout" image="registry.k8s.io/pause:3.9"
time="2024-03-07T20:40:09+08:00" level=fatal msg="pulling image: rpc error: code = DeadlineExceeded desc = failed to pull and unpack image \"registry.k8s.io/pause:3.9\": failed to resolve reference \"registry.k8s.io/pause:3.9\": failed to do request: Head \"https://us-west2-docker.pkg.dev/v2/k8s-artifacts-prod/images/pause/manifests/3.9\": dial tcp 142.250.157.82:443: i/o timeout"
, error: exit status 1
[ERROR ImagePull]: failed to pull image registry.k8s.io/etcd:3.5.9-0: output: E0307 20:42:43.494045 29980 remote_image.go:171] "PullImage from image service failed" err="rpc error: code = DeadlineExceeded desc = failed to pull and unpack image \"registry.k8s.io/etcd:3.5.9-0\": failed to resolve reference \"registry.k8s.io/etcd:3.5.9-0\": failed to do request: Head \"https://us-west2-docker.pkg.dev/v2/k8s-artifacts-prod/images/etcd/manifests/3.5.9-0\": dial tcp 64.233.187.82:443: i/o timeout" image="registry.k8s.io/etcd:3.5.9-0"
time="2024-03-07T20:42:43+08:00" level=fatal msg="pulling image: rpc error: code = DeadlineExceeded desc = failed to pull and unpack image \"registry.k8s.io/etcd:3.5.9-0\": failed to resolve reference \"registry.k8s.io/etcd:3.5.9-0\": failed to do request: Head \"https://us-west2-docker.pkg.dev/v2/k8s-artifacts-prod/images/etcd/manifests/3.5.9-0\": dial tcp 64.233.187.82:443: i/o timeout"
, error: exit status 1
[ERROR ImagePull]: failed to pull image registry.k8s.io/coredns/coredns:v1.10.1: output: E0307 20:45:20.049924 30018 remote_image.go:171] "PullImage from image service failed" err="rpc error: code = DeadlineExceeded desc = failed to pull and unpack image \"registry.k8s.io/coredns/coredns:v1.10.1\": failed to resolve reference \"registry.k8s.io/coredns/coredns:v1.10.1\": failed to do request: Head \"https://us-west2-docker.pkg.dev/v2/k8s-artifacts-prod/images/coredns/coredns/manifests/v1.10.1\": dial tcp 64.233.189.82:443: i/o timeout" image="registry.k8s.io/coredns/coredns:v1.10.1"
time="2024-03-07T20:45:20+08:00" level=fatal msg="pulling image: rpc error: code = DeadlineExceeded desc = failed to pull and unpack image \"registry.k8s.io/coredns/coredns:v1.10.1\": failed to resolve reference \"registry.k8s.io/coredns/coredns:v1.10.1\": failed to do request: Head \"https://us-west2-docker.pkg.dev/v2/k8s-artifacts-prod/images/coredns/coredns/manifests/v1.10.1\": dial tcp 64.233.189.82:443: i/o timeout"
, error: exit status 1
[preflight] If you know what you are doing, you can make a check non-fatal with `--ignore-preflight-errors=...`
To see the stack trace of this error execute with --v=5 or higher |
Oh, I found where the problem is. I didn't config kubeadm correctly, i need to config the repository namespace in kubeadm, below is the config yaml file. apiVersion: kubeadm.k8s.io/v1beta3
bootstrapTokens:
- groups:
- system:bootstrappers:kubeadm:default-node-token
token: dy42df.qrsmyv39pncjqanc
ttl: 24h0m0s
usages:
- signing
- authentication
kind: InitConfiguration
localAPIEndpoint:
advertiseAddress: 192.168.1.241
bindPort: 6443
nodeRegistration:
criSocket: unix:///var/run/containerd/containerd.sock
imagePullPolicy: IfNotPresent
name: master
taints:
- effect: NoSchedule
key: node-role.kubernetes.io/control-plane
---
apiServer:
certSANs:
- master
- xx.xx.xx.xx # ip address
- 192.168.1.241
- 10.96.0.1
timeoutForControlPlane: 4m0s
apiVersion: kubeadm.k8s.io/v1beta3
certificatesDir: /etc/kubernetes/pki
clusterName: kubernetes
controlPlaneEndpoint: xx.xx.xx.xx:6443 # ip address
controllerManager: {}
dns: {}
etcd:
local:
dataDir: /var/lib/etcd
imageRepository: registry.cn-hangzhou.aliyuncs.com/google_containers
kind: ClusterConfiguration
kubernetesVersion: v1.28.2
networking:
dnsDomain: cluster.local
podSubnet: 10.244.0.0/16
serviceSubnet: 10.96.0.0/12
scheduler: {}
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Versions
kubeadm version (use
v1.28.2):Environment:
v1.28.2):Linux 10.0.0.9 5.4.119-1-tlinux4-0010.3 #1 SMP Thu Jan 5 17:31:23 CST 2023 x86_64 x86_64 x86_64 GNU/Linux):What happened?
The text was updated successfully, but these errors were encountered: