Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Upgrade Go from 1.19.12 and 1.20.7 to 1.19.13 and 1.20.8 #1129

Merged
merged 3 commits into from
Sep 14, 2023
Merged

Upgrade Go from 1.19.12 and 1.20.7 to 1.19.13 and 1.20.8 #1129

merged 3 commits into from
Sep 14, 2023

Conversation

flatcar-infra
Copy link

@flatcar-infra flatcar-infra commented Sep 11, 2023 •
edited by dongsupark
Loading

@flatcar-infra flatcar-infra temporarily deployed to development September 11, 2023 07:23 — with GitHub Actions Inactive
@dongsupark dongsupark temporarily deployed to development September 11, 2023 07:34 — with GitHub Actions Inactive
@dongsupark dongsupark force-pushed the go-1.19.13-and-1.20.8-main branch from 524eb7c to 39abf69 Compare September 12, 2023 13:18
@dongsupark dongsupark temporarily deployed to development September 12, 2023 13:18 — with GitHub Actions Inactive
@github-actions
Copy link

Test report for 3724.0.0+nightly-20230911-2100 / amd64 arm64

Platforms tested : qemu_uefi-amd64 qemu_update-amd64 qemu_uefi-arm64

ok bpf.execsnoop 🟢 Succeeded: qemu_uefi-amd64 (1)

ok bpf.local-gadget 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok cl.basic 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok cl.cgroupv1 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok cl.cloudinit.basic 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok cl.cloudinit.multipart-mime 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok cl.cloudinit.script 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok cl.disk.raid0.data 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok cl.disk.raid0.root 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok cl.disk.raid1.data 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok cl.disk.raid1.root 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok cl.etcd-member.discovery 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok cl.etcd-member.etcdctlv3 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok cl.etcd-member.v2-backup-restore 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok cl.filesystem 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok cl.flannel.udp 🟢 Succeeded: qemu_uefi-amd64 (1)

ok cl.flannel.vxlan 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok cl.ignition.instantiated.enable-unit 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok cl.ignition.kargs 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok cl.ignition.luks 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok cl.ignition.oem.indirect 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok cl.ignition.oem.indirect.new 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok cl.ignition.oem.regular 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok cl.ignition.oem.regular.new 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok cl.ignition.oem.reuse 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok cl.ignition.oem.wipe 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok cl.ignition.symlink 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok cl.ignition.translation 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok cl.ignition.v1.btrfsroot 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok cl.ignition.v1.ext4root 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok cl.ignition.v1.groups 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok cl.ignition.v1.once 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok cl.ignition.v1.sethostname 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok cl.ignition.v1.users 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok cl.ignition.v1.xfsroot 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok cl.ignition.v2.btrfsroot 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok cl.ignition.v2.ext4root 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok cl.ignition.v2.users 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok cl.ignition.v2.xfsroot 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok cl.ignition.v2_1.ext4checkexisting 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok cl.ignition.v2_1.swap 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok cl.ignition.v2_1.vfat 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok cl.install.cloudinit 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok cl.internet 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok cl.locksmith.cluster 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok cl.misc.falco 🟢 Succeeded: qemu_uefi-amd64 (1)

ok cl.network.initramfs.second-boot 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok cl.network.listeners 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok cl.network.wireguard 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok cl.omaha.ping 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok cl.osreset.ignition-rerun 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok cl.overlay.cleanup 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok cl.swap_activation 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok cl.sysext.boot 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok cl.sysext.fallbackdownload # SKIP 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok cl.toolbox.dnf-install 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok cl.update.badverity 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok cl.update.grubnop 🟢 Succeeded: qemu_uefi-amd64 (1)

ok cl.update.reboot 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok cl.users.shells 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok cl.verity 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok coreos.auth.verify 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok coreos.ignition.groups 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok coreos.ignition.once 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok coreos.ignition.resource.local 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok coreos.ignition.resource.remote 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok coreos.ignition.resource.s3.versioned 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok coreos.ignition.security.tls 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok coreos.ignition.sethostname 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok coreos.ignition.systemd.enable-service 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok coreos.locksmith.reboot 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok coreos.locksmith.tls 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok coreos.selinux.boolean 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok coreos.selinux.enforce 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok coreos.tls.fetch-urls 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok coreos.update.badusr 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok devcontainer.docker 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok devcontainer.systemd-nspawn 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok docker.base 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok docker.btrfs-storage 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok docker.containerd-restart 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok docker.lib-coreos-dockerd-compat 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok docker.network 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok docker.selinux 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok docker.torcx-manifest-pkgs 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok docker.userns 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok kubeadm.v1.24.14.calico.base 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (4) ❌ Failed: qemu_uefi-arm64 (1, 2, 3)

                Diagnostic output for qemu_uefi-arm64, run 3 
    L1: " Error: _cluster.go:117: I0912 17:46:47.625459    1488 version.go:256] remote version is much newer: v1.28.1; falling back to: stable-1.24"
    L2: "cluster.go:117: [config/images] Pulled registry.k8s.io/kube-apiserver:v1.24.17"
    L3: "cluster.go:117: [config/images] Pulled registry.k8s.io/kube-controller-manager:v1.24.17"
    L4: "cluster.go:117: [config/images] Pulled registry.k8s.io/kube-scheduler:v1.24.17"
    L5: "cluster.go:117: [config/images] Pulled registry.k8s.io/kube-proxy:v1.24.17"
    L6: "cluster.go:117: [config/images] Pulled registry.k8s.io/pause:3.7"
    L7: "cluster.go:117: [config/images] Pulled registry.k8s.io/etcd:3.5.6-0"
    L8: "cluster.go:117: [config/images] Pulled registry.k8s.io/coredns/coredns:v1.8.6"
    L9: "cluster.go:117: I0912 17:47:00.640405    1652 version.go:256] remote version is much newer: v1.28.1; falling back to: stable-1.24"
    L10: "cluster.go:117: [init] Using Kubernetes version: v1.24.17"
    L11: "cluster.go:117: [preflight] Running pre-flight checks"
    L12: "cluster.go:117: [preflight] Pulling images required for setting up a Kubernetes cluster"
    L13: "cluster.go:117: [preflight] This might take a minute or two, depending on the speed of your internet connection"
    L14: "cluster.go:117: [preflight] You can also perform this action in beforehand using _kubeadm config images pull_"
    L15: "cluster.go:117: [certs] Using certificateDir folder __/etc/kubernetes/pki__"
    L16: "cluster.go:117: [certs] Generating __ca__ certificate and key"
    L17: "cluster.go:117: [certs] Generating __apiserver__ certificate and key"
    L18: "cluster.go:117: [certs] apiserver serving cert is signed for DNS names [kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local localhost] and IPs [10.96.0.1 10.0.0.5?]"
    L19: "cluster.go:117: [certs] Generating __apiserver-kubelet-client__ certificate and key"
    L20: "cluster.go:117: [certs] Generating __front-proxy-ca__ certificate and key"
    L21: "cluster.go:117: [certs] Generating __front-proxy-client__ certificate and key"
    L22: "cluster.go:117: [certs] External etcd mode: Skipping etcd/ca certificate authority generation"
    L23: "cluster.go:117: [certs] External etcd mode: Skipping etcd/server certificate generation"
    L24: "cluster.go:117: [certs] External etcd mode: Skipping etcd/peer certificate generation"
    L25: "cluster.go:117: [certs] External etcd mode: Skipping etcd/healthcheck-client certificate generation"
    L26: "cluster.go:117: [certs] External etcd mode: Skipping apiserver-etcd-client certificate generation"
    L27: "cluster.go:117: [certs] Generating __sa__ key and public key"
    L28: "cluster.go:117: [kubeconfig] Using kubeconfig folder __/etc/kubernetes__"
    L29: "cluster.go:117: [kubeconfig] Writing __admin.conf__ kubeconfig file"
    L30: "cluster.go:117: [kubeconfig] Writing __kubelet.conf__ kubeconfig file"
    L31: "cluster.go:117: [kubeconfig] Writing __controller-manager.conf__ kubeconfig file"
    L32: "cluster.go:117: [kubeconfig] Writing __scheduler.conf__ kubeconfig file"
    L33: "cluster.go:117: [kubelet-start] Writing kubelet environment file with flags to file __/var/lib/kubelet/kubeadm-flags.env__"
    L34: "cluster.go:117: [kubelet-start] Writing kubelet configuration to file __/var/lib/kubelet/config.yaml__"
    L35: "cluster.go:117: [kubelet-start] Starting the kubelet"
    L36: "cluster.go:117: [control-plane] Using manifest folder __/etc/kubernetes/manifests__"
    L37: "cluster.go:117: [control-plane] Creating static Pod manifest for __kube-apiserver__"
    L38: "cluster.go:117: [control-plane] Creating static Pod manifest for __kube-controller-manager__"
    L39: "cluster.go:117: [control-plane] Creating static Pod manifest for __kube-scheduler__"
    L40: "cluster.go:117: [wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory __/etc/kubernetes/manifests__. This can take up to 30m0s"
    L41: "cluster.go:117: [apiclient] All control plane components are healthy after 7.003638 seconds"
    L42: "cluster.go:117: [upload-config] Storing the configuration used in ConfigMap __kubeadm-config__ in the __kube-system__ Namespace"
    L43: "cluster.go:117: [kubelet] Creating a ConfigMap __kubelet-config__ in namespace kube-system with the configuration for the kubelets in the cluster"
    L44: "cluster.go:117: [upload-certs] Skipping phase. Please see --upload-certs"
    L45: "cluster.go:117: [mark-control-plane] Marking the node localhost as control-plane by adding the labels: [node-role.kubernetes.io/control-plane node.kubernetes.io/exclude-from-external-load-balancers]"
    L46: "cluster.go:117: [mark-control-plane] Marking the node localhost as control-plane by adding the taints [node-role.kubernetes.io/master:NoSchedule node-role.kubernetes.io/control-plane:NoSchedule]"
    L47: "cluster.go:117: [bootstrap-token] Using token: u46307.9soxvqnk72eehqsa"
    L48: "cluster.go:117: [bootstrap-token] Configuring bootstrap tokens, cluster-info ConfigMap, RBAC Roles"
    L49: "cluster.go:117: [bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to get nodes"
    L50: "cluster.go:117: [bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials"
    L51: "cluster.go:117: [bootstrap-token] Configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token"
    L52: "cluster.go:117: [bootstrap-token] Configured RBAC rules to allow certificate rotation for all node client certificates in the cluster"
    L53: "cluster.go:117: [bootstrap-token] Creating the __cluster-info__ ConfigMap in the __kube-public__ namespace"
    L54: "cluster.go:117: [kubelet-finalize] Updating __/etc/kubernetes/kubelet.conf__ to point to a rotatable kubelet client certificate and key"
    L55: "cluster.go:117: [addons] Applied essential addon: CoreDNS"
    L56: "cluster.go:117: [addons] Applied essential addon: kube-proxy"
    L57: "cluster.go:117: "
    L58: "cluster.go:117: Your Kubernetes control-plane has initialized successfully!"
    L59: "cluster.go:117: "
    L60: "cluster.go:117: To start using your cluster, you need to run the following as a regular user:"
    L61: "cluster.go:117: "
    L62: "cluster.go:117:   mkdir -p $HOME/.kube"
    L63: "cluster.go:117:   sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config"
    L64: "cluster.go:117:   sudo chown $(id -u):$(id -g) $HOME/.kube/config"
    L65: "cluster.go:117: "
    L66: "cluster.go:117: Alternatively, if you are the root user, you can run:"
    L67: "cluster.go:117: "
    L68: "cluster.go:117:   export KUBECONFIG=/etc/kubernetes/admin.conf"
    L69: "cluster.go:117: "
    L70: "cluster.go:117: You should now deploy a pod network to the cluster."
    L71: "cluster.go:117: Run __kubectl apply -f [podnetwork].yaml__ with one of the options listed at:"
    L72: "cluster.go:117:   https://kubernetes.io/docs/concepts/cluster-administration/addons/"
    L73: "cluster.go:117: "
    L74: "cluster.go:117: Then you can join any number of worker nodes by running the following on each as root:"
    L75: "cluster.go:117: "
    L76: "cluster.go:117: kubeadm join 10.0.0.5:6443 --token u46307.9soxvqnk72eehqsa _"
    L77: "cluster.go:117:  --discovery-token-ca-cert-hash sha256:317e659aa39d5a04d882bc76ced0007d46cd00eb883fa911f6b6714c1c2c92bc "
    L78: "cluster.go:117: namespace/tigera-operator created"
    L79: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/bgpconfigurations.crd.projectcalico.org created"
    L80: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/bgppeers.crd.projectcalico.org created"
    L81: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/blockaffinities.crd.projectcalico.org created"
    L82: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/caliconodestatuses.crd.projectcalico.org created"
    L83: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/clusterinformations.crd.projectcalico.org created"
    L84: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/felixconfigurations.crd.projectcalico.org created"
    L85: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/globalnetworkpolicies.crd.projectcalico.org created"
    L86: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/globalnetworksets.crd.projectcalico.org created"
    L87: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/hostendpoints.crd.projectcalico.org created"
    L88: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/ipamblocks.crd.projectcalico.org created"
    L89: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/ipamconfigs.crd.projectcalico.org created"
    L90: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/ipamhandles.crd.projectcalico.org created"
    L91: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/ippools.crd.projectcalico.org created"
    L92: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/ipreservations.crd.projectcalico.org created"
    L93: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/kubecontrollersconfigurations.crd.projectcalico.org created"
    L94: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/networkpolicies.crd.projectcalico.org created"
    L95: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/networksets.crd.projectcalico.org created"
    L96: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/apiservers.operator.tigera.io created"
    L97: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/imagesets.operator.tigera.io created"
    L98: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/installations.operator.tigera.io created"
    L99: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/tigerastatuses.operator.tigera.io created"
    L100: "cluster.go:117: serviceaccount/tigera-operator created"
    L101: "cluster.go:117: clusterrole.rbac.authorization.k8s.io/tigera-operator created"
    L102: "cluster.go:117: clusterrolebinding.rbac.authorization.k8s.io/tigera-operator created"
    L103: "cluster.go:117: deployment.apps/tigera-operator created"
    L104: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/installations.operator.tigera.io condition met"
    L105: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/apiservers.operator.tigera.io condition met"
    L106: "cluster.go:117: installation.operator.tigera.io/default created"
    L107: "cluster.go:117: apiserver.operator.tigera.io/default created"
    L108: "cluster.go:117: Created symlink /etc/systemd/system/multi-user.target.wants/kubelet.service ??? /etc/systemd/system/kubelet.service."
    L109: "--- FAIL: kubeadm.v1.24.14.calico.base/nginx_deployment (93.55s)"
    L110: "kubeadm.go:313: nginx is not deployed: ready replicas should be equal to 1: null_"
    L111: " "
                Diagnostic output for qemu_uefi-arm64, run 2 
    L1: " Error: _cluster.go:117: I0912 17:40:38.817958    1494 version.go:256] remote version is much newer: v1.28.1; falling back to: stable-1.24"
    L2: "cluster.go:117: [config/images] Pulled registry.k8s.io/kube-apiserver:v1.24.17"
    L3: "cluster.go:117: [config/images] Pulled registry.k8s.io/kube-controller-manager:v1.24.17"
    L4: "cluster.go:117: [config/images] Pulled registry.k8s.io/kube-scheduler:v1.24.17"
    L5: "cluster.go:117: [config/images] Pulled registry.k8s.io/kube-proxy:v1.24.17"
    L6: "cluster.go:117: [config/images] Pulled registry.k8s.io/pause:3.7"
    L7: "cluster.go:117: [config/images] Pulled registry.k8s.io/etcd:3.5.6-0"
    L8: "cluster.go:117: [config/images] Pulled registry.k8s.io/coredns/coredns:v1.8.6"
    L9: "cluster.go:117: I0912 17:40:50.391867    1659 version.go:256] remote version is much newer: v1.28.1; falling back to: stable-1.24"
    L10: "cluster.go:117: [init] Using Kubernetes version: v1.24.17"
    L11: "cluster.go:117: [preflight] Running pre-flight checks"
    L12: "cluster.go:117: [preflight] Pulling images required for setting up a Kubernetes cluster"
    L13: "cluster.go:117: [preflight] This might take a minute or two, depending on the speed of your internet connection"
    L14: "cluster.go:117: [preflight] You can also perform this action in beforehand using _kubeadm config images pull_"
    L15: "cluster.go:117: [certs] Using certificateDir folder __/etc/kubernetes/pki__"
    L16: "cluster.go:117: [certs] Generating __ca__ certificate and key"
    L17: "cluster.go:117: [certs] Generating __apiserver__ certificate and key"
    L18: "cluster.go:117: [certs] apiserver serving cert is signed for DNS names [kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local localhost] and IPs [10.96.0.1 10.0.0.7?]"
    L19: "cluster.go:117: [certs] Generating __apiserver-kubelet-client__ certificate and key"
    L20: "cluster.go:117: [certs] Generating __front-proxy-ca__ certificate and key"
    L21: "cluster.go:117: [certs] Generating __front-proxy-client__ certificate and key"
    L22: "cluster.go:117: [certs] External etcd mode: Skipping etcd/ca certificate authority generation"
    L23: "cluster.go:117: [certs] External etcd mode: Skipping etcd/server certificate generation"
    L24: "cluster.go:117: [certs] External etcd mode: Skipping etcd/peer certificate generation"
    L25: "cluster.go:117: [certs] External etcd mode: Skipping etcd/healthcheck-client certificate generation"
    L26: "cluster.go:117: [certs] External etcd mode: Skipping apiserver-etcd-client certificate generation"
    L27: "cluster.go:117: [certs] Generating __sa__ key and public key"
    L28: "cluster.go:117: [kubeconfig] Using kubeconfig folder __/etc/kubernetes__"
    L29: "cluster.go:117: [kubeconfig] Writing __admin.conf__ kubeconfig file"
    L30: "cluster.go:117: [kubeconfig] Writing __kubelet.conf__ kubeconfig file"
    L31: "cluster.go:117: [kubeconfig] Writing __controller-manager.conf__ kubeconfig file"
    L32: "cluster.go:117: [kubeconfig] Writing __scheduler.conf__ kubeconfig file"
    L33: "cluster.go:117: [kubelet-start] Writing kubelet environment file with flags to file __/var/lib/kubelet/kubeadm-flags.env__"
    L34: "cluster.go:117: [kubelet-start] Writing kubelet configuration to file __/var/lib/kubelet/config.yaml__"
    L35: "cluster.go:117: [kubelet-start] Starting the kubelet"
    L36: "cluster.go:117: [control-plane] Using manifest folder __/etc/kubernetes/manifests__"
    L37: "cluster.go:117: [control-plane] Creating static Pod manifest for __kube-apiserver__"
    L38: "cluster.go:117: [control-plane] Creating static Pod manifest for __kube-controller-manager__"
    L39: "cluster.go:117: [control-plane] Creating static Pod manifest for __kube-scheduler__"
    L40: "cluster.go:117: [wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory __/etc/kubernetes/manifests__. This can take up to 30m0s"
    L41: "cluster.go:117: [apiclient] All control plane components are healthy after 8.005008 seconds"
    L42: "cluster.go:117: [upload-config] Storing the configuration used in ConfigMap __kubeadm-config__ in the __kube-system__ Namespace"
    L43: "cluster.go:117: [kubelet] Creating a ConfigMap __kubelet-config__ in namespace kube-system with the configuration for the kubelets in the cluster"
    L44: "cluster.go:117: [upload-certs] Skipping phase. Please see --upload-certs"
    L45: "cluster.go:117: [mark-control-plane] Marking the node localhost as control-plane by adding the labels: [node-role.kubernetes.io/control-plane node.kubernetes.io/exclude-from-external-load-balancers]"
    L46: "cluster.go:117: [mark-control-plane] Marking the node localhost as control-plane by adding the taints [node-role.kubernetes.io/master:NoSchedule node-role.kubernetes.io/control-plane:NoSchedule]"
    L47: "cluster.go:117: [bootstrap-token] Using token: wlzvpa.o3otzj7vy1mw56hn"
    L48: "cluster.go:117: [bootstrap-token] Configuring bootstrap tokens, cluster-info ConfigMap, RBAC Roles"
    L49: "cluster.go:117: [bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to get nodes"
    L50: "cluster.go:117: [bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials"
    L51: "cluster.go:117: [bootstrap-token] Configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token"
    L52: "cluster.go:117: [bootstrap-token] Configured RBAC rules to allow certificate rotation for all node client certificates in the cluster"
    L53: "cluster.go:117: [bootstrap-token] Creating the __cluster-info__ ConfigMap in the __kube-public__ namespace"
    L54: "cluster.go:117: [kubelet-finalize] Updating __/etc/kubernetes/kubelet.conf__ to point to a rotatable kubelet client certificate and key"
    L55: "cluster.go:117: [addons] Applied essential addon: CoreDNS"
    L56: "cluster.go:117: [addons] Applied essential addon: kube-proxy"
    L57: "cluster.go:117: "
    L58: "cluster.go:117: Your Kubernetes control-plane has initialized successfully!"
    L59: "cluster.go:117: "
    L60: "cluster.go:117: To start using your cluster, you need to run the following as a regular user:"
    L61: "cluster.go:117: "
    L62: "cluster.go:117:   mkdir -p $HOME/.kube"
    L63: "cluster.go:117:   sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config"
    L64: "cluster.go:117:   sudo chown $(id -u):$(id -g) $HOME/.kube/config"
    L65: "cluster.go:117: "
    L66: "cluster.go:117: Alternatively, if you are the root user, you can run:"
    L67: "cluster.go:117: "
    L68: "cluster.go:117:   export KUBECONFIG=/etc/kubernetes/admin.conf"
    L69: "cluster.go:117: "
    L70: "cluster.go:117: You should now deploy a pod network to the cluster."
    L71: "cluster.go:117: Run __kubectl apply -f [podnetwork].yaml__ with one of the options listed at:"
    L72: "cluster.go:117:   https://kubernetes.io/docs/concepts/cluster-administration/addons/"
    L73: "cluster.go:117: "
    L74: "cluster.go:117: Then you can join any number of worker nodes by running the following on each as root:"
    L75: "cluster.go:117: "
    L76: "cluster.go:117: kubeadm join 10.0.0.7:6443 --token wlzvpa.o3otzj7vy1mw56hn _"
    L77: "cluster.go:117:  --discovery-token-ca-cert-hash sha256:b00f332ac2c13488f342eb850c8c4b09e6a70b9703ed215a3866ed755aa77a73 "
    L78: "cluster.go:117: namespace/tigera-operator created"
    L79: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/bgpconfigurations.crd.projectcalico.org created"
    L80: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/bgppeers.crd.projectcalico.org created"
    L81: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/blockaffinities.crd.projectcalico.org created"
    L82: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/caliconodestatuses.crd.projectcalico.org created"
    L83: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/clusterinformations.crd.projectcalico.org created"
    L84: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/felixconfigurations.crd.projectcalico.org created"
    L85: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/globalnetworkpolicies.crd.projectcalico.org created"
    L86: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/globalnetworksets.crd.projectcalico.org created"
    L87: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/hostendpoints.crd.projectcalico.org created"
    L88: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/ipamblocks.crd.projectcalico.org created"
    L89: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/ipamconfigs.crd.projectcalico.org created"
    L90: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/ipamhandles.crd.projectcalico.org created"
    L91: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/ippools.crd.projectcalico.org created"
    L92: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/ipreservations.crd.projectcalico.org created"
    L93: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/kubecontrollersconfigurations.crd.projectcalico.org created"
    L94: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/networkpolicies.crd.projectcalico.org created"
    L95: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/networksets.crd.projectcalico.org created"
    L96: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/apiservers.operator.tigera.io created"
    L97: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/imagesets.operator.tigera.io created"
    L98: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/installations.operator.tigera.io created"
    L99: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/tigerastatuses.operator.tigera.io created"
    L100: "cluster.go:117: serviceaccount/tigera-operator created"
    L101: "cluster.go:117: clusterrole.rbac.authorization.k8s.io/tigera-operator created"
    L102: "cluster.go:117: clusterrolebinding.rbac.authorization.k8s.io/tigera-operator created"
    L103: "cluster.go:117: deployment.apps/tigera-operator created"
    L104: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/installations.operator.tigera.io condition met"
    L105: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/apiservers.operator.tigera.io condition met"
    L106: "cluster.go:117: installation.operator.tigera.io/default created"
    L107: "cluster.go:117: apiserver.operator.tigera.io/default created"
    L108: "cluster.go:117: Created symlink /etc/systemd/system/multi-user.target.wants/kubelet.service ??? /etc/systemd/system/kubelet.service."
    L109: "--- FAIL: kubeadm.v1.24.14.calico.base/nginx_deployment (93.51s)"
    L110: "kubeadm.go:313: nginx is not deployed: ready replicas should be equal to 1: null_"
    L111: " "
                Diagnostic output for qemu_uefi-arm64, run 1 
    L1: "  "
    L2: " Error: _cluster.go:117: I0912 17:33:16.758269    1488 version.go:256] remote version is much newer: v1.28.1; falling back to: stable-1.24"
    L3: "cluster.go:117: [config/images] Pulled registry.k8s.io/kube-apiserver:v1.24.17"
    L4: "cluster.go:117: [config/images] Pulled registry.k8s.io/kube-controller-manager:v1.24.17"
    L5: "cluster.go:117: [config/images] Pulled registry.k8s.io/kube-scheduler:v1.24.17"
    L6: "cluster.go:117: [config/images] Pulled registry.k8s.io/kube-proxy:v1.24.17"
    L7: "cluster.go:117: [config/images] Pulled registry.k8s.io/pause:3.7"
    L8: "cluster.go:117: [config/images] Pulled registry.k8s.io/etcd:3.5.6-0"
    L9: "cluster.go:117: [config/images] Pulled registry.k8s.io/coredns/coredns:v1.8.6"
    L10: "cluster.go:117: I0912 17:33:29.491276    1652 version.go:256] remote version is much newer: v1.28.1; falling back to: stable-1.24"
    L11: "cluster.go:117: [init] Using Kubernetes version: v1.24.17"
    L12: "cluster.go:117: [preflight] Running pre-flight checks"
    L13: "cluster.go:117: [preflight] Pulling images required for setting up a Kubernetes cluster"
    L14: "cluster.go:117: [preflight] This might take a minute or two, depending on the speed of your internet connection"
    L15: "cluster.go:117: [preflight] You can also perform this action in beforehand using _kubeadm config images pull_"
    L16: "cluster.go:117: [certs] Using certificateDir folder __/etc/kubernetes/pki__"
    L17: "cluster.go:117: [certs] Generating __ca__ certificate and key"
    L18: "cluster.go:117: [certs] Generating __apiserver__ certificate and key"
    L19: "cluster.go:117: [certs] apiserver serving cert is signed for DNS names [kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local localhost] and IPs [10.96.0.1 10.0.0.1?50]"
    L20: "cluster.go:117: [certs] Generating __apiserver-kubelet-client__ certificate and key"
    L21: "cluster.go:117: [certs] Generating __front-proxy-ca__ certificate and key"
    L22: "cluster.go:117: [certs] Generating __front-proxy-client__ certificate and key"
    L23: "cluster.go:117: [certs] External etcd mode: Skipping etcd/ca certificate authority generation"
    L24: "cluster.go:117: [certs] External etcd mode: Skipping etcd/server certificate generation"
    L25: "cluster.go:117: [certs] External etcd mode: Skipping etcd/peer certificate generation"
    L26: "cluster.go:117: [certs] External etcd mode: Skipping etcd/healthcheck-client certificate generation"
    L27: "cluster.go:117: [certs] External etcd mode: Skipping apiserver-etcd-client certificate generation"
    L28: "cluster.go:117: [certs] Generating __sa__ key and public key"
    L29: "cluster.go:117: [kubeconfig] Using kubeconfig folder __/etc/kubernetes__"
    L30: "cluster.go:117: [kubeconfig] Writing __admin.conf__ kubeconfig file"
    L31: "cluster.go:117: [kubeconfig] Writing __kubelet.conf__ kubeconfig file"
    L32: "cluster.go:117: [kubeconfig] Writing __controller-manager.conf__ kubeconfig file"
    L33: "cluster.go:117: [kubeconfig] Writing __scheduler.conf__ kubeconfig file"
    L34: "cluster.go:117: [kubelet-start] Writing kubelet environment file with flags to file __/var/lib/kubelet/kubeadm-flags.env__"
    L35: "cluster.go:117: [kubelet-start] Writing kubelet configuration to file __/var/lib/kubelet/config.yaml__"
    L36: "cluster.go:117: [kubelet-start] Starting the kubelet"
    L37: "cluster.go:117: [control-plane] Using manifest folder __/etc/kubernetes/manifests__"
    L38: "cluster.go:117: [control-plane] Creating static Pod manifest for __kube-apiserver__"
    L39: "cluster.go:117: [control-plane] Creating static Pod manifest for __kube-controller-manager__"
    L40: "cluster.go:117: [control-plane] Creating static Pod manifest for __kube-scheduler__"
    L41: "cluster.go:117: [wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory __/etc/kubernetes/manifests__. This can take up to 30m0s"
    L42: "cluster.go:117: [apiclient] All control plane components are healthy after 7.004519 seconds"
    L43: "cluster.go:117: [upload-config] Storing the configuration used in ConfigMap __kubeadm-config__ in the __kube-system__ Namespace"
    L44: "cluster.go:117: [kubelet] Creating a ConfigMap __kubelet-config__ in namespace kube-system with the configuration for the kubelets in the cluster"
    L45: "cluster.go:117: [upload-certs] Skipping phase. Please see --upload-certs"
    L46: "cluster.go:117: [mark-control-plane] Marking the node localhost as control-plane by adding the labels: [node-role.kubernetes.io/control-plane node.kubernetes.io/exclude-from-external-load-balancers]"
    L47: "cluster.go:117: [mark-control-plane] Marking the node localhost as control-plane by adding the taints [node-role.kubernetes.io/master:NoSchedule node-role.kubernetes.io/control-plane:NoSchedule]"
    L48: "cluster.go:117: [bootstrap-token] Using token: 15l1au.bcn5f616lqlvp6lh"
    L49: "cluster.go:117: [bootstrap-token] Configuring bootstrap tokens, cluster-info ConfigMap, RBAC Roles"
    L50: "cluster.go:117: [bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to get nodes"
    L51: "cluster.go:117: [bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials"
    L52: "cluster.go:117: [bootstrap-token] Configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token"
    L53: "cluster.go:117: [bootstrap-token] Configured RBAC rules to allow certificate rotation for all node client certificates in the cluster"
    L54: "cluster.go:117: [bootstrap-token] Creating the __cluster-info__ ConfigMap in the __kube-public__ namespace"
    L55: "cluster.go:117: [kubelet-finalize] Updating __/etc/kubernetes/kubelet.conf__ to point to a rotatable kubelet client certificate and key"
    L56: "cluster.go:117: [addons] Applied essential addon: CoreDNS"
    L57: "cluster.go:117: [addons] Applied essential addon: kube-proxy"
    L58: "cluster.go:117: "
    L59: "cluster.go:117: Your Kubernetes control-plane has initialized successfully!"
    L60: "cluster.go:117: "
    L61: "cluster.go:117: To start using your cluster, you need to run the following as a regular user:"
    L62: "cluster.go:117: "
    L63: "cluster.go:117:   mkdir -p $HOME/.kube"
    L64: "cluster.go:117:   sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config"
    L65: "cluster.go:117:   sudo chown $(id -u):$(id -g) $HOME/.kube/config"
    L66: "cluster.go:117: "
    L67: "cluster.go:117: Alternatively, if you are the root user, you can run:"
    L68: "cluster.go:117: "
    L69: "cluster.go:117:   export KUBECONFIG=/etc/kubernetes/admin.conf"
    L70: "cluster.go:117: "
    L71: "cluster.go:117: You should now deploy a pod network to the cluster."
    L72: "cluster.go:117: Run __kubectl apply -f [podnetwork].yaml__ with one of the options listed at:"
    L73: "cluster.go:117:   https://kubernetes.io/docs/concepts/cluster-administration/addons/"
    L74: "cluster.go:117: "
    L75: "cluster.go:117: Then you can join any number of worker nodes by running the following on each as root:"
    L76: "cluster.go:117: "
    L77: "cluster.go:117: kubeadm join 10.0.0.150:6443 --token 15l1au.bcn5f616lqlvp6lh _"
    L78: "cluster.go:117:  --discovery-token-ca-cert-hash sha256:7b14e87e8064167eb4f0b915b3a444240dcafbb597e96878abf653c485f20e98 "
    L79: "cluster.go:117: namespace/tigera-operator created"
    L80: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/bgpconfigurations.crd.projectcalico.org created"
    L81: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/bgppeers.crd.projectcalico.org created"
    L82: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/blockaffinities.crd.projectcalico.org created"
    L83: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/caliconodestatuses.crd.projectcalico.org created"
    L84: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/clusterinformations.crd.projectcalico.org created"
    L85: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/felixconfigurations.crd.projectcalico.org created"
    L86: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/globalnetworkpolicies.crd.projectcalico.org created"
    L87: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/globalnetworksets.crd.projectcalico.org created"
    L88: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/hostendpoints.crd.projectcalico.org created"
    L89: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/ipamblocks.crd.projectcalico.org created"
    L90: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/ipamconfigs.crd.projectcalico.org created"
    L91: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/ipamhandles.crd.projectcalico.org created"
    L92: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/ippools.crd.projectcalico.org created"
    L93: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/ipreservations.crd.projectcalico.org created"
    L94: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/kubecontrollersconfigurations.crd.projectcalico.org created"
    L95: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/networkpolicies.crd.projectcalico.org created"
    L96: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/networksets.crd.projectcalico.org created"
    L97: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/apiservers.operator.tigera.io created"
    L98: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/imagesets.operator.tigera.io created"
    L99: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/installations.operator.tigera.io created"
    L100: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/tigerastatuses.operator.tigera.io created"
    L101: "cluster.go:117: serviceaccount/tigera-operator created"
    L102: "cluster.go:117: clusterrole.rbac.authorization.k8s.io/tigera-operator created"
    L103: "cluster.go:117: clusterrolebinding.rbac.authorization.k8s.io/tigera-operator created"
    L104: "cluster.go:117: deployment.apps/tigera-operator created"
    L105: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/installations.operator.tigera.io condition met"
    L106: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/apiservers.operator.tigera.io condition met"
    L107: "cluster.go:117: installation.operator.tigera.io/default created"
    L108: "cluster.go:117: apiserver.operator.tigera.io/default created"
    L109: "cluster.go:117: Created symlink /etc/systemd/system/multi-user.target.wants/kubelet.service ??? /etc/systemd/system/kubelet.service."
    L110: "--- FAIL: kubeadm.v1.24.14.calico.base/nginx_deployment (93.59s)"
    L111: "kubeadm.go:313: nginx is not deployed: ready replicas should be equal to 1: null_"
    L112: " "

not ok kubeadm.v1.24.14.calico.cgroupv1.base 🟢 Succeeded: qemu_uefi-amd64 (3) ❌ Failed: qemu_uefi-amd64 (1, 2); qemu_uefi-arm64 (1, 2, 3, 4, 5)

                Diagnostic output for qemu_uefi-arm64, run 5 
    L1: " Error: _cluster.go:117: I0912 17:54:50.591465    1601 version.go:256] remote version is much newer: v1.28.1; falling back to: stable-1.24"
    L2: "cluster.go:117: [config/images] Pulled registry.k8s.io/kube-apiserver:v1.24.17"
    L3: "cluster.go:117: [config/images] Pulled registry.k8s.io/kube-controller-manager:v1.24.17"
    L4: "cluster.go:117: [config/images] Pulled registry.k8s.io/kube-scheduler:v1.24.17"
    L5: "cluster.go:117: [config/images] Pulled registry.k8s.io/kube-proxy:v1.24.17"
    L6: "cluster.go:117: [config/images] Pulled registry.k8s.io/pause:3.7"
    L7: "cluster.go:117: [config/images] Pulled registry.k8s.io/etcd:3.5.6-0"
    L8: "cluster.go:117: [config/images] Pulled registry.k8s.io/coredns/coredns:v1.8.6"
    L9: "cluster.go:117: I0912 17:55:03.592144    1767 version.go:256] remote version is much newer: v1.28.1; falling back to: stable-1.24"
    L10: "cluster.go:117: [init] Using Kubernetes version: v1.24.17"
    L11: "cluster.go:117: [preflight] Running pre-flight checks"
    L12: "cluster.go:117: [preflight] Pulling images required for setting up a Kubernetes cluster"
    L13: "cluster.go:117: [preflight] This might take a minute or two, depending on the speed of your internet connection"
    L14: "cluster.go:117: [preflight] You can also perform this action in beforehand using _kubeadm config images pull_"
    L15: "cluster.go:117: [certs] Using certificateDir folder __/etc/kubernetes/pki__"
    L16: "cluster.go:117: [certs] Generating __ca__ certificate and key"
    L17: "cluster.go:117: [certs] Generating __apiserver__ certificate and key"
    L18: "cluster.go:117: [certs] apiserver serving cert is signed for DNS names [kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local localhost] and IPs [10.96.0.1 10.0.0.3?]"
    L19: "cluster.go:117: [certs] Generating __apiserver-kubelet-client__ certificate and key"
    L20: "cluster.go:117: [certs] Generating __front-proxy-ca__ certificate and key"
    L21: "cluster.go:117: [certs] Generating __front-proxy-client__ certificate and key"
    L22: "cluster.go:117: [certs] External etcd mode: Skipping etcd/ca certificate authority generation"
    L23: "cluster.go:117: [certs] External etcd mode: Skipping etcd/server certificate generation"
    L24: "cluster.go:117: [certs] External etcd mode: Skipping etcd/peer certificate generation"
    L25: "cluster.go:117: [certs] External etcd mode: Skipping etcd/healthcheck-client certificate generation"
    L26: "cluster.go:117: [certs] External etcd mode: Skipping apiserver-etcd-client certificate generation"
    L27: "cluster.go:117: [certs] Generating __sa__ key and public key"
    L28: "cluster.go:117: [kubeconfig] Using kubeconfig folder __/etc/kubernetes__"
    L29: "cluster.go:117: [kubeconfig] Writing __admin.conf__ kubeconfig file"
    L30: "cluster.go:117: [kubeconfig] Writing __kubelet.conf__ kubeconfig file"
    L31: "cluster.go:117: [kubeconfig] Writing __controller-manager.conf__ kubeconfig file"
    L32: "cluster.go:117: [kubeconfig] Writing __scheduler.conf__ kubeconfig file"
    L33: "cluster.go:117: [kubelet-start] Writing kubelet environment file with flags to file __/var/lib/kubelet/kubeadm-flags.env__"
    L34: "cluster.go:117: [kubelet-start] Writing kubelet configuration to file __/var/lib/kubelet/config.yaml__"
    L35: "cluster.go:117: [kubelet-start] Starting the kubelet"
    L36: "cluster.go:117: [control-plane] Using manifest folder __/etc/kubernetes/manifests__"
    L37: "cluster.go:117: [control-plane] Creating static Pod manifest for __kube-apiserver__"
    L38: "cluster.go:117: [control-plane] Creating static Pod manifest for __kube-controller-manager__"
    L39: "cluster.go:117: [control-plane] Creating static Pod manifest for __kube-scheduler__"
    L40: "cluster.go:117: [wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory __/etc/kubernetes/manifests__. This can take up to 30m0s"
    L41: "cluster.go:117: [apiclient] All control plane components are healthy after 6.505617 seconds"
    L42: "cluster.go:117: [upload-config] Storing the configuration used in ConfigMap __kubeadm-config__ in the __kube-system__ Namespace"
    L43: "cluster.go:117: [kubelet] Creating a ConfigMap __kubelet-config__ in namespace kube-system with the configuration for the kubelets in the cluster"
    L44: "cluster.go:117: [upload-certs] Skipping phase. Please see --upload-certs"
    L45: "cluster.go:117: [mark-control-plane] Marking the node localhost as control-plane by adding the labels: [node-role.kubernetes.io/control-plane node.kubernetes.io/exclude-from-external-load-balancers]"
    L46: "cluster.go:117: [mark-control-plane] Marking the node localhost as control-plane by adding the taints [node-role.kubernetes.io/master:NoSchedule node-role.kubernetes.io/control-plane:NoSchedule]"
    L47: "cluster.go:117: [bootstrap-token] Using token: 2azgat.fhh4a1bsbt0mnfog"
    L48: "cluster.go:117: [bootstrap-token] Configuring bootstrap tokens, cluster-info ConfigMap, RBAC Roles"
    L49: "cluster.go:117: [bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to get nodes"
    L50: "cluster.go:117: [bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials"
    L51: "cluster.go:117: [bootstrap-token] Configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token"
    L52: "cluster.go:117: [bootstrap-token] Configured RBAC rules to allow certificate rotation for all node client certificates in the cluster"
    L53: "cluster.go:117: [bootstrap-token] Creating the __cluster-info__ ConfigMap in the __kube-public__ namespace"
    L54: "cluster.go:117: [kubelet-finalize] Updating __/etc/kubernetes/kubelet.conf__ to point to a rotatable kubelet client certificate and key"
    L55: "cluster.go:117: [addons] Applied essential addon: CoreDNS"
    L56: "cluster.go:117: [addons] Applied essential addon: kube-proxy"
    L57: "cluster.go:117: "
    L58: "cluster.go:117: Your Kubernetes control-plane has initialized successfully!"
    L59: "cluster.go:117: "
    L60: "cluster.go:117: To start using your cluster, you need to run the following as a regular user:"
    L61: "cluster.go:117: "
    L62: "cluster.go:117:   mkdir -p $HOME/.kube"
    L63: "cluster.go:117:   sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config"
    L64: "cluster.go:117:   sudo chown $(id -u):$(id -g) $HOME/.kube/config"
    L65: "cluster.go:117: "
    L66: "cluster.go:117: Alternatively, if you are the root user, you can run:"
    L67: "cluster.go:117: "
    L68: "cluster.go:117:   export KUBECONFIG=/etc/kubernetes/admin.conf"
    L69: "cluster.go:117: "
    L70: "cluster.go:117: You should now deploy a pod network to the cluster."
    L71: "cluster.go:117: Run __kubectl apply -f [podnetwork].yaml__ with one of the options listed at:"
    L72: "cluster.go:117:   https://kubernetes.io/docs/concepts/cluster-administration/addons/"
    L73: "cluster.go:117: "
    L74: "cluster.go:117: Then you can join any number of worker nodes by running the following on each as root:"
    L75: "cluster.go:117: "
    L76: "cluster.go:117: kubeadm join 10.0.0.3:6443 --token 2azgat.fhh4a1bsbt0mnfog _"
    L77: "cluster.go:117:  --discovery-token-ca-cert-hash sha256:f324ec61b5a6a8852ef7f26c1b7072984de52af0e26aaca843560b504c23dc9d "
    L78: "cluster.go:117: namespace/tigera-operator created"
    L79: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/bgpconfigurations.crd.projectcalico.org created"
    L80: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/bgppeers.crd.projectcalico.org created"
    L81: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/blockaffinities.crd.projectcalico.org created"
    L82: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/caliconodestatuses.crd.projectcalico.org created"
    L83: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/clusterinformations.crd.projectcalico.org created"
    L84: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/felixconfigurations.crd.projectcalico.org created"
    L85: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/globalnetworkpolicies.crd.projectcalico.org created"
    L86: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/globalnetworksets.crd.projectcalico.org created"
    L87: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/hostendpoints.crd.projectcalico.org created"
    L88: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/ipamblocks.crd.projectcalico.org created"
    L89: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/ipamconfigs.crd.projectcalico.org created"
    L90: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/ipamhandles.crd.projectcalico.org created"
    L91: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/ippools.crd.projectcalico.org created"
    L92: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/ipreservations.crd.projectcalico.org created"
    L93: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/kubecontrollersconfigurations.crd.projectcalico.org created"
    L94: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/networkpolicies.crd.projectcalico.org created"
    L95: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/networksets.crd.projectcalico.org created"
    L96: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/apiservers.operator.tigera.io created"
    L97: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/imagesets.operator.tigera.io created"
    L98: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/installations.operator.tigera.io created"
    L99: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/tigerastatuses.operator.tigera.io created"
    L100: "cluster.go:117: serviceaccount/tigera-operator created"
    L101: "cluster.go:117: clusterrole.rbac.authorization.k8s.io/tigera-operator created"
    L102: "cluster.go:117: clusterrolebinding.rbac.authorization.k8s.io/tigera-operator created"
    L103: "cluster.go:117: deployment.apps/tigera-operator created"
    L104: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/installations.operator.tigera.io condition met"
    L105: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/apiservers.operator.tigera.io condition met"
    L106: "cluster.go:117: installation.operator.tigera.io/default created"
    L107: "cluster.go:117: apiserver.operator.tigera.io/default created"
    L108: "cluster.go:117: Created symlink /etc/systemd/system/multi-user.target.wants/kubelet.service ??? /etc/systemd/system/kubelet.service."
    L109: "--- FAIL: kubeadm.v1.24.14.calico.cgroupv1.base/nginx_deployment (93.65s)"
    L110: "kubeadm.go:313: nginx is not deployed: ready replicas should be equal to 1: null_"
    L111: " "
                Diagnostic output for qemu_uefi-arm64, run 4 
    L1: " Error: _cluster.go:117: I0912 17:50:56.121783    1604 version.go:256] remote version is much newer: v1.28.1; falling back to: stable-1.24"
    L2: "cluster.go:117: [config/images] Pulled registry.k8s.io/kube-apiserver:v1.24.17"
    L3: "cluster.go:117: [config/images] Pulled registry.k8s.io/kube-controller-manager:v1.24.17"
    L4: "cluster.go:117: [config/images] Pulled registry.k8s.io/kube-scheduler:v1.24.17"
    L5: "cluster.go:117: [config/images] Pulled registry.k8s.io/kube-proxy:v1.24.17"
    L6: "cluster.go:117: [config/images] Pulled registry.k8s.io/pause:3.7"
    L7: "cluster.go:117: [config/images] Pulled registry.k8s.io/etcd:3.5.6-0"
    L8: "cluster.go:117: [config/images] Pulled registry.k8s.io/coredns/coredns:v1.8.6"
    L9: "cluster.go:117: I0912 17:51:07.340717    1771 version.go:256] remote version is much newer: v1.28.1; falling back to: stable-1.24"
    L10: "cluster.go:117: [init] Using Kubernetes version: v1.24.17"
    L11: "cluster.go:117: [preflight] Running pre-flight checks"
    L12: "cluster.go:117: [preflight] Pulling images required for setting up a Kubernetes cluster"
    L13: "cluster.go:117: [preflight] This might take a minute or two, depending on the speed of your internet connection"
    L14: "cluster.go:117: [preflight] You can also perform this action in beforehand using _kubeadm config images pull_"
    L15: "cluster.go:117: [certs] Using certificateDir folder __/etc/kubernetes/pki__"
    L16: "cluster.go:117: [certs] Generating __ca__ certificate and key"
    L17: "cluster.go:117: [certs] Generating __apiserver__ certificate and key"
    L18: "cluster.go:117: [certs] apiserver serving cert is signed for DNS names [kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local localhost] and IPs [10.96.0.1 10.0.0.4?]"
    L19: "cluster.go:117: [certs] Generating __apiserver-kubelet-client__ certificate and key"
    L20: "cluster.go:117: [certs] Generating __front-proxy-ca__ certificate and key"
    L21: "cluster.go:117: [certs] Generating __front-proxy-client__ certificate and key"
    L22: "cluster.go:117: [certs] External etcd mode: Skipping etcd/ca certificate authority generation"
    L23: "cluster.go:117: [certs] External etcd mode: Skipping etcd/server certificate generation"
    L24: "cluster.go:117: [certs] External etcd mode: Skipping etcd/peer certificate generation"
    L25: "cluster.go:117: [certs] External etcd mode: Skipping etcd/healthcheck-client certificate generation"
    L26: "cluster.go:117: [certs] External etcd mode: Skipping apiserver-etcd-client certificate generation"
    L27: "cluster.go:117: [certs] Generating __sa__ key and public key"
    L28: "cluster.go:117: [kubeconfig] Using kubeconfig folder __/etc/kubernetes__"
    L29: "cluster.go:117: [kubeconfig] Writing __admin.conf__ kubeconfig file"
    L30: "cluster.go:117: [kubeconfig] Writing __kubelet.conf__ kubeconfig file"
    L31: "cluster.go:117: [kubeconfig] Writing __controller-manager.conf__ kubeconfig file"
    L32: "cluster.go:117: [kubeconfig] Writing __scheduler.conf__ kubeconfig file"
    L33: "cluster.go:117: [kubelet-start] Writing kubelet environment file with flags to file __/var/lib/kubelet/kubeadm-flags.env__"
    L34: "cluster.go:117: [kubelet-start] Writing kubelet configuration to file __/var/lib/kubelet/config.yaml__"
    L35: "cluster.go:117: [kubelet-start] Starting the kubelet"
    L36: "cluster.go:117: [control-plane] Using manifest folder __/etc/kubernetes/manifests__"
    L37: "cluster.go:117: [control-plane] Creating static Pod manifest for __kube-apiserver__"
    L38: "cluster.go:117: [control-plane] Creating static Pod manifest for __kube-controller-manager__"
    L39: "cluster.go:117: [control-plane] Creating static Pod manifest for __kube-scheduler__"
    L40: "cluster.go:117: [wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory __/etc/kubernetes/manifests__. This can take up to 30m0s"
    L41: "cluster.go:117: [apiclient] All control plane components are healthy after 7.004859 seconds"
    L42: "cluster.go:117: [upload-config] Storing the configuration used in ConfigMap __kubeadm-config__ in the __kube-system__ Namespace"
    L43: "cluster.go:117: [kubelet] Creating a ConfigMap __kubelet-config__ in namespace kube-system with the configuration for the kubelets in the cluster"
    L44: "cluster.go:117: [upload-certs] Skipping phase. Please see --upload-certs"
    L45: "cluster.go:117: [mark-control-plane] Marking the node localhost as control-plane by adding the labels: [node-role.kubernetes.io/control-plane node.kubernetes.io/exclude-from-external-load-balancers]"
    L46: "cluster.go:117: [mark-control-plane] Marking the node localhost as control-plane by adding the taints [node-role.kubernetes.io/master:NoSchedule node-role.kubernetes.io/control-plane:NoSchedule]"
    L47: "cluster.go:117: [bootstrap-token] Using token: gvmddf.qlsi5s018ztp8tsp"
    L48: "cluster.go:117: [bootstrap-token] Configuring bootstrap tokens, cluster-info ConfigMap, RBAC Roles"
    L49: "cluster.go:117: [bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to get nodes"
    L50: "cluster.go:117: [bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials"
    L51: "cluster.go:117: [bootstrap-token] Configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token"
    L52: "cluster.go:117: [bootstrap-token] Configured RBAC rules to allow certificate rotation for all node client certificates in the cluster"
    L53: "cluster.go:117: [bootstrap-token] Creating the __cluster-info__ ConfigMap in the __kube-public__ namespace"
    L54: "cluster.go:117: [kubelet-finalize] Updating __/etc/kubernetes/kubelet.conf__ to point to a rotatable kubelet client certificate and key"
    L55: "cluster.go:117: [addons] Applied essential addon: CoreDNS"
    L56: "cluster.go:117: [addons] Applied essential addon: kube-proxy"
    L57: "cluster.go:117: "
    L58: "cluster.go:117: Your Kubernetes control-plane has initialized successfully!"
    L59: "cluster.go:117: "
    L60: "cluster.go:117: To start using your cluster, you need to run the following as a regular user:"
    L61: "cluster.go:117: "
    L62: "cluster.go:117:   mkdir -p $HOME/.kube"
    L63: "cluster.go:117:   sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config"
    L64: "cluster.go:117:   sudo chown $(id -u):$(id -g) $HOME/.kube/config"
    L65: "cluster.go:117: "
    L66: "cluster.go:117: Alternatively, if you are the root user, you can run:"
    L67: "cluster.go:117: "
    L68: "cluster.go:117:   export KUBECONFIG=/etc/kubernetes/admin.conf"
    L69: "cluster.go:117: "
    L70: "cluster.go:117: You should now deploy a pod network to the cluster."
    L71: "cluster.go:117: Run __kubectl apply -f [podnetwork].yaml__ with one of the options listed at:"
    L72: "cluster.go:117:   https://kubernetes.io/docs/concepts/cluster-administration/addons/"
    L73: "cluster.go:117: "
    L74: "cluster.go:117: Then you can join any number of worker nodes by running the following on each as root:"
    L75: "cluster.go:117: "
    L76: "cluster.go:117: kubeadm join 10.0.0.4:6443 --token gvmddf.qlsi5s018ztp8tsp _"
    L77: "cluster.go:117:  --discovery-token-ca-cert-hash sha256:78517eb67bfdedc6b40aa10dc4f71f24c4483775a61f6896b61522d6f83a7e32 "
    L78: "cluster.go:117: namespace/tigera-operator created"
    L79: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/bgpconfigurations.crd.projectcalico.org created"
    L80: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/bgppeers.crd.projectcalico.org created"
    L81: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/blockaffinities.crd.projectcalico.org created"
    L82: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/caliconodestatuses.crd.projectcalico.org created"
    L83: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/clusterinformations.crd.projectcalico.org created"
    L84: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/felixconfigurations.crd.projectcalico.org created"
    L85: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/globalnetworkpolicies.crd.projectcalico.org created"
    L86: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/globalnetworksets.crd.projectcalico.org created"
    L87: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/hostendpoints.crd.projectcalico.org created"
    L88: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/ipamblocks.crd.projectcalico.org created"
    L89: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/ipamconfigs.crd.projectcalico.org created"
    L90: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/ipamhandles.crd.projectcalico.org created"
    L91: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/ippools.crd.projectcalico.org created"
    L92: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/ipreservations.crd.projectcalico.org created"
    L93: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/kubecontrollersconfigurations.crd.projectcalico.org created"
    L94: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/networkpolicies.crd.projectcalico.org created"
    L95: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/networksets.crd.projectcalico.org created"
    L96: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/apiservers.operator.tigera.io created"
    L97: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/imagesets.operator.tigera.io created"
    L98: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/installations.operator.tigera.io created"
    L99: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/tigerastatuses.operator.tigera.io created"
    L100: "cluster.go:117: serviceaccount/tigera-operator created"
    L101: "cluster.go:117: clusterrole.rbac.authorization.k8s.io/tigera-operator created"
    L102: "cluster.go:117: clusterrolebinding.rbac.authorization.k8s.io/tigera-operator created"
    L103: "cluster.go:117: deployment.apps/tigera-operator created"
    L104: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/installations.operator.tigera.io condition met"
    L105: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/apiservers.operator.tigera.io condition met"
    L106: "cluster.go:117: installation.operator.tigera.io/default created"
    L107: "cluster.go:117: apiserver.operator.tigera.io/default created"
    L108: "cluster.go:117: Created symlink /etc/systemd/system/multi-user.target.wants/kubelet.service ??? /etc/systemd/system/kubelet.service."
    L109: "--- FAIL: kubeadm.v1.24.14.calico.cgroupv1.base/nginx_deployment (93.65s)"
    L110: "kubeadm.go:313: nginx is not deployed: ready replicas should be equal to 1: null_"
    L111: " "
                Diagnostic output for qemu_uefi-arm64, run 3 
    L1: "  "
    L2: " Error: _cluster.go:117: I0912 17:46:47.515098    1626 version.go:256] remote version is much newer: v1.28.1; falling back to: stable-1.24"
    L3: "cluster.go:117: [config/images] Pulled registry.k8s.io/kube-apiserver:v1.24.17"
    L4: "cluster.go:117: [config/images] Pulled registry.k8s.io/kube-controller-manager:v1.24.17"
    L5: "cluster.go:117: [config/images] Pulled registry.k8s.io/kube-scheduler:v1.24.17"
    L6: "cluster.go:117: [config/images] Pulled registry.k8s.io/kube-proxy:v1.24.17"
    L7: "cluster.go:117: [config/images] Pulled registry.k8s.io/pause:3.7"
    L8: "cluster.go:117: [config/images] Pulled registry.k8s.io/etcd:3.5.6-0"
    L9: "cluster.go:117: [config/images] Pulled registry.k8s.io/coredns/coredns:v1.8.6"
    L10: "cluster.go:117: I0912 17:46:59.866138    1793 version.go:256] remote version is much newer: v1.28.1; falling back to: stable-1.24"
    L11: "cluster.go:117: [init] Using Kubernetes version: v1.24.17"
    L12: "cluster.go:117: [preflight] Running pre-flight checks"
    L13: "cluster.go:117: [preflight] Pulling images required for setting up a Kubernetes cluster"
    L14: "cluster.go:117: [preflight] This might take a minute or two, depending on the speed of your internet connection"
    L15: "cluster.go:117: [preflight] You can also perform this action in beforehand using _kubeadm config images pull_"
    L16: "cluster.go:117: [certs] Using certificateDir folder __/etc/kubernetes/pki__"
    L17: "cluster.go:117: [certs] Generating __ca__ certificate and key"
    L18: "cluster.go:117: [certs] Generating __apiserver__ certificate and key"
    L19: "cluster.go:117: [certs] apiserver serving cert is signed for DNS names [kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local localhost] and IPs [10.96.0.1 10.0.0.4?]"
    L20: "cluster.go:117: [certs] Generating __apiserver-kubelet-client__ certificate and key"
    L21: "cluster.go:117: [certs] Generating __front-proxy-ca__ certificate and key"
    L22: "cluster.go:117: [certs] Generating __front-proxy-client__ certificate and key"
    L23: "cluster.go:117: [certs] External etcd mode: Skipping etcd/ca certificate authority generation"
    L24: "cluster.go:117: [certs] External etcd mode: Skipping etcd/server certificate generation"
    L25: "cluster.go:117: [certs] External etcd mode: Skipping etcd/peer certificate generation"
    L26: "cluster.go:117: [certs] External etcd mode: Skipping etcd/healthcheck-client certificate generation"
    L27: "cluster.go:117: [certs] External etcd mode: Skipping apiserver-etcd-client certificate generation"
    L28: "cluster.go:117: [certs] Generating __sa__ key and public key"
    L29: "cluster.go:117: [kubeconfig] Using kubeconfig folder __/etc/kubernetes__"
    L30: "cluster.go:117: [kubeconfig] Writing __admin.conf__ kubeconfig file"
    L31: "cluster.go:117: [kubeconfig] Writing __kubelet.conf__ kubeconfig file"
    L32: "cluster.go:117: [kubeconfig] Writing __controller-manager.conf__ kubeconfig file"
    L33: "cluster.go:117: [kubeconfig] Writing __scheduler.conf__ kubeconfig file"
    L34: "cluster.go:117: [kubelet-start] Writing kubelet environment file with flags to file __/var/lib/kubelet/kubeadm-flags.env__"
    L35: "cluster.go:117: [kubelet-start] Writing kubelet configuration to file __/var/lib/kubelet/config.yaml__"
    L36: "cluster.go:117: [kubelet-start] Starting the kubelet"
    L37: "cluster.go:117: [control-plane] Using manifest folder __/etc/kubernetes/manifests__"
    L38: "cluster.go:117: [control-plane] Creating static Pod manifest for __kube-apiserver__"
    L39: "cluster.go:117: [control-plane] Creating static Pod manifest for __kube-controller-manager__"
    L40: "cluster.go:117: [control-plane] Creating static Pod manifest for __kube-scheduler__"
    L41: "cluster.go:117: [wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory __/etc/kubernetes/manifests__. This can take up to 30m0s"
    L42: "cluster.go:117: [apiclient] All control plane components are healthy after 8.005214 seconds"
    L43: "cluster.go:117: [upload-config] Storing the configuration used in ConfigMap __kubeadm-config__ in the __kube-system__ Namespace"
    L44: "cluster.go:117: [kubelet] Creating a ConfigMap __kubelet-config__ in namespace kube-system with the configuration for the kubelets in the cluster"
    L45: "cluster.go:117: [upload-certs] Skipping phase. Please see --upload-certs"
    L46: "cluster.go:117: [mark-control-plane] Marking the node localhost as control-plane by adding the labels: [node-role.kubernetes.io/control-plane node.kubernetes.io/exclude-from-external-load-balancers]"
    L47: "cluster.go:117: [mark-control-plane] Marking the node localhost as control-plane by adding the taints [node-role.kubernetes.io/master:NoSchedule node-role.kubernetes.io/control-plane:NoSchedule]"
    L48: "cluster.go:117: [bootstrap-token] Using token: urnepj.tuo7u1e9eycdhs7j"
    L49: "cluster.go:117: [bootstrap-token] Configuring bootstrap tokens, cluster-info ConfigMap, RBAC Roles"
    L50: "cluster.go:117: [bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to get nodes"
    L51: "cluster.go:117: [bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials"
    L52: "cluster.go:117: [bootstrap-token] Configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token"
    L53: "cluster.go:117: [bootstrap-token] Configured RBAC rules to allow certificate rotation for all node client certificates in the cluster"
    L54: "cluster.go:117: [bootstrap-token] Creating the __cluster-info__ ConfigMap in the __kube-public__ namespace"
    L55: "cluster.go:117: [kubelet-finalize] Updating __/etc/kubernetes/kubelet.conf__ to point to a rotatable kubelet client certificate and key"
    L56: "cluster.go:117: [addons] Applied essential addon: CoreDNS"
    L57: "cluster.go:117: [addons] Applied essential addon: kube-proxy"
    L58: "cluster.go:117: "
    L59: "cluster.go:117: Your Kubernetes control-plane has initialized successfully!"
    L60: "cluster.go:117: "
    L61: "cluster.go:117: To start using your cluster, you need to run the following as a regular user:"
    L62: "cluster.go:117: "
    L63: "cluster.go:117:   mkdir -p $HOME/.kube"
    L64: "cluster.go:117:   sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config"
    L65: "cluster.go:117:   sudo chown $(id -u):$(id -g) $HOME/.kube/config"
    L66: "cluster.go:117: "
    L67: "cluster.go:117: Alternatively, if you are the root user, you can run:"
    L68: "cluster.go:117: "
    L69: "cluster.go:117:   export KUBECONFIG=/etc/kubernetes/admin.conf"
    L70: "cluster.go:117: "
    L71: "cluster.go:117: You should now deploy a pod network to the cluster."
    L72: "cluster.go:117: Run __kubectl apply -f [podnetwork].yaml__ with one of the options listed at:"
    L73: "cluster.go:117:   https://kubernetes.io/docs/concepts/cluster-administration/addons/"
    L74: "cluster.go:117: "
    L75: "cluster.go:117: Then you can join any number of worker nodes by running the following on each as root:"
    L76: "cluster.go:117: "
    L77: "cluster.go:117: kubeadm join 10.0.0.4:6443 --token urnepj.tuo7u1e9eycdhs7j _"
    L78: "cluster.go:117:  --discovery-token-ca-cert-hash sha256:5ec94ac51400f2db878543841400e9b87a1822f8fb075fa17a3578e98741ee8f "
    L79: "cluster.go:117: namespace/tigera-operator created"
    L80: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/bgpconfigurations.crd.projectcalico.org created"
    L81: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/bgppeers.crd.projectcalico.org created"
    L82: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/blockaffinities.crd.projectcalico.org created"
    L83: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/caliconodestatuses.crd.projectcalico.org created"
    L84: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/clusterinformations.crd.projectcalico.org created"
    L85: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/felixconfigurations.crd.projectcalico.org created"
    L86: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/globalnetworkpolicies.crd.projectcalico.org created"
    L87: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/globalnetworksets.crd.projectcalico.org created"
    L88: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/hostendpoints.crd.projectcalico.org created"
    L89: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/ipamblocks.crd.projectcalico.org created"
    L90: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/ipamconfigs.crd.projectcalico.org created"
    L91: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/ipamhandles.crd.projectcalico.org created"
    L92: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/ippools.crd.projectcalico.org created"
    L93: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/ipreservations.crd.projectcalico.org created"
    L94: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/kubecontrollersconfigurations.crd.projectcalico.org created"
    L95: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/networkpolicies.crd.projectcalico.org created"
    L96: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/networksets.crd.projectcalico.org created"
    L97: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/apiservers.operator.tigera.io created"
    L98: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/imagesets.operator.tigera.io created"
    L99: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/installations.operator.tigera.io created"
    L100: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/tigerastatuses.operator.tigera.io created"
    L101: "cluster.go:117: serviceaccount/tigera-operator created"
    L102: "cluster.go:117: clusterrole.rbac.authorization.k8s.io/tigera-operator created"
    L103: "cluster.go:117: clusterrolebinding.rbac.authorization.k8s.io/tigera-operator created"
    L104: "cluster.go:117: deployment.apps/tigera-operator created"
    L105: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/installations.operator.tigera.io condition met"
    L106: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/apiservers.operator.tigera.io condition met"
    L107: "cluster.go:117: installation.operator.tigera.io/default created"
    L108: "cluster.go:117: apiserver.operator.tigera.io/default created"
    L109: "cluster.go:117: Created symlink /etc/systemd/system/multi-user.target.wants/kubelet.service ??? /etc/systemd/system/kubelet.service."
    L110: "--- FAIL: kubeadm.v1.24.14.calico.cgroupv1.base/nginx_deployment (93.75s)"
    L111: "kubeadm.go:313: nginx is not deployed: ready replicas should be equal to 1: null_"
    L112: " "
                Diagnostic output for qemu_uefi-arm64, run 2 
    L1: " Error: _cluster.go:117: I0912 17:37:57.067779    1671 version.go:256] remote version is much newer: v1.28.1; falling back to: stable-1.24"
    L2: "cluster.go:117: [config/images] Pulled registry.k8s.io/kube-apiserver:v1.24.17"
    L3: "cluster.go:117: [config/images] Pulled registry.k8s.io/kube-controller-manager:v1.24.17"
    L4: "cluster.go:117: [config/images] Pulled registry.k8s.io/kube-scheduler:v1.24.17"
    L5: "cluster.go:117: [config/images] Pulled registry.k8s.io/kube-proxy:v1.24.17"
    L6: "cluster.go:117: [config/images] Pulled registry.k8s.io/pause:3.7"
    L7: "cluster.go:117: [config/images] Pulled registry.k8s.io/etcd:3.5.6-0"
    L8: "cluster.go:117: [config/images] Pulled registry.k8s.io/coredns/coredns:v1.8.6"
    L9: "cluster.go:117: I0912 17:38:07.150823    1828 version.go:256] remote version is much newer: v1.28.1; falling back to: stable-1.24"
    L10: "cluster.go:117: [init] Using Kubernetes version: v1.24.17"
    L11: "cluster.go:117: [preflight] Running pre-flight checks"
    L12: "cluster.go:117: [preflight] Pulling images required for setting up a Kubernetes cluster"
    L13: "cluster.go:117: [preflight] This might take a minute or two, depending on the speed of your internet connection"
    L14: "cluster.go:117: [preflight] You can also perform this action in beforehand using _kubeadm config images pull_"
    L15: "cluster.go:117: [certs] Using certificateDir folder __/etc/kubernetes/pki__"
    L16: "cluster.go:117: [certs] Generating __ca__ certificate and key"
    L17: "cluster.go:117: [certs] Generating __apiserver__ certificate and key"
    L18: "cluster.go:117: [certs] apiserver serving cert is signed for DNS names [kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local localhost] and IPs [10.96.0.1 10.0.0.6?]"
    L19: "cluster.go:117: [certs] Generating __apiserver-kubelet-client__ certificate and key"
    L20: "cluster.go:117: [certs] Generating __front-proxy-ca__ certificate and key"
    L21: "cluster.go:117: [certs] Generating __front-proxy-client__ certificate and key"
    L22: "cluster.go:117: [certs] External etcd mode: Skipping etcd/ca certificate authority generation"
    L23: "cluster.go:117: [certs] External etcd mode: Skipping etcd/server certificate generation"
    L24: "cluster.go:117: [certs] External etcd mode: Skipping etcd/peer certificate generation"
    L25: "cluster.go:117: [certs] External etcd mode: Skipping etcd/healthcheck-client certificate generation"
    L26: "cluster.go:117: [certs] External etcd mode: Skipping apiserver-etcd-client certificate generation"
    L27: "cluster.go:117: [certs] Generating __sa__ key and public key"
    L28: "cluster.go:117: [kubeconfig] Using kubeconfig folder __/etc/kubernetes__"
    L29: "cluster.go:117: [kubeconfig] Writing __admin.conf__ kubeconfig file"
    L30: "cluster.go:117: [kubeconfig] Writing __kubelet.conf__ kubeconfig file"
    L31: "cluster.go:117: [kubeconfig] Writing __controller-manager.conf__ kubeconfig file"
    L32: "cluster.go:117: [kubeconfig] Writing __scheduler.conf__ kubeconfig file"
    L33: "cluster.go:117: [kubelet-start] Writing kubelet environment file with flags to file __/var/lib/kubelet/kubeadm-flags.env__"
    L34: "cluster.go:117: [kubelet-start] Writing kubelet configuration to file __/var/lib/kubelet/config.yaml__"
    L35: "cluster.go:117: [kubelet-start] Starting the kubelet"
    L36: "cluster.go:117: [control-plane] Using manifest folder __/etc/kubernetes/manifests__"
    L37: "cluster.go:117: [control-plane] Creating static Pod manifest for __kube-apiserver__"
    L38: "cluster.go:117: [control-plane] Creating static Pod manifest for __kube-controller-manager__"
    L39: "cluster.go:117: [control-plane] Creating static Pod manifest for __kube-scheduler__"
    L40: "cluster.go:117: [wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory __/etc/kubernetes/manifests__. This can take up to 30m0s"
    L41: "cluster.go:117: [apiclient] All control plane components are healthy after 5.002095 seconds"
    L42: "cluster.go:117: [upload-config] Storing the configuration used in ConfigMap __kubeadm-config__ in the __kube-system__ Namespace"
    L43: "cluster.go:117: [kubelet] Creating a ConfigMap __kubelet-config__ in namespace kube-system with the configuration for the kubelets in the cluster"
    L44: "cluster.go:117: [upload-certs] Skipping phase. Please see --upload-certs"
    L45: "cluster.go:117: [mark-control-plane] Marking the node localhost as control-plane by adding the labels: [node-role.kubernetes.io/control-plane node.kubernetes.io/exclude-from-external-load-balancers]"
    L46: "cluster.go:117: [mark-control-plane] Marking the node localhost as control-plane by adding the taints [node-role.kubernetes.io/master:NoSchedule node-role.kubernetes.io/control-plane:NoSchedule]"
    L47: "cluster.go:117: [bootstrap-token] Using token: 9r9b2z.m5wxmh4dz3tgihc5"
    L48: "cluster.go:117: [bootstrap-token] Configuring bootstrap tokens, cluster-info ConfigMap, RBAC Roles"
    L49: "cluster.go:117: [bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to get nodes"
    L50: "cluster.go:117: [bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials"
    L51: "cluster.go:117: [bootstrap-token] Configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token"
    L52: "cluster.go:117: [bootstrap-token] Configured RBAC rules to allow certificate rotation for all node client certificates in the cluster"
    L53: "cluster.go:117: [bootstrap-token] Creating the __cluster-info__ ConfigMap in the __kube-public__ namespace"
    L54: "cluster.go:117: [kubelet-finalize] Updating __/etc/kubernetes/kubelet.conf__ to point to a rotatable kubelet client certificate and key"
    L55: "cluster.go:117: [addons] Applied essential addon: CoreDNS"
    L56: "cluster.go:117: [addons] Applied essential addon: kube-proxy"
    L57: "cluster.go:117: "
    L58: "cluster.go:117: Your Kubernetes control-plane has initialized successfully!"
    L59: "cluster.go:117: "
    L60: "cluster.go:117: To start using your cluster, you need to run the following as a regular user:"
    L61: "cluster.go:117: "
    L62: "cluster.go:117:   mkdir -p $HOME/.kube"
    L63: "cluster.go:117:   sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config"
    L64: "cluster.go:117:   sudo chown $(id -u):$(id -g) $HOME/.kube/config"
    L65: "cluster.go:117: "
    L66: "cluster.go:117: Alternatively, if you are the root user, you can run:"
    L67: "cluster.go:117: "
    L68: "cluster.go:117:   export KUBECONFIG=/etc/kubernetes/admin.conf"
    L69: "cluster.go:117: "
    L70: "cluster.go:117: You should now deploy a pod network to the cluster."
    L71: "cluster.go:117: Run __kubectl apply -f [podnetwork].yaml__ with one of the options listed at:"
    L72: "cluster.go:117:   https://kubernetes.io/docs/concepts/cluster-administration/addons/"
    L73: "cluster.go:117: "
    L74: "cluster.go:117: Then you can join any number of worker nodes by running the following on each as root:"
    L75: "cluster.go:117: "
    L76: "cluster.go:117: kubeadm join 10.0.0.6:6443 --token 9r9b2z.m5wxmh4dz3tgihc5 _"
    L77: "cluster.go:117:  --discovery-token-ca-cert-hash sha256:57eff127f71cb934b8be3ab4293b9ec68ea49298ca70b129a94023e8fc5029ac "
    L78: "cluster.go:117: namespace/tigera-operator created"
    L79: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/bgpconfigurations.crd.projectcalico.org created"
    L80: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/bgppeers.crd.projectcalico.org created"
    L81: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/blockaffinities.crd.projectcalico.org created"
    L82: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/caliconodestatuses.crd.projectcalico.org created"
    L83: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/clusterinformations.crd.projectcalico.org created"
    L84: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/felixconfigurations.crd.projectcalico.org created"
    L85: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/globalnetworkpolicies.crd.projectcalico.org created"
    L86: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/globalnetworksets.crd.projectcalico.org created"
    L87: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/hostendpoints.crd.projectcalico.org created"
    L88: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/ipamblocks.crd.projectcalico.org created"
    L89: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/ipamconfigs.crd.projectcalico.org created"
    L90: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/ipamhandles.crd.projectcalico.org created"
    L91: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/ippools.crd.projectcalico.org created"
    L92: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/ipreservations.crd.projectcalico.org created"
    L93: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/kubecontrollersconfigurations.crd.projectcalico.org created"
    L94: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/networkpolicies.crd.projectcalico.org created"
    L95: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/networksets.crd.projectcalico.org created"
    L96: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/apiservers.operator.tigera.io created"
    L97: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/imagesets.operator.tigera.io created"
    L98: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/installations.operator.tigera.io created"
    L99: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/tigerastatuses.operator.tigera.io created"
    L100: "cluster.go:117: serviceaccount/tigera-operator created"
    L101: "cluster.go:117: clusterrole.rbac.authorization.k8s.io/tigera-operator created"
    L102: "cluster.go:117: clusterrolebinding.rbac.authorization.k8s.io/tigera-operator created"
    L103: "cluster.go:117: deployment.apps/tigera-operator created"
    L104: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/installations.operator.tigera.io condition met"
    L105: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/apiservers.operator.tigera.io condition met"
    L106: "cluster.go:117: installation.operator.tigera.io/default created"
    L107: "cluster.go:117: apiserver.operator.tigera.io/default created"
    L108: "cluster.go:117: Created symlink /etc/systemd/system/multi-user.target.wants/kubelet.service ??? /etc/systemd/system/kubelet.service."
    L109: "--- FAIL: kubeadm.v1.24.14.calico.cgroupv1.base/nginx_deployment (92.57s)"
    L110: "kubeadm.go:313: nginx is not deployed: ready replicas should be equal to 1: null_"
    L111: " "
    L112: " Error: _cluster.go:117: I0912 17:40:38.662589    1604 version.go:256] remote version is much newer: v1.28.1; falling back to: stable-1.24"
    L113: "cluster.go:117: [config/images] Pulled registry.k8s.io/kube-apiserver:v1.24.17"
    L114: "cluster.go:117: [config/images] Pulled registry.k8s.io/kube-controller-manager:v1.24.17"
    L115: "cluster.go:117: [config/images] Pulled registry.k8s.io/kube-scheduler:v1.24.17"
    L116: "cluster.go:117: [config/images] Pulled registry.k8s.io/kube-proxy:v1.24.17"
    L117: "cluster.go:117: [config/images] Pulled registry.k8s.io/pause:3.7"
    L118: "cluster.go:117: [config/images] Pulled registry.k8s.io/etcd:3.5.6-0"
    L119: "cluster.go:117: [config/images] Pulled registry.k8s.io/coredns/coredns:v1.8.6"
    L120: "cluster.go:117: I0912 17:40:50.204092    1773 version.go:256] remote version is much newer: v1.28.1; falling back to: stable-1.24"
    L121: "cluster.go:117: [init] Using Kubernetes version: v1.24.17"
    L122: "cluster.go:117: [preflight] Running pre-flight checks"
    L123: "cluster.go:117: [preflight] Pulling images required for setting up a Kubernetes cluster"
    L124: "cluster.go:117: [preflight] This might take a minute or two, depending on the speed of your internet connection"
    L125: "cluster.go:117: [preflight] You can also perform this action in beforehand using _kubeadm config images pull_"
    L126: "cluster.go:117: [certs] Using certificateDir folder __/etc/kubernetes/pki__"
    L127: "cluster.go:117: [certs] Generating __ca__ certificate and key"
    L128: "cluster.go:117: [certs] Generating __apiserver__ certificate and key"
    L129: "cluster.go:117: [certs] apiserver serving cert is signed for DNS names [kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local localhost] and IPs [10.96.0.1 10.0.0.9?]"
    L130: "cluster.go:117: [certs] Generating __apiserver-kubelet-client__ certificate and key"
    L131: "cluster.go:117: [certs] Generating __front-proxy-ca__ certificate and key"
    L132: "cluster.go:117: [certs] Generating __front-proxy-client__ certificate and key"
    L133: "cluster.go:117: [certs] External etcd mode: Skipping etcd/ca certificate authority generation"
    L134: "cluster.go:117: [certs] External etcd mode: Skipping etcd/server certificate generation"
    L135: "cluster.go:117: [certs] External etcd mode: Skipping etcd/peer certificate generation"
    L136: "cluster.go:117: [certs] External etcd mode: Skipping etcd/healthcheck-client certificate generation"
    L137: "cluster.go:117: [certs] External etcd mode: Skipping apiserver-etcd-client certificate generation"
    L138: "cluster.go:117: [certs] Generating __sa__ key and public key"
    L139: "cluster.go:117: [kubeconfig] Using kubeconfig folder __/etc/kubernetes__"
    L140: "cluster.go:117: [kubeconfig] Writing __admin.conf__ kubeconfig file"
    L141: "cluster.go:117: [kubeconfig] Writing __kubelet.conf__ kubeconfig file"
    L142: "cluster.go:117: [kubeconfig] Writing __controller-manager.conf__ kubeconfig file"
    L143: "cluster.go:117: [kubeconfig] Writing __scheduler.conf__ kubeconfig file"
    L144: "cluster.go:117: [kubelet-start] Writing kubelet environment file with flags to file __/var/lib/kubelet/kubeadm-flags.env__"
    L145: "cluster.go:117: [kubelet-start] Writing kubelet configuration to file __/var/lib/kubelet/config.yaml__"
    L146: "cluster.go:117: [kubelet-start] Starting the kubelet"
    L147: "cluster.go:117: [control-plane] Using manifest folder __/etc/kubernetes/manifests__"
    L148: "cluster.go:117: [control-plane] Creating static Pod manifest for __kube-apiserver__"
    L149: "cluster.go:117: [control-plane] Creating static Pod manifest for __kube-controller-manager__"
    L150: "cluster.go:117: [control-plane] Creating static Pod manifest for __kube-scheduler__"
    L151: "cluster.go:117: [wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory __/etc/kubernetes/manifests__. This can take up to 30m0s"
    L152: "cluster.go:117: [apiclient] All control plane components are healthy after 7.503559 seconds"
    L153: "cluster.go:117: [upload-config] Storing the configuration used in ConfigMap __kubeadm-config__ in the __kube-system__ Namespace"
    L154: "cluster.go:117: [kubelet] Creating a ConfigMap __kubelet-config__ in namespace kube-system with the configuration for the kubelets in the cluster"
    L155: "cluster.go:117: [upload-certs] Skipping phase. Please see --upload-certs"
    L156: "cluster.go:117: [mark-control-plane] Marking the node localhost as control-plane by adding the labels: [node-role.kubernetes.io/control-plane node.kubernetes.io/exclude-from-external-load-balancers]"
    L157: "cluster.go:117: [mark-control-plane] Marking the node localhost as control-plane by adding the taints [node-role.kubernetes.io/master:NoSchedule node-role.kubernetes.io/control-plane:NoSchedule]"
    L158: "cluster.go:117: [bootstrap-token] Using token: zc6wf9.23nki2q6s0v2ssfp"
    L159: "cluster.go:117: [bootstrap-token] Configuring bootstrap tokens, cluster-info ConfigMap, RBAC Roles"
    L160: "cluster.go:117: [bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to get nodes"
    L161: "cluster.go:117: [bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials"
    L162: "cluster.go:117: [bootstrap-token] Configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token"
    L163: "cluster.go:117: [bootstrap-token] Configured RBAC rules to allow certificate rotation for all node client certificates in the cluster"
    L164: "cluster.go:117: [bootstrap-token] Creating the __cluster-info__ ConfigMap in the __kube-public__ namespace"
    L165: "cluster.go:117: [kubelet-finalize] Updating __/etc/kubernetes/kubelet.conf__ to point to a rotatable kubelet client certificate and key"
    L166: "cluster.go:117: [addons] Applied essential addon: CoreDNS"
    L167: "cluster.go:117: [addons] Applied essential addon: kube-proxy"
    L168: "cluster.go:117: "
    L169: "cluster.go:117: Your Kubernetes control-plane has initialized successfully!"
    L170: "cluster.go:117: "
    L171: "cluster.go:117: To start using your cluster, you need to run the following as a regular user:"
    L172: "cluster.go:117: "
    L173: "cluster.go:117:   mkdir -p $HOME/.kube"
    L174: "cluster.go:117:   sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config"
    L175: "cluster.go:117:   sudo chown $(id -u):$(id -g) $HOME/.kube/config"
    L176: "cluster.go:117: "
    L177: "cluster.go:117: Alternatively, if you are the root user, you can run:"
    L178: "cluster.go:117: "
    L179: "cluster.go:117:   export KUBECONFIG=/etc/kubernetes/admin.conf"
    L180: "cluster.go:117: "
    L181: "cluster.go:117: You should now deploy a pod network to the cluster."
    L182: "cluster.go:117: Run __kubectl apply -f [podnetwork].yaml__ with one of the options listed at:"
    L183: "cluster.go:117:   https://kubernetes.io/docs/concepts/cluster-administration/addons/"
    L184: "cluster.go:117: "
    L185: "cluster.go:117: Then you can join any number of worker nodes by running the following on each as root:"
    L186: "cluster.go:117: "
    L187: "cluster.go:117: kubeadm join 10.0.0.9:6443 --token zc6wf9.23nki2q6s0v2ssfp _"
    L188: "cluster.go:117:  --discovery-token-ca-cert-hash sha256:4aa543f5c7f09c73b8aeb3717cfdbc9a8f28a04740fde43ce0ad2b9a2dff01c0 "
    L189: "cluster.go:117: namespace/tigera-operator created"
    L190: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/bgpconfigurations.crd.projectcalico.org created"
    L191: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/bgppeers.crd.projectcalico.org created"
    L192: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/blockaffinities.crd.projectcalico.org created"
    L193: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/caliconodestatuses.crd.projectcalico.org created"
    L194: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/clusterinformations.crd.projectcalico.org created"
    L195: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/felixconfigurations.crd.projectcalico.org created"
    L196: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/globalnetworkpolicies.crd.projectcalico.org created"
    L197: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/globalnetworksets.crd.projectcalico.org created"
    L198: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/hostendpoints.crd.projectcalico.org created"
    L199: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/ipamblocks.crd.projectcalico.org created"
    L200: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/ipamconfigs.crd.projectcalico.org created"
    L201: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/ipamhandles.crd.projectcalico.org created"
    L202: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/ippools.crd.projectcalico.org created"
    L203: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/ipreservations.crd.projectcalico.org created"
    L204: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/kubecontrollersconfigurations.crd.projectcalico.org created"
    L205: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/networkpolicies.crd.projectcalico.org created"
    L206: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/networksets.crd.projectcalico.org created"
    L207: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/apiservers.operator.tigera.io created"
    L208: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/imagesets.operator.tigera.io created"
    L209: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/installations.operator.tigera.io created"
    L210: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/tigerastatuses.operator.tigera.io created"
    L211: "cluster.go:117: serviceaccount/tigera-operator created"
    L212: "cluster.go:117: clusterrole.rbac.authorization.k8s.io/tigera-operator created"
    L213: "cluster.go:117: clusterrolebinding.rbac.authorization.k8s.io/tigera-operator created"
    L214: "cluster.go:117: deployment.apps/tigera-operator created"
    L215: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/installations.operator.tigera.io condition met"
    L216: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/apiservers.operator.tigera.io condition met"
    L217: "cluster.go:117: installation.operator.tigera.io/default created"
    L218: "cluster.go:117: apiserver.operator.tigera.io/default created"
    L219: "cluster.go:117: Created symlink /etc/systemd/system/multi-user.target.wants/kubelet.service ??? /etc/systemd/system/kubelet.service."
    L220: "--- FAIL: kubeadm.v1.24.14.calico.cgroupv1.base/nginx_deployment (93.66s)"
    L221: "kubeadm.go:313: nginx is not deployed: ready replicas should be equal to 1: null_"
    L222: " "
                Diagnostic output for qemu_uefi-amd64, run 2 
    L1: " Error: _cluster.go:117: I0912 17:37:57.067779    1671 version.go:256] remote version is much newer: v1.28.1; falling back to: stable-1.24"
    L2: "cluster.go:117: [config/images] Pulled registry.k8s.io/kube-apiserver:v1.24.17"
    L3: "cluster.go:117: [config/images] Pulled registry.k8s.io/kube-controller-manager:v1.24.17"
    L4: "cluster.go:117: [config/images] Pulled registry.k8s.io/kube-scheduler:v1.24.17"
    L5: "cluster.go:117: [config/images] Pulled registry.k8s.io/kube-proxy:v1.24.17"
    L6: "cluster.go:117: [config/images] Pulled registry.k8s.io/pause:3.7"
    L7: "cluster.go:117: [config/images] Pulled registry.k8s.io/etcd:3.5.6-0"
    L8: "cluster.go:117: [config/images] Pulled registry.k8s.io/coredns/coredns:v1.8.6"
    L9: "cluster.go:117: I0912 17:38:07.150823    1828 version.go:256] remote version is much newer: v1.28.1; falling back to: stable-1.24"
    L10: "cluster.go:117: [init] Using Kubernetes version: v1.24.17"
    L11: "cluster.go:117: [preflight] Running pre-flight checks"
    L12: "cluster.go:117: [preflight] Pulling images required for setting up a Kubernetes cluster"
    L13: "cluster.go:117: [preflight] This might take a minute or two, depending on the speed of your internet connection"
    L14: "cluster.go:117: [preflight] You can also perform this action in beforehand using _kubeadm config images pull_"
    L15: "cluster.go:117: [certs] Using certificateDir folder __/etc/kubernetes/pki__"
    L16: "cluster.go:117: [certs] Generating __ca__ certificate and key"
    L17: "cluster.go:117: [certs] Generating __apiserver__ certificate and key"
    L18: "cluster.go:117: [certs] apiserver serving cert is signed for DNS names [kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local localhost] and IPs [10.96.0.1 10.0.0.6?]"
    L19: "cluster.go:117: [certs] Generating __apiserver-kubelet-client__ certificate and key"
    L20: "cluster.go:117: [certs] Generating __front-proxy-ca__ certificate and key"
    L21: "cluster.go:117: [certs] Generating __front-proxy-client__ certificate and key"
    L22: "cluster.go:117: [certs] External etcd mode: Skipping etcd/ca certificate authority generation"
    L23: "cluster.go:117: [certs] External etcd mode: Skipping etcd/server certificate generation"
    L24: "cluster.go:117: [certs] External etcd mode: Skipping etcd/peer certificate generation"
    L25: "cluster.go:117: [certs] External etcd mode: Skipping etcd/healthcheck-client certificate generation"
    L26: "cluster.go:117: [certs] External etcd mode: Skipping apiserver-etcd-client certificate generation"
    L27: "cluster.go:117: [certs] Generating __sa__ key and public key"
    L28: "cluster.go:117: [kubeconfig] Using kubeconfig folder __/etc/kubernetes__"
    L29: "cluster.go:117: [kubeconfig] Writing __admin.conf__ kubeconfig file"
    L30: "cluster.go:117: [kubeconfig] Writing __kubelet.conf__ kubeconfig file"
    L31: "cluster.go:117: [kubeconfig] Writing __controller-manager.conf__ kubeconfig file"
    L32: "cluster.go:117: [kubeconfig] Writing __scheduler.conf__ kubeconfig file"
    L33: "cluster.go:117: [kubelet-start] Writing kubelet environment file with flags to file __/var/lib/kubelet/kubeadm-flags.env__"
    L34: "cluster.go:117: [kubelet-start] Writing kubelet configuration to file __/var/lib/kubelet/config.yaml__"
    L35: "cluster.go:117: [kubelet-start] Starting the kubelet"
    L36: "cluster.go:117: [control-plane] Using manifest folder __/etc/kubernetes/manifests__"
    L37: "cluster.go:117: [control-plane] Creating static Pod manifest for __kube-apiserver__"
    L38: "cluster.go:117: [control-plane] Creating static Pod manifest for __kube-controller-manager__"
    L39: "cluster.go:117: [control-plane] Creating static Pod manifest for __kube-scheduler__"
    L40: "cluster.go:117: [wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory __/etc/kubernetes/manifests__. This can take up to 30m0s"
    L41: "cluster.go:117: [apiclient] All control plane components are healthy after 5.002095 seconds"
    L42: "cluster.go:117: [upload-config] Storing the configuration used in ConfigMap __kubeadm-config__ in the __kube-system__ Namespace"
    L43: "cluster.go:117: [kubelet] Creating a ConfigMap __kubelet-config__ in namespace kube-system with the configuration for the kubelets in the cluster"
    L44: "cluster.go:117: [upload-certs] Skipping phase. Please see --upload-certs"
    L45: "cluster.go:117: [mark-control-plane] Marking the node localhost as control-plane by adding the labels: [node-role.kubernetes.io/control-plane node.kubernetes.io/exclude-from-external-load-balancers]"
    L46: "cluster.go:117: [mark-control-plane] Marking the node localhost as control-plane by adding the taints [node-role.kubernetes.io/master:NoSchedule node-role.kubernetes.io/control-plane:NoSchedule]"
    L47: "cluster.go:117: [bootstrap-token] Using token: 9r9b2z.m5wxmh4dz3tgihc5"
    L48: "cluster.go:117: [bootstrap-token] Configuring bootstrap tokens, cluster-info ConfigMap, RBAC Roles"
    L49: "cluster.go:117: [bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to get nodes"
    L50: "cluster.go:117: [bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials"
    L51: "cluster.go:117: [bootstrap-token] Configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token"
    L52: "cluster.go:117: [bootstrap-token] Configured RBAC rules to allow certificate rotation for all node client certificates in the cluster"
    L53: "cluster.go:117: [bootstrap-token] Creating the __cluster-info__ ConfigMap in the __kube-public__ namespace"
    L54: "cluster.go:117: [kubelet-finalize] Updating __/etc/kubernetes/kubelet.conf__ to point to a rotatable kubelet client certificate and key"
    L55: "cluster.go:117: [addons] Applied essential addon: CoreDNS"
    L56: "cluster.go:117: [addons] Applied essential addon: kube-proxy"
    L57: "cluster.go:117: "
    L58: "cluster.go:117: Your Kubernetes control-plane has initialized successfully!"
    L59: "cluster.go:117: "
    L60: "cluster.go:117: To start using your cluster, you need to run the following as a regular user:"
    L61: "cluster.go:117: "
    L62: "cluster.go:117:   mkdir -p $HOME/.kube"
    L63: "cluster.go:117:   sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config"
    L64: "cluster.go:117:   sudo chown $(id -u):$(id -g) $HOME/.kube/config"
    L65: "cluster.go:117: "
    L66: "cluster.go:117: Alternatively, if you are the root user, you can run:"
    L67: "cluster.go:117: "
    L68: "cluster.go:117:   export KUBECONFIG=/etc/kubernetes/admin.conf"
    L69: "cluster.go:117: "
    L70: "cluster.go:117: You should now deploy a pod network to the cluster."
    L71: "cluster.go:117: Run __kubectl apply -f [podnetwork].yaml__ with one of the options listed at:"
    L72: "cluster.go:117:   https://kubernetes.io/docs/concepts/cluster-administration/addons/"
    L73: "cluster.go:117: "
    L74: "cluster.go:117: Then you can join any number of worker nodes by running the following on each as root:"
    L75: "cluster.go:117: "
    L76: "cluster.go:117: kubeadm join 10.0.0.6:6443 --token 9r9b2z.m5wxmh4dz3tgihc5 _"
    L77: "cluster.go:117:  --discovery-token-ca-cert-hash sha256:57eff127f71cb934b8be3ab4293b9ec68ea49298ca70b129a94023e8fc5029ac "
    L78: "cluster.go:117: namespace/tigera-operator created"
    L79: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/bgpconfigurations.crd.projectcalico.org created"
    L80: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/bgppeers.crd.projectcalico.org created"
    L81: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/blockaffinities.crd.projectcalico.org created"
    L82: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/caliconodestatuses.crd.projectcalico.org created"
    L83: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/clusterinformations.crd.projectcalico.org created"
    L84: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/felixconfigurations.crd.projectcalico.org created"
    L85: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/globalnetworkpolicies.crd.projectcalico.org created"
    L86: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/globalnetworksets.crd.projectcalico.org created"
    L87: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/hostendpoints.crd.projectcalico.org created"
    L88: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/ipamblocks.crd.projectcalico.org created"
    L89: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/ipamconfigs.crd.projectcalico.org created"
    L90: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/ipamhandles.crd.projectcalico.org created"
    L91: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/ippools.crd.projectcalico.org created"
    L92: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/ipreservations.crd.projectcalico.org created"
    L93: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/kubecontrollersconfigurations.crd.projectcalico.org created"
    L94: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/networkpolicies.crd.projectcalico.org created"
    L95: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/networksets.crd.projectcalico.org created"
    L96: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/apiservers.operator.tigera.io created"
    L97: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/imagesets.operator.tigera.io created"
    L98: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/installations.operator.tigera.io created"
    L99: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/tigerastatuses.operator.tigera.io created"
    L100: "cluster.go:117: serviceaccount/tigera-operator created"
    L101: "cluster.go:117: clusterrole.rbac.authorization.k8s.io/tigera-operator created"
    L102: "cluster.go:117: clusterrolebinding.rbac.authorization.k8s.io/tigera-operator created"
    L103: "cluster.go:117: deployment.apps/tigera-operator created"
    L104: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/installations.operator.tigera.io condition met"
    L105: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/apiservers.operator.tigera.io condition met"
    L106: "cluster.go:117: installation.operator.tigera.io/default created"
    L107: "cluster.go:117: apiserver.operator.tigera.io/default created"
    L108: "cluster.go:117: Created symlink /etc/systemd/system/multi-user.target.wants/kubelet.service ??? /etc/systemd/system/kubelet.service."
    L109: "--- FAIL: kubeadm.v1.24.14.calico.cgroupv1.base/nginx_deployment (92.57s)"
    L110: "kubeadm.go:313: nginx is not deployed: ready replicas should be equal to 1: null_"
    L111: " "
    L112: " Error: _cluster.go:117: I0912 17:40:38.662589    1604 version.go:256] remote version is much newer: v1.28.1; falling back to: stable-1.24"
    L113: "cluster.go:117: [config/images] Pulled registry.k8s.io/kube-apiserver:v1.24.17"
    L114: "cluster.go:117: [config/images] Pulled registry.k8s.io/kube-controller-manager:v1.24.17"
    L115: "cluster.go:117: [config/images] Pulled registry.k8s.io/kube-scheduler:v1.24.17"
    L116: "cluster.go:117: [config/images] Pulled registry.k8s.io/kube-proxy:v1.24.17"
    L117: "cluster.go:117: [config/images] Pulled registry.k8s.io/pause:3.7"
    L118: "cluster.go:117: [config/images] Pulled registry.k8s.io/etcd:3.5.6-0"
    L119: "cluster.go:117: [config/images] Pulled registry.k8s.io/coredns/coredns:v1.8.6"
    L120: "cluster.go:117: I0912 17:40:50.204092    1773 version.go:256] remote version is much newer: v1.28.1; falling back to: stable-1.24"
    L121: "cluster.go:117: [init] Using Kubernetes version: v1.24.17"
    L122: "cluster.go:117: [preflight] Running pre-flight checks"
    L123: "cluster.go:117: [preflight] Pulling images required for setting up a Kubernetes cluster"
    L124: "cluster.go:117: [preflight] This might take a minute or two, depending on the speed of your internet connection"
    L125: "cluster.go:117: [preflight] You can also perform this action in beforehand using _kubeadm config images pull_"
    L126: "cluster.go:117: [certs] Using certificateDir folder __/etc/kubernetes/pki__"
    L127: "cluster.go:117: [certs] Generating __ca__ certificate and key"
    L128: "cluster.go:117: [certs] Generating __apiserver__ certificate and key"
    L129: "cluster.go:117: [certs] apiserver serving cert is signed for DNS names [kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local localhost] and IPs [10.96.0.1 10.0.0.9?]"
    L130: "cluster.go:117: [certs] Generating __apiserver-kubelet-client__ certificate and key"
    L131: "cluster.go:117: [certs] Generating __front-proxy-ca__ certificate and key"
    L132: "cluster.go:117: [certs] Generating __front-proxy-client__ certificate and key"
    L133: "cluster.go:117: [certs] External etcd mode: Skipping etcd/ca certificate authority generation"
    L134: "cluster.go:117: [certs] External etcd mode: Skipping etcd/server certificate generation"
    L135: "cluster.go:117: [certs] External etcd mode: Skipping etcd/peer certificate generation"
    L136: "cluster.go:117: [certs] External etcd mode: Skipping etcd/healthcheck-client certificate generation"
    L137: "cluster.go:117: [certs] External etcd mode: Skipping apiserver-etcd-client certificate generation"
    L138: "cluster.go:117: [certs] Generating __sa__ key and public key"
    L139: "cluster.go:117: [kubeconfig] Using kubeconfig folder __/etc/kubernetes__"
    L140: "cluster.go:117: [kubeconfig] Writing __admin.conf__ kubeconfig file"
    L141: "cluster.go:117: [kubeconfig] Writing __kubelet.conf__ kubeconfig file"
    L142: "cluster.go:117: [kubeconfig] Writing __controller-manager.conf__ kubeconfig file"
    L143: "cluster.go:117: [kubeconfig] Writing __scheduler.conf__ kubeconfig file"
    L144: "cluster.go:117: [kubelet-start] Writing kubelet environment file with flags to file __/var/lib/kubelet/kubeadm-flags.env__"
    L145: "cluster.go:117: [kubelet-start] Writing kubelet configuration to file __/var/lib/kubelet/config.yaml__"
    L146: "cluster.go:117: [kubelet-start] Starting the kubelet"
    L147: "cluster.go:117: [control-plane] Using manifest folder __/etc/kubernetes/manifests__"
    L148: "cluster.go:117: [control-plane] Creating static Pod manifest for __kube-apiserver__"
    L149: "cluster.go:117: [control-plane] Creating static Pod manifest for __kube-controller-manager__"
    L150: "cluster.go:117: [control-plane] Creating static Pod manifest for __kube-scheduler__"
    L151: "cluster.go:117: [wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory __/etc/kubernetes/manifests__. This can take up to 30m0s"
    L152: "cluster.go:117: [apiclient] All control plane components are healthy after 7.503559 seconds"
    L153: "cluster.go:117: [upload-config] Storing the configuration used in ConfigMap __kubeadm-config__ in the __kube-system__ Namespace"
    L154: "cluster.go:117: [kubelet] Creating a ConfigMap __kubelet-config__ in namespace kube-system with the configuration for the kubelets in the cluster"
    L155: "cluster.go:117: [upload-certs] Skipping phase. Please see --upload-certs"
    L156: "cluster.go:117: [mark-control-plane] Marking the node localhost as control-plane by adding the labels: [node-role.kubernetes.io/control-plane node.kubernetes.io/exclude-from-external-load-balancers]"
    L157: "cluster.go:117: [mark-control-plane] Marking the node localhost as control-plane by adding the taints [node-role.kubernetes.io/master:NoSchedule node-role.kubernetes.io/control-plane:NoSchedule]"
    L158: "cluster.go:117: [bootstrap-token] Using token: zc6wf9.23nki2q6s0v2ssfp"
    L159: "cluster.go:117: [bootstrap-token] Configuring bootstrap tokens, cluster-info ConfigMap, RBAC Roles"
    L160: "cluster.go:117: [bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to get nodes"
    L161: "cluster.go:117: [bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials"
    L162: "cluster.go:117: [bootstrap-token] Configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token"
    L163: "cluster.go:117: [bootstrap-token] Configured RBAC rules to allow certificate rotation for all node client certificates in the cluster"
    L164: "cluster.go:117: [bootstrap-token] Creating the __cluster-info__ ConfigMap in the __kube-public__ namespace"
    L165: "cluster.go:117: [kubelet-finalize] Updating __/etc/kubernetes/kubelet.conf__ to point to a rotatable kubelet client certificate and key"
    L166: "cluster.go:117: [addons] Applied essential addon: CoreDNS"
    L167: "cluster.go:117: [addons] Applied essential addon: kube-proxy"
    L168: "cluster.go:117: "
    L169: "cluster.go:117: Your Kubernetes control-plane has initialized successfully!"
    L170: "cluster.go:117: "
    L171: "cluster.go:117: To start using your cluster, you need to run the following as a regular user:"
    L172: "cluster.go:117: "
    L173: "cluster.go:117:   mkdir -p $HOME/.kube"
    L174: "cluster.go:117:   sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config"
    L175: "cluster.go:117:   sudo chown $(id -u):$(id -g) $HOME/.kube/config"
    L176: "cluster.go:117: "
    L177: "cluster.go:117: Alternatively, if you are the root user, you can run:"
    L178: "cluster.go:117: "
    L179: "cluster.go:117:   export KUBECONFIG=/etc/kubernetes/admin.conf"
    L180: "cluster.go:117: "
    L181: "cluster.go:117: You should now deploy a pod network to the cluster."
    L182: "cluster.go:117: Run __kubectl apply -f [podnetwork].yaml__ with one of the options listed at:"
    L183: "cluster.go:117:   https://kubernetes.io/docs/concepts/cluster-administration/addons/"
    L184: "cluster.go:117: "
    L185: "cluster.go:117: Then you can join any number of worker nodes by running the following on each as root:"
    L186: "cluster.go:117: "
    L187: "cluster.go:117: kubeadm join 10.0.0.9:6443 --token zc6wf9.23nki2q6s0v2ssfp _"
    L188: "cluster.go:117:  --discovery-token-ca-cert-hash sha256:4aa543f5c7f09c73b8aeb3717cfdbc9a8f28a04740fde43ce0ad2b9a2dff01c0 "
    L189: "cluster.go:117: namespace/tigera-operator created"
    L190: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/bgpconfigurations.crd.projectcalico.org created"
    L191: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/bgppeers.crd.projectcalico.org created"
    L192: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/blockaffinities.crd.projectcalico.org created"
    L193: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/caliconodestatuses.crd.projectcalico.org created"
    L194: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/clusterinformations.crd.projectcalico.org created"
    L195: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/felixconfigurations.crd.projectcalico.org created"
    L196: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/globalnetworkpolicies.crd.projectcalico.org created"
    L197: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/globalnetworksets.crd.projectcalico.org created"
    L198: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/hostendpoints.crd.projectcalico.org created"
    L199: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/ipamblocks.crd.projectcalico.org created"
    L200: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/ipamconfigs.crd.projectcalico.org created"
    L201: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/ipamhandles.crd.projectcalico.org created"
    L202: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/ippools.crd.projectcalico.org created"
    L203: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/ipreservations.crd.projectcalico.org created"
    L204: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/kubecontrollersconfigurations.crd.projectcalico.org created"
    L205: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/networkpolicies.crd.projectcalico.org created"
    L206: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/networksets.crd.projectcalico.org created"
    L207: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/apiservers.operator.tigera.io created"
    L208: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/imagesets.operator.tigera.io created"
    L209: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/installations.operator.tigera.io created"
    L210: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/tigerastatuses.operator.tigera.io created"
    L211: "cluster.go:117: serviceaccount/tigera-operator created"
    L212: "cluster.go:117: clusterrole.rbac.authorization.k8s.io/tigera-operator created"
    L213: "cluster.go:117: clusterrolebinding.rbac.authorization.k8s.io/tigera-operator created"
    L214: "cluster.go:117: deployment.apps/tigera-operator created"
    L215: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/installations.operator.tigera.io condition met"
    L216: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/apiservers.operator.tigera.io condition met"
    L217: "cluster.go:117: installation.operator.tigera.io/default created"
    L218: "cluster.go:117: apiserver.operator.tigera.io/default created"
    L219: "cluster.go:117: Created symlink /etc/systemd/system/multi-user.target.wants/kubelet.service ??? /etc/systemd/system/kubelet.service."
    L220: "--- FAIL: kubeadm.v1.24.14.calico.cgroupv1.base/nginx_deployment (93.66s)"
    L221: "kubeadm.go:313: nginx is not deployed: ready replicas should be equal to 1: null_"
    L222: " "
                Diagnostic output for qemu_uefi-arm64, run 1 
    L1: " Error: _cluster.go:117: I0912 17:25:11.817989    1660 version.go:256] remote version is much newer: v1.28.1; falling back to: stable-1.24"
    L2: "cluster.go:117: [config/images] Pulled registry.k8s.io/kube-apiserver:v1.24.17"
    L3: "cluster.go:117: [config/images] Pulled registry.k8s.io/kube-controller-manager:v1.24.17"
    L4: "cluster.go:117: [config/images] Pulled registry.k8s.io/kube-scheduler:v1.24.17"
    L5: "cluster.go:117: [config/images] Pulled registry.k8s.io/kube-proxy:v1.24.17"
    L6: "cluster.go:117: [config/images] Pulled registry.k8s.io/pause:3.7"
    L7: "cluster.go:117: [config/images] Pulled registry.k8s.io/etcd:3.5.6-0"
    L8: "cluster.go:117: [config/images] Pulled registry.k8s.io/coredns/coredns:v1.8.6"
    L9: "cluster.go:117: I0912 17:25:21.589130    1819 version.go:256] remote version is much newer: v1.28.1; falling back to: stable-1.24"
    L10: "cluster.go:117: [init] Using Kubernetes version: v1.24.17"
    L11: "cluster.go:117: [preflight] Running pre-flight checks"
    L12: "cluster.go:117: [preflight] Pulling images required for setting up a Kubernetes cluster"
    L13: "cluster.go:117: [preflight] This might take a minute or two, depending on the speed of your internet connection"
    L14: "cluster.go:117: [preflight] You can also perform this action in beforehand using _kubeadm config images pull_"
    L15: "cluster.go:117: [certs] Using certificateDir folder __/etc/kubernetes/pki__"
    L16: "cluster.go:117: [certs] Generating __ca__ certificate and key"
    L17: "cluster.go:117: [certs] Generating __apiserver__ certificate and key"
    L18: "cluster.go:117: [certs] apiserver serving cert is signed for DNS names [kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local localhost] and IPs [10.96.0.1 10.0.0.6?7]"
    L19: "cluster.go:117: [certs] Generating __apiserver-kubelet-client__ certificate and key"
    L20: "cluster.go:117: [certs] Generating __front-proxy-ca__ certificate and key"
    L21: "cluster.go:117: [certs] Generating __front-proxy-client__ certificate and key"
    L22: "cluster.go:117: [certs] External etcd mode: Skipping etcd/ca certificate authority generation"
    L23: "cluster.go:117: [certs] External etcd mode: Skipping etcd/server certificate generation"
    L24: "cluster.go:117: [certs] External etcd mode: Skipping etcd/peer certificate generation"
    L25: "cluster.go:117: [certs] External etcd mode: Skipping etcd/healthcheck-client certificate generation"
    L26: "cluster.go:117: [certs] External etcd mode: Skipping apiserver-etcd-client certificate generation"
    L27: "cluster.go:117: [certs] Generating __sa__ key and public key"
    L28: "cluster.go:117: [kubeconfig] Using kubeconfig folder __/etc/kubernetes__"
    L29: "cluster.go:117: [kubeconfig] Writing __admin.conf__ kubeconfig file"
    L30: "cluster.go:117: [kubeconfig] Writing __kubelet.conf__ kubeconfig file"
    L31: "cluster.go:117: [kubeconfig] Writing __controller-manager.conf__ kubeconfig file"
    L32: "cluster.go:117: [kubeconfig] Writing __scheduler.conf__ kubeconfig file"
    L33: "cluster.go:117: [kubelet-start] Writing kubelet environment file with flags to file __/var/lib/kubelet/kubeadm-flags.env__"
    L34: "cluster.go:117: [kubelet-start] Writing kubelet configuration to file __/var/lib/kubelet/config.yaml__"
    L35: "cluster.go:117: [kubelet-start] Starting the kubelet"
    L36: "cluster.go:117: [control-plane] Using manifest folder __/etc/kubernetes/manifests__"
    L37: "cluster.go:117: [control-plane] Creating static Pod manifest for __kube-apiserver__"
    L38: "cluster.go:117: [control-plane] Creating static Pod manifest for __kube-controller-manager__"
    L39: "cluster.go:117: [control-plane] Creating static Pod manifest for __kube-scheduler__"
    L40: "cluster.go:117: [wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory __/etc/kubernetes/manifests__. This can take up to 30m0s"
    L41: "cluster.go:117: [apiclient] All control plane components are healthy after 5.502474 seconds"
    L42: "cluster.go:117: [upload-config] Storing the configuration used in ConfigMap __kubeadm-config__ in the __kube-system__ Namespace"
    L43: "cluster.go:117: [kubelet] Creating a ConfigMap __kubelet-config__ in namespace kube-system with the configuration for the kubelets in the cluster"
    L44: "cluster.go:117: [upload-certs] Skipping phase. Please see --upload-certs"
    L45: "cluster.go:117: [mark-control-plane] Marking the node localhost as control-plane by adding the labels: [node-role.kubernetes.io/control-plane node.kubernetes.io/exclude-from-external-load-balancers]"
    L46: "cluster.go:117: [mark-control-plane] Marking the node localhost as control-plane by adding the taints [node-role.kubernetes.io/master:NoSchedule node-role.kubernetes.io/control-plane:NoSchedule]"
    L47: "cluster.go:117: [bootstrap-token] Using token: e4xzsk.6bte5tipsvt4c9ov"
    L48: "cluster.go:117: [bootstrap-token] Configuring bootstrap tokens, cluster-info ConfigMap, RBAC Roles"
    L49: "cluster.go:117: [bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to get nodes"
    L50: "cluster.go:117: [bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials"
    L51: "cluster.go:117: [bootstrap-token] Configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token"
    L52: "cluster.go:117: [bootstrap-token] Configured RBAC rules to allow certificate rotation for all node client certificates in the cluster"
    L53: "cluster.go:117: [bootstrap-token] Creating the __cluster-info__ ConfigMap in the __kube-public__ namespace"
    L54: "cluster.go:117: [kubelet-finalize] Updating __/etc/kubernetes/kubelet.conf__ to point to a rotatable kubelet client certificate and key"
    L55: "cluster.go:117: [addons] Applied essential addon: CoreDNS"
    L56: "cluster.go:117: [addons] Applied essential addon: kube-proxy"
    L57: "cluster.go:117: "
    L58: "cluster.go:117: Your Kubernetes control-plane has initialized successfully!"
    L59: "cluster.go:117: "
    L60: "cluster.go:117: To start using your cluster, you need to run the following as a regular user:"
    L61: "cluster.go:117: "
    L62: "cluster.go:117:   mkdir -p $HOME/.kube"
    L63: "cluster.go:117:   sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config"
    L64: "cluster.go:117:   sudo chown $(id -u):$(id -g) $HOME/.kube/config"
    L65: "cluster.go:117: "
    L66: "cluster.go:117: Alternatively, if you are the root user, you can run:"
    L67: "cluster.go:117: "
    L68: "cluster.go:117:   export KUBECONFIG=/etc/kubernetes/admin.conf"
    L69: "cluster.go:117: "
    L70: "cluster.go:117: You should now deploy a pod network to the cluster."
    L71: "cluster.go:117: Run __kubectl apply -f [podnetwork].yaml__ with one of the options listed at:"
    L72: "cluster.go:117:   https://kubernetes.io/docs/concepts/cluster-administration/addons/"
    L73: "cluster.go:117: "
    L74: "cluster.go:117: Then you can join any number of worker nodes by running the following on each as root:"
    L75: "cluster.go:117: "
    L76: "cluster.go:117: kubeadm join 10.0.0.67:6443 --token e4xzsk.6bte5tipsvt4c9ov _"
    L77: "cluster.go:117:  --discovery-token-ca-cert-hash sha256:9b3ee6c535df13592d35f0a85efe240d59aee46fb9e78135143feafd9be5b3c0 "
    L78: "cluster.go:117: namespace/tigera-operator created"
    L79: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/bgpconfigurations.crd.projectcalico.org created"
    L80: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/bgppeers.crd.projectcalico.org created"
    L81: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/blockaffinities.crd.projectcalico.org created"
    L82: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/caliconodestatuses.crd.projectcalico.org created"
    L83: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/clusterinformations.crd.projectcalico.org created"
    L84: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/felixconfigurations.crd.projectcalico.org created"
    L85: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/globalnetworkpolicies.crd.projectcalico.org created"
    L86: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/globalnetworksets.crd.projectcalico.org created"
    L87: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/hostendpoints.crd.projectcalico.org created"
    L88: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/ipamblocks.crd.projectcalico.org created"
    L89: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/ipamconfigs.crd.projectcalico.org created"
    L90: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/ipamhandles.crd.projectcalico.org created"
    L91: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/ippools.crd.projectcalico.org created"
    L92: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/ipreservations.crd.projectcalico.org created"
    L93: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/kubecontrollersconfigurations.crd.projectcalico.org created"
    L94: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/networkpolicies.crd.projectcalico.org created"
    L95: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/networksets.crd.projectcalico.org created"
    L96: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/apiservers.operator.tigera.io created"
    L97: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/imagesets.operator.tigera.io created"
    L98: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/installations.operator.tigera.io created"
    L99: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/tigerastatuses.operator.tigera.io created"
    L100: "cluster.go:117: serviceaccount/tigera-operator created"
    L101: "cluster.go:117: clusterrole.rbac.authorization.k8s.io/tigera-operator created"
    L102: "cluster.go:117: clusterrolebinding.rbac.authorization.k8s.io/tigera-operator created"
    L103: "cluster.go:117: deployment.apps/tigera-operator created"
    L104: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/installations.operator.tigera.io condition met"
    L105: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/apiservers.operator.tigera.io condition met"
    L106: "cluster.go:117: installation.operator.tigera.io/default created"
    L107: "cluster.go:117: apiserver.operator.tigera.io/default created"
    L108: "cluster.go:117: Created symlink /etc/systemd/system/multi-user.target.wants/kubelet.service ??? /etc/systemd/system/kubelet.service."
    L109: "--- FAIL: kubeadm.v1.24.14.calico.cgroupv1.base/nginx_deployment (92.58s)"
    L110: "kubeadm.go:313: nginx is not deployed: ready replicas should be equal to 1: null_"
    L111: " "
    L112: " Error: _cluster.go:117: I0912 17:32:42.476709    1611 version.go:256] remote version is much newer: v1.28.1; falling back to: stable-1.24"
    L113: "cluster.go:117: [config/images] Pulled registry.k8s.io/kube-apiserver:v1.24.17"
    L114: "cluster.go:117: [config/images] Pulled registry.k8s.io/kube-controller-manager:v1.24.17"
    L115: "cluster.go:117: [config/images] Pulled registry.k8s.io/kube-scheduler:v1.24.17"
    L116: "cluster.go:117: [config/images] Pulled registry.k8s.io/kube-proxy:v1.24.17"
    L117: "cluster.go:117: [config/images] Pulled registry.k8s.io/pause:3.7"
    L118: "cluster.go:117: [config/images] Pulled registry.k8s.io/etcd:3.5.6-0"
    L119: "cluster.go:117: [config/images] Pulled registry.k8s.io/coredns/coredns:v1.8.6"
    L120: "cluster.go:117: I0912 17:32:55.025968    1777 version.go:256] remote version is much newer: v1.28.1; falling back to: stable-1.24"
    L121: "cluster.go:117: [init] Using Kubernetes version: v1.24.17"
    L122: "cluster.go:117: [preflight] Running pre-flight checks"
    L123: "cluster.go:117: [preflight] Pulling images required for setting up a Kubernetes cluster"
    L124: "cluster.go:117: [preflight] This might take a minute or two, depending on the speed of your internet connection"
    L125: "cluster.go:117: [preflight] You can also perform this action in beforehand using _kubeadm config images pull_"
    L126: "cluster.go:117: [certs] Using certificateDir folder __/etc/kubernetes/pki__"
    L127: "cluster.go:117: [certs] Generating __ca__ certificate and key"
    L128: "cluster.go:117: [certs] Generating __apiserver__ certificate and key"
    L129: "cluster.go:117: [certs] apiserver serving cert is signed for DNS names [kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local localhost] and IPs [10.96.0.1 10.0.0.1?44]"
    L130: "cluster.go:117: [certs] Generating __apiserver-kubelet-client__ certificate and key"
    L131: "cluster.go:117: [certs] Generating __front-proxy-ca__ certificate and key"
    L132: "cluster.go:117: [certs] Generating __front-proxy-client__ certificate and key"
    L133: "cluster.go:117: [certs] External etcd mode: Skipping etcd/ca certificate authority generation"
    L134: "cluster.go:117: [certs] External etcd mode: Skipping etcd/server certificate generation"
    L135: "cluster.go:117: [certs] External etcd mode: Skipping etcd/peer certificate generation"
    L136: "cluster.go:117: [certs] External etcd mode: Skipping etcd/healthcheck-client certificate generation"
    L137: "cluster.go:117: [certs] External etcd mode: Skipping apiserver-etcd-client certificate generation"
    L138: "cluster.go:117: [certs] Generating __sa__ key and public key"
    L139: "cluster.go:117: [kubeconfig] Using kubeconfig folder __/etc/kubernetes__"
    L140: "cluster.go:117: [kubeconfig] Writing __admin.conf__ kubeconfig file"
    L141: "cluster.go:117: [kubeconfig] Writing __kubelet.conf__ kubeconfig file"
    L142: "cluster.go:117: [kubeconfig] Writing __controller-manager.conf__ kubeconfig file"
    L143: "cluster.go:117: [kubeconfig] Writing __scheduler.conf__ kubeconfig file"
    L144: "cluster.go:117: [kubelet-start] Writing kubelet environment file with flags to file __/var/lib/kubelet/kubeadm-flags.env__"
    L145: "cluster.go:117: [kubelet-start] Writing kubelet configuration to file __/var/lib/kubelet/config.yaml__"
    L146: "cluster.go:117: [kubelet-start] Starting the kubelet"
    L147: "cluster.go:117: [control-plane] Using manifest folder __/etc/kubernetes/manifests__"
    L148: "cluster.go:117: [control-plane] Creating static Pod manifest for __kube-apiserver__"
    L149: "cluster.go:117: [control-plane] Creating static Pod manifest for __kube-controller-manager__"
    L150: "cluster.go:117: [control-plane] Creating static Pod manifest for __kube-scheduler__"
    L151: "cluster.go:117: [wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory __/etc/kubernetes/manifests__. This can take up to 30m0s"
    L152: "cluster.go:117: [apiclient] All control plane components are healthy after 6.503611 seconds"
    L153: "cluster.go:117: [upload-config] Storing the configuration used in ConfigMap __kubeadm-config__ in the __kube-system__ Namespace"
    L154: "cluster.go:117: [kubelet] Creating a ConfigMap __kubelet-config__ in namespace kube-system with the configuration for the kubelets in the cluster"
    L155: "cluster.go:117: [upload-certs] Skipping phase. Please see --upload-certs"
    L156: "cluster.go:117: [mark-control-plane] Marking the node localhost as control-plane by adding the labels: [node-role.kubernetes.io/control-plane node.kubernetes.io/exclude-from-external-load-balancers]"
    L157: "cluster.go:117: [mark-control-plane] Marking the node localhost as control-plane by adding the taints [node-role.kubernetes.io/master:NoSchedule node-role.kubernetes.io/control-plane:NoSchedule]"
    L158: "cluster.go:117: [bootstrap-token] Using token: 63krmi.e1iymnhpbo7q4ako"
    L159: "cluster.go:117: [bootstrap-token] Configuring bootstrap tokens, cluster-info ConfigMap, RBAC Roles"
    L160: "cluster.go:117: [bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to get nodes"
    L161: "cluster.go:117: [bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials"
    L162: "cluster.go:117: [bootstrap-token] Configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token"
    L163: "cluster.go:117: [bootstrap-token] Configured RBAC rules to allow certificate rotation for all node client certificates in the cluster"
    L164: "cluster.go:117: [bootstrap-token] Creating the __cluster-info__ ConfigMap in the __kube-public__ namespace"
    L165: "cluster.go:117: [kubelet-finalize] Updating __/etc/kubernetes/kubelet.conf__ to point to a rotatable kubelet client certificate and key"
    L166: "cluster.go:117: [addons] Applied essential addon: CoreDNS"
    L167: "cluster.go:117: [addons] Applied essential addon: kube-proxy"
    L168: "cluster.go:117: "
    L169: "cluster.go:117: Your Kubernetes control-plane has initialized successfully!"
    L170: "cluster.go:117: "
    L171: "cluster.go:117: To start using your cluster, you need to run the following as a regular user:"
    L172: "cluster.go:117: "
    L173: "cluster.go:117:   mkdir -p $HOME/.kube"
    L174: "cluster.go:117:   sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config"
    L175: "cluster.go:117:   sudo chown $(id -u):$(id -g) $HOME/.kube/config"
    L176: "cluster.go:117: "
    L177: "cluster.go:117: Alternatively, if you are the root user, you can run:"
    L178: "cluster.go:117: "
    L179: "cluster.go:117:   export KUBECONFIG=/etc/kubernetes/admin.conf"
    L180: "cluster.go:117: "
    L181: "cluster.go:117: You should now deploy a pod network to the cluster."
    L182: "cluster.go:117: Run __kubectl apply -f [podnetwork].yaml__ with one of the options listed at:"
    L183: "cluster.go:117:   https://kubernetes.io/docs/concepts/cluster-administration/addons/"
    L184: "cluster.go:117: "
    L185: "cluster.go:117: Then you can join any number of worker nodes by running the following on each as root:"
    L186: "cluster.go:117: "
    L187: "cluster.go:117: kubeadm join 10.0.0.144:6443 --token 63krmi.e1iymnhpbo7q4ako _"
    L188: "cluster.go:117:  --discovery-token-ca-cert-hash sha256:b7dea931c02ea1bf909019e199ace5b1cbcd52c6c2091335edca83fdfbcee444 "
    L189: "cluster.go:117: namespace/tigera-operator created"
    L190: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/bgpconfigurations.crd.projectcalico.org created"
    L191: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/bgppeers.crd.projectcalico.org created"
    L192: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/blockaffinities.crd.projectcalico.org created"
    L193: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/caliconodestatuses.crd.projectcalico.org created"
    L194: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/clusterinformations.crd.projectcalico.org created"
    L195: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/felixconfigurations.crd.projectcalico.org created"
    L196: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/globalnetworkpolicies.crd.projectcalico.org created"
    L197: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/globalnetworksets.crd.projectcalico.org created"
    L198: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/hostendpoints.crd.projectcalico.org created"
    L199: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/ipamblocks.crd.projectcalico.org created"
    L200: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/ipamconfigs.crd.projectcalico.org created"
    L201: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/ipamhandles.crd.projectcalico.org created"
    L202: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/ippools.crd.projectcalico.org created"
    L203: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/ipreservations.crd.projectcalico.org created"
    L204: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/kubecontrollersconfigurations.crd.projectcalico.org created"
    L205: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/networkpolicies.crd.projectcalico.org created"
    L206: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/networksets.crd.projectcalico.org created"
    L207: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/apiservers.operator.tigera.io created"
    L208: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/imagesets.operator.tigera.io created"
    L209: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/installations.operator.tigera.io created"
    L210: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/tigerastatuses.operator.tigera.io created"
    L211: "cluster.go:117: serviceaccount/tigera-operator created"
    L212: "cluster.go:117: clusterrole.rbac.authorization.k8s.io/tigera-operator created"
    L213: "cluster.go:117: clusterrolebinding.rbac.authorization.k8s.io/tigera-operator created"
    L214: "cluster.go:117: deployment.apps/tigera-operator created"
    L215: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/installations.operator.tigera.io condition met"
    L216: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/apiservers.operator.tigera.io condition met"
    L217: "cluster.go:117: installation.operator.tigera.io/default created"
    L218: "cluster.go:117: apiserver.operator.tigera.io/default created"
    L219: "cluster.go:117: Created symlink /etc/systemd/system/multi-user.target.wants/kubelet.service ??? /etc/systemd/system/kubelet.service."
    L220: "--- FAIL: kubeadm.v1.24.14.calico.cgroupv1.base/nginx_deployment (93.55s)"
    L221: "kubeadm.go:313: nginx is not deployed: ready replicas should be equal to 1: null_"
    L222: " "
                Diagnostic output for qemu_uefi-amd64, run 1 
    L1: " Error: _cluster.go:117: I0912 17:25:11.817989    1660 version.go:256] remote version is much newer: v1.28.1; falling back to: stable-1.24"
    L2: "cluster.go:117: [config/images] Pulled registry.k8s.io/kube-apiserver:v1.24.17"
    L3: "cluster.go:117: [config/images] Pulled registry.k8s.io/kube-controller-manager:v1.24.17"
    L4: "cluster.go:117: [config/images] Pulled registry.k8s.io/kube-scheduler:v1.24.17"
    L5: "cluster.go:117: [config/images] Pulled registry.k8s.io/kube-proxy:v1.24.17"
    L6: "cluster.go:117: [config/images] Pulled registry.k8s.io/pause:3.7"
    L7: "cluster.go:117: [config/images] Pulled registry.k8s.io/etcd:3.5.6-0"
    L8: "cluster.go:117: [config/images] Pulled registry.k8s.io/coredns/coredns:v1.8.6"
    L9: "cluster.go:117: I0912 17:25:21.589130    1819 version.go:256] remote version is much newer: v1.28.1; falling back to: stable-1.24"
    L10: "cluster.go:117: [init] Using Kubernetes version: v1.24.17"
    L11: "cluster.go:117: [preflight] Running pre-flight checks"
    L12: "cluster.go:117: [preflight] Pulling images required for setting up a Kubernetes cluster"
    L13: "cluster.go:117: [preflight] This might take a minute or two, depending on the speed of your internet connection"
    L14: "cluster.go:117: [preflight] You can also perform this action in beforehand using _kubeadm config images pull_"
    L15: "cluster.go:117: [certs] Using certificateDir folder __/etc/kubernetes/pki__"
    L16: "cluster.go:117: [certs] Generating __ca__ certificate and key"
    L17: "cluster.go:117: [certs] Generating __apiserver__ certificate and key"
    L18: "cluster.go:117: [certs] apiserver serving cert is signed for DNS names [kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local localhost] and IPs [10.96.0.1 10.0.0.6?7]"
    L19: "cluster.go:117: [certs] Generating __apiserver-kubelet-client__ certificate and key"
    L20: "cluster.go:117: [certs] Generating __front-proxy-ca__ certificate and key"
    L21: "cluster.go:117: [certs] Generating __front-proxy-client__ certificate and key"
    L22: "cluster.go:117: [certs] External etcd mode: Skipping etcd/ca certificate authority generation"
    L23: "cluster.go:117: [certs] External etcd mode: Skipping etcd/server certificate generation"
    L24: "cluster.go:117: [certs] External etcd mode: Skipping etcd/peer certificate generation"
    L25: "cluster.go:117: [certs] External etcd mode: Skipping etcd/healthcheck-client certificate generation"
    L26: "cluster.go:117: [certs] External etcd mode: Skipping apiserver-etcd-client certificate generation"
    L27: "cluster.go:117: [certs] Generating __sa__ key and public key"
    L28: "cluster.go:117: [kubeconfig] Using kubeconfig folder __/etc/kubernetes__"
    L29: "cluster.go:117: [kubeconfig] Writing __admin.conf__ kubeconfig file"
    L30: "cluster.go:117: [kubeconfig] Writing __kubelet.conf__ kubeconfig file"
    L31: "cluster.go:117: [kubeconfig] Writing __controller-manager.conf__ kubeconfig file"
    L32: "cluster.go:117: [kubeconfig] Writing __scheduler.conf__ kubeconfig file"
    L33: "cluster.go:117: [kubelet-start] Writing kubelet environment file with flags to file __/var/lib/kubelet/kubeadm-flags.env__"
    L34: "cluster.go:117: [kubelet-start] Writing kubelet configuration to file __/var/lib/kubelet/config.yaml__"
    L35: "cluster.go:117: [kubelet-start] Starting the kubelet"
    L36: "cluster.go:117: [control-plane] Using manifest folder __/etc/kubernetes/manifests__"
    L37: "cluster.go:117: [control-plane] Creating static Pod manifest for __kube-apiserver__"
    L38: "cluster.go:117: [control-plane] Creating static Pod manifest for __kube-controller-manager__"
    L39: "cluster.go:117: [control-plane] Creating static Pod manifest for __kube-scheduler__"
    L40: "cluster.go:117: [wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory __/etc/kubernetes/manifests__. This can take up to 30m0s"
    L41: "cluster.go:117: [apiclient] All control plane components are healthy after 5.502474 seconds"
    L42: "cluster.go:117: [upload-config] Storing the configuration used in ConfigMap __kubeadm-config__ in the __kube-system__ Namespace"
    L43: "cluster.go:117: [kubelet] Creating a ConfigMap __kubelet-config__ in namespace kube-system with the configuration for the kubelets in the cluster"
    L44: "cluster.go:117: [upload-certs] Skipping phase. Please see --upload-certs"
    L45: "cluster.go:117: [mark-control-plane] Marking the node localhost as control-plane by adding the labels: [node-role.kubernetes.io/control-plane node.kubernetes.io/exclude-from-external-load-balancers]"
    L46: "cluster.go:117: [mark-control-plane] Marking the node localhost as control-plane by adding the taints [node-role.kubernetes.io/master:NoSchedule node-role.kubernetes.io/control-plane:NoSchedule]"
    L47: "cluster.go:117: [bootstrap-token] Using token: e4xzsk.6bte5tipsvt4c9ov"
    L48: "cluster.go:117: [bootstrap-token] Configuring bootstrap tokens, cluster-info ConfigMap, RBAC Roles"
    L49: "cluster.go:117: [bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to get nodes"
    L50: "cluster.go:117: [bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials"
    L51: "cluster.go:117: [bootstrap-token] Configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token"
    L52: "cluster.go:117: [bootstrap-token] Configured RBAC rules to allow certificate rotation for all node client certificates in the cluster"
    L53: "cluster.go:117: [bootstrap-token] Creating the __cluster-info__ ConfigMap in the __kube-public__ namespace"
    L54: "cluster.go:117: [kubelet-finalize] Updating __/etc/kubernetes/kubelet.conf__ to point to a rotatable kubelet client certificate and key"
    L55: "cluster.go:117: [addons] Applied essential addon: CoreDNS"
    L56: "cluster.go:117: [addons] Applied essential addon: kube-proxy"
    L57: "cluster.go:117: "
    L58: "cluster.go:117: Your Kubernetes control-plane has initialized successfully!"
    L59: "cluster.go:117: "
    L60: "cluster.go:117: To start using your cluster, you need to run the following as a regular user:"
    L61: "cluster.go:117: "
    L62: "cluster.go:117:   mkdir -p $HOME/.kube"
    L63: "cluster.go:117:   sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config"
    L64: "cluster.go:117:   sudo chown $(id -u):$(id -g) $HOME/.kube/config"
    L65: "cluster.go:117: "
    L66: "cluster.go:117: Alternatively, if you are the root user, you can run:"
    L67: "cluster.go:117: "
    L68: "cluster.go:117:   export KUBECONFIG=/etc/kubernetes/admin.conf"
    L69: "cluster.go:117: "
    L70: "cluster.go:117: You should now deploy a pod network to the cluster."
    L71: "cluster.go:117: Run __kubectl apply -f [podnetwork].yaml__ with one of the options listed at:"
    L72: "cluster.go:117:   https://kubernetes.io/docs/concepts/cluster-administration/addons/"
    L73: "cluster.go:117: "
    L74: "cluster.go:117: Then you can join any number of worker nodes by running the following on each as root:"
    L75: "cluster.go:117: "
    L76: "cluster.go:117: kubeadm join 10.0.0.67:6443 --token e4xzsk.6bte5tipsvt4c9ov _"
    L77: "cluster.go:117:  --discovery-token-ca-cert-hash sha256:9b3ee6c535df13592d35f0a85efe240d59aee46fb9e78135143feafd9be5b3c0 "
    L78: "cluster.go:117: namespace/tigera-operator created"
    L79: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/bgpconfigurations.crd.projectcalico.org created"
    L80: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/bgppeers.crd.projectcalico.org created"
    L81: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/blockaffinities.crd.projectcalico.org created"
    L82: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/caliconodestatuses.crd.projectcalico.org created"
    L83: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/clusterinformations.crd.projectcalico.org created"
    L84: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/felixconfigurations.crd.projectcalico.org created"
    L85: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/globalnetworkpolicies.crd.projectcalico.org created"
    L86: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/globalnetworksets.crd.projectcalico.org created"
    L87: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/hostendpoints.crd.projectcalico.org created"
    L88: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/ipamblocks.crd.projectcalico.org created"
    L89: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/ipamconfigs.crd.projectcalico.org created"
    L90: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/ipamhandles.crd.projectcalico.org created"
    L91: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/ippools.crd.projectcalico.org created"
    L92: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/ipreservations.crd.projectcalico.org created"
    L93: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/kubecontrollersconfigurations.crd.projectcalico.org created"
    L94: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/networkpolicies.crd.projectcalico.org created"
    L95: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/networksets.crd.projectcalico.org created"
    L96: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/apiservers.operator.tigera.io created"
    L97: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/imagesets.operator.tigera.io created"
    L98: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/installations.operator.tigera.io created"
    L99: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/tigerastatuses.operator.tigera.io created"
    L100: "cluster.go:117: serviceaccount/tigera-operator created"
    L101: "cluster.go:117: clusterrole.rbac.authorization.k8s.io/tigera-operator created"
    L102: "cluster.go:117: clusterrolebinding.rbac.authorization.k8s.io/tigera-operator created"
    L103: "cluster.go:117: deployment.apps/tigera-operator created"
    L104: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/installations.operator.tigera.io condition met"
    L105: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/apiservers.operator.tigera.io condition met"
    L106: "cluster.go:117: installation.operator.tigera.io/default created"
    L107: "cluster.go:117: apiserver.operator.tigera.io/default created"
    L108: "cluster.go:117: Created symlink /etc/systemd/system/multi-user.target.wants/kubelet.service ??? /etc/systemd/system/kubelet.service."
    L109: "--- FAIL: kubeadm.v1.24.14.calico.cgroupv1.base/nginx_deployment (92.58s)"
    L110: "kubeadm.go:313: nginx is not deployed: ready replicas should be equal to 1: null_"
    L111: " "
    L112: " Error: _cluster.go:117: I0912 17:32:42.476709    1611 version.go:256] remote version is much newer: v1.28.1; falling back to: stable-1.24"
    L113: "cluster.go:117: [config/images] Pulled registry.k8s.io/kube-apiserver:v1.24.17"
    L114: "cluster.go:117: [config/images] Pulled registry.k8s.io/kube-controller-manager:v1.24.17"
    L115: "cluster.go:117: [config/images] Pulled registry.k8s.io/kube-scheduler:v1.24.17"
    L116: "cluster.go:117: [config/images] Pulled registry.k8s.io/kube-proxy:v1.24.17"
    L117: "cluster.go:117: [config/images] Pulled registry.k8s.io/pause:3.7"
    L118: "cluster.go:117: [config/images] Pulled registry.k8s.io/etcd:3.5.6-0"
    L119: "cluster.go:117: [config/images] Pulled registry.k8s.io/coredns/coredns:v1.8.6"
    L120: "cluster.go:117: I0912 17:32:55.025968    1777 version.go:256] remote version is much newer: v1.28.1; falling back to: stable-1.24"
    L121: "cluster.go:117: [init] Using Kubernetes version: v1.24.17"
    L122: "cluster.go:117: [preflight] Running pre-flight checks"
    L123: "cluster.go:117: [preflight] Pulling images required for setting up a Kubernetes cluster"
    L124: "cluster.go:117: [preflight] This might take a minute or two, depending on the speed of your internet connection"
    L125: "cluster.go:117: [preflight] You can also perform this action in beforehand using _kubeadm config images pull_"
    L126: "cluster.go:117: [certs] Using certificateDir folder __/etc/kubernetes/pki__"
    L127: "cluster.go:117: [certs] Generating __ca__ certificate and key"
    L128: "cluster.go:117: [certs] Generating __apiserver__ certificate and key"
    L129: "cluster.go:117: [certs] apiserver serving cert is signed for DNS names [kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local localhost] and IPs [10.96.0.1 10.0.0.1?44]"
    L130: "cluster.go:117: [certs] Generating __apiserver-kubelet-client__ certificate and key"
    L131: "cluster.go:117: [certs] Generating __front-proxy-ca__ certificate and key"
    L132: "cluster.go:117: [certs] Generating __front-proxy-client__ certificate and key"
    L133: "cluster.go:117: [certs] External etcd mode: Skipping etcd/ca certificate authority generation"
    L134: "cluster.go:117: [certs] External etcd mode: Skipping etcd/server certificate generation"
    L135: "cluster.go:117: [certs] External etcd mode: Skipping etcd/peer certificate generation"
    L136: "cluster.go:117: [certs] External etcd mode: Skipping etcd/healthcheck-client certificate generation"
    L137: "cluster.go:117: [certs] External etcd mode: Skipping apiserver-etcd-client certificate generation"
    L138: "cluster.go:117: [certs] Generating __sa__ key and public key"
    L139: "cluster.go:117: [kubeconfig] Using kubeconfig folder __/etc/kubernetes__"
    L140: "cluster.go:117: [kubeconfig] Writing __admin.conf__ kubeconfig file"
    L141: "cluster.go:117: [kubeconfig] Writing __kubelet.conf__ kubeconfig file"
    L142: "cluster.go:117: [kubeconfig] Writing __controller-manager.conf__ kubeconfig file"
    L143: "cluster.go:117: [kubeconfig] Writing __scheduler.conf__ kubeconfig file"
    L144: "cluster.go:117: [kubelet-start] Writing kubelet environment file with flags to file __/var/lib/kubelet/kubeadm-flags.env__"
    L145: "cluster.go:117: [kubelet-start] Writing kubelet configuration to file __/var/lib/kubelet/config.yaml__"
    L146: "cluster.go:117: [kubelet-start] Starting the kubelet"
    L147: "cluster.go:117: [control-plane] Using manifest folder __/etc/kubernetes/manifests__"
    L148: "cluster.go:117: [control-plane] Creating static Pod manifest for __kube-apiserver__"
    L149: "cluster.go:117: [control-plane] Creating static Pod manifest for __kube-controller-manager__"
    L150: "cluster.go:117: [control-plane] Creating static Pod manifest for __kube-scheduler__"
    L151: "cluster.go:117: [wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory __/etc/kubernetes/manifests__. This can take up to 30m0s"
    L152: "cluster.go:117: [apiclient] All control plane components are healthy after 6.503611 seconds"
    L153: "cluster.go:117: [upload-config] Storing the configuration used in ConfigMap __kubeadm-config__ in the __kube-system__ Namespace"
    L154: "cluster.go:117: [kubelet] Creating a ConfigMap __kubelet-config__ in namespace kube-system with the configuration for the kubelets in the cluster"
    L155: "cluster.go:117: [upload-certs] Skipping phase. Please see --upload-certs"
    L156: "cluster.go:117: [mark-control-plane] Marking the node localhost as control-plane by adding the labels: [node-role.kubernetes.io/control-plane node.kubernetes.io/exclude-from-external-load-balancers]"
    L157: "cluster.go:117: [mark-control-plane] Marking the node localhost as control-plane by adding the taints [node-role.kubernetes.io/master:NoSchedule node-role.kubernetes.io/control-plane:NoSchedule]"
    L158: "cluster.go:117: [bootstrap-token] Using token: 63krmi.e1iymnhpbo7q4ako"
    L159: "cluster.go:117: [bootstrap-token] Configuring bootstrap tokens, cluster-info ConfigMap, RBAC Roles"
    L160: "cluster.go:117: [bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to get nodes"
    L161: "cluster.go:117: [bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials"
    L162: "cluster.go:117: [bootstrap-token] Configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token"
    L163: "cluster.go:117: [bootstrap-token] Configured RBAC rules to allow certificate rotation for all node client certificates in the cluster"
    L164: "cluster.go:117: [bootstrap-token] Creating the __cluster-info__ ConfigMap in the __kube-public__ namespace"
    L165: "cluster.go:117: [kubelet-finalize] Updating __/etc/kubernetes/kubelet.conf__ to point to a rotatable kubelet client certificate and key"
    L166: "cluster.go:117: [addons] Applied essential addon: CoreDNS"
    L167: "cluster.go:117: [addons] Applied essential addon: kube-proxy"
    L168: "cluster.go:117: "
    L169: "cluster.go:117: Your Kubernetes control-plane has initialized successfully!"
    L170: "cluster.go:117: "
    L171: "cluster.go:117: To start using your cluster, you need to run the following as a regular user:"
    L172: "cluster.go:117: "
    L173: "cluster.go:117:   mkdir -p $HOME/.kube"
    L174: "cluster.go:117:   sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config"
    L175: "cluster.go:117:   sudo chown $(id -u):$(id -g) $HOME/.kube/config"
    L176: "cluster.go:117: "
    L177: "cluster.go:117: Alternatively, if you are the root user, you can run:"
    L178: "cluster.go:117: "
    L179: "cluster.go:117:   export KUBECONFIG=/etc/kubernetes/admin.conf"
    L180: "cluster.go:117: "
    L181: "cluster.go:117: You should now deploy a pod network to the cluster."
    L182: "cluster.go:117: Run __kubectl apply -f [podnetwork].yaml__ with one of the options listed at:"
    L183: "cluster.go:117:   https://kubernetes.io/docs/concepts/cluster-administration/addons/"
    L184: "cluster.go:117: "
    L185: "cluster.go:117: Then you can join any number of worker nodes by running the following on each as root:"
    L186: "cluster.go:117: "
    L187: "cluster.go:117: kubeadm join 10.0.0.144:6443 --token 63krmi.e1iymnhpbo7q4ako _"
    L188: "cluster.go:117:  --discovery-token-ca-cert-hash sha256:b7dea931c02ea1bf909019e199ace5b1cbcd52c6c2091335edca83fdfbcee444 "
    L189: "cluster.go:117: namespace/tigera-operator created"
    L190: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/bgpconfigurations.crd.projectcalico.org created"
    L191: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/bgppeers.crd.projectcalico.org created"
    L192: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/blockaffinities.crd.projectcalico.org created"
    L193: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/caliconodestatuses.crd.projectcalico.org created"
    L194: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/clusterinformations.crd.projectcalico.org created"
    L195: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/felixconfigurations.crd.projectcalico.org created"
    L196: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/globalnetworkpolicies.crd.projectcalico.org created"
    L197: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/globalnetworksets.crd.projectcalico.org created"
    L198: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/hostendpoints.crd.projectcalico.org created"
    L199: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/ipamblocks.crd.projectcalico.org created"
    L200: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/ipamconfigs.crd.projectcalico.org created"
    L201: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/ipamhandles.crd.projectcalico.org created"
    L202: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/ippools.crd.projectcalico.org created"
    L203: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/ipreservations.crd.projectcalico.org created"
    L204: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/kubecontrollersconfigurations.crd.projectcalico.org created"
    L205: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/networkpolicies.crd.projectcalico.org created"
    L206: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/networksets.crd.projectcalico.org created"
    L207: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/apiservers.operator.tigera.io created"
    L208: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/imagesets.operator.tigera.io created"
    L209: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/installations.operator.tigera.io created"
    L210: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/tigerastatuses.operator.tigera.io created"
    L211: "cluster.go:117: serviceaccount/tigera-operator created"
    L212: "cluster.go:117: clusterrole.rbac.authorization.k8s.io/tigera-operator created"
    L213: "cluster.go:117: clusterrolebinding.rbac.authorization.k8s.io/tigera-operator created"
    L214: "cluster.go:117: deployment.apps/tigera-operator created"
    L215: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/installations.operator.tigera.io condition met"
    L216: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/apiservers.operator.tigera.io condition met"
    L217: "cluster.go:117: installation.operator.tigera.io/default created"
    L218: "cluster.go:117: apiserver.operator.tigera.io/default created"
    L219: "cluster.go:117: Created symlink /etc/systemd/system/multi-user.target.wants/kubelet.service ??? /etc/systemd/system/kubelet.service."
    L220: "--- FAIL: kubeadm.v1.24.14.calico.cgroupv1.base/nginx_deployment (93.55s)"
    L221: "kubeadm.go:313: nginx is not deployed: ready replicas should be equal to 1: null_"
    L222: " "

ok kubeadm.v1.24.14.cilium.base 🟢 Succeeded: qemu_uefi-amd64 (2); qemu_uefi-arm64 (2) ❌ Failed: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

                Diagnostic output for qemu_uefi-arm64, run 1 
    L1: " Error: _cluster.go:117: I0912 17:24:26.255769    1490 version.go:256] remote version is much newer: v1.28.1; falling back to: stable-1.24"
    L2: "cluster.go:117: [config/images] Pulled registry.k8s.io/kube-apiserver:v1.24.17"
    L3: "cluster.go:117: [config/images] Pulled registry.k8s.io/kube-controller-manager:v1.24.17"
    L4: "cluster.go:117: [config/images] Pulled registry.k8s.io/kube-scheduler:v1.24.17"
    L5: "cluster.go:117: [config/images] Pulled registry.k8s.io/kube-proxy:v1.24.17"
    L6: "cluster.go:117: [config/images] Pulled registry.k8s.io/pause:3.7"
    L7: "cluster.go:117: [config/images] Pulled registry.k8s.io/etcd:3.5.6-0"
    L8: "cluster.go:117: [config/images] Pulled registry.k8s.io/coredns/coredns:v1.8.6"
    L9: "cluster.go:117: I0912 17:24:36.758555    1652 version.go:256] remote version is much newer: v1.28.1; falling back to: stable-1.24"
    L10: "cluster.go:117: [init] Using Kubernetes version: v1.24.17"
    L11: "cluster.go:117: [preflight] Running pre-flight checks"
    L12: "cluster.go:117: [preflight] Pulling images required for setting up a Kubernetes cluster"
    L13: "cluster.go:117: [preflight] This might take a minute or two, depending on the speed of your internet connection"
    L14: "cluster.go:117: [preflight] You can also perform this action in beforehand using _kubeadm config images pull_"
    L15: "cluster.go:117: [certs] Using certificateDir folder __/etc/kubernetes/pki__"
    L16: "cluster.go:117: [certs] Generating __ca__ certificate and key"
    L17: "cluster.go:117: [certs] Generating __apiserver__ certificate and key"
    L18: "cluster.go:117: [certs] apiserver serving cert is signed for DNS names [kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local localhost] and IPs [10.96.0.1 10.0.0.5?9]"
    L19: "cluster.go:117: [certs] Generating __apiserver-kubelet-client__ certificate and key"
    L20: "cluster.go:117: [certs] Generating __front-proxy-ca__ certificate and key"
    L21: "cluster.go:117: [certs] Generating __front-proxy-client__ certificate and key"
    L22: "cluster.go:117: [certs] External etcd mode: Skipping etcd/ca certificate authority generation"
    L23: "cluster.go:117: [certs] External etcd mode: Skipping etcd/server certificate generation"
    L24: "cluster.go:117: [certs] External etcd mode: Skipping etcd/peer certificate generation"
    L25: "cluster.go:117: [certs] External etcd mode: Skipping etcd/healthcheck-client certificate generation"
    L26: "cluster.go:117: [certs] External etcd mode: Skipping apiserver-etcd-client certificate generation"
    L27: "cluster.go:117: [certs] Generating __sa__ key and public key"
    L28: "cluster.go:117: [kubeconfig] Using kubeconfig folder __/etc/kubernetes__"
    L29: "cluster.go:117: [kubeconfig] Writing __admin.conf__ kubeconfig file"
    L30: "cluster.go:117: [kubeconfig] Writing __kubelet.conf__ kubeconfig file"
    L31: "cluster.go:117: [kubeconfig] Writing __controller-manager.conf__ kubeconfig file"
    L32: "cluster.go:117: [kubeconfig] Writing __scheduler.conf__ kubeconfig file"
    L33: "cluster.go:117: [kubelet-start] Writing kubelet environment file with flags to file __/var/lib/kubelet/kubeadm-flags.env__"
    L34: "cluster.go:117: [kubelet-start] Writing kubelet configuration to file __/var/lib/kubelet/config.yaml__"
    L35: "cluster.go:117: [kubelet-start] Starting the kubelet"
    L36: "cluster.go:117: [control-plane] Using manifest folder __/etc/kubernetes/manifests__"
    L37: "cluster.go:117: [control-plane] Creating static Pod manifest for __kube-apiserver__"
    L38: "cluster.go:117: [control-plane] Creating static Pod manifest for __kube-controller-manager__"
    L39: "cluster.go:117: [control-plane] Creating static Pod manifest for __kube-scheduler__"
    L40: "cluster.go:117: [wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory __/etc/kubernetes/manifests__. This can take up to 30m0s"
    L41: "cluster.go:117: [apiclient] All control plane components are healthy after 5.503525 seconds"
    L42: "cluster.go:117: [upload-config] Storing the configuration used in ConfigMap __kubeadm-config__ in the __kube-system__ Namespace"
    L43: "cluster.go:117: [kubelet] Creating a ConfigMap __kubelet-config__ in namespace kube-system with the configuration for the kubelets in the cluster"
    L44: "cluster.go:117: [upload-certs] Skipping phase. Please see --upload-certs"
    L45: "cluster.go:117: [mark-control-plane] Marking the node localhost as control-plane by adding the labels: [node-role.kubernetes.io/control-plane node.kubernetes.io/exclude-from-external-load-balancers]"
    L46: "cluster.go:117: [mark-control-plane] Marking the node localhost as control-plane by adding the taints [node-role.kubernetes.io/master:NoSchedule node-role.kubernetes.io/control-plane:NoSchedule]"
    L47: "cluster.go:117: [bootstrap-token] Using token: kmcj73.hmpunky8spjnwtio"
    L48: "cluster.go:117: [bootstrap-token] Configuring bootstrap tokens, cluster-info ConfigMap, RBAC Roles"
    L49: "cluster.go:117: [bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to get nodes"
    L50: "cluster.go:117: [bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials"
    L51: "cluster.go:117: [bootstrap-token] Configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token"
    L52: "cluster.go:117: [bootstrap-token] Configured RBAC rules to allow certificate rotation for all node client certificates in the cluster"
    L53: "cluster.go:117: [bootstrap-token] Creating the __cluster-info__ ConfigMap in the __kube-public__ namespace"
    L54: "cluster.go:117: [kubelet-finalize] Updating __/etc/kubernetes/kubelet.conf__ to point to a rotatable kubelet client certificate and key"
    L55: "cluster.go:117: [addons] Applied essential addon: CoreDNS"
    L56: "cluster.go:117: [addons] Applied essential addon: kube-proxy"
    L57: "cluster.go:117: "
    L58: "cluster.go:117: Your Kubernetes control-plane has initialized successfully!"
    L59: "cluster.go:117: "
    L60: "cluster.go:117: To start using your cluster, you need to run the following as a regular user:"
    L61: "cluster.go:117: "
    L62: "cluster.go:117:   mkdir -p $HOME/.kube"
    L63: "cluster.go:117:   sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config"
    L64: "cluster.go:117:   sudo chown $(id -u):$(id -g) $HOME/.kube/config"
    L65: "cluster.go:117: "
    L66: "cluster.go:117: Alternatively, if you are the root user, you can run:"
    L67: "cluster.go:117: "
    L68: "cluster.go:117:   export KUBECONFIG=/etc/kubernetes/admin.conf"
    L69: "cluster.go:117: "
    L70: "cluster.go:117: You should now deploy a pod network to the cluster."
    L71: "cluster.go:117: Run __kubectl apply -f [podnetwork].yaml__ with one of the options listed at:"
    L72: "cluster.go:117:   https://kubernetes.io/docs/concepts/cluster-administration/addons/"
    L73: "cluster.go:117: "
    L74: "cluster.go:117: Then you can join any number of worker nodes by running the following on each as root:"
    L75: "cluster.go:117: "
    L76: "cluster.go:117: kubeadm join 10.0.0.59:6443 --token kmcj73.hmpunky8spjnwtio _"
    L77: "cluster.go:117:  --discovery-token-ca-cert-hash sha256:31099eb48390fdab96ef62e6b1a9bc94a6f88f86e25d0f967dc03c6bd431cacd "
    L78: "cluster.go:117: i  Using Cilium version 1.12.1"
    L79: "cluster.go:117: ? Auto-detected cluster name: kubernetes"
    L80: "cluster.go:117: ? Auto-detected datapath mode: tunnel"
    L81: "cluster.go:117: ? Auto-detected kube-proxy has been installed"
    L82: "cluster.go:117: i  helm template --namespace kube-system cilium cilium/cilium --version 1.12.1 --set cluster.id=0,cluster.name=kubernetes,encryption.nodeEncryption=false,extraConfig.cluster-pool-ipv4-?cidr=192.168.0.0/17,extraConfig.enable-endpoint-routes=true,kubeProxyReplacement=disabled,operator.replicas=1,serviceAccounts.cilium.name=cilium,serviceAccounts.operator.name=cilium-operator,tunnel=vx?lan"
    L83: "cluster.go:117: i  Storing helm values file in kube-system/cilium-cli-helm-values Secret"
    L84: "cluster.go:117: ? Created CA in secret cilium-ca"
    L85: "cluster.go:117: ? Generating certificates for Hubble..."
    L86: "cluster.go:117: ? Creating Service accounts..."
    L87: "cluster.go:117: ? Creating Cluster roles..."
    L88: "cluster.go:117: ? Creating ConfigMap for Cilium version 1.12.1..."
    L89: "cluster.go:117: i Manual overwrite in ConfigMap: enable-endpoint-routes=true"
    L90: "cluster.go:117: i Manual overwrite in ConfigMap: cluster-pool-ipv4-cidr=192.168.0.0/17"
    L91: "cluster.go:117: ? Creating Agent DaemonSet..."
    L92: "cluster.go:117: ? Creating Operator Deployment..."
    L93: "cluster.go:117: ? Waiting for Cilium to be installed and ready..."
    L94: "cluster.go:117: ? Cilium was successfully installed! Run _cilium status_ to view installation health"
    L95: "cluster.go:117: daemonset.apps/cilium patched"
    L96: "cluster.go:117: ?[33m    /??_"
    L97: "cluster.go:117: ?[36m /???[33m___/?[32m??_?[0m    Cilium:         ?[32mOK?[0m"
    L98: "cluster.go:117: ?[36m ___?[31m/??_?[32m__/?[0m    Operator:       ?[32mOK?[0m"
    L99: "cluster.go:117: ?[32m /???[31m___/?[35m??_?[0m    Hubble:         ?[36mdisabled?[0m"
    L100: "cluster.go:117: ?[32m ___?[34m/??_?[35m__/?[0m    ClusterMesh:    ?[36mdisabled?[0m"
    L101: "cluster.go:117: ?[34m    ___/"
    L102: "cluster.go:117: ?[0m"
    L103: "cluster.go:117: Deployment       cilium-operator    "
    L104: "cluster.go:117: DaemonSet        cilium             "
    L105: "cluster.go:117: Containers:      cilium             "
    L106: "cluster.go:117:                  cilium-operator    "
    L107: "cluster.go:117: Cluster Pods:    0/0 managed by Cilium"
    L108: "cluster.go:117: Created symlink /etc/systemd/system/multi-user.target.wants/kubelet.service ??? /etc/systemd/system/kubelet.service."
    L109: "--- FAIL: kubeadm.v1.24.14.cilium.base/node_readiness (91.72s)"
    L110: "kubeadm.go:295: nodes are not ready: ready nodes should be equal to 2: 1"
    L111: "--- FAIL: kubeadm.v1.24.14.cilium.base/IPSec_encryption (65.22s)"
    L112: "cluster.go:117: Error: Unable to determine status:  timeout while waiting for status to become successful: context deadline exceeded"
    L113: "cluster.go:130: __/opt/bin/cilium status --wait --wait-duration 1m__ failed: output ?[33m    /????_"
    L114: "?[36m /?????[33m___/?[32m????_?[0m    Cilium:         ?[31m1 errors?[0m, ?[33m1 warnings?[0m"
    L115: "?[36m ___?[31m/????_?[32m__/?[0m    Operator:       ?[32mOK?[0m"
    L116: "?[32m /?????[31m___/?[35m????_?[0m    Hubble:         ?[36mdisabled?[0m"
    L117: "?[32m ___?[34m/????_?[35m__/?[0m    ClusterMesh:    ?[36mdisabled?[0m"
    L118: "?[34m    ___/"
    L119: "?[0m"
    L120: "Deployment        cilium-operator    Desired: 1, Ready: ?[32m1/1?[0m, Available: ?[32m1/1?[0m"
    L121: "DaemonSet         cilium             Desired: 2, Ready: ?[33m1/2?[0m, Available: ?[33m1/2?[0m, Unavailable: ?[31m1/2?[0m"
    L122: "Containers:       cilium             Pending: ?[32m1?[0m, Running: ?[32m1?[0m"
    L123: "cilium-operator    Running: ?[32m1?[0m"
    L124: "Cluster Pods:     3/3 managed by Cilium"
    L125: "Image versions    cilium             quay.io/cilium/cilium:v1.12.1@sha256:ea2db1ee21b88127b5c18a96ad155c25485d0815a667ef77c2b7c7f31cab601b: 2"
    L126: "cilium-operator    quay.io/cilium/operator-generic:v1.12.1@sha256:93d5aaeda37d59e6c4325ff05030d7b48fabde6576478e3fdbfb9bb4a68ec4a1: 1"
    L127: "Errors:           cilium             cilium          1 pods of DaemonSet cilium are not ready"
    L128: "Warnings:         cilium             cilium-g5lx9    pod is pending, status Process exited with status 1_"
    L129: " "
    L130: " Error: _cluster.go:117: I0912 17:30:33.043531    1458 version.go:256] remote version is much newer: v1.28.1; falling back to: stable-1.24"
    L131: "cluster.go:117: [config/images] Pulled registry.k8s.io/kube-apiserver:v1.24.17"
    L132: "cluster.go:117: [config/images] Pulled registry.k8s.io/kube-controller-manager:v1.24.17"
    L133: "cluster.go:117: [config/images] Pulled registry.k8s.io/kube-scheduler:v1.24.17"
    L134: "cluster.go:117: [config/images] Pulled registry.k8s.io/kube-proxy:v1.24.17"
    L135: "cluster.go:117: [config/images] Pulled registry.k8s.io/pause:3.7"
    L136: "cluster.go:117: [config/images] Pulled registry.k8s.io/etcd:3.5.6-0"
    L137: "cluster.go:117: [config/images] Pulled registry.k8s.io/coredns/coredns:v1.8.6"
    L138: "cluster.go:117: I0912 17:30:48.437281    1624 version.go:256] remote version is much newer: v1.28.1; falling back to: stable-1.24"
    L139: "cluster.go:117: [init] Using Kubernetes version: v1.24.17"
    L140: "cluster.go:117: [preflight] Running pre-flight checks"
    L141: "cluster.go:117: [preflight] Pulling images required for setting up a Kubernetes cluster"
    L142: "cluster.go:117: [preflight] This might take a minute or two, depending on the speed of your internet connection"
    L143: "cluster.go:117: [preflight] You can also perform this action in beforehand using _kubeadm config images pull_"
    L144: "cluster.go:117: [certs] Using certificateDir folder __/etc/kubernetes/pki__"
    L145: "cluster.go:117: [certs] Generating __ca__ certificate and key"
    L146: "cluster.go:117: [certs] Generating __apiserver__ certificate and key"
    L147: "cluster.go:117: [certs] apiserver serving cert is signed for DNS names [kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local localhost] and IPs [10.96.0.1 10.0.0.1?22]"
    L148: "cluster.go:117: [certs] Generating __apiserver-kubelet-client__ certificate and key"
    L149: "cluster.go:117: [certs] Generating __front-proxy-ca__ certificate and key"
    L150: "cluster.go:117: [certs] Generating __front-proxy-client__ certificate and key"
    L151: "cluster.go:117: [certs] External etcd mode: Skipping etcd/ca certificate authority generation"
    L152: "cluster.go:117: [certs] External etcd mode: Skipping etcd/server certificate generation"
    L153: "cluster.go:117: [certs] External etcd mode: Skipping etcd/peer certificate generation"
    L154: "cluster.go:117: [certs] External etcd mode: Skipping etcd/healthcheck-client certificate generation"
    L155: "cluster.go:117: [certs] External etcd mode: Skipping apiserver-etcd-client certificate generation"
    L156: "cluster.go:117: [certs] Generating __sa__ key and public key"
    L157: "cluster.go:117: [kubeconfig] Using kubeconfig folder __/etc/kubernetes__"
    L158: "cluster.go:117: [kubeconfig] Writing __admin.conf__ kubeconfig file"
    L159: "cluster.go:117: [kubeconfig] Writing __kubelet.conf__ kubeconfig file"
    L160: "cluster.go:117: [kubeconfig] Writing __controller-manager.conf__ kubeconfig file"
    L161: "cluster.go:117: [kubeconfig] Writing __scheduler.conf__ kubeconfig file"
    L162: "cluster.go:117: [kubelet-start] Writing kubelet environment file with flags to file __/var/lib/kubelet/kubeadm-flags.env__"
    L163: "cluster.go:117: [kubelet-start] Writing kubelet configuration to file __/var/lib/kubelet/config.yaml__"
    L164: "cluster.go:117: [kubelet-start] Starting the kubelet"
    L165: "cluster.go:117: [control-plane] Using manifest folder __/etc/kubernetes/manifests__"
    L166: "cluster.go:117: [control-plane] Creating static Pod manifest for __kube-apiserver__"
    L167: "cluster.go:117: [control-plane] Creating static Pod manifest for __kube-controller-manager__"
    L168: "cluster.go:117: [control-plane] Creating static Pod manifest for __kube-scheduler__"
    L169: "cluster.go:117: [wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory __/etc/kubernetes/manifests__. This can take up to 30m0s"
    L170: "cluster.go:117: [apiclient] All control plane components are healthy after 7.004698 seconds"
    L171: "cluster.go:117: [upload-config] Storing the configuration used in ConfigMap __kubeadm-config__ in the __kube-system__ Namespace"
    L172: "cluster.go:117: [kubelet] Creating a ConfigMap __kubelet-config__ in namespace kube-system with the configuration for the kubelets in the cluster"
    L173: "cluster.go:117: [upload-certs] Skipping phase. Please see --upload-certs"
    L174: "cluster.go:117: [mark-control-plane] Marking the node localhost as control-plane by adding the labels: [node-role.kubernetes.io/control-plane node.kubernetes.io/exclude-from-external-load-balancers]"
    L175: "cluster.go:117: [mark-control-plane] Marking the node localhost as control-plane by adding the taints [node-role.kubernetes.io/master:NoSchedule node-role.kubernetes.io/control-plane:NoSchedule]"
    L176: "cluster.go:117: [bootstrap-token] Using token: 6geadt.u5via8zhggicyv4o"
    L177: "cluster.go:117: [bootstrap-token] Configuring bootstrap tokens, cluster-info ConfigMap, RBAC Roles"
    L178: "cluster.go:117: [bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to get nodes"
    L179: "cluster.go:117: [bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials"
    L180: "cluster.go:117: [bootstrap-token] Configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token"
    L181: "cluster.go:117: [bootstrap-token] Configured RBAC rules to allow certificate rotation for all node client certificates in the cluster"
    L182: "cluster.go:117: [bootstrap-token] Creating the __cluster-info__ ConfigMap in the __kube-public__ namespace"
    L183: "cluster.go:117: [kubelet-finalize] Updating __/etc/kubernetes/kubelet.conf__ to point to a rotatable kubelet client certificate and key"
    L184: "cluster.go:117: [addons] Applied essential addon: CoreDNS"
    L185: "cluster.go:117: [addons] Applied essential addon: kube-proxy"
    L186: "cluster.go:117: "
    L187: "cluster.go:117: Your Kubernetes control-plane has initialized successfully!"
    L188: "cluster.go:117: "
    L189: "cluster.go:117: To start using your cluster, you need to run the following as a regular user:"
    L190: "cluster.go:117: "
    L191: "cluster.go:117:   mkdir -p $HOME/.kube"
    L192: "cluster.go:117:   sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config"
    L193: "cluster.go:117:   sudo chown $(id -u):$(id -g) $HOME/.kube/config"
    L194: "cluster.go:117: "
    L195: "cluster.go:117: Alternatively, if you are the root user, you can run:"
    L196: "cluster.go:117: "
    L197: "cluster.go:117:   export KUBECONFIG=/etc/kubernetes/admin.conf"
    L198: "cluster.go:117: "
    L199: "cluster.go:117: You should now deploy a pod network to the cluster."
    L200: "cluster.go:117: Run __kubectl apply -f [podnetwork].yaml__ with one of the options listed at:"
    L201: "cluster.go:117:   https://kubernetes.io/docs/concepts/cluster-administration/addons/"
    L202: "cluster.go:117: "
    L203: "cluster.go:117: Then you can join any number of worker nodes by running the following on each as root:"
    L204: "cluster.go:117: "
    L205: "cluster.go:117: kubeadm join 10.0.0.122:6443 --token 6geadt.u5via8zhggicyv4o _"
    L206: "cluster.go:117:  --discovery-token-ca-cert-hash sha256:309deb859bd04b8a61b475a9d412c1a31ccb71da5dc3b391b4aea02a0c296211 "
    L207: "cluster.go:117: i  Using Cilium version 1.12.1"
    L208: "cluster.go:117: ? Auto-detected cluster name: kubernetes"
    L209: "cluster.go:117: ? Auto-detected datapath mode: tunnel"
    L210: "cluster.go:117: ? Auto-detected kube-proxy has been installed"
    L211: "cluster.go:117: i  helm template --namespace kube-system cilium cilium/cilium --version 1.12.1 --set cluster.id=0,cluster.name=kubernetes,encryption.nodeEncryption=false,extraConfig.cluster-pool-ipv4-?cidr=192.168.0.0/17,extraConfig.enable-endpoint-routes=true,kubeProxyReplacement=disabled,operator.replicas=1,serviceAccounts.cilium.name=cilium,serviceAccounts.operator.name=cilium-operator,tunnel=vx?lan"
    L212: "cluster.go:117: i  Storing helm values file in kube-system/cilium-cli-helm-values Secret"
    L213: "cluster.go:117: ? Created CA in secret cilium-ca"
    L214: "cluster.go:117: ? Generating certificates for Hubble..."
    L215: "cluster.go:117: ? Creating Service accounts..."
    L216: "cluster.go:117: ? Creating Cluster roles..."
    L217: "cluster.go:117: ? Creating ConfigMap for Cilium version 1.12.1..."
    L218: "cluster.go:117: i Manual overwrite in ConfigMap: cluster-pool-ipv4-cidr=192.168.0.0/17"
    L219: "cluster.go:117: i Manual overwrite in ConfigMap: enable-endpoint-routes=true"
    L220: "cluster.go:117: ? Creating Agent DaemonSet..."
    L221: "cluster.go:117: ? Creating Operator Deployment..."
    L222: "cluster.go:117: ? Waiting for Cilium to be installed and ready..."
    L223: "cluster.go:117: ? Cilium was successfully installed! Run _cilium status_ to view installation health"
    L224: "cluster.go:117: daemonset.apps/cilium patched"
    L225: "cluster.go:117: ?[33m    /??_"
    L226: "cluster.go:117: ?[36m /???[33m___/?[32m??_?[0m    Cilium:         ?[32mOK?[0m"
    L227: "cluster.go:117: ?[36m ___?[31m/??_?[32m__/?[0m    Operator:       ?[32mOK?[0m"
    L228: "cluster.go:117: ?[32m /???[31m___/?[35m??_?[0m    Hubble:         ?[36mdisabled?[0m"
    L229: "cluster.go:117: ?[32m ___?[34m/??_?[35m__/?[0m    ClusterMesh:    ?[36mdisabled?[0m"
    L230: "cluster.go:117: ?[34m    ___/"
    L231: "cluster.go:117: ?[0m"
    L232: "cluster.go:117: Deployment       cilium-operator    "
    L233: "cluster.go:117: DaemonSet        cilium             "
    L234: "cluster.go:117: Containers:      cilium             "
    L235: "cluster.go:117:                  cilium-operator    "
    L236: "cluster.go:117: Cluster Pods:    0/0 managed by Cilium"
    L237: "cluster.go:117: Created symlink /etc/systemd/system/multi-user.target.wants/kubelet.service ??? /etc/systemd/system/kubelet.service."
    L238: "harness.go:582: Found emergency shell on machine fb78e0fb-bf86-4151-ad76-227fd0e05ae9 console"
    L239: "harness.go:582: Found systemd unit failed to start (?[0;1;39mignition-f???es.service?[0m - Ignition (files). ) on machine fb78e0fb-bf86-4151-ad76-227fd0e05ae9 console"
    L240: "harness.go:582: Found systemd dependency unit failed to start (?[0;1;39migni???0m - Ignition (record completion). ) on machine fb78e0fb-bf86-4151-ad76-227fd0e05ae9 console_"
    L241: " "
                Diagnostic output for qemu_uefi-amd64, run 1 
    L1: " Error: _cluster.go:117: I0912 17:24:26.255769    1490 version.go:256] remote version is much newer: v1.28.1; falling back to: stable-1.24"
    L2: "cluster.go:117: [config/images] Pulled registry.k8s.io/kube-apiserver:v1.24.17"
    L3: "cluster.go:117: [config/images] Pulled registry.k8s.io/kube-controller-manager:v1.24.17"
    L4: "cluster.go:117: [config/images] Pulled registry.k8s.io/kube-scheduler:v1.24.17"
    L5: "cluster.go:117: [config/images] Pulled registry.k8s.io/kube-proxy:v1.24.17"
    L6: "cluster.go:117: [config/images] Pulled registry.k8s.io/pause:3.7"
    L7: "cluster.go:117: [config/images] Pulled registry.k8s.io/etcd:3.5.6-0"
    L8: "cluster.go:117: [config/images] Pulled registry.k8s.io/coredns/coredns:v1.8.6"
    L9: "cluster.go:117: I0912 17:24:36.758555    1652 version.go:256] remote version is much newer: v1.28.1; falling back to: stable-1.24"
    L10: "cluster.go:117: [init] Using Kubernetes version: v1.24.17"
    L11: "cluster.go:117: [preflight] Running pre-flight checks"
    L12: "cluster.go:117: [preflight] Pulling images required for setting up a Kubernetes cluster"
    L13: "cluster.go:117: [preflight] This might take a minute or two, depending on the speed of your internet connection"
    L14: "cluster.go:117: [preflight] You can also perform this action in beforehand using _kubeadm config images pull_"
    L15: "cluster.go:117: [certs] Using certificateDir folder __/etc/kubernetes/pki__"
    L16: "cluster.go:117: [certs] Generating __ca__ certificate and key"
    L17: "cluster.go:117: [certs] Generating __apiserver__ certificate and key"
    L18: "cluster.go:117: [certs] apiserver serving cert is signed for DNS names [kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local localhost] and IPs [10.96.0.1 10.0.0.5?9]"
    L19: "cluster.go:117: [certs] Generating __apiserver-kubelet-client__ certificate and key"
    L20: "cluster.go:117: [certs] Generating __front-proxy-ca__ certificate and key"
    L21: "cluster.go:117: [certs] Generating __front-proxy-client__ certificate and key"
    L22: "cluster.go:117: [certs] External etcd mode: Skipping etcd/ca certificate authority generation"
    L23: "cluster.go:117: [certs] External etcd mode: Skipping etcd/server certificate generation"
    L24: "cluster.go:117: [certs] External etcd mode: Skipping etcd/peer certificate generation"
    L25: "cluster.go:117: [certs] External etcd mode: Skipping etcd/healthcheck-client certificate generation"
    L26: "cluster.go:117: [certs] External etcd mode: Skipping apiserver-etcd-client certificate generation"
    L27: "cluster.go:117: [certs] Generating __sa__ key and public key"
    L28: "cluster.go:117: [kubeconfig] Using kubeconfig folder __/etc/kubernetes__"
    L29: "cluster.go:117: [kubeconfig] Writing __admin.conf__ kubeconfig file"
    L30: "cluster.go:117: [kubeconfig] Writing __kubelet.conf__ kubeconfig file"
    L31: "cluster.go:117: [kubeconfig] Writing __controller-manager.conf__ kubeconfig file"
    L32: "cluster.go:117: [kubeconfig] Writing __scheduler.conf__ kubeconfig file"
    L33: "cluster.go:117: [kubelet-start] Writing kubelet environment file with flags to file __/var/lib/kubelet/kubeadm-flags.env__"
    L34: "cluster.go:117: [kubelet-start] Writing kubelet configuration to file __/var/lib/kubelet/config.yaml__"
    L35: "cluster.go:117: [kubelet-start] Starting the kubelet"
    L36: "cluster.go:117: [control-plane] Using manifest folder __/etc/kubernetes/manifests__"
    L37: "cluster.go:117: [control-plane] Creating static Pod manifest for __kube-apiserver__"
    L38: "cluster.go:117: [control-plane] Creating static Pod manifest for __kube-controller-manager__"
    L39: "cluster.go:117: [control-plane] Creating static Pod manifest for __kube-scheduler__"
    L40: "cluster.go:117: [wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory __/etc/kubernetes/manifests__. This can take up to 30m0s"
    L41: "cluster.go:117: [apiclient] All control plane components are healthy after 5.503525 seconds"
    L42: "cluster.go:117: [upload-config] Storing the configuration used in ConfigMap __kubeadm-config__ in the __kube-system__ Namespace"
    L43: "cluster.go:117: [kubelet] Creating a ConfigMap __kubelet-config__ in namespace kube-system with the configuration for the kubelets in the cluster"
    L44: "cluster.go:117: [upload-certs] Skipping phase. Please see --upload-certs"
    L45: "cluster.go:117: [mark-control-plane] Marking the node localhost as control-plane by adding the labels: [node-role.kubernetes.io/control-plane node.kubernetes.io/exclude-from-external-load-balancers]"
    L46: "cluster.go:117: [mark-control-plane] Marking the node localhost as control-plane by adding the taints [node-role.kubernetes.io/master:NoSchedule node-role.kubernetes.io/control-plane:NoSchedule]"
    L47: "cluster.go:117: [bootstrap-token] Using token: kmcj73.hmpunky8spjnwtio"
    L48: "cluster.go:117: [bootstrap-token] Configuring bootstrap tokens, cluster-info ConfigMap, RBAC Roles"
    L49: "cluster.go:117: [bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to get nodes"
    L50: "cluster.go:117: [bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials"
    L51: "cluster.go:117: [bootstrap-token] Configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token"
    L52: "cluster.go:117: [bootstrap-token] Configured RBAC rules to allow certificate rotation for all node client certificates in the cluster"
    L53: "cluster.go:117: [bootstrap-token] Creating the __cluster-info__ ConfigMap in the __kube-public__ namespace"
    L54: "cluster.go:117: [kubelet-finalize] Updating __/etc/kubernetes/kubelet.conf__ to point to a rotatable kubelet client certificate and key"
    L55: "cluster.go:117: [addons] Applied essential addon: CoreDNS"
    L56: "cluster.go:117: [addons] Applied essential addon: kube-proxy"
    L57: "cluster.go:117: "
    L58: "cluster.go:117: Your Kubernetes control-plane has initialized successfully!"
    L59: "cluster.go:117: "
    L60: "cluster.go:117: To start using your cluster, you need to run the following as a regular user:"
    L61: "cluster.go:117: "
    L62: "cluster.go:117:   mkdir -p $HOME/.kube"
    L63: "cluster.go:117:   sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config"
    L64: "cluster.go:117:   sudo chown $(id -u):$(id -g) $HOME/.kube/config"
    L65: "cluster.go:117: "
    L66: "cluster.go:117: Alternatively, if you are the root user, you can run:"
    L67: "cluster.go:117: "
    L68: "cluster.go:117:   export KUBECONFIG=/etc/kubernetes/admin.conf"
    L69: "cluster.go:117: "
    L70: "cluster.go:117: You should now deploy a pod network to the cluster."
    L71: "cluster.go:117: Run __kubectl apply -f [podnetwork].yaml__ with one of the options listed at:"
    L72: "cluster.go:117:   https://kubernetes.io/docs/concepts/cluster-administration/addons/"
    L73: "cluster.go:117: "
    L74: "cluster.go:117: Then you can join any number of worker nodes by running the following on each as root:"
    L75: "cluster.go:117: "
    L76: "cluster.go:117: kubeadm join 10.0.0.59:6443 --token kmcj73.hmpunky8spjnwtio _"
    L77: "cluster.go:117:  --discovery-token-ca-cert-hash sha256:31099eb48390fdab96ef62e6b1a9bc94a6f88f86e25d0f967dc03c6bd431cacd "
    L78: "cluster.go:117: i  Using Cilium version 1.12.1"
    L79: "cluster.go:117: ? Auto-detected cluster name: kubernetes"
    L80: "cluster.go:117: ? Auto-detected datapath mode: tunnel"
    L81: "cluster.go:117: ? Auto-detected kube-proxy has been installed"
    L82: "cluster.go:117: i  helm template --namespace kube-system cilium cilium/cilium --version 1.12.1 --set cluster.id=0,cluster.name=kubernetes,encryption.nodeEncryption=false,extraConfig.cluster-pool-ipv4-?cidr=192.168.0.0/17,extraConfig.enable-endpoint-routes=true,kubeProxyReplacement=disabled,operator.replicas=1,serviceAccounts.cilium.name=cilium,serviceAccounts.operator.name=cilium-operator,tunnel=vx?lan"
    L83: "cluster.go:117: i  Storing helm values file in kube-system/cilium-cli-helm-values Secret"
    L84: "cluster.go:117: ? Created CA in secret cilium-ca"
    L85: "cluster.go:117: ? Generating certificates for Hubble..."
    L86: "cluster.go:117: ? Creating Service accounts..."
    L87: "cluster.go:117: ? Creating Cluster roles..."
    L88: "cluster.go:117: ? Creating ConfigMap for Cilium version 1.12.1..."
    L89: "cluster.go:117: i Manual overwrite in ConfigMap: enable-endpoint-routes=true"
    L90: "cluster.go:117: i Manual overwrite in ConfigMap: cluster-pool-ipv4-cidr=192.168.0.0/17"
    L91: "cluster.go:117: ? Creating Agent DaemonSet..."
    L92: "cluster.go:117: ? Creating Operator Deployment..."
    L93: "cluster.go:117: ? Waiting for Cilium to be installed and ready..."
    L94: "cluster.go:117: ? Cilium was successfully installed! Run _cilium status_ to view installation health"
    L95: "cluster.go:117: daemonset.apps/cilium patched"
    L96: "cluster.go:117: ?[33m    /??_"
    L97: "cluster.go:117: ?[36m /???[33m___/?[32m??_?[0m    Cilium:         ?[32mOK?[0m"
    L98: "cluster.go:117: ?[36m ___?[31m/??_?[32m__/?[0m    Operator:       ?[32mOK?[0m"
    L99: "cluster.go:117: ?[32m /???[31m___/?[35m??_?[0m    Hubble:         ?[36mdisabled?[0m"
    L100: "cluster.go:117: ?[32m ___?[34m/??_?[35m__/?[0m    ClusterMesh:    ?[36mdisabled?[0m"
    L101: "cluster.go:117: ?[34m    ___/"
    L102: "cluster.go:117: ?[0m"
    L103: "cluster.go:117: Deployment       cilium-operator    "
    L104: "cluster.go:117: DaemonSet        cilium             "
    L105: "cluster.go:117: Containers:      cilium             "
    L106: "cluster.go:117:                  cilium-operator    "
    L107: "cluster.go:117: Cluster Pods:    0/0 managed by Cilium"
    L108: "cluster.go:117: Created symlink /etc/systemd/system/multi-user.target.wants/kubelet.service ??? /etc/systemd/system/kubelet.service."
    L109: "--- FAIL: kubeadm.v1.24.14.cilium.base/node_readiness (91.72s)"
    L110: "kubeadm.go:295: nodes are not ready: ready nodes should be equal to 2: 1"
    L111: "--- FAIL: kubeadm.v1.24.14.cilium.base/IPSec_encryption (65.22s)"
    L112: "cluster.go:117: Error: Unable to determine status:  timeout while waiting for status to become successful: context deadline exceeded"
    L113: "cluster.go:130: __/opt/bin/cilium status --wait --wait-duration 1m__ failed: output ?[33m    /????_"
    L114: "?[36m /?????[33m___/?[32m????_?[0m    Cilium:         ?[31m1 errors?[0m, ?[33m1 warnings?[0m"
    L115: "?[36m ___?[31m/????_?[32m__/?[0m    Operator:       ?[32mOK?[0m"
    L116: "?[32m /?????[31m___/?[35m????_?[0m    Hubble:         ?[36mdisabled?[0m"
    L117: "?[32m ___?[34m/????_?[35m__/?[0m    ClusterMesh:    ?[36mdisabled?[0m"
    L118: "?[34m    ___/"
    L119: "?[0m"
    L120: "Deployment        cilium-operator    Desired: 1, Ready: ?[32m1/1?[0m, Available: ?[32m1/1?[0m"
    L121: "DaemonSet         cilium             Desired: 2, Ready: ?[33m1/2?[0m, Available: ?[33m1/2?[0m, Unavailable: ?[31m1/2?[0m"
    L122: "Containers:       cilium             Pending: ?[32m1?[0m, Running: ?[32m1?[0m"
    L123: "cilium-operator    Running: ?[32m1?[0m"
    L124: "Cluster Pods:     3/3 managed by Cilium"
    L125: "Image versions    cilium             quay.io/cilium/cilium:v1.12.1@sha256:ea2db1ee21b88127b5c18a96ad155c25485d0815a667ef77c2b7c7f31cab601b: 2"
    L126: "cilium-operator    quay.io/cilium/operator-generic:v1.12.1@sha256:93d5aaeda37d59e6c4325ff05030d7b48fabde6576478e3fdbfb9bb4a68ec4a1: 1"
    L127: "Errors:           cilium             cilium          1 pods of DaemonSet cilium are not ready"
    L128: "Warnings:         cilium             cilium-g5lx9    pod is pending, status Process exited with status 1_"
    L129: " "
    L130: " Error: _cluster.go:117: I0912 17:30:33.043531    1458 version.go:256] remote version is much newer: v1.28.1; falling back to: stable-1.24"
    L131: "cluster.go:117: [config/images] Pulled registry.k8s.io/kube-apiserver:v1.24.17"
    L132: "cluster.go:117: [config/images] Pulled registry.k8s.io/kube-controller-manager:v1.24.17"
    L133: "cluster.go:117: [config/images] Pulled registry.k8s.io/kube-scheduler:v1.24.17"
    L134: "cluster.go:117: [config/images] Pulled registry.k8s.io/kube-proxy:v1.24.17"
    L135: "cluster.go:117: [config/images] Pulled registry.k8s.io/pause:3.7"
    L136: "cluster.go:117: [config/images] Pulled registry.k8s.io/etcd:3.5.6-0"
    L137: "cluster.go:117: [config/images] Pulled registry.k8s.io/coredns/coredns:v1.8.6"
    L138: "cluster.go:117: I0912 17:30:48.437281    1624 version.go:256] remote version is much newer: v1.28.1; falling back to: stable-1.24"
    L139: "cluster.go:117: [init] Using Kubernetes version: v1.24.17"
    L140: "cluster.go:117: [preflight] Running pre-flight checks"
    L141: "cluster.go:117: [preflight] Pulling images required for setting up a Kubernetes cluster"
    L142: "cluster.go:117: [preflight] This might take a minute or two, depending on the speed of your internet connection"
    L143: "cluster.go:117: [preflight] You can also perform this action in beforehand using _kubeadm config images pull_"
    L144: "cluster.go:117: [certs] Using certificateDir folder __/etc/kubernetes/pki__"
    L145: "cluster.go:117: [certs] Generating __ca__ certificate and key"
    L146: "cluster.go:117: [certs] Generating __apiserver__ certificate and key"
    L147: "cluster.go:117: [certs] apiserver serving cert is signed for DNS names [kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local localhost] and IPs [10.96.0.1 10.0.0.1?22]"
    L148: "cluster.go:117: [certs] Generating __apiserver-kubelet-client__ certificate and key"
    L149: "cluster.go:117: [certs] Generating __front-proxy-ca__ certificate and key"
    L150: "cluster.go:117: [certs] Generating __front-proxy-client__ certificate and key"
    L151: "cluster.go:117: [certs] External etcd mode: Skipping etcd/ca certificate authority generation"
    L152: "cluster.go:117: [certs] External etcd mode: Skipping etcd/server certificate generation"
    L153: "cluster.go:117: [certs] External etcd mode: Skipping etcd/peer certificate generation"
    L154: "cluster.go:117: [certs] External etcd mode: Skipping etcd/healthcheck-client certificate generation"
    L155: "cluster.go:117: [certs] External etcd mode: Skipping apiserver-etcd-client certificate generation"
    L156: "cluster.go:117: [certs] Generating __sa__ key and public key"
    L157: "cluster.go:117: [kubeconfig] Using kubeconfig folder __/etc/kubernetes__"
    L158: "cluster.go:117: [kubeconfig] Writing __admin.conf__ kubeconfig file"
    L159: "cluster.go:117: [kubeconfig] Writing __kubelet.conf__ kubeconfig file"
    L160: "cluster.go:117: [kubeconfig] Writing __controller-manager.conf__ kubeconfig file"
    L161: "cluster.go:117: [kubeconfig] Writing __scheduler.conf__ kubeconfig file"
    L162: "cluster.go:117: [kubelet-start] Writing kubelet environment file with flags to file __/var/lib/kubelet/kubeadm-flags.env__"
    L163: "cluster.go:117: [kubelet-start] Writing kubelet configuration to file __/var/lib/kubelet/config.yaml__"
    L164: "cluster.go:117: [kubelet-start] Starting the kubelet"
    L165: "cluster.go:117: [control-plane] Using manifest folder __/etc/kubernetes/manifests__"
    L166: "cluster.go:117: [control-plane] Creating static Pod manifest for __kube-apiserver__"
    L167: "cluster.go:117: [control-plane] Creating static Pod manifest for __kube-controller-manager__"
    L168: "cluster.go:117: [control-plane] Creating static Pod manifest for __kube-scheduler__"
    L169: "cluster.go:117: [wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory __/etc/kubernetes/manifests__. This can take up to 30m0s"
    L170: "cluster.go:117: [apiclient] All control plane components are healthy after 7.004698 seconds"
    L171: "cluster.go:117: [upload-config] Storing the configuration used in ConfigMap __kubeadm-config__ in the __kube-system__ Namespace"
    L172: "cluster.go:117: [kubelet] Creating a ConfigMap __kubelet-config__ in namespace kube-system with the configuration for the kubelets in the cluster"
    L173: "cluster.go:117: [upload-certs] Skipping phase. Please see --upload-certs"
    L174: "cluster.go:117: [mark-control-plane] Marking the node localhost as control-plane by adding the labels: [node-role.kubernetes.io/control-plane node.kubernetes.io/exclude-from-external-load-balancers]"
    L175: "cluster.go:117: [mark-control-plane] Marking the node localhost as control-plane by adding the taints [node-role.kubernetes.io/master:NoSchedule node-role.kubernetes.io/control-plane:NoSchedule]"
    L176: "cluster.go:117: [bootstrap-token] Using token: 6geadt.u5via8zhggicyv4o"
    L177: "cluster.go:117: [bootstrap-token] Configuring bootstrap tokens, cluster-info ConfigMap, RBAC Roles"
    L178: "cluster.go:117: [bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to get nodes"
    L179: "cluster.go:117: [bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials"
    L180: "cluster.go:117: [bootstrap-token] Configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token"
    L181: "cluster.go:117: [bootstrap-token] Configured RBAC rules to allow certificate rotation for all node client certificates in the cluster"
    L182: "cluster.go:117: [bootstrap-token] Creating the __cluster-info__ ConfigMap in the __kube-public__ namespace"
    L183: "cluster.go:117: [kubelet-finalize] Updating __/etc/kubernetes/kubelet.conf__ to point to a rotatable kubelet client certificate and key"
    L184: "cluster.go:117: [addons] Applied essential addon: CoreDNS"
    L185: "cluster.go:117: [addons] Applied essential addon: kube-proxy"
    L186: "cluster.go:117: "
    L187: "cluster.go:117: Your Kubernetes control-plane has initialized successfully!"
    L188: "cluster.go:117: "
    L189: "cluster.go:117: To start using your cluster, you need to run the following as a regular user:"
    L190: "cluster.go:117: "
    L191: "cluster.go:117:   mkdir -p $HOME/.kube"
    L192: "cluster.go:117:   sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config"
    L193: "cluster.go:117:   sudo chown $(id -u):$(id -g) $HOME/.kube/config"
    L194: "cluster.go:117: "
    L195: "cluster.go:117: Alternatively, if you are the root user, you can run:"
    L196: "cluster.go:117: "
    L197: "cluster.go:117:   export KUBECONFIG=/etc/kubernetes/admin.conf"
    L198: "cluster.go:117: "
    L199: "cluster.go:117: You should now deploy a pod network to the cluster."
    L200: "cluster.go:117: Run __kubectl apply -f [podnetwork].yaml__ with one of the options listed at:"
    L201: "cluster.go:117:   https://kubernetes.io/docs/concepts/cluster-administration/addons/"
    L202: "cluster.go:117: "
    L203: "cluster.go:117: Then you can join any number of worker nodes by running the following on each as root:"
    L204: "cluster.go:117: "
    L205: "cluster.go:117: kubeadm join 10.0.0.122:6443 --token 6geadt.u5via8zhggicyv4o _"
    L206: "cluster.go:117:  --discovery-token-ca-cert-hash sha256:309deb859bd04b8a61b475a9d412c1a31ccb71da5dc3b391b4aea02a0c296211 "
    L207: "cluster.go:117: i  Using Cilium version 1.12.1"
    L208: "cluster.go:117: ? Auto-detected cluster name: kubernetes"
    L209: "cluster.go:117: ? Auto-detected datapath mode: tunnel"
    L210: "cluster.go:117: ? Auto-detected kube-proxy has been installed"
    L211: "cluster.go:117: i  helm template --namespace kube-system cilium cilium/cilium --version 1.12.1 --set cluster.id=0,cluster.name=kubernetes,encryption.nodeEncryption=false,extraConfig.cluster-pool-ipv4-?cidr=192.168.0.0/17,extraConfig.enable-endpoint-routes=true,kubeProxyReplacement=disabled,operator.replicas=1,serviceAccounts.cilium.name=cilium,serviceAccounts.operator.name=cilium-operator,tunnel=vx?lan"
    L212: "cluster.go:117: i  Storing helm values file in kube-system/cilium-cli-helm-values Secret"
    L213: "cluster.go:117: ? Created CA in secret cilium-ca"
    L214: "cluster.go:117: ? Generating certificates for Hubble..."
    L215: "cluster.go:117: ? Creating Service accounts..."
    L216: "cluster.go:117: ? Creating Cluster roles..."
    L217: "cluster.go:117: ? Creating ConfigMap for Cilium version 1.12.1..."
    L218: "cluster.go:117: i Manual overwrite in ConfigMap: cluster-pool-ipv4-cidr=192.168.0.0/17"
    L219: "cluster.go:117: i Manual overwrite in ConfigMap: enable-endpoint-routes=true"
    L220: "cluster.go:117: ? Creating Agent DaemonSet..."
    L221: "cluster.go:117: ? Creating Operator Deployment..."
    L222: "cluster.go:117: ? Waiting for Cilium to be installed and ready..."
    L223: "cluster.go:117: ? Cilium was successfully installed! Run _cilium status_ to view installation health"
    L224: "cluster.go:117: daemonset.apps/cilium patched"
    L225: "cluster.go:117: ?[33m    /??_"
    L226: "cluster.go:117: ?[36m /???[33m___/?[32m??_?[0m    Cilium:         ?[32mOK?[0m"
    L227: "cluster.go:117: ?[36m ___?[31m/??_?[32m__/?[0m    Operator:       ?[32mOK?[0m"
    L228: "cluster.go:117: ?[32m /???[31m___/?[35m??_?[0m    Hubble:         ?[36mdisabled?[0m"
    L229: "cluster.go:117: ?[32m ___?[34m/??_?[35m__/?[0m    ClusterMesh:    ?[36mdisabled?[0m"
    L230: "cluster.go:117: ?[34m    ___/"
    L231: "cluster.go:117: ?[0m"
    L232: "cluster.go:117: Deployment       cilium-operator    "
    L233: "cluster.go:117: DaemonSet        cilium             "
    L234: "cluster.go:117: Containers:      cilium             "
    L235: "cluster.go:117:                  cilium-operator    "
    L236: "cluster.go:117: Cluster Pods:    0/0 managed by Cilium"
    L237: "cluster.go:117: Created symlink /etc/systemd/system/multi-user.target.wants/kubelet.service ??? /etc/systemd/system/kubelet.service."
    L238: "harness.go:582: Found emergency shell on machine fb78e0fb-bf86-4151-ad76-227fd0e05ae9 console"
    L239: "harness.go:582: Found systemd unit failed to start (?[0;1;39mignition-f???es.service?[0m - Ignition (files). ) on machine fb78e0fb-bf86-4151-ad76-227fd0e05ae9 console"
    L240: "harness.go:582: Found systemd dependency unit failed to start (?[0;1;39migni???0m - Ignition (record completion). ) on machine fb78e0fb-bf86-4151-ad76-227fd0e05ae9 console_"
    L241: " "

ok kubeadm.v1.24.14.cilium.cgroupv1.base 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok kubeadm.v1.24.14.flannel.base 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok kubeadm.v1.24.14.flannel.cgroupv1.base 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok kubeadm.v1.25.10.calico.base 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (2) ❌ Failed: qemu_uefi-arm64 (1)

                Diagnostic output for qemu_uefi-arm64, run 1 
    L1: "  "
    L2: " Error: _cluster.go:117: I0912 17:35:50.850564    1480 version.go:256] remote version is much newer: v1.28.1; falling back to: stable-1.25"
    L3: "cluster.go:117: [config/images] Pulled registry.k8s.io/kube-apiserver:v1.25.13"
    L4: "cluster.go:117: [config/images] Pulled registry.k8s.io/kube-controller-manager:v1.25.13"
    L5: "cluster.go:117: [config/images] Pulled registry.k8s.io/kube-scheduler:v1.25.13"
    L6: "cluster.go:117: [config/images] Pulled registry.k8s.io/kube-proxy:v1.25.13"
    L7: "cluster.go:117: [config/images] Pulled registry.k8s.io/pause:3.8"
    L8: "cluster.go:117: [config/images] Pulled registry.k8s.io/etcd:3.5.6-0"
    L9: "cluster.go:117: [config/images] Pulled registry.k8s.io/coredns/coredns:v1.9.3"
    L10: "cluster.go:117: I0912 17:36:05.768494    1643 version.go:256] remote version is much newer: v1.28.1; falling back to: stable-1.25"
    L11: "cluster.go:117: [init] Using Kubernetes version: v1.25.13"
    L12: "cluster.go:117: [preflight] Running pre-flight checks"
    L13: "cluster.go:117: [preflight] Pulling images required for setting up a Kubernetes cluster"
    L14: "cluster.go:117: [preflight] This might take a minute or two, depending on the speed of your internet connection"
    L15: "cluster.go:117: [preflight] You can also perform this action in beforehand using _kubeadm config images pull_"
    L16: "cluster.go:117: [certs] Using certificateDir folder __/etc/kubernetes/pki__"
    L17: "cluster.go:117: [certs] Generating __ca__ certificate and key"
    L18: "cluster.go:117: [certs] Generating __apiserver__ certificate and key"
    L19: "cluster.go:117: [certs] apiserver serving cert is signed for DNS names [kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local localhost] and IPs [10.96.0.1 10.0.0.1?24]"
    L20: "cluster.go:117: [certs] Generating __apiserver-kubelet-client__ certificate and key"
    L21: "cluster.go:117: [certs] Generating __front-proxy-ca__ certificate and key"
    L22: "cluster.go:117: [certs] Generating __front-proxy-client__ certificate and key"
    L23: "cluster.go:117: [certs] External etcd mode: Skipping etcd/ca certificate authority generation"
    L24: "cluster.go:117: [certs] External etcd mode: Skipping etcd/server certificate generation"
    L25: "cluster.go:117: [certs] External etcd mode: Skipping etcd/peer certificate generation"
    L26: "cluster.go:117: [certs] External etcd mode: Skipping etcd/healthcheck-client certificate generation"
    L27: "cluster.go:117: [certs] External etcd mode: Skipping apiserver-etcd-client certificate generation"
    L28: "cluster.go:117: [certs] Generating __sa__ key and public key"
    L29: "cluster.go:117: [kubeconfig] Using kubeconfig folder __/etc/kubernetes__"
    L30: "cluster.go:117: [kubeconfig] Writing __admin.conf__ kubeconfig file"
    L31: "cluster.go:117: [kubeconfig] Writing __kubelet.conf__ kubeconfig file"
    L32: "cluster.go:117: [kubeconfig] Writing __controller-manager.conf__ kubeconfig file"
    L33: "cluster.go:117: [kubeconfig] Writing __scheduler.conf__ kubeconfig file"
    L34: "cluster.go:117: [kubelet-start] Writing kubelet environment file with flags to file __/var/lib/kubelet/kubeadm-flags.env__"
    L35: "cluster.go:117: [kubelet-start] Writing kubelet configuration to file __/var/lib/kubelet/config.yaml__"
    L36: "cluster.go:117: [kubelet-start] Starting the kubelet"
    L37: "cluster.go:117: [control-plane] Using manifest folder __/etc/kubernetes/manifests__"
    L38: "cluster.go:117: [control-plane] Creating static Pod manifest for __kube-apiserver__"
    L39: "cluster.go:117: [control-plane] Creating static Pod manifest for __kube-controller-manager__"
    L40: "cluster.go:117: [control-plane] Creating static Pod manifest for __kube-scheduler__"
    L41: "cluster.go:117: [wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory __/etc/kubernetes/manifests__. This can take up to 30m0s"
    L42: "cluster.go:117: [apiclient] All control plane components are healthy after 6.504241 seconds"
    L43: "cluster.go:117: [upload-config] Storing the configuration used in ConfigMap __kubeadm-config__ in the __kube-system__ Namespace"
    L44: "cluster.go:117: [kubelet] Creating a ConfigMap __kubelet-config__ in namespace kube-system with the configuration for the kubelets in the cluster"
    L45: "cluster.go:117: [upload-certs] Skipping phase. Please see --upload-certs"
    L46: "cluster.go:117: [mark-control-plane] Marking the node localhost as control-plane by adding the labels: [node-role.kubernetes.io/control-plane node.kubernetes.io/exclude-from-external-load-balancers]"
    L47: "cluster.go:117: [mark-control-plane] Marking the node localhost as control-plane by adding the taints [node-role.kubernetes.io/control-plane:NoSchedule]"
    L48: "cluster.go:117: [bootstrap-token] Using token: dvxda3.sfwy6tifwsmsldoj"
    L49: "cluster.go:117: [bootstrap-token] Configuring bootstrap tokens, cluster-info ConfigMap, RBAC Roles"
    L50: "cluster.go:117: [bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to get nodes"
    L51: "cluster.go:117: [bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials"
    L52: "cluster.go:117: [bootstrap-token] Configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token"
    L53: "cluster.go:117: [bootstrap-token] Configured RBAC rules to allow certificate rotation for all node client certificates in the cluster"
    L54: "cluster.go:117: [bootstrap-token] Creating the __cluster-info__ ConfigMap in the __kube-public__ namespace"
    L55: "cluster.go:117: [kubelet-finalize] Updating __/etc/kubernetes/kubelet.conf__ to point to a rotatable kubelet client certificate and key"
    L56: "cluster.go:117: [addons] Applied essential addon: CoreDNS"
    L57: "cluster.go:117: [addons] Applied essential addon: kube-proxy"
    L58: "cluster.go:117: "
    L59: "cluster.go:117: Your Kubernetes control-plane has initialized successfully!"
    L60: "cluster.go:117: "
    L61: "cluster.go:117: To start using your cluster, you need to run the following as a regular user:"
    L62: "cluster.go:117: "
    L63: "cluster.go:117:   mkdir -p $HOME/.kube"
    L64: "cluster.go:117:   sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config"
    L65: "cluster.go:117:   sudo chown $(id -u):$(id -g) $HOME/.kube/config"
    L66: "cluster.go:117: "
    L67: "cluster.go:117: Alternatively, if you are the root user, you can run:"
    L68: "cluster.go:117: "
    L69: "cluster.go:117:   export KUBECONFIG=/etc/kubernetes/admin.conf"
    L70: "cluster.go:117: "
    L71: "cluster.go:117: You should now deploy a pod network to the cluster."
    L72: "cluster.go:117: Run __kubectl apply -f [podnetwork].yaml__ with one of the options listed at:"
    L73: "cluster.go:117:   https://kubernetes.io/docs/concepts/cluster-administration/addons/"
    L74: "cluster.go:117: "
    L75: "cluster.go:117: Then you can join any number of worker nodes by running the following on each as root:"
    L76: "cluster.go:117: "
    L77: "cluster.go:117: kubeadm join 10.0.0.124:6443 --token dvxda3.sfwy6tifwsmsldoj _"
    L78: "cluster.go:117:  --discovery-token-ca-cert-hash sha256:6a078fcf8f6220620a0acf2232791a3e52365379e04dbfafaf5b6011fa1e867d "
    L79: "cluster.go:117: namespace/tigera-operator created"
    L80: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/bgpconfigurations.crd.projectcalico.org created"
    L81: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/bgppeers.crd.projectcalico.org created"
    L82: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/blockaffinities.crd.projectcalico.org created"
    L83: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/caliconodestatuses.crd.projectcalico.org created"
    L84: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/clusterinformations.crd.projectcalico.org created"
    L85: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/felixconfigurations.crd.projectcalico.org created"
    L86: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/globalnetworkpolicies.crd.projectcalico.org created"
    L87: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/globalnetworksets.crd.projectcalico.org created"
    L88: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/hostendpoints.crd.projectcalico.org created"
    L89: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/ipamblocks.crd.projectcalico.org created"
    L90: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/ipamconfigs.crd.projectcalico.org created"
    L91: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/ipamhandles.crd.projectcalico.org created"
    L92: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/ippools.crd.projectcalico.org created"
    L93: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/ipreservations.crd.projectcalico.org created"
    L94: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/kubecontrollersconfigurations.crd.projectcalico.org created"
    L95: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/networkpolicies.crd.projectcalico.org created"
    L96: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/networksets.crd.projectcalico.org created"
    L97: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/apiservers.operator.tigera.io created"
    L98: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/imagesets.operator.tigera.io created"
    L99: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/installations.operator.tigera.io created"
    L100: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/tigerastatuses.operator.tigera.io created"
    L101: "cluster.go:117: serviceaccount/tigera-operator created"
    L102: "cluster.go:117: clusterrole.rbac.authorization.k8s.io/tigera-operator created"
    L103: "cluster.go:117: clusterrolebinding.rbac.authorization.k8s.io/tigera-operator created"
    L104: "cluster.go:117: deployment.apps/tigera-operator created"
    L105: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/installations.operator.tigera.io condition met"
    L106: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/apiservers.operator.tigera.io condition met"
    L107: "cluster.go:117: installation.operator.tigera.io/default created"
    L108: "cluster.go:117: apiserver.operator.tigera.io/default created"
    L109: "cluster.go:117: Created symlink /etc/systemd/system/multi-user.target.wants/kubelet.service ??? /etc/systemd/system/kubelet.service."
    L110: "harness.go:582: Found emergency shell on machine 4d20d192-57ae-4d13-85a2-ac77584e31ed console"
    L111: "harness.go:582: Found systemd unit failed to start (?[0;1;39mignition-f???es.service?[0m - Ignition (files). ) on machine 4d20d192-57ae-4d13-85a2-ac77584e31ed console"
    L112: "harness.go:582: Found systemd dependency unit failed to start (?[0;1;39migni???te.target?[0m - Ignition Complete. ) on machine 4d20d192-57ae-4d13-85a2-ac77584e31ed console_"
    L113: " "

ok kubeadm.v1.25.10.cilium.base 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok kubeadm.v1.25.10.flannel.base 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok kubeadm.v1.26.5.calico.base 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok kubeadm.v1.26.5.cilium.base 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok kubeadm.v1.26.5.flannel.base 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok kubeadm.v1.27.2.calico.base 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok kubeadm.v1.27.2.cilium.base 🟢 Succeeded: qemu_uefi-amd64 (2); qemu_uefi-arm64 (1) ❌ Failed: qemu_uefi-amd64 (1)

                Diagnostic output for qemu_uefi-amd64, run 1 
    L1: " Error: _cluster.go:117: I0912 17:20:59.618064    1498 version.go:256] remote version is much newer: v1.28.1; falling back to: stable-1.27"
    L2: "cluster.go:117: W0912 17:20:59.839590    1498 images.go:80] could not find officially supported version of etcd for Kubernetes v1.27.5, falling back to the nearest etcd version (3.5.7-0)"
    L3: "cluster.go:117: [config/images] Pulled registry.k8s.io/kube-apiserver:v1.27.5"
    L4: "cluster.go:117: [config/images] Pulled registry.k8s.io/kube-controller-manager:v1.27.5"
    L5: "cluster.go:117: [config/images] Pulled registry.k8s.io/kube-scheduler:v1.27.5"
    L6: "cluster.go:117: [config/images] Pulled registry.k8s.io/kube-proxy:v1.27.5"
    L7: "cluster.go:117: [config/images] Pulled registry.k8s.io/pause:3.9"
    L8: "cluster.go:117: [config/images] Pulled registry.k8s.io/etcd:3.5.7-0"
    L9: "cluster.go:117: [config/images] Pulled registry.k8s.io/coredns/coredns:v1.10.1"
    L10: "cluster.go:117: I0912 17:21:10.449164    1660 version.go:256] remote version is much newer: v1.28.1; falling back to: stable-1.27"
    L11: "cluster.go:117: [init] Using Kubernetes version: v1.27.5"
    L12: "cluster.go:117: [preflight] Running pre-flight checks"
    L13: "cluster.go:117: [preflight] Pulling images required for setting up a Kubernetes cluster"
    L14: "cluster.go:117: [preflight] This might take a minute or two, depending on the speed of your internet connection"
    L15: "cluster.go:117: [preflight] You can also perform this action in beforehand using _kubeadm config images pull_"
    L16: "cluster.go:117: W0912 17:21:10.722390    1660 checks.go:835] detected that the sandbox image __registry.k8s.io/pause:3.8__ of the container runtime is inconsistent with that used by kubeadm. It is rec?ommended that using __registry.k8s.io/pause:3.9__ as the CRI sandbox image."
    L17: "cluster.go:117: [certs] Using certificateDir folder __/etc/kubernetes/pki__"
    L18: "cluster.go:117: [certs] Generating __ca__ certificate and key"
    L19: "cluster.go:117: [certs] Generating __apiserver__ certificate and key"
    L20: "cluster.go:117: [certs] apiserver serving cert is signed for DNS names [kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local localhost] and IPs [10.96.0.1 10.0.0.1?7]"
    L21: "cluster.go:117: [certs] Generating __apiserver-kubelet-client__ certificate and key"
    L22: "cluster.go:117: [certs] Generating __front-proxy-ca__ certificate and key"
    L23: "cluster.go:117: [certs] Generating __front-proxy-client__ certificate and key"
    L24: "cluster.go:117: [certs] External etcd mode: Skipping etcd/ca certificate authority generation"
    L25: "cluster.go:117: [certs] External etcd mode: Skipping etcd/server certificate generation"
    L26: "cluster.go:117: [certs] External etcd mode: Skipping etcd/peer certificate generation"
    L27: "cluster.go:117: [certs] External etcd mode: Skipping etcd/healthcheck-client certificate generation"
    L28: "cluster.go:117: [certs] External etcd mode: Skipping apiserver-etcd-client certificate generation"
    L29: "cluster.go:117: [certs] Generating __sa__ key and public key"
    L30: "cluster.go:117: [kubeconfig] Using kubeconfig folder __/etc/kubernetes__"
    L31: "cluster.go:117: [kubeconfig] Writing __admin.conf__ kubeconfig file"
    L32: "cluster.go:117: [kubeconfig] Writing __kubelet.conf__ kubeconfig file"
    L33: "cluster.go:117: [kubeconfig] Writing __controller-manager.conf__ kubeconfig file"
    L34: "cluster.go:117: [kubeconfig] Writing __scheduler.conf__ kubeconfig file"
    L35: "cluster.go:117: [kubelet-start] Writing kubelet environment file with flags to file __/var/lib/kubelet/kubeadm-flags.env__"
    L36: "cluster.go:117: [kubelet-start] Writing kubelet configuration to file __/var/lib/kubelet/config.yaml__"
    L37: "cluster.go:117: [kubelet-start] Starting the kubelet"
    L38: "cluster.go:117: [control-plane] Using manifest folder __/etc/kubernetes/manifests__"
    L39: "cluster.go:117: [control-plane] Creating static Pod manifest for __kube-apiserver__"
    L40: "cluster.go:117: [control-plane] Creating static Pod manifest for __kube-controller-manager__"
    L41: "cluster.go:117: [control-plane] Creating static Pod manifest for __kube-scheduler__"
    L42: "cluster.go:117: [wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory __/etc/kubernetes/manifests__. This can take up to 30m0s"
    L43: "cluster.go:117: [apiclient] All control plane components are healthy after 4.002550 seconds"
    L44: "cluster.go:117: [upload-config] Storing the configuration used in ConfigMap __kubeadm-config__ in the __kube-system__ Namespace"
    L45: "cluster.go:117: [kubelet] Creating a ConfigMap __kubelet-config__ in namespace kube-system with the configuration for the kubelets in the cluster"
    L46: "cluster.go:117: [upload-certs] Skipping phase. Please see --upload-certs"
    L47: "cluster.go:117: [mark-control-plane] Marking the node localhost as control-plane by adding the labels: [node-role.kubernetes.io/control-plane node.kubernetes.io/exclude-from-external-load-balancers]"
    L48: "cluster.go:117: [mark-control-plane] Marking the node localhost as control-plane by adding the taints [node-role.kubernetes.io/control-plane:NoSchedule]"
    L49: "cluster.go:117: [bootstrap-token] Using token: 1iilj3.4u9jyzg1e3d7aqz9"
    L50: "cluster.go:117: [bootstrap-token] Configuring bootstrap tokens, cluster-info ConfigMap, RBAC Roles"
    L51: "cluster.go:117: [bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to get nodes"
    L52: "cluster.go:117: [bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials"
    L53: "cluster.go:117: [bootstrap-token] Configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token"
    L54: "cluster.go:117: [bootstrap-token] Configured RBAC rules to allow certificate rotation for all node client certificates in the cluster"
    L55: "cluster.go:117: [bootstrap-token] Creating the __cluster-info__ ConfigMap in the __kube-public__ namespace"
    L56: "cluster.go:117: [kubelet-finalize] Updating __/etc/kubernetes/kubelet.conf__ to point to a rotatable kubelet client certificate and key"
    L57: "cluster.go:117: [addons] Applied essential addon: CoreDNS"
    L58: "cluster.go:117: [addons] Applied essential addon: kube-proxy"
    L59: "cluster.go:117: "
    L60: "cluster.go:117: Your Kubernetes control-plane has initialized successfully!"
    L61: "cluster.go:117: "
    L62: "cluster.go:117: To start using your cluster, you need to run the following as a regular user:"
    L63: "cluster.go:117: "
    L64: "cluster.go:117:   mkdir -p $HOME/.kube"
    L65: "cluster.go:117:   sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config"
    L66: "cluster.go:117:   sudo chown $(id -u):$(id -g) $HOME/.kube/config"
    L67: "cluster.go:117: "
    L68: "cluster.go:117: Alternatively, if you are the root user, you can run:"
    L69: "cluster.go:117: "
    L70: "cluster.go:117:   export KUBECONFIG=/etc/kubernetes/admin.conf"
    L71: "cluster.go:117: "
    L72: "cluster.go:117: You should now deploy a pod network to the cluster."
    L73: "cluster.go:117: Run __kubectl apply -f [podnetwork].yaml__ with one of the options listed at:"
    L74: "cluster.go:117:   https://kubernetes.io/docs/concepts/cluster-administration/addons/"
    L75: "cluster.go:117: "
    L76: "cluster.go:117: Then you can join any number of worker nodes by running the following on each as root:"
    L77: "cluster.go:117: "
    L78: "cluster.go:117: kubeadm join 10.0.0.17:6443 --token 1iilj3.4u9jyzg1e3d7aqz9 _"
    L79: "cluster.go:117:  --discovery-token-ca-cert-hash sha256:540fa4aa6c45c2486c20048392d15fc21e6c0bb96cd0fb8975ece53131d4c7bb "
    L80: "cluster.go:117: i  Using Cilium version 1.12.5"
    L81: "cluster.go:117: ? Auto-detected cluster name: kubernetes"
    L82: "cluster.go:117: ? Auto-detected datapath mode: tunnel"
    L83: "cluster.go:117: ? Auto-detected kube-proxy has been installed"
    L84: "cluster.go:117: i  helm template --namespace kube-system cilium cilium/cilium --version 1.12.5 --set cluster.id=0,cluster.name=kubernetes,encryption.nodeEncryption=false,extraConfig.cluster-pool-ipv4-?cidr=192.168.0.0/17,extraConfig.enable-endpoint-routes=true,kubeProxyReplacement=disabled,operator.replicas=1,serviceAccounts.cilium.name=cilium,serviceAccounts.operator.name=cilium-operator,tunnel=vx?lan"
    L85: "cluster.go:117: i  Storing helm values file in kube-system/cilium-cli-helm-values Secret"
    L86: "cluster.go:117: ? Created CA in secret cilium-ca"
    L87: "cluster.go:117: ? Generating certificates for Hubble..."
    L88: "cluster.go:117: ? Creating Service accounts..."
    L89: "cluster.go:117: ? Creating Cluster roles..."
    L90: "cluster.go:117: ? Creating ConfigMap for Cilium version 1.12.5..."
    L91: "cluster.go:117: i  Manual overwrite in ConfigMap: enable-endpoint-routes=true"
    L92: "cluster.go:117: i  Manual overwrite in ConfigMap: cluster-pool-ipv4-cidr=192.168.0.0/17"
    L93: "cluster.go:117: ? Creating Agent DaemonSet..."
    L94: "cluster.go:117: ? Creating Operator Deployment..."
    L95: "cluster.go:117: ? Waiting for Cilium to be installed and ready..."
    L96: "cluster.go:117: ? Cilium was successfully installed! Run _cilium status_ to view installation health"
    L97: "cluster.go:117: daemonset.apps/cilium patched"
    L98: "cluster.go:117: ?[33m    /??_"
    L99: "cluster.go:117: ?[36m /???[33m___/?[32m??_?[0m    Cilium:         ?[32mOK?[0m"
    L100: "cluster.go:117: ?[36m ___?[31m/??_?[32m__/?[0m    Operator:       ?[32mOK?[0m"
    L101: "cluster.go:117: ?[32m /???[31m___/?[35m??_?[0m    Hubble:         ?[36mdisabled?[0m"
    L102: "cluster.go:117: ?[32m ___?[34m/??_?[35m__/?[0m    ClusterMesh:    ?[36mdisabled?[0m"
    L103: "cluster.go:117: ?[34m    ___/"
    L104: "cluster.go:117: ?[0m"
    L105: "cluster.go:117: Deployment       cilium-operator    "
    L106: "cluster.go:117: DaemonSet        cilium             "
    L107: "cluster.go:117: Containers:      cilium             "
    L108: "cluster.go:117:                  cilium-operator    "
    L109: "cluster.go:117: Cluster Pods:    0/0 managed by Cilium"
    L110: "cluster.go:117: Created symlink /etc/systemd/system/multi-user.target.wants/kubelet.service ??? /etc/systemd/system/kubelet.service."
    L111: "--- FAIL: kubeadm.v1.27.2.cilium.base/node_readiness (91.77s)"
    L112: "kubeadm.go:295: nodes are not ready: ready nodes should be equal to 2: 1_"
    L113: " "
    L114: "  "

ok kubeadm.v1.27.2.flannel.base 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok linux.nfs.v3 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok linux.nfs.v4 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok linux.ntp 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok packages 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok skipped qemu update tests 🟢 Succeeded: qemu_update-amd64 (1)

ok systemd.journal.remote 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok systemd.journal.user 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok systemd.sysext.custom-docker 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok systemd.sysext.custom-oem 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok systemd.sysext.simple 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok systemd.sysusers.gshadow 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok torcx.enable-service 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

dongsupark
dongsupark approved these changes Sep 14, 2023
View reviewed changes
Copy link
Member

@dongsupark dongsupark left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CI passed

@dongsupark dongsupark merged commit fc3edbc into main Sep 14, 2023
@dongsupark dongsupark deleted the go-1.19.13-and-1.20.8-main branch September 14, 2023 07:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@dongsupark dongsupark dongsupark approved these changes

Assignees
No one assigned
Labels
main
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

update: go
2 participants
@flatcar-infra @dongsupark