Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

hyperv: cannot use "fe80::215:5dff:fe45:3110" as the bind address for the API Server (IPv6) #3963

Closed
xtracoder opened this issue Mar 25, 2019 · 20 comments · Fixed by #6241
Closed

hyperv: cannot use "fe80::215:5dff:fe45:3110" as the bind address for the API Server (IPv6) #3963

xtracoder opened this issue Mar 25, 2019 · 20 comments · Fixed by #6241
Assignees
@tstromberg
Labels
area/networking networking issues co/hyperv HyperV related issues help wanted Denotes an issue that needs help from a contributor. Must meet "help wanted" guidelines. kind/bug Categorizes issue or PR as related to a bug. priority/backlog Higher priority than priority/awaiting-more-evidence. priority/important-soon Must be staffed and worked on either currently, or very soon, ideally in time for the next release. top-10-issues Top 10 support issues
Milestone
v1.7.0

Comments

@xtracoder
Copy link

Problem

minikube start fails with
cannot use "fe80::215:5dff:fe45:3110" as the bind address for the API Server

command-lines

minikube config set vm-driver hyperv && minikube config set hyperv-virtual-switch MyMiniKube
minikube start --alsologtostderr --logtostderr

The full output of the command that failed

C:\Sdk\Cloud\Minikube>minikube config set vm-driver hyperv && minikube config set hyperv-virtual-switch MyMiniKube
!   These changes will take effect upon a minikube delete and then a minikube start

C:\Sdk\Cloud\Minikube>minikube start --alsologtostderr --logtostderr
I0325 23:45:43.917153    5632 notify.go:121] Checking for updates...
o   minikube v0.35.0 on windows (amd64)
I0325 23:45:44.059218    5632 start.go:582] Saving config:
{
    "MachineConfig": {
        "MinikubeISO": "https://storage.googleapis.com/minikube/iso/minikube-v0.35.0.iso",
        "Memory": 2048,
        "CPUs": 2,
        "DiskSize": 20000,
        "VMDriver": "hyperv",
        "ContainerRuntime": "docker",
        "HyperkitVpnKitSock": "",
        "HyperkitVSockPorts": [],
        "XhyveDiskDriver": "ahci-hd",
        "DockerEnv": null,
        "InsecureRegistry": null,
        "RegistryMirror": null,
        "HostOnlyCIDR": "192.168.99.1/24",
        "HypervVirtualSwitch": "MyMiniKube",
        "KvmNetwork": "default",
        "DockerOpt": null,
        "DisableDriverMounts": false,
        "NFSShare": [],
        "NFSSharesRoot": "/nfsshares",
        "UUID": "",
        "GPU": false,
        "NoVTXCheck": false
    },
    "KubernetesConfig": {
        "KubernetesVersion": "v1.13.4",
        "NodeIP": "",
        "NodePort": 8443,
        "NodeName": "minikube",
        "APIServerName": "minikubeCA",
        "APIServerNames": null,
        "APIServerIPs": null,
        "DNSDomain": "cluster.local",
        "ContainerRuntime": "docker",
        "CRISocket": "",
        "NetworkPlugin": "",
        "FeatureGates": "",
        "ServiceCIDR": "10.96.0.0/12",
        "ExtraOptions": null,
        "ShouldLoadCachedImages": false,
        "EnableDefaultCNI": false
    }
}
I0325 23:45:44.065214    5632 cluster.go:70] Machine does not exist... provisioning new machine
I0325 23:45:44.066218    5632 cluster.go:71] Provisioning machine with config: {MinikubeISO:https://storage.googleapis.com/minikube/iso/minikube-v0.35.0.iso Memory:2048 CPUs:2 DiskSize:20000 VMDriver:hyperv ContainerRuntime:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] XhyveDiskDriver:ahci-hd DockerEnv:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.99.1/24 HypervVirtualSwitch:MyMiniKube KvmNetwork:default Downloader:{} DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: GPU:false NoVTXCheck:false}
>   Creating hyperv VM (CPUs=2, Memory=2048MB, Disk=20000MB) ...
@   Downloading Minikube ISO ...
 184.42 MB / 184.42 MB [============================================] 100.00% 0s
I0325 23:47:06.246340    5632 ssh_runner.go:101] SSH: sudo rm -f /etc/docker/ca.pem
I0325 23:47:06.291438    5632 ssh_runner.go:101] SSH: sudo mkdir -p /etc/docker
I0325 23:47:06.300435    5632 ssh_runner.go:101] SSH: sudo rm -f /etc/docker/server.pem
I0325 23:47:06.304475    5632 ssh_runner.go:101] SSH: sudo mkdir -p /etc/docker
I0325 23:47:06.313440    5632 ssh_runner.go:101] SSH: sudo rm -f /etc/docker/server-key.pem
I0325 23:47:06.317468    5632 ssh_runner.go:101] SSH: sudo mkdir -p /etc/docker
-   "minikube" IP address is fe80::215:5dff:fe45:3110
I0325 23:47:18.759185    5632 start.go:582] Saving config:
{
    "MachineConfig": {
        "MinikubeISO": "https://storage.googleapis.com/minikube/iso/minikube-v0.35.0.iso",
        "Memory": 2048,
        "CPUs": 2,
        "DiskSize": 20000,
        "VMDriver": "hyperv",
        "ContainerRuntime": "docker",
        "HyperkitVpnKitSock": "",
        "HyperkitVSockPorts": [],
        "XhyveDiskDriver": "ahci-hd",
        "DockerEnv": null,
        "InsecureRegistry": null,
        "RegistryMirror": null,
        "HostOnlyCIDR": "192.168.99.1/24",
        "HypervVirtualSwitch": "MyMiniKube",
        "KvmNetwork": "default",
        "DockerOpt": null,
        "DisableDriverMounts": false,
        "NFSShare": [],
        "NFSSharesRoot": "/nfsshares",
        "UUID": "",
        "GPU": false,
        "NoVTXCheck": false
    },
    "KubernetesConfig": {
        "KubernetesVersion": "v1.13.4",
        "NodeIP": "fe80::215:5dff:fe45:3110",
        "NodePort": 8443,
        "NodeName": "minikube",
        "APIServerName": "minikubeCA",
        "APIServerNames": null,
        "APIServerIPs": null,
        "DNSDomain": "cluster.local",
        "ContainerRuntime": "docker",
        "CRISocket": "",
        "NetworkPlugin": "",
        "FeatureGates": "",
        "ServiceCIDR": "10.96.0.0/12",
        "ExtraOptions": null,
        "ShouldLoadCachedImages": false,
        "EnableDefaultCNI": false
    }
}
-   Configuring Docker as the container runtime ...
I0325 23:47:21.159696    5632 ssh_runner.go:101] SSH: systemctl is-active --quiet service containerd
I0325 23:47:21.206061    5632 ssh_runner.go:101] SSH: sudo systemctl stop containerd
I0325 23:47:21.216025    5632 ssh_runner.go:101] SSH: systemctl is-active --quiet service containerd
I0325 23:47:21.221040    5632 ssh_runner.go:101] SSH: systemctl is-active --quiet service crio
I0325 23:47:21.225054    5632 ssh_runner.go:101] SSH: systemctl is-active --quiet service rkt-api
I0325 23:47:21.230054    5632 ssh_runner.go:101] SSH: sudo systemctl stop rkt-api
I0325 23:47:21.238091    5632 ssh_runner.go:101] SSH: sudo systemctl stop rkt-metadata
I0325 23:47:21.246053    5632 ssh_runner.go:101] SSH: systemctl is-active --quiet service rkt-api
I0325 23:47:21.253626    5632 ssh_runner.go:101] SSH: sudo systemctl restart docker
-   Preparing Kubernetes environment ...
I0325 23:47:23.965028    5632 kubeadm.go:394] kubelet v1.13.4 config:

[Unit]
Wants=docker.socket

[Service]
ExecStart=
ExecStart=/usr/bin/kubelet --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --hostname-override=minikube --allow-privileged=true --cluster-dns=10.96.0.10 --authorization-mode=Webhook --fail-swap-on=false --pod-manifest-path=/etc/kubernetes/manifests --cluster-domain=cluster.local --client-ca-file=/var/lib/minikube/certs/ca.crt --cgroup-driver=cgroupfs --container-runtime=docker --kubeconfig=/etc/kubernetes/kubelet.conf

[Install]
@   Downloading kubeadm v1.13.4
@   Downloading kubelet v1.13.4
I0325 23:47:26.723958    5632 ssh_runner.go:101] SSH: sudo rm -f /usr/bin/kubeadm
I0325 23:47:26.727959    5632 ssh_runner.go:101] SSH: sudo mkdir -p /usr/bin
I0325 23:47:28.147484    5632 ssh_runner.go:101] SSH: sudo rm -f /usr/bin/kubelet
I0325 23:47:28.151458    5632 ssh_runner.go:101] SSH: sudo mkdir -p /usr/bin
I0325 23:47:28.906470    5632 ssh_runner.go:101] SSH: sudo rm -f /lib/systemd/system/kubelet.service
I0325 23:47:28.909505    5632 ssh_runner.go:101] SSH: sudo mkdir -p /lib/systemd/system
I0325 23:47:28.916471    5632 ssh_runner.go:101] SSH: sudo rm -f /etc/systemd/system/kubelet.service.d/10-kubeadm.conf
I0325 23:47:28.920478    5632 ssh_runner.go:101] SSH: sudo mkdir -p /etc/systemd/system/kubelet.service.d
I0325 23:47:28.927503    5632 ssh_runner.go:101] SSH: sudo rm -f /var/lib/kubeadm.yaml
I0325 23:47:28.931472    5632 ssh_runner.go:101] SSH: sudo mkdir -p /var/lib
I0325 23:47:28.939514    5632 ssh_runner.go:101] SSH: sudo rm -f /etc/kubernetes/addons/storageclass.yaml
I0325 23:47:28.943504    5632 ssh_runner.go:101] SSH: sudo mkdir -p /etc/kubernetes/addons
I0325 23:47:28.960504    5632 ssh_runner.go:101] SSH: sudo rm -f /etc/kubernetes/addons/storage-provisioner.yaml
I0325 23:47:28.964471    5632 ssh_runner.go:101] SSH: sudo mkdir -p /etc/kubernetes/addons
I0325 23:47:28.971469    5632 ssh_runner.go:101] SSH: sudo rm -f /etc/kubernetes/manifests/addon-manager.yaml
I0325 23:47:28.986472    5632 ssh_runner.go:101] SSH: sudo mkdir -p /etc/kubernetes/manifests/
I0325 23:47:28.994521    5632 ssh_runner.go:101] SSH:
sudo systemctl daemon-reload &&
sudo systemctl enable kubelet &&
sudo systemctl start kubelet
I0325 23:47:29.048502    5632 utils.go:224] ! Created symlink /etc/systemd/system/multi-user.target.wants/kubelet.service → /usr/lib/systemd/system/kubelet.service.
I0325 23:47:29.107501    5632 certs.go:47] Setting up certificates for IP: fe80::215:5dff:fe45:3110
I0325 23:47:29.828517    5632 ssh_runner.go:101] SSH: sudo rm -f /var/lib/minikube/certs/ca.crt
I0325 23:47:29.833473    5632 ssh_runner.go:101] SSH: sudo mkdir -p /var/lib/minikube/certs/
I0325 23:47:29.840473    5632 ssh_runner.go:101] SSH: sudo rm -f /var/lib/minikube/certs/ca.key
I0325 23:47:29.843475    5632 ssh_runner.go:101] SSH: sudo mkdir -p /var/lib/minikube/certs/
I0325 23:47:29.850505    5632 ssh_runner.go:101] SSH: sudo rm -f /var/lib/minikube/certs/apiserver.crt
I0325 23:47:29.854473    5632 ssh_runner.go:101] SSH: sudo mkdir -p /var/lib/minikube/certs/
I0325 23:47:29.861474    5632 ssh_runner.go:101] SSH: sudo rm -f /var/lib/minikube/certs/apiserver.key
I0325 23:47:29.865513    5632 ssh_runner.go:101] SSH: sudo mkdir -p /var/lib/minikube/certs/
I0325 23:47:29.873487    5632 ssh_runner.go:101] SSH: sudo rm -f /var/lib/minikube/certs/proxy-client-ca.crt
I0325 23:47:29.878473    5632 ssh_runner.go:101] SSH: sudo mkdir -p /var/lib/minikube/certs/
I0325 23:47:29.891624    5632 ssh_runner.go:101] SSH: sudo rm -f /var/lib/minikube/certs/proxy-client-ca.key
I0325 23:47:29.896599    5632 ssh_runner.go:101] SSH: sudo mkdir -p /var/lib/minikube/certs/
I0325 23:47:29.903631    5632 ssh_runner.go:101] SSH: sudo rm -f /var/lib/minikube/certs/proxy-client.crt
I0325 23:47:29.916629    5632 ssh_runner.go:101] SSH: sudo mkdir -p /var/lib/minikube/certs/
I0325 23:47:29.923631    5632 ssh_runner.go:101] SSH: sudo rm -f /var/lib/minikube/certs/proxy-client.key
I0325 23:47:29.927661    5632 ssh_runner.go:101] SSH: sudo mkdir -p /var/lib/minikube/certs/
I0325 23:47:29.943303    5632 ssh_runner.go:101] SSH: sudo rm -f /var/lib/minikube/kubeconfig
I0325 23:47:29.950260    5632 ssh_runner.go:101] SSH: sudo mkdir -p /var/lib/minikube
I0325 23:47:32.335454    5632 config.go:125] Using kubeconfig:  C:\Users\User/.kube/config
-   Pulling images required by Kubernetes v1.13.4 ...
I0325 23:47:32.337158    5632 ssh_runner.go:101] SSH: sudo kubeadm config images pull --config /var/lib/kubeadm.yaml
I0325 23:47:32.405277    5632 utils.go:224] ! failed to pull image "k8s.gcr.io/kube-apiserver:v1.13.4": output: Error response from daemon: Get https://k8s.gcr.io/v2/: dial tcp: lookup k8s.gcr.io on [2001:4860:4860::8888]:53: dial udp [2001:4860:4860::8888]:53: connect: network is unreachable
I0325 23:47:32.406292    5632 utils.go:224] ! , error: exit status 1
X   Unable to pull images, which may be OK: running cmd: sudo kubeadm config images pull --config /var/lib/kubeadm.yaml: command failed: sudo kubeadm config images pull --config /var/lib/kubeadm.yaml
stdout:
stderr: failed to pull image "k8s.gcr.io/kube-apiserver:v1.13.4": output: Error response from daemon: Get https://k8s.gcr.io/v2/: dial tcp: lookup k8s.gcr.io on [2001:4860:4860::8888]:53: dial udp [2001:4860:4860::8888]:53: connect: network is unreachable
, error: exit status 1
: Process exited with status 1
-   Launching Kubernetes v1.13.4 using kubeadm ...
I0325 23:47:32.430283    5632 ssh_runner.go:137] Run with output:
sudo /usr/bin/kubeadm init --config /var/lib/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests --ignore-preflight-errors=DirAvailable--data-minikube --ignore-preflight-errors=Port-10250 --ignore-preflight-errors=FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml --ignore-preflight-errors=FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml --ignore-preflight-errors=FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml --ignore-preflight-errors=FileAvailable--etc-kubernetes-manifests-etcd.yaml --ignore-preflight-errors=Swap --ignore-preflight-errors=CRI

I0325 23:47:32.453757    5632 utils.go:224] ! cannot use "fe80::215:5dff:fe45:3110" as the bind address for the API Server
I0325 23:47:32.453757    5632 ssh_runner.go:137] Run with output: docker ps -a --filter="name=k8s_kube-apiserver" --format="{{.ID}}"
I0325 23:47:32.497245    5632 logs.go:144] 0 containers: []
W0325 23:47:32.497245    5632 logs.go:146] No container was found matching "kube-apiserver"
I0325 23:47:32.498219    5632 ssh_runner.go:137] Run with output: docker ps -a --filter="name=k8s_coredns" --format="{{.ID}}"
I0325 23:47:32.543380    5632 logs.go:144] 0 containers: []
W0325 23:47:32.543380    5632 logs.go:146] No container was found matching "coredns"
I0325 23:47:32.548382    5632 ssh_runner.go:137] Run with output: docker ps -a --filter="name=k8s_kube-scheduler" --format="{{.ID}}"
I0325 23:47:32.588379    5632 logs.go:144] 0 containers: []
W0325 23:47:32.588379    5632 logs.go:146] No container was found matching "kube-scheduler"
I0325 23:47:32.599378    5632 logs.go:69] Gathering logs for kubelet ...
I0325 23:47:32.603392    5632 ssh_runner.go:137] Run with output: journalctl -u kubelet -n 200
I0325 23:47:32.607378    5632 utils.go:224] > -- Logs begin at Mon 2019-03-25 20:46:28 UTC, end at Mon 2019-03-25 23:46:27 UTC. --
I0325 23:47:32.607378    5632 utils.go:224] > Mar 25 20:47:29 minikube systemd[1]: Started kubelet: The Kubernetes Node Agent.
I0325 23:47:32.607378    5632 utils.go:224] > Mar 25 20:47:29 minikube kubelet[2786]: Flag --allow-privileged has been deprecated, will be removed in a future version
I0325 23:47:32.608379    5632 utils.go:224] > Mar 25 20:47:29 minikube kubelet[2786]: Flag --cluster-dns has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
I0325 23:47:32.608379    5632 utils.go:224] > Mar 25 20:47:29 minikube kubelet[2786]: Flag --authorization-mode has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
I0325 23:47:32.609393    5632 utils.go:224] > Mar 25 20:47:29 minikube kubelet[2786]: Flag --fail-swap-on has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
I0325 23:47:32.616379    5632 utils.go:224] > Mar 25 20:47:29 minikube kubelet[2786]: Flag --pod-manifest-path has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
I0325 23:47:32.631379    5632 utils.go:224] > Mar 25 20:47:29 minikube kubelet[2786]: Flag --cluster-domain has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
I0325 23:47:32.631379    5632 utils.go:224] > Mar 25 20:47:29 minikube kubelet[2786]: Flag --client-ca-file has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
I0325 23:47:32.635400    5632 utils.go:224] > Mar 25 20:47:29 minikube kubelet[2786]: Flag --cgroup-driver has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
I0325 23:47:32.636379    5632 utils.go:224] > Mar 25 20:47:29 minikube kubelet[2786]: F0325 20:47:29.371243    2786 server.go:244] unable to load client CA file /var/lib/minikube/certs/ca.crt: open /var/lib/minikube/certs/ca.crt: no such file or directory
I0325 23:47:32.637378    5632 utils.go:224] > Mar 25 20:47:29 minikube systemd[1]: kubelet.service: Main process exited, code=exited, status=255/n/a
I0325 23:47:32.637378    5632 utils.go:224] > Mar 25 20:47:29 minikube systemd[1]: kubelet.service: Failed with result 'exit-code'.
W0325 23:47:32.638390    5632 exit.go:87] Error starting cluster: kubeadm init:
sudo /usr/bin/kubeadm init --config /var/lib/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests --ignore-preflight-errors=DirAvailable--data-minikube --ignore-preflight-errors=Port-10250 --ignore-preflight-errors=FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml --ignore-preflight-errors=FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml --ignore-preflight-errors=FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml --ignore-preflight-errors=FileAvailable--etc-kubernetes-manifests-etcd.yaml --ignore-preflight-errors=Swap --ignore-preflight-errors=CRI

cannot use "fe80::215:5dff:fe45:3110" as the bind address for the API Server

: Process exited with status 1
!   Error starting cluster: kubeadm init:
sudo /usr/bin/kubeadm init --config /var/lib/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests --ignore-preflight-errors=DirAvailable--data-minikube --ignore-preflight-errors=Port-10250 --ignore-preflight-errors=FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml --ignore-preflight-errors=FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml --ignore-preflight-errors=FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml --ignore-preflight-errors=FileAvailable--etc-kubernetes-manifests-etcd.yaml --ignore-preflight-errors=Swap --ignore-preflight-errors=CRI

cannot use "fe80::215:5dff:fe45:3110" as the bind address for the API Server

: Process exited with status 1

*   Sorry that minikube crashed. If this was unexpected, we would love to hear from you:
-   https://github.com/kubernetes/minikube/issues/new
Press any key to continue . . .

The output of the "minikube logs" command, if applicable

==> kubelet <==
-- Logs begin at Mon 2019-03-25 20:46:28 UTC, end at Mon 2019-03-25 23:46:27 UTC. --
Mar 25 21:02:31 minikube systemd[1]: kubelet.service: Scheduled restart job, restart counter is at 88.
Mar 25 21:02:31 minikube systemd[1]: Stopped kubelet: The Kubernetes Node Agent.
Mar 25 21:02:31 minikube systemd[1]: Started kubelet: The Kubernetes Node Agent.
Mar 25 21:02:31 minikube kubelet[3947]: Flag --allow-privileged has been deprecated, will be removed in a future version
Mar 25 21:02:31 minikube kubelet[3947]: Flag --cluster-dns has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Mar 25 21:02:31 minikube kubelet[3947]: Flag --authorization-mode has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Mar 25 21:02:31 minikube kubelet[3947]: Flag --fail-swap-on has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Mar 25 21:02:31 minikube kubelet[3947]: Flag --pod-manifest-path has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Mar 25 21:02:31 minikube kubelet[3947]: Flag --cluster-domain has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Mar 25 21:02:31 minikube kubelet[3947]: Flag --client-ca-file has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Mar 25 21:02:31 minikube kubelet[3947]: Flag --cgroup-driver has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Mar 25 21:02:31 minikube kubelet[3947]: I0325 21:02:31.453825    3947 server.go:407] Version: v1.13.4
Mar 25 21:02:31 minikube kubelet[3947]: I0325 21:02:31.453968    3947 plugins.go:103] No cloud provider specified.
Mar 25 21:02:31 minikube kubelet[3947]: F0325 21:02:31.453991    3947 server.go:261] failed to run Kubelet: unable to load bootstrap kubeconfig: stat /etc/kubernetes/bootstrap-kubelet.conf: no such file or directory
Mar 25 21:02:31 minikube systemd[1]: kubelet.service: Main process exited, code=exited, status=255/n/a
Mar 25 21:02:31 minikube systemd[1]: kubelet.service: Failed with result 'exit-code'.
Mar 25 21:02:41 minikube systemd[1]: kubelet.service: Service hold-off time over, scheduling restart.
Mar 25 21:02:41 minikube systemd[1]: kubelet.service: Scheduled restart job, restart counter is at 89.
Mar 25 21:02:41 minikube systemd[1]: Stopped kubelet: The Kubernetes Node Agent.
Mar 25 21:02:41 minikube systemd[1]: Started kubelet: The Kubernetes Node Agent.
Mar 25 21:02:41 minikube kubelet[3960]: Flag --allow-privileged has been deprecated, will be removed in a future version
Mar 25 21:02:41 minikube kubelet[3960]: Flag --cluster-dns has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Mar 25 21:02:41 minikube kubelet[3960]: Flag --authorization-mode has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Mar 25 21:02:41 minikube kubelet[3960]: Flag --fail-swap-on has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Mar 25 21:02:41 minikube kubelet[3960]: Flag --pod-manifest-path has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Mar 25 21:02:41 minikube kubelet[3960]: Flag --cluster-domain has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Mar 25 21:02:41 minikube kubelet[3960]: Flag --client-ca-file has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Mar 25 21:02:41 minikube kubelet[3960]: Flag --cgroup-driver has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Mar 25 21:02:41 minikube kubelet[3960]: I0325 21:02:41.702605    3960 server.go:407] Version: v1.13.4
Mar 25 21:02:41 minikube kubelet[3960]: I0325 21:02:41.702746    3960 plugins.go:103] No cloud provider specified.
Mar 25 21:02:41 minikube kubelet[3960]: F0325 21:02:41.702790    3960 server.go:261] failed to run Kubelet: unable to load bootstrap kubeconfig: stat /etc/kubernetes/bootstrap-kubelet.conf: no such file or directory
Mar 25 21:02:41 minikube systemd[1]: kubelet.service: Main process exited, code=exited, status=255/n/a
Mar 25 21:02:41 minikube systemd[1]: kubelet.service: Failed with result 'exit-code'.
Mar 25 21:02:51 minikube systemd[1]: kubelet.service: Service hold-off time over, scheduling restart.
Mar 25 21:02:51 minikube systemd[1]: kubelet.service: Scheduled restart job, restart counter is at 90.
Mar 25 21:02:51 minikube systemd[1]: Stopped kubelet: The Kubernetes Node Agent.
Mar 25 21:02:51 minikube systemd[1]: Started kubelet: The Kubernetes Node Agent.
Mar 25 21:02:51 minikube kubelet[3973]: Flag --allow-privileged has been deprecated, will be removed in a future version
Mar 25 21:02:51 minikube kubelet[3973]: Flag --cluster-dns has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Mar 25 21:02:51 minikube kubelet[3973]: Flag --authorization-mode has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Mar 25 21:02:51 minikube kubelet[3973]: Flag --fail-swap-on has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Mar 25 21:02:51 minikube kubelet[3973]: Flag --pod-manifest-path has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Mar 25 21:02:51 minikube kubelet[3973]: Flag --cluster-domain has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Mar 25 21:02:51 minikube kubelet[3973]: Flag --client-ca-file has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Mar 25 21:02:51 minikube kubelet[3973]: Flag --cgroup-driver has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Mar 25 21:02:51 minikube kubelet[3973]: I0325 21:02:51.953366    3973 server.go:407] Version: v1.13.4
Mar 25 21:02:51 minikube kubelet[3973]: I0325 21:02:51.953466    3973 plugins.go:103] No cloud provider specified.
Mar 25 21:02:51 minikube kubelet[3973]: F0325 21:02:51.953484    3973 server.go:261] failed to run Kubelet: unable to load bootstrap kubeconfig: stat /etc/kubernetes/bootstrap-kubelet.conf: no such file or directory
Mar 25 21:02:51 minikube systemd[1]: kubelet.service: Main process exited, code=exited, status=255/n/a
Mar 25 21:02:51 minikube systemd[1]: kubelet.service: Failed with result 'exit-code'.

Which operating system version was used

Windows 1809 Build 17763.379

Notes

It looks like IPv4 is not initialized within virtual machine.
This gets fixed when I connect to console in HyperV, login via docker/tcuser and execute ping k8s.gcr.io.

After that when I ssh into virtual host and execute sudo kubeadm config images pull --config /var/lib/kubeadm.yaml it finishes OK:

$ sudo kubeadm config images pull --config /var/lib/kubeadm.yaml
[config/images] Pulled k8s.gcr.io/kube-apiserver:v1.13.4
[config/images] Pulled k8s.gcr.io/kube-controller-manager:v1.13.4
[config/images] Pulled k8s.gcr.io/kube-scheduler:v1.13.4
[config/images] Pulled k8s.gcr.io/kube-proxy:v1.13.4
[config/images] Pulled k8s.gcr.io/pause:3.1
[config/images] Pulled k8s.gcr.io/etcd:3.2.24
[config/images] Pulled k8s.gcr.io/coredns:1.2.6

But ... problem with cannot use "fe80::215:5dff:fe45:3110" as the bind address for the API Server still present when i manually re-execute sudo /usr/bin/kubeadm init <... see full command in the log above ...>
@balopat balopat added co/hyperv HyperV related issues kind/bug Categorizes issue or PR as related to a bug. area/networking networking issues priority/backlog Higher priority than priority/awaiting-more-evidence. labels Apr 4, 2019
@tstromberg tstromberg changed the title cannot use "fe80::215:5dff:fe45:3110" as the bind address for the API Server hyperv: cannot use "fe80::215:5dff:fe45:3110" as the bind address for the API Server May 23, 2019
@tstromberg tstromberg added the r/2019q2 Issue was last reviewed 2019q2 label May 23, 2019
@jwardle
Copy link

jwardle commented Jun 4, 2019

Also seeing the same issue, same Wwin v1809 version. Has anyone been able to resolve it?

@dhiwakarK
Copy link

Seems to be same for me - I'm trying to spin off Minikube in Azure Windows Server 2019; and am using similar command as above mentioned for spinning off the minikube using HyperV

minikube start --vm-driver hyperv --hyperv-virtual-switch "NATSwitch" --alsologtostderr --logtostderr

_I0605 16:53:33.682694 5220 utils.go:240] > [Jun 5 16:53] systemd-fstab-generator[4344]: Ignoring "noauto" for root device
I0605 16:53:33.682694 5220 utils.go:240] > [ +21.257636] systemd-fstab-generator[4724]: Ignoring "noauto" for root device
W0605 16:53:33.682694 5220 exit.go:100] Error restarting cluster: running cmd: sudo kubeadm init phase certs all --config
/var/lib/kubeadm.yaml: command failed: sudo kubeadm init phase certs all --config /var/lib/kubeadm.yaml
stdout:
stderr: cannot use "fe80::215:5dff:fe01:402" as the bind address for the API Server
: Process exited with status 1
X Error restarting cluster: running cmd: sudo kubeadm init phase certs all --config /var/lib/kubeadm.yaml: command failed: sudo kubeadm init
phase certs all --config /var/lib/kubeadm.yaml
stdout:
stderr: cannot use "fe80::215:5dff:fe01:402" as the bind address for the API Server
: Process exited with status 1

@bdschaap
Copy link

bdschaap commented Jun 7, 2019

I'm experiencing the same issue running an Azure VM with Windows Server 2019.

@subodhsohoni
Copy link

I am also getting the same error / issue while running the command on Azure VM Windows Server 2016 Datacenter build 1607.

@odytrice
Copy link

Had the same problem, was able to work around it using the "Default Switch" instead. Hope someone finds this useful

@jasondunken
Copy link

I was getting similar error while using either a user-defined vSwitch or the default vSwitch. Disabling IPv6 in the vAdapter settings, running 'minikube delete' and then restarting resolved the issue.

@tstromberg tstromberg mentioned this issue Jul 16, 2019
Closed
@tstromberg tstromberg added the help wanted Denotes an issue that needs help from a contributor. Must meet "help wanted" guidelines. label Jul 16, 2019
This was referenced Jul 16, 2019
Closed
Closed
Closed
@laozc
Copy link
Contributor

laozc commented Jul 17, 2019

I think in some case that when an IPv4 address cannot be assigned this could happen.
I could deliver a fix to use the default switch by default.

@tstromberg tstromberg mentioned this issue Aug 13, 2019
Closed
@venkatasandu
Copy link

I was getting similar error while using either a user-defined vSwitch or the default vSwitch. Disabling IPv6 in the vAdapter settings, running 'minikube delete' and then restarting resolved the issue.

followed this suggestion. Thank you. this is working fine.

@FarhadMohseni
Copy link

in addition to @odytrice ,s comment , use minikube delete and restart with "Default Switch"

@tstromberg tstromberg mentioned this issue Aug 25, 2019
Closed
@tstromberg tstromberg mentioned this issue Sep 19, 2019
Closed
@sharifelgamal
Copy link
Collaborator

Currently, the best workaround if you see IPv6 addresses trying to get passed through is to disable IPv6 in your switch settings. We're looking into either better documentation or a way to fix it on our end to avoid this happening at all.

The good news is the minikube 1.4 now picked the default switch automatically and no longer requires a user-defined external switch at all.

@tstromberg tstromberg removed the r/2019q2 Issue was last reviewed 2019q2 label Sep 20, 2019
@tstromberg
Copy link
Contributor

Apparently this can still happen with the default switch. Anyone have a tutorial on how to disable IPv6 for the default switch in Hyper-V?

@tstromberg tstromberg changed the title hyperv: cannot use "fe80::215:5dff:fe45:3110" as the bind address for the API Server hyperv: cannot use "fe80::215:5dff:fe45:3110" as the bind address for the API Server (IPv6) Sep 28, 2019
@tstromberg tstromberg mentioned this issue Sep 28, 2019
Closed
@abshaheen
Copy link

Apparently this can still happen with the default switch. Anyone have a tutorial on how to disable IPv6 for the default switch in Hyper-V?

I found this link useful to explain the workaround

https://medium.com/@JockDaRock/disabling-ipv6-on-network-adapter-windows-10-5fad010bca75

@Bowman74
Copy link

Bowman74 commented Nov 5, 2019

Whenever I encounter this I remove the shared virtual switch from the main network adapter and re-add it. I then disable and re-enable the main network adapter and everything starts working when I attempt to do a kubectl start again.

Not ideal but it gets me back up and running every time.

This was referenced Nov 6, 2019
Open
Closed
@brainfull
Copy link

I was getting that exact error when I was rebooting my Window 10 Pro laptop. On my side I use ICS with internal VM Switch. Even if ICS seems configured correctly after reboot, starting minikube would give me that exact same error. That is why I need to reset the ICS setting after each reboot. Actually I ended up automating a call to Set-ICS.ps1 before each 'minikube start' to ensure I don't get that problem anymore. This script that you can see below is disabling ICS and re-enabling it automatically.

  1. Use internal VM Switch. You can set it up easily with the following powershell. It will take care of creating the VM switch if it doesn't exist and establish ICS between your internet connection and the internal VM Switch.
    Set-ICS.ps1.txt
    Open Powershell and call the script. In the following example it creates a VM Switch named 'minikube':
    ./Set-ICS.ps1 -VMSwitch minikube Enabled

  2. Create your minikube VM. Open Powershell and call the following command. In the following example it creates a VM named 'minikube' using the VM switch named 'minikube':
    minikube start --vm-driver hyperv --hyperv-virtual-switch minikube

  3. From that point on, your VM 'minikube' is available internally in your computer under the hostname (VM Name).mshome.net, if you followed the previous instructions that is 'minikube.mshome.net'. It is ICS DHCP server that takes care of defining that hostname under C:\Windows\System32\drivers\etc\hosts.ics

I hope it gets you to a working solution for your setup. Definitely there is a lack of support about minikube on Windows. But I am committed to use it since it allows for greater productivity overall.

Have a look at this issue if you wonder why I use an internal VM Switch #5072 .

@tstromberg tstromberg mentioned this issue Nov 6, 2019
Closed
@tstromberg tstromberg added the top-10-issues Top 10 support issues label Nov 6, 2019
@brainfull brainfull mentioned this issue Nov 7, 2019
Closed
@gbraad
Copy link
Contributor

gbraad commented Nov 19, 2019

Apparently this can still happen with the default switch. Anyone have a tutorial on how to disable IPv6 for the default switch in Hyper-V?

Can you try:

PS> Set-NetIPInterface -InterfaceAlias 'vEthernet (Default Switch)' -AddressFamily IPv6 -dhcp disable

@chenmin1992
Copy link

chenmin1992 commented Dec 16, 2019 •
edited
Loading

Still suffering from this issue. :(
How to prevent VM from binding an ipv6 IP?
Is anyone still working on this issue?
I have tried all those methods, but nothing worked.

reg add "HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Tcpip6\Parameters" /v DisabledComponents /t REG_DWORD /d 0xffffffff /f
netsh interface ipv6 set global randomizeidentifiers=disable
Disable-NetAdapterBinding -Name "*" -ComponentID ms_tcpip6

@tstromberg tstromberg mentioned this issue Jan 8, 2020
Closed
@tstromberg tstromberg added the priority/important-soon Must be staffed and worked on either currently, or very soon, ideally in time for the next release. label Jan 8, 2020
@tstromberg tstromberg added this to the v1.7.0 milestone Jan 8, 2020
@tstromberg tstromberg self-assigned this Jan 8, 2020
@afbjorklund afbjorklund mentioned this issue Jan 9, 2020
Merged
@medyagh
Copy link
Member

medyagh commented Jan 10, 2020

I believe part of this is our kubeadm templates that uses the ipv4 format.

@dschafhauser
Copy link

dschafhauser commented Jan 13, 2020 •
edited
Loading

I'm a bit confused about the fix #6241. Don't the build logs for Hyper-V on Windows exhibit exactly the problem of this issue?

build log snippet 
[...]
I0109 13:12:56.357630    6564 main.go:110] libmachine: [stderr =====>] : 
I0109 13:12:56.357630    6564 main.go:110] libmachine: [executing ==>] : C:\WINDOWS\System32\WindowsPowerShell\v1.0\powershell.exe -NoProfile -NonInteractive (( Hyper-V\Get-VM old-docker-20200109T131143.4985152-1008 ).networkadapters[0]).ipaddresses[0]
I0109 13:12:57.236539    6564 main.go:110] libmachine: [stdout =====>] : 2401:fa00:480:7a01:215:5dff:fe6a:4579

I0109 13:12:57.236539    6564 main.go:110] libmachine: [stderr =====>] : 
I0109 13:12:57.239545    6564 main.go:110] libmachine: Using SSH client type: native
I0109 13:12:57.240543    6564 main.go:110] libmachine: &{{{<nil> 0 [] [] []} docker [0x7c3b10] 0x7c3ae0 <nil>  [] 0s} 2401:fa00:480:7a01:215:5dff:fe6a:4579 22 <nil> <nil>}
I0109 13:12:57.240543    6564 main.go:110] libmachine: About to run SSH command:
sudo hostname old-docker-20200109T131143.4985152-1008 && echo "old-docker-20200109T131143.4985152-1008" | sudo tee /etc/hostname
I0109 13:12:57.240543    6564 main.go:110] libmachine: Error dialing TCP: dial tcp [2401:fa00:480:7a01:215:5dff:fe6a:4579]:22: connectex: A socket operation was attempted to an unreachable network.
I0109 13:13:00.243006    6564 main.go:110] libmachine: Error dialing TCP: dial tcp [2401:fa00:480:7a01:215:5dff:fe6a:4579]:22: connectex: A socket operation was attempted to an unreachable network.
I0109 13:13:03.243550    6564 main.go:110] libmachine: Error dialing TCP: dial tcp [2401:fa00:480:7a01:215:5dff:fe6a:4579]:22: connectex: A socket operation was attempted to an unreachable network.
I0109 13:13:06.248793    6564 main.go:110] libmachine: Error dialing TCP: dial tcp [2401:fa00:480:7a01:215:5dff:fe6a:4579]:22: connectex: A socket operation was attempted to an unreachable network.
I0109 13:13:09.250487    6564 main.go:110] libmachine: Error dialing TCP: dial tcp [2401:fa00:480:7a01:215:5dff:fe6a:4579]:22: connectex: A socket operation was attempted to an unreachable network.
I0109 13:13:12.251298    6564 main.go:110] libmachine: Error dialing TCP: dial tcp [2401:fa00:480:7a01:215:5dff:fe6a:4579]:22: connectex: A socket operation was attempted to an unreachable network.
I0109 13:13:15.251678    6564 main.go:110] libmachine: Error dialing TCP: dial tcp [2401:fa00:480:7a01:215:5dff:fe6a:4579]:22: connectex: A socket operation was attempted to an unreachable network.
I0109 13:13:18.252431    6564 main.go:110] libmachine: Error dialing TCP: dial tcp [2401:fa00:480:7a01:215:5dff:fe6a:4579]:22: connectex: A socket operation was attempted to an unreachable network.
I0109 13:13:21.252659    6564 main.go:110] libmachine: Error dialing TCP: dial tcp [2401:fa00:480:7a01:215:5dff:fe6a:4579]:22: connectex: A socket operation was attempted to an unreachable network.
[...]

I've also tried the windows executable of this build myself and the issue still persists.

Edit: More precisely, the build logs reflect the error described in #5774, which is also the exact error I am facing currently.

Edit2: Hm, if I understand it correctly, then the built executable and the tests apparently don't use the "minikube.iso" of #6241 (containing the fix), which explains my observations above.

So, I tried starting minikube by providing the iso manually:
minikube start --logtostderr --iso-url "https://storage.googleapis.com/minikube-builds/6241/minikube.iso"
This yields an endless loop of the following log entries for me:

[...]
I0113 21:42:46.714846    6912 main.go:110] libmachine: [stderr =====>] :
I0113 21:42:46.716810    6912 main.go:110] libmachine: [executing ==>] : C:\WINDOWS\System32\WindowsPowerShell\v1.0\powershell.exe -NoProfile -NonInteractive (( Hyper-V\Get-VM minikube ).networkadapters[0]).ipaddresses[0]
I0113 21:42:47.879545    6912 main.go:110] libmachine: [stdout =====>] :
I0113 21:42:47.879545    6912 main.go:110] libmachine: [stderr =====>] :
I0113 21:42:48.881802    6912 main.go:110] libmachine: [executing ==>] : C:\WINDOWS\System32\WindowsPowerShell\v1.0\powershell.exe -NoProfile -NonInteractive ( Hyper-V\Get-VM minikube ).state
I0113 21:42:49.782521    6912 main.go:110] libmachine: [stdout =====>] : Running
[...]

I am not familiar with what is happening underneath, but from what I gathered from this and related issues is, that on my machine only an IPv6 address is provided to minikube, but never an IPv4 address.
Are there some ways for me to troubleshoot this?

@alexndrk
Copy link

alexndrk commented Feb 7, 2020

/reopen
Hello! I am facing the cimilar issue. Endless loop of IP gathering as dschafhauser said. Diabled IPV6 in iso image seems like not enogth solution for this

@k8s-ci-robot
Copy link
Contributor

@AlexKulizhnikov: You can't reopen an issue/PR unless you authored it or you are a collaborator.

In response to this:

/reopen
Hello! I am facing the cimilar issue. Endless loop of IP gathering as dschafhauser said. Diabled IPV6 in iso image seems like not enogth solution for this

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@tstromberg tstromberg

Labels
area/networking networking issues co/hyperv HyperV related issues help wanted Denotes an issue that needs help from a contributor. Must meet "help wanted" guidelines. kind/bug Categorizes issue or PR as related to a bug. priority/backlog Higher priority than priority/awaiting-more-evidence. priority/important-soon Must be staffed and worked on either currently, or very soon, ideally in time for the next release. top-10-issues Top 10 support issues
Projects
None yet
Milestone
v1.7.0