Error starting kubernetes - waiting for services

[dandavids]

Actual Behavior

###Kubernetes Error
Error starting Kubernetes
Error: C:\Users\17422\AppData\Local\Programs\Rancher Desktop\resources\resources\win32\bin\kubectl.exe exited with code 255

Context:
Waiting for services

Some recent logfile lines:
2022-03-09T19:54:37.955Z: Running command wsl --distribution rancher-desktop --exec busybox cp /mnt/c/Users/17422/AppData/Local/Temp/1/rd-k3s-dnR9yg/k3s /etc/conf.d/k3s...
2022-03-09T19:54:38.664Z: Running command wsl --distribution rancher-desktop --exec busybox chmod 644 /etc/conf.d/k3s...
2022-03-09T19:54:39.372Z: Running command wsl --distribution rancher-desktop --exec /sbin/rc-update --update...
2022-03-09T19:54:40.041Z: Running command wsl --distribution rancher-desktop --exec /usr/local/bin/wsl-service k3s start...
2022-03-09T19:54:54.695Z: Running command wsl --distribution rancher-desktop --exec cat /proc/net/route...
2022-03-09T19:54:56.279Z: Running command wsl --distribution rancher-desktop --exec cat /proc/net/fib_trie...
2022-03-09T19:55:18.939Z: Running command wsl --distribution rancher-desktop --exec /bin/sh -c if test -r /etc/rancher/k3s/k3s.yaml; then echo yes; else echo no; fi...
2022-03-09T19:55:19.998Z: Running command wsl --distribution rancher-desktop --exec wslpath -a -u C:\Users\17422\AppData\Local\Programs\Rancher Desktop\resources\resources\linux\wsl-helper...
2022-03-09T19:55:21.182Z: Running command wsl --distribution rancher-desktop --exec /mnt/c/Users/17422/AppData/Local/Programs/Rancher Desktop/resources/resources/linux/wsl-helper k3s kubeconfig...

Steps to Reproduce

Download Rancher Desktop version 1.1.1
Install Rancher Desktop
Choose option to start/run Rancher Desktop at end of installation.

Result

Please see attached image of error and description of error above under ###Actual Behavior

Expected Behavior

Rancher desktop up and running.

Additional Information

Please see attached screenshot

Rancher Desktop Version

1.1.1

Rancher Desktop K8s Version

1.20.15

Which container runtime are you using?

containerd (nerdctl)

What operating system are you using?

Windows

Operating System / Build Version

Windows 10 Enterprise 64bit Build:20h2 (19042)

What CPU architecture are you using?

x64

Linux only: what package format did you use to install Rancher Desktop?

No response

Windows User Only

On VPN - company.
McAfee and Sentinalone endpoint security.
Working behind company proxy here - I've asked our engineering team to add all of the below endpoints to proxy auth bypass rule. Got below info from here - #1598

https://update.k3s.io/v1-release/channels (to get the preferred k3s versions)
https://api.github.com/repos/k3s-io/k3s/releases (to get all possible k3s versions)
https://github.com/k3s-io/k3s/releases/download/* (actual K3s artifacts)
https://objects.githubusercontent.com/ (redirect of the above)
https://desktop.version.rancher.io/v1/checkupgrade (upgrade checks)

[mook-as]

Could you attach background.log? (You can find it by going to the Troubleshooting tab, and clicking on the button to show logs.)

The log we do include is, unfortunately, a red herring here; the actual error was shown in the screen shot:

kubectl.exe exited with code 255

I suspect this may be because of flavio/kuberlr#22 — in which case we need to use a newer kuberlr.

Thanks!

[dandavids]

Thank you @mook-as! Please see attached background.log
background.log

[mook-as]

Hmm, that just has:

request to https://desktop.version.rancher.io/v1/checkupgrade failed, reason: getaddrinfo ENOTFOUND desktop.version.rancher.io

That should just mean you can't fetch updates; it shouldn't have any effect on kubectl.

Does any of the other logs have anything relevant?

[NullpointerForever]

I have the same issue. I'm using VPN Cisco and working behind the company proxy. When I disconnect from my VPN it works.

[venkatfromgithub]

I have the similar issue in Rancher Desktop 1.1.1 with VPN Cisco (4.8.03036)....Unable to connect to the (Kubernetes) server: Service Unavailable. Can you please help me on this?

[iliuta]

Hi,

Running into similar issue here.
I installed Rancher Desktop 1.1.1 on a Windows 11 box and trying to setup it with a company Cisco VPN connection (Cisco any connect) behind a proxy.

We're using dockerd in order to be able to use docker CLI.

I used wsl-vpnkit in order to make network connection work under VPN and it seems okay, I am able to download stuff from the internet from a Linux prompt.

I set up the proxy connection by adding the appropriate env vars (https_proxy, HTTPS_PROXY) in /etc/rc.conf, as it follows:

wsl -d rancher-desktop
vi /etc/rc.conf  

rc_env_allow="http_proxy http_proxy no_proxy HTTP_PROXY HTTPS_PROXY" 
export HTTP_PROXY="http://username:***@proxy.company.com:3128"
export HTTPS_PROXY="http://username:***@proxy.company.com:3128"
export http_proxy="http://username:***@proxy.company.com:3128"
export https_proxy="http://username:***@proxy.company.com:3128"

Rancher desktop starts but is waiting for Kubernetes.
dockerd is available during this startup and I am able to perform docker pull & docker run
Kubernetes services seem ok as well as "docker ps" command states:

PS C:\Users\adria> docker ps
CONTAINER ID   IMAGE                        COMMAND                  CREATED              STATUS              PORTS     NAMES
c49877d16ebd   f73640fb5061                 "/metrics-server --c…"   52 seconds ago       Up 52 seconds                 k8s_metrics-server_metrics-server-ff9dbcb6c-w2hf6_kube-system_aaaedaa2-84e5-458c-9bb9-8ee61cf80c06_30
bbbcb0ef70ca   746788bcc27e                 "entry"                  About a minute ago   Up About a minute             k8s_lb-port-443_svclb-traefik-cs6kk_kube-system_34a3b58d-474f-40ce-aa16-9d8233902162_26
a48fa7c716f7   746788bcc27e                 "entry"                  About a minute ago   Up About a minute             k8s_lb-port-80_svclb-traefik-cs6kk_kube-system_34a3b58d-474f-40ce-aa16-9d8233902162_26
fcfb5b306be5   a4ca41631cc7                 "/coredns -conf /etc…"   About a minute ago   Up About a minute             k8s_coredns_coredns-96cc4f57d-kbt9q_kube-system_e1b4ab08-1c9f-4d08-8d2e-c06abca7940b_26
008ab8b39fe5   fb9b574e03c3                 "local-path-provisio…"   About a minute ago   Up About a minute             k8s_local-path-provisioner_local-path-provisioner-84bb864455-xv9jb_kube-system_768eb1e8-f593-46bd-9802-0b19090109c7_29
8cba7bd9ddcd   eddc86c9f0a0                 "/entrypoint.sh --gl…"   About a minute ago   Up About a minute             k8s_traefik_traefik-56c4b88c4b-f88mh_kube-system_78514c26-8910-42b7-92ae-18f32b5b984f_26
7394a9ad89eb   rancher/mirrored-pause:3.6   "/pause"                 About a minute ago   Up About a minute             k8s_POD_coredns-96cc4f57d-kbt9q_kube-system_e1b4ab08-1c9f-4d08-8d2e-c06abca7940b_202
3b48960c38c8   rancher/mirrored-pause:3.6   "/pause"                 About a minute ago   Up About a minute             k8s_POD_metrics-server-ff9dbcb6c-w2hf6_kube-system_aaaedaa2-84e5-458c-9bb9-8ee61cf80c06_198
1becf7c6d8d1   rancher/mirrored-pause:3.6   "/pause"                 About a minute ago   Up About a minute             k8s_POD_local-path-provisioner-84bb864455-xv9jb_kube-system_768eb1e8-f593-46bd-9802-0b19090109c7_184
18acb981145d   rancher/mirrored-pause:3.6   "/pause"                 About a minute ago   Up About a minute             k8s_POD_svclb-traefik-cs6kk_kube-system_34a3b58d-474f-40ce-aa16-9d8233902162_118
fe3bb0ca75e3   rancher/mirrored-pause:3.6   "/pause"                 About a minute ago   Up About a minute             k8s_POD_traefik-56c4b88c4b-f88mh_kube-system_78514c26-8910-42b7-92ae-18f32b5b984f_201

After a while (more than 5 min), I get this error in the Rancher Desktop GUI:

Error: connect ETIMEDOUT 172.31.151.118:6443

2022-03-18T08:22:41.952Z: Running command wsl --distribution Ubuntu --exec readlink -f $HOME/.docker/cli-plugins/docker-compose...
2022-03-18T08:22:41.952Z: Capturing output: wsl.exe --distribution Ubuntu --exec readlink -f $HOME/.docker/cli-plugins/docker-compose
2022-03-18T08:22:42.013Z: WSL: executing: readlink -f $HOME/.docker/cli-plugins/docker-compose: Error: wsl.exe exited with code 1
2022-03-18T08:22:42.013Z: Failed to readlink/rm $HOME/.docker/cli-plugins/docker-compose Error: wsl.exe exited with code 1
    at ChildProcess.<anonymous> (C:\Users\adria\AppData\Local\Programs\Rancher Desktop\resources\app.asar\dist\app\background.js:1:8692)
    at ChildProcess.emit (node:events:394:28)
    at Process.ChildProcess._handle.onexit (node:internal/child_process:290:12) {
  stdout: ''
}

Kubectl command is unable to connect to the kube cluster on Rancher desktop:

PS C:\Users\adria> kubectl get pods
I0318 09:59:40.149959    1452 versioner.go:56] Remote kubernetes server unreachable
Unable to connect to the server: dial tcp 172.31.151.118:6443: connectex: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.

[chrissssss]

Hi! I'm experiencing the same problem - somehow rancher/kubectl tries to connect to the kubernetes cluster daemon using the proxy - you have to put the IP of WSL into the NO_PROXY variable.

Kind of a problem: this ip changes often - so I would like to find out which ip subnet wsl is choosing from, and that you can put in here. Help for that appreciated ;)

For your problem, you can at least work around putting the wsl ip in your NO_PROXY variable (and check that after each restart that it hasnt changed)

Currently I'm not quite sure if it's the same problem going on here, since the error reads:

Kubernetes Error
Rancher Desktop 1.4.1 - win32 (x64)
Error Starting Kubernetes
TypeError [ERR_INVALID_PROTOCOL]: Protocol "http:" not supported. Expected "https:"