kubectl logs broken in in v0.5.0-rc1 (41520d73)

[runningman84]

Describe the bug
Fetching logs from containers is not possible anymore in v0.5.0-rc1 (41520d7)

To Reproduce
Replace k3s binary with v0.5.0-rc1 (41520d7) release

Expected behavior
Working log output like version v.0.4.0

Screenshots
broken version

root@virtnuc1:~# kubectl get pods
No resources found.
root@virtnuc1:~# kubectl get pods -n hass
NAME                               READY   STATUS    RESTARTS   AGE
appdaemon-548c996d67-gpwmz         2/2     Running   12         14d
home-assistant-74995bbb79-79jlz    1/1     Running   4          13d
mosquitto-558b78c877-x96rl         1/1     Running   6          14d
svclb-mosquitto-5dbcb699dc-hw5tr   1/1     Running   6          15d
svclb-mosquitto-rmxgh              0/1     Pending   0          75m
root@virtnuc1:~# kubectl logs home-assistant-74995bbb79-79jlz -n hass
Error from server (Forbidden): Forbidden (user=kubernetes, verb=get, resource=nodes, subresource=proxy) ( pods/log home-assistant-74995bbb79-79jlz)
root@virtnuc1:~# kubectl logs appdaemon-548c996d67-gpwmz -n hass
Error from server (BadRequest): a container name must be specified for pod appdaemon-548c996d67-gpwmz, choose one of: [git-sync appdaemon] or one of the init containers: [git-sync-init]
root@virtnuc1:~# kubectl logs appdaemon-548c996d67-gpwmz -n hass -c appdaemon
Error from server (Forbidden): Forbidden (user=kubernetes, verb=get, resource=nodes, subresource=proxy) ( pods/log appdaemon-548c996d67-gpwmz)
root@virtnuc1:~# k3s 
NAME:
   k3s - Kubernetes, but small and simple

USAGE:
   k3s [global options] command [command options] [arguments...]

VERSION:
   v0.5.0-rc1 (41520d73)

COMMANDS:
     server   Run management server
     agent    Run node agent
     kubectl  Run kubectl
     crictl   Run crictl
     help, h  Shows a list of commands or help for one command

GLOBAL OPTIONS:
   --debug        Turn on debug logs
   --help, -h     show help
   --version, -v  print the version

working version

root@virtnuc1:~# kubectl logs appdaemon-548c996d67-gpwmz -n hass -c appdaemon
2019-04-27 20:54:25.402986 WARNING AppDaemon: HASS: Disconnected from Home Assistant, retrying in 5 seconds
2019-04-27 20:54:35.410632 WARNING AppDaemon: HASS: Disconnected from Home Assistant, retrying in 5 seconds
2019-04-27 20:54:45.418221 WARNING AppDaemon: HASS: Disconnected from Home Assistant, retrying in 5 seconds
2019-04-27 20:54:55.425748 WARNING AppDaemon: HASS: Disconnected from Home Assistant, retrying in 5 seconds
2019-04-27 20:55:05.435796 WARNING AppDaemon: HASS: Disconnected from Home Assistant, retrying in 5 seconds
2019-04-27 20:55:15.441602 WARNING AppDaemon: HASS: Disconnected from Home Assistant, retrying in 5 seconds
root@virtnuc1:~# k3s
NAME:
   k3s - Kubernetes, but small and simple

USAGE:
   k3s [global options] command [command options] [arguments...]

VERSION:
   v0.4.0 (7822549f)

COMMANDS:
     server   Run management server
     agent    Run node agent
     kubectl  Run kubectl
     crictl   Run crictl
     help, h  Shows a list of commands or help for one command

GLOBAL OPTIONS:
   --debug        Turn on debug logs
   --help, -h     show help
   --version, -v  print the version

Add any other context about the problem here.

[erikwilson]

Thanks for reporting this @runningman84. This is because we have changed how certs are generated. If you remove token-node.crt I think it should be regenerated and work.

[erikwilson]

The agent certs will also need to be regenerated to use unmodified metrics server.

[erikwilson]

Should also be fixed when #359 is updated & merged.

[dnoland1]

Was able to reproduce with v0.5.0-rc1:

vagrant@k3s-node1:~$ k3s kubectl logs -n kube-system coredns-695688789-9s476
Error from server (Forbidden): Forbidden (user=kubernetes, verb=get, resource=nodes, subresource=proxy) ( pods/log coredns-695688789-9s476)

Upgraded to v0.5.0-rc4 using curl -sfL https://get.k3s.io | INSTALL_K3S_VERSION=v0.5.0-rc4 sh -

Can see logs now:

root@k3s-node1:~# k3s kubectl logs -n kube-system coredns-695688789-9s476 --tail=5
E0503 23:40:00.221015       1 reflector.go:205] github.com/coredns/coredns/plugin/kubernetes/controller.go:317: Failed to list *v1.Endpoints: Get https://10.43.0.1:443/api/v1/endpoints?limit=500&resourceVersion=0: dial tcp 10.43.0.1:443: connect: connection refused
E0503 23:40:00.222305       1 reflector.go:205] github.com/coredns/coredns/plugin/kubernetes/controller.go:315: Failed to list *v1.Service: Get https://10.43.0.1:443/api/v1/services?limit=500&resourceVersion=0: dial tcp 10.43.0.1:443: connect: connection refused
E0503 23:40:01.217936       1 reflector.go:205] github.com/coredns/coredns/plugin/kubernetes/controller.go:322: Failed to list *v1.Namespace: Get https://10.43.0.1:443/api/v1/namespaces?limit=500&resourceVersion=0: dial tcp 10.43.0.1:443: connect: connection refused
E0503 23:40:01.234247       1 reflector.go:205] github.com/coredns/coredns/plugin/kubernetes/controller.go:317: Failed to list *v1.Endpoints: Get https://10.43.0.1:443/api/v1/endpoints?limit=500&resourceVersion=0: dial tcp 10.43.0.1:443: connect: connection refused
E0503 23:40:01.234294       1 reflector.go:205] github.com/coredns/coredns/plugin/kubernetes/controller.go:315: Failed to list *v1.Service: Get https://10.43.0.1:443/api/v1/services?limit=500&resourceVersion=0: dial tcp 10.43.0.1:443: connect: connection refused

[erikwilson]

This should be fixed in 0.5.0-rc4