[Help] Can't connect to k3s from outside network

[minhnguyenvato]

After change my ~/.kube/config to k3s generated certificate
Then I run kubectl get pod and received this error message

Unable to connect to the server: x509: certificate is valid for 127.0.0.1, 172.18.0.2, not 192.168.1.200

[iwilltry42]

Hey @minhnguyenvato, can you post the command you used to create the cluster and the resulting kubeconfig here please?

[minhnguyenvan95]

k3d command k3d create -n "d1" --workers 7

kubeconfig

apiVersion: v1
clusters:
- cluster:
    certificate-authority-data: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUM1RENDQWN5Z0F3SUJBZ0lCQURBTkJna3Foa2lHOXcwQkFRc0ZBREFqTVJBd0RnWURWUVFLRXdkck0zTXQKYjNKbk1ROHdEUVlEVlFRREV3WnJNM010WTJFd0hoY05NVGt3TlRBNU1EZzBOelV5V2hjTk1qa3dOVEEyTURnMApOelV5V2pBak1SQXdEZ1lEVlFRS0V3ZHJNM010YjNKbk1ROHdEUVlEVlFRREV3WnJNM010WTJFd2dnRWlNQTBHCkNTcUdTSWIzRFFFQkFRVUFBNElCRHdBd2dnRUtBb0lCQVFEQkljWitoNjR1UlhwTVhvNHIvU1NCSExWejEyekIKR3ZicHpKQlg2WXlNZCtkWFF4OXQvRjlYdFVrZ2FCeHlNTnpadHVQc3gzUTVqRk9UaXRuZ290bWZ5YWNueUtwcQoxL2tONzhicUYwa0pDTkJkYlJTZk5qTVVYazhhRHlaeUtBbFNOWDJHeTJGVHF6dHN3UENpOTh5dW50blFlTlpLCnJmRnlhem8vd2RNTTFMb2VYeDlqMG15YUQxOVh4cDdLbUZMVFJ4bjJ0TnFPK3Vjc1I0R1ZBZjg2cE1OZEJzdTcKVW1oQW45cHNqeFFRdTlsSFh5dWZRZDMrR2k1Q1RicjBtWmtUWnRwUGZsNVFFVUV5VUwrUE5sSS9sUkJLaHZtMgpJYXNkcStGMmRCdlh2THV4UHNvbGNvUHVxRVM0NWJNOW96K2UvaUZTRFdDZ2FkQnpCcDV6d0RvbEFnTUJBQUdqCkl6QWhNQTRHQTFVZER3RUIvd1FFQXdJQ3BEQVBCZ05WSFJNQkFmOEVCVEFEQVFIL01BMEdDU3FHU0liM0RRRUIKQ3dVQUE0SUJBUUMvZFdFU2FFMDBYQ2ExUHVVeW55TEhyNjRsRGJPeDNla2FZL2cvY0dCVFU4eG5menpXRG91cApmRlB1dUFTNkltMUprWkgySDdlZGtIQmVEVlFmei9WWUR0NlZrMkluQXRnYit0bGliaEI3UkQ4RFYzNE1Ecis3CjJxVURVL2xrVHN2OXBvalc2U2ZUZVJRSXM0eWpITlBxUWhPWERWd0dpbzNIWXRmRU8vS1pucHZ4N2JUeFkydWwKdGQ4MndMdURpalVqNWFxRXZGRStyYkE2YkZ5cXh6by9vWUFOS1NOUlhRcm9VOHJoanpWUHE3c1ROVDBqVmZSYwprNDQxNjQ1SkUvUHBOMUJLaS9sNEZZWlFtVVJqQVpOYW9TcWV0R2pxWmlGOUFYbFdhd3FPaXFxR3pxVXk5YWx3ClBHWjZmSnFlK3FITVYvYm1Sb0YvRGZ6V1NEcGtFZHI4Ci0tLS0tRU5EIENFUlRJRklDQVRFLS0tLS0K
    server: https://localhost:6443
  name: default
contexts:
- context:
    cluster: default
    user: default
  name: default
current-context: default
kind: Config
preferences: {}
users:
- name: default
  user:
    password: a551485914de49a9a41ef4790f5bcd5e
    username: admin

[iwilltry42]

So that's the kubeconfig that you've got via k3d get-kubeconfig -n d1, right?
And you copied that whole thing over your ~/.kube/config ?

[iwilltry42]

Does it work for you if you just do export KUBECONFIG="$(bin/k3d get-kubeconfig --name='test')" instead of copying the kubeconfig into ~/.kube/config?

[minhnguyenvan95]

I copy the cluster, context, user from , that's the kubeconfig. Then using kubectx to switch to that context.
I think I should passing some k3s args to allow external connect , do you know which args are ?

https://github.com/rancher/k3d/blob/32cc70b59901c927b47157b44aaae6944de99815/cli/commands.go#L94

[minhnguyenvan95]

I've replace server: https://localhost:6443 with external ip address server: https://192.168.1.200:6443 in my ~/.kube/config

[iwilltry42]

Ah, sorry, I totally misunderstood you and didn't get that you were running k3d on a remote server.
You have two options (that I know of) here:

1. skip certificate verification on client side via kubectl --insecure-skip-tls-verify get nodes (not recommended)
2. add remote host's IP as a SAN for the server certificate: k3d create -x --tls-san="1.2.3.4"

There might be other options such as changing the server bind address, but I didn't test those.

Does this solve your issue?

[minhnguyenvan95]

Thank you very much . Option 2 is works for me .
Can I ask another question , How to binding multiple address for tls-san .
For example : --tls-san="1.2.3.4,domain.com,anotherdomain.com"

[iwilltry42]

@minhnguyenvan95 not sure if k3s supports lists there, but you might just want to use the --tls-san flag multiple times.
From k3d: k3d create -p 6550 --workers 2 -x --tls-san="1.2.3.4" -x --tls-san="4.3.2.1"

[nunix]

@iwilltry42 you just made my day. Crazy demo incoming (on twitter 😉)

[abdullahainun]

Ah, sorry, I totally misunderstood you and didn't get that you were running k3d on a remote server.
You have two options (that I know of) here:

1. skip certificate verification on client side via kubectl --insecure-skip-tls-verify get nodes (not recommended)
2. add remote host's IP as a SAN for the server certificate: k3d create -x --tls-san="1.2.3.4"

There might be other options such as changing the server bind address, but I didn't test those.

Does this solve your issue?

thanks sir

[louis030195]

I see the solution changed since the CLI changed,

I tried to put in k3d config:

...
options:
...
  k3s: # options passed on to K3s itself
    extraArgs: # additional arguments pasised to the `k3s server|agent` command; same as `--k3s-arg`
      - arg: --tls-san="MY_EXTERNAL_IP"

With this, the error still shows, so for now I use kubectl --insecure-skip-tls-verify

EDIT: nevermind, somehow the error is gone, so for the newcomer, what I wrote is the solution on latest k3d

Update (@iwilltry42): Here's the CLI way in k3d v5: k3d cluster create --k3s-arg "--tls-san=MY_EXTERNAL_IP"@server:*

[newbe36524]

I see the solution changed since the CLI changed,

I tried to put in k3d config:

...
options:
...
  k3s: # options passed on to K3s itself
    extraArgs: # additional arguments pasised to the `k3s server|agent` command; same as `--k3s-arg`
      - arg: --tls-san="MY_EXTERNAL_IP"

With this, the error still shows, so for now I use kubectl --insecure-skip-tls-verify

EDIT: nevermind, somehow the error is gone, so for the newcomer, what I wrote is the solution on latest k3d

Update (@iwilltry42): Here's the CLI way in k3d v5: k3d cluster create --k3s-arg "--tls-san=MY_EXTERNAL_IP"@server:*

As I have tested, it should work when expanding the quotes.

...
options:
...
  k3s: # options passed on to K3s itself
    extraArgs: # additional arguments pasised to the `k3s server|agent` command; same as `--k3s-arg`
      - arg: "--tls-san=MY_EXTERNAL_IP"