[WSL2] K3s and Kubernetes support

[ibuildthecloud]

k3s is simple, lightweight distribution of Kubernetes. To run k3s on WSL2 the following kernel settings are missing. These will be needed for most any Kubernetes setup, so this is not specific to k3s.

CONFIG_BRIDGE_NETFILTER=y
CONFIG_NETFILTER_XT_MATCH_COMMENT=y
CONFIG_NETFILTER_XT_MATCH_MULTIPORT=y
CONFIG_NETFILTER_XT_MATCH_OWNER=y
CONFIG_NETFILTER_XT_MATCH_PHYSDEV=y
CONFIG_VXLAN=y
CONFIG_GENEVE=y

I was able to get k3s and rio (which uses istio, knative, prometheus, grafana, kiali, and lot of other things) with these steps. https://gist.github.com/ibuildthecloud/1b7d6940552ada6d37f54c71a89f7d00

While I only included the CONFIG_NETFILTER_XT_MATCH_* parameters Kubernetes specifically needs, it might be worth it to just include all the default xt_match options. It is going to continually pop up that people want different iptables modules.

[benhillis]

Thanks for posting, I've kicked off a new kernel build with those features enabled.

[onomatopellan]

Do notice after replacing the original kernel with anything new, wsl.exe won't convert installs between WSL1 and WSL2 anymore. It freezes and a reboot is needed in order to run WSL2 again.

Using the original kernel install conversions work again.

@benhillis Is this the expected behavior? Why is the kernel included in WSL2 50Mb in size? Is the kernel truly just a renamed bzImage file?

[Biswa96]

Did you compile it from WSL2 Linux kernel source code or from kernel.org? @simonferquel did some testing with home-made ones.

Why is the kernel included in WSL2 50Mb in size?

Probable answer:

root@A:~# cd /mnt/c/Windows/System32/lxss/tools/
root@A:/mnt/c/Windows/System32/lxss/tools# file kernel
kernel: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), statically linked, BuildID[sha1]=1cdcda82d3a405575308724a5e303c5738893915, not stripped

[onomatopellan]

@Biswa96 I compiled it from the WSL2 docs link.

The WSL2 kernel is 50 Mb but real size in disk is 14Mb so maybe is uncompressed? The most similar file I found after compiling is vmlinux.bin which is 21Mb in size.

[Biswa96]

@onomatopellan Try with the WSL2-Linux-Kernel. WSL2 and WSL1 <--> WSL2 conversion both works.

[benhillis]

@Biswa96 - does converting work as well?

[onomatopellan]

Indeed replacing kernel with compiled bzImage (7Mb) from git and now conversions and new installs work. :)

[benc-uk]

Is this coming to the default "stock" WSL2 kernel?
I'd love to run k3s but I really don't fancy building my own kernel :)

[Biswa96]

The configurations are added in the wsl2 Linux kernel repository. If you don't want to build the kernel wait for next 2 or 3 Windows 10 insider builds.

[benc-uk]

Thanks, I can wait for a build or three

[daniel-kun]

Any update on this?

[markfarnan]

Just looked through /pro/config.gz on my install.
Looks like all the requested kernel settings are enabled in WSL 2 in Build 18995

[benc-uk]

Yes K3s seems to work in 18995 ! 😎
This is in WSL2, Kernel: 4.19.72-microsoft-standard

 $ sudo ./k3s kubectl get po --all-namespaces                                                                           
NAMESPACE     NAME                         READY   STATUS      RESTARTS   AGE
kube-system   coredns-66f496764-nksvz      1/1     Running     0          39s
kube-system   helm-install-traefik-kx7cx   0/1     Completed   0          39s
kube-system   svclb-traefik-tskxx          3/3     Running     0          16s
kube-system   traefik-d869575c8-xqccw      0/1     Running     0          16s
 $ sudo ./k3s kubectl get nodes                                                                                         
NAME        STATUS   ROLES    AGE   VERSION
becolemsb   Ready    master   59s   v1.15.4-k3s.1

[GaboFDC]

Yes K3s seems to work in 18995 ! 😎

How did you manage to install it? Latest script looks for systems, so I cant install k3s in wsl

[onomatopellan]

@GaboFDC for systemd see https://github.com/arkane-systems/genie