[release-1.21] Support dual-stack in klipper-lb

[brandond]

• Backport of Support dual-stack in klipper-lb #4021 on branch release-1.21

Is your feature request related to a problem? Please describe.

Dual-stack services in load-balanced mode are not working because klipper-lb does not support it:

+ iptables -t nat -I PREROUTING '!' -s 2001:cafe:42:1::def/32 -p TCP --dport 8080 -j DNAT --to 2001:cafe:42:1::def:8080
iptables v1.6.2: Port `cafe:42:1::def:8080' not valid

Describe the solution you'd like

klipper-lb is able to apply an ipv6 config

Additional context

#1405

[manuelbuil]

Is this backport really needed or a nice to have?

[brandond]

@cwayne18 set the milestone for the parent issue to 1.21 but it needs to land here first. If a backport is not needed then we can close this.

[manuelbuil]

Still not clear if dual-stack in k3s needs to be backported to 1.21. Waiting for @cwayne18 to come back with an answer

[ShylajaDevadiga]

While validating using commit id k3s version v1.21.6+k3s-93cf545a, k3s fails to start due to flannel incompatibility.

Nov 16 07:09:07 ip-192-168-17-194 k3s[15617]: time="2021-11-16T07:09:07.638730349Z" level=fatal msg="flannel CNI and network policy enforcement are not compatible with dual-stack operation; server must be restarted with --flannel-backend=n>
Nov 16 07:09:07 ip-192-168-17-194 systemd[1]: k3s.service: Main process exited, code=exited, status=1/FAILURE
Nov 16 07:09:07 ip-192-168-17-194 systemd[1]: k3s.service: Failed with result 'exit-code'.
Nov 16 07:09:07 ip-192-168-17-194 systemd[1]: Failed to start Lightweight Kubernetes.

Above error is fixed in #4503

Deployment for testing dual-stack fails
svclb-my-service-v6-rwbgt 0/1 CrashLoopBackOff

$ k3s -v
k3s version v1.21.6+k3s-6854470a (6854470a)

$ kubectl logs  svclb-my-service-v6-rwbgt
+ trap exit TERM INT
/usr/bin/entry: line 6: can't create /proc/sys/net/ipv4/ip_forward: Read-only file system
+ echo 1
+ true
+ cat /proc/sys/net/ipv4/ip_forward
+ '[' 1 '!=' 1 ]
+ iptables -t nat -I PREROUTING '!' -s 2001:cafe:42:1::4909/32 -p TCP --dport 8082 -j DNAT --to 2001:cafe:42:1::4909:8082
iptables v1.6.2: Port `cafe:42:1::4909:8082' not valid

Try `iptables -h' or 'iptables --help' for more information.

[ShylajaDevadiga]

Validated using commit id 5b45697 on release 1.21 branch

$ k3s -v
k3s version v1.21.6+k3s-5b456972 (5b456972)

$ kubectl get pods -A -o wide |grep my-service
default       svclb-my-service-v4-xjjkw                 1/1     Running     0          8m6s    10.42.1.5    ip-192-168-28-131   <none>           <none>
default       svclb-my-service-require-dual-snp6d       1/1     Running     0          8m5s    10.42.0.18   ip-192-168-12-124   <none>           <none>
default       svclb-my-service-v6-kfdqg                 1/1     Running     0          8m6s    10.42.0.17   ip-192-168-12-124   <none>           <none>
default       svclb-my-service-v4-7n4nj                 1/1     Running     0          8m6s    10.42.0.16   ip-192-168-12-124   <none>           <none>
default       svclb-my-service-prefer-dual-l8j9t        1/1     Running     0          8m5s    10.42.0.19   ip-192-168-12-124   <none>           <none>
default       svclb-my-service-v6-hqg8v                 1/1     Running     0          8m6s    10.42.1.6    ip-192-168-28-131   <none>           <none>
default       svclb-my-service-require-dual-6hw4z       1/1     Running     0          8m5s    10.42.1.7    ip-192-168-28-131   <none>           <none>
default       svclb-my-service-prefer-dual-9q6h4        1/1     Running     0          8m5s    10.42.1.8    ip-192-168-28-131   <none>           <none>

$ kubectl get svc |grep my-service
my-service-v4             LoadBalancer   10.43.70.140           192.168.12.124,192.168.28.131                                                                                 8081:32312/TCP   8m23s
my-service-v6             LoadBalancer   2001:cafe:42:1::b04d   2600:1f1c:ab4:ee48:dce0:6fdf:ef7c:5a8e,2600:1f1c:ab4:ee48:de91:a663:bccc:bb8c                                 8082:32244/TCP   8m23s
my-service-require-dual   LoadBalancer   10.43.21.99            192.168.12.124,192.168.28.131,2600:1f1c:ab4:ee48:dce0:6fdf:ef7c:5a8e,2600:1f1c:ab4:ee48:de91:a663:bccc:bb8c   8083:30987/TCP   8m23s
my-service-prefer-dual    LoadBalancer   10.43.201.97           192.168.12.124,192.168.28.131,2600:1f1c:ab4:ee48:dce0:6fdf:ef7c:5a8e,2600:1f1c:ab4:ee48:de91:a663:bccc:bb8c   8084:32750/TCP   8m22s

$ kubectl exec -it svclb-my-service-v6-kfdqg -- ping 10.42.1.5 
PING 10.42.1.5 (10.42.1.5): 56 data bytes
64 bytes from 10.42.1.5: seq=0 ttl=62 time=0.805 ms

$ kubectl exec -it svclb-my-service-v4-xjjkw -- ping 10.42.0.18
PING 10.42.0.18 (10.42.0.18): 56 data bytes
64 bytes from 10.42.0.18: seq=0 ttl=62 time=0.557 ms

$ curl http://[2600:1f1c:ab4:ee48:de91:a663:bccc:bb8c]:8084
<html><body><h1>It works!</h1></body></html>

$ curl http://192.168.28.131:8084
<html><body><h1>It works!</h1></body></html>