Consider switching default CNI to Calico

[sandys]

hi ,
so we have begun using k3s in production. We have had intermittent issues with flannel - especially with spot instances coming up and going down. we have also seen this issue - flannel-io/flannel#1245

What is most worrying is that flannel seems to be effectively unmaintained. There also seems to be compatibility issues with k8s 1.17.2

Calico is one of the only plugins supported by AWS ( https://docs.aws.amazon.com/eks/latest/userguide/alternate-cni-plugins.html) , AKS ( https://docs.microsoft.com/en-us/azure/aks/use-network-policies#network-policy-options-in-aks ) and GKE ( https://cloud.google.com/kubernetes-engine/docs/concepts/network-overview )

Calico is also part of CNCF - https://landscape.cncf.io/selected=project-calico with an Apache 2.0 license.

Wanted to open a placeholder issue for any consideration to choose calico as default

[blaggacao]

calico can advertise BGP routes and thereby — given appropriate router hardware — effectively implement bare metal external lbs without the need for metalLB, iirc.

[ghost]

Wireguard support for Calico was announced four days before this was opened — positive. But if other CNIs can already be swapped in what's the advantage of changing the default when it would take away time from adding support other CNIs such as Cilium? I see 21 thumbs up for the default adjustment but no attention from maintainers on this issue—which seems a bit strange. Are 90% of users switching the default? Or is this more of a nice-to-have consideration issue?

See also:

• https://rancher.com/blog/2019/2019-03-21-comparing-kubernetes-cni-providers-flannel-calico-canal-and-weave/
• https://k3d.io/usage/guides/calico/
• https://docs.projectcalico.org/getting-started/kubernetes/k3s/

[sandys]

Calico can now announce service lb ip addresses to BGP. Wireguard support is stable. https://docs.projectcalico.org/archive/v3.18/networking/advertise-service-ips#advertise-service-load-balancer-ip-addresses

[billimek]

It would be a really great user experience if we could at least choose Calico as the CNI as part of the k3s bootstrapping/installation instead of needing to install it out-of-band.

Do any of the k3s maintainers have insights if this is even a possibility in the future, or should we continue to use workarounds like we do today?

[brandond]

We don't currently bundle 'alternatives' for any of K3s' opinionated defaults. Disabling the packaged component and installing your own is not a workaround, it is the supported method to replace something that we've selected (usually for its simplicity and ease of use) with something better suited to your specific use case.

At some point we may wholesale change out components, but we don't currently have that on the roadmap.

[stale]

This repository uses a bot to automatically label issues which have not had any activity (commit/comment/label) for 180 days. This helps us manage the community issues better. If the issue is still relevant, please add a comment to the issue so the bot can remove the label and we know it is still valid. If it is no longer relevant (or possibly fixed in the latest release), the bot will automatically close the issue in 14 days. Thank you for your contributions.

[Bessonov]

activity

[stale]

This repository uses a bot to automatically label issues which have not had any activity (commit/comment/label) for 180 days. This helps us manage the community issues better. If the issue is still relevant, please add a comment to the issue so the bot can remove the label and we know it is still valid. If it is no longer relevant (or possibly fixed in the latest release), the bot will automatically close the issue in 14 days. Thank you for your contributions.

[Bessonov]

activity

[stale]

This repository uses a bot to automatically label issues which have not had any activity (commit/comment/label) for 180 days. This helps us manage the community issues better. If the issue is still relevant, please add a comment to the issue so the bot can remove the label and we know it is still valid. If it is no longer relevant (or possibly fixed in the latest release), the bot will automatically close the issue in 14 days. Thank you for your contributions.

[Bessonov]

activity

[cwayne18]

At this point we do not plan to change the default CNI any time soon. You should be able to disable flannel and use other CNIs, but k3s by design is opinionated. I'm going to close this out for now.