kindnetd remove wrong routes

[chrischdi]

This adds logic to kindnetd to remove routes which do not match the expected gateway address for an (Pod)CIDR.

I'm currently working at

• Implements test for self-hosted upgrades cluster-api#3728

Cluster API uses kindnetd as CNI for lots of e2e tests.

As of today, kindnetd only adds new routes and does never delete orphaned routes.

This could lead to the following situation in the workload Cluster created by Cluster API when kindnetd is used as CNI:

• a new Node joins the cluster which:
  • re-uses a PodCIDR of a deleted node
  • but has a different internal IP address compared to the deleted node
• All nodes which already had the route via the old node won't get the new route and pod to pod communication is broken

Broken pod to pod communication may include kube-apiserver to webhook communication, which is the case I detected during my tests.

[neolit123]

i was not aware capi uses kindnet for e2e.
node rotation is out of scope for kind, but for kubeadm e2e (also uses kindnet) we wanted to eventually test node addition / removal mainly around stacked etcd sustainability.

in any case, unclear if this is desired.

/cc @aojea

[chrischdi]

Thank you very much for your review and guidance @aojea 👍

I also inlined the func again because as you outlined correctly, we can't just do an early return.

[aojea]

/lgtm
/hold

because we have to build the image to run it in the CI, this code is not being exercised right now on the CI

@BenTheElder can you PTAL

[BenTheElder]

kind is generally -1 on adding code paths to support things we don't support (a tautology if you will!) but this change is small and seems reasonable and useful for multiple projects in-org.

+1 to antonio's review thus far :-)
/approve

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: BenTheElder, chrischdi

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [BenTheElder]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[chrischdi]

/test pull-kind-e2e-kubernetes-1-23

[BenTheElder]

/assign
rebasing, bumping kindneted

[BenTheElder]

OK, all of the prow CI jobs (not github actions) should run with this now.

[aojea]

/lgtm
/hold cancel
Thanks