Dual Stack Additions

[aauren]

@mrueg @murali-reddy

This is PR is primarily the work of @vadorovsky and @thomasferrandiz done in #1280 and #1343 to make the network policy controller (NPC) dual-stack compatible. This was done by keeping track of two different iptables and ipset handlers within the NPC. For rules/sets that are IP agnostic, rules/sets are written using both handlers. For rules/sets that are IP specific, they check which family the IP belongs to and then use the appropriate handlers. This PR replaces both of the previous PRs.

In addition to the NPC changes, in order to test this well, I found that I required dual-stack capabilities in the network routes controller (NRC). This had previously been IPv6 compatible, but not dual-stack compatible (meaning that you could only use IPv4 or IPv6). I refactored this to make the controller truly dual stack.

I expect that this will have a marginal impact on NPC performance as the initial section of the NPC's logic that execs out to ensure that default rules exist and are valid still uses the old style List / Insert logic. However, all subsequent network policy rules are written using the newer iptables-save / iptables-restore which greatly reduces processing time. It still requires 2 such calls as IPv4 and IPv6 are managed using different iptables commands (iptables vs ip6tables). But this is the best we can do here without switching to nftables or BPF.

My goal with this PR is to:

1. Provide maintainers a chance to review the work, while acknowledging that this is a humongous PR and there is little chance of a decent review given its size.
2. Give any other members of the community a chance to review or add thoughts
3. Begin a conversation about my intent with this work (see below)
4. Give anyone who is interested a chance to test the work before it gets merged

You'll notice that this PR does not target the master branch. This is an incredibly large change with over 5k lines of code having been modified. Additionally, there are a few nuances and options that are not fully backward compatible. As such, I would like to eventually incorporate this into a kube-router 2.0 release and release several release candidates over the next month or two to allow community members to test.

I see the flow going something like this:

1. Continue to test PR (so far I've only done basic testing) while giving maintainers and community members a chance to review
2. Add documentation to the docs about config items that have changed or are no longer backward compatible
3. When confident, merge the PR and create a v2.0.0-rc1 release candidate (this would not affect the latest tag which could introduce breaking changes to regular clusters)
4. Encourage community members to try out and test the release candidate
5. Make an additional PR to convert the network services controller (NSC) to also be dual-stack
6. Incorporate fixes and NSC dual-stack additions into subsequent v2.0.0-rc's
7. Continue to incorporate fixes and feedback from the community in subsequent RCs
8. When confident, merge the prep-v2.0 branch into master and cut an official v2.0.0 release of kube-router

Let me know if anyone has any thoughts here.

[aauren]

This PR also fixes #1023

[aauren]

This PR also fixes #1032 by parsing next-hop from the NLRI MP REACH attribute: https://github.com/cloudnativelabs/kube-router/pull/1386/files#diff-0c7a66931bb103d41704081acb720a2e87222dbd9732538d0bd911cb3106e9bcR194

[aauren]

Update, I got a chance to test these changes a bit more, and realized that yet still more testing is needed. Specifically around peering with global peers.

• I currently have some staged changes that catch the unit tests up to the added BGP changes
• Now that we have the localips annotation that was added in master, there was some better ways to approach peering that needed to be considered
• Realized that whether or not to advertise IPv6 routes shouldn't be dependent upon the primary IP address of the kube-router node, but should instead by based upon the command-line switch kube-router was started with. Maybe some day down the road it should be based upon more in-depth peer config, but without a more expressive peering configuration i.e. Restructure node peer annotations #1393 this should be postponed.

Additionally, upon merging this PR, I intend to update the IPv6 documentation on the main branch (https://github.com/cloudnativelabs/kube-router/blob/master/docs/ipv6.md) to give updates on where we stand as a project on IPv6 support when using the functionality introduced with this branch. Hopefully, with a container built and updated docs provided more people will help out with testing these changes.

For now, I plan to keep the staged changes where they are until I finish testing.