Enable public ingress on nerc-ocp-prod #146
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
The routes added by these files are technically correct, but due to the way loadbalancer addresses are handled the rules don't apply to traffic in a useful fashion. Traffic coming from a service is modified by a masquerade rule in the POSTROUTING chain to have the correct source address. Because this happens *after* a routing decision has already been made, the outbound request never matches the routing policy rules defined by these files.
There's no need to reproduce this file for every machineconfig directory.
Configure `net.ipv4.conf.all.rp_filter` on all worker nodes for "loose" mode (`rp_filter=2`). This permits asymmetric routing, which is necessary right now for cluster access via the public network.
This commit enables symmetric routing for requests on the public
address range. This requires a few changes:
1. Add custom netfilter rules that:
- Add a connection mark to incoming requests on the public network.
- Transfer the connection mark to the fwmark on reply packets
originating from the serviceNetwork cidr rnage.
2. Add policy routing rules that match the fwmark so that replies
to public requests use a custom route table.
3. Add rules to the custom table that direct replies to public
requests out the appropriate default gateway.
computate
approved these changes
Nov 14, 2022
dystewart
approved these changes
Nov 14, 2022
naved001
approved these changes
Nov 14, 2022
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This pull request applies the configuration we developed to enable public ingress on the nerc-ocp-prod cluster. There are broadly three changes:
Enable loose
rp_filtermode (this prevents the kernel from simply dropping packets that ingress on the public network)Add some custom netfilter rules to managing connection and packet marks on public ingress traffic.
Add policy routing rules that utilize the packet marks to direct replies to the appropriate default gateway.
Closes nerc-project/operations#16