feat(load-balancer): Set IPMode to "Proxy" if load balancer is configured to use proxy protocol (#727) #783
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
lukasmetzner
temporarily deployed
to
e2e-robot
GitHub Actions
Inactive
|
As Kubernetes version 1.28 is no longer supported we can skip the version check. |
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #783 +/- ##
==========================================
+ Coverage 70.29% 70.41% +0.11%
==========================================
Files 31 31
Lines 3205 3231 +26
==========================================
+ Hits 2253 2275 +22
- Misses 779 782 +3
- Partials 173 174 +1 ☔ View full report in Codecov by Sentry. |
jooola
approved these changes
Nov 12, 2024
Seems like we still support it though, so we might need to drop support for 1.28 before releasing this? |
…xy traffic KEP-1860 introduced a new field `service.status.loadBalancer.ingress[].ipMode`: VIP | Proxy to indicate the behavior of the Load Balancer. Currently users on IPVS-based networking setups can not access the Load Balancer IPs when they enable PROXY protocol.
lukasmetzner
force-pushed
the
ipmode-proxy-if-using-proxy-protocol
branch
from
2d1b1e0 to
83fab0d
Compare
lukasmetzner
temporarily deployed
to
e2e-robot
GitHub Actions
Inactive
1 task
lukasmetzner
added a commit
that referenced
this pull request
Nov 19, 2024
…ured to use proxy protocol (#727) (#783) [KEP-1860](https://github.com/kubernetes/enhancements/tree/master/keps/sig-network/1860-kube-proxy-IP-node-binding) introduced a new field `service.status.loadBalancer.ingress[].ipMode`: VIP | Proxy to indicate the behavior of the Load Balancer. Currently users on IPVS-based networking setups can not access the Load Balancer IPs when they enable PROXY protocol. Expected behavior Users should always be able to access their services from inside of the cluster, even if they use IPVS and PROXY protocol. IIUC we should set the IP Mode to Proxy for the IPs we return in the status. --------- Co-authored-by: simonhammes <[email protected]>
lukasmetzner
pushed a commit
that referenced
this pull request
Nov 26, 2024
<!-- section-start changelog --> ### Feature Highlights & Upgrade Notes #### Load Balancer IPs set to Private IPs If networking support is enabled, the load balancer IPs are now populated with the private IPs, unless the `load-balancer.hetzner.cloud/disable-private-ingress` annotation is set to `true`. Please make sure that you configured the annotation according to your needs, for example if you are using `external-dns`. #### Provided-By Label We introduced a the label `instance.hetzner.cloud/provided-by`, which will be automatically added to all **new** nodes. This label can have the values `cloud` or `robot` to distinguish between our products. We use this label in the csi-driver to ensure the daemonset is only running on cloud nodes. We recommend to add this label to your existing nodes with the appropriate value. - `kubectl label node $CLOUD_NODE_NAME instance.hetzner.cloud/provided-by=cloud` - `kubectl label node $ROBOT_NODE_NAME instance.hetzner.cloud/provided-by=robot` #### Load Balancer IPMode Proxy Kubernetes KEP-1860 added a new field to the Load Balancer Service Status that allows us to mark if the IP address we add should be considered as a Proxy (always send traffic here) and VIP (allow optimization by keeping the traffic in the cluster). Previously Kubernetes considered all IPs as VIP, which caused issues when when the PROXY protocol was in use. We have previously recommended to use the annotation `load-balancer.hetzner.cloud/hostname` to workaround this problem. We now set the new field to `Proxy` if the PROXY protocol is active so the issue should no longer appear. If you only added the `load-balancer.hetzner.cloud/hostname` annotation for this problem, you can remove it after upgrading. Further information: - kubernetes/enhancements#1860 - #160 (comment) ### Features - **service**: Specify private ip for loadbalancer (#724) - add support & tests for Kubernetes 1.31 (#747) - **helm**: allow setting extra pod volumes via chart values (#744) - **instance**: add label to distinguish servers from Cloud and Robot (#764) - emit event when robot server name and node name mismatch (#773) - **load-balancer**: Set IPMode to "Proxy" if load balancer is configured to use proxy protocol (#727) (#783) - **routes**: emit warning if cluster cidr is misconfigured (#793) - **load-balancer**: ignore nodes that don't use known provider IDs (#780) - drop tests for kubernetes v1.27 and v1.28 ### Bug Fixes - populate ingress private ip when disable-private-ingress is false (#715) - wrong version logged on startup (#729) - invalid characters in label instance-type of robot servers (#770) - no events are emitted as broadcaster has no sink configured (#774) ### Kubernetes Support This version was tested with Kubernetes 1.29 - 1.31. Furthermore, we dropped v1.27 and v1.28 support. <!-- section-end changelog --> --- <details> <summary><h4>PR by <a href="https://github.com/apricote/releaser-pleaser">releaser-pleaser</a> 🤖</h4></summary> If you want to modify the proposed release, add you overrides here. You can learn more about the options in the docs. ## Release Notes ### Prefix / Start This will be added to the start of the release notes. ```rp-prefix ### Feature Highlights & Upgrade Notes #### Load Balancer IPs set to Private IPs If networking support is enabled, the load balancer IPs are now populated with the private IPs, unless the `load-balancer.hetzner.cloud/disable-private-ingress` annotation is set to `true`. Please make sure that you configured the annotation according to your needs, for example if you are using `external-dns`. #### Provided-By Label We introduced a the label `instance.hetzner.cloud/provided-by`, which will be automatically added to all **new** nodes. This label can have the values `cloud` or `robot` to distinguish between our products. We use this label in the csi-driver to ensure the daemonset is only running on cloud nodes. We recommend to add this label to your existing nodes with the appropriate value. - `kubectl label node $CLOUD_NODE_NAME instance.hetzner.cloud/provided-by=cloud` - `kubectl label node $ROBOT_NODE_NAME instance.hetzner.cloud/provided-by=robot` #### Load Balancer IPMode Proxy Kubernetes KEP-1860 added a new field to the Load Balancer Service Status that allows us to mark if the IP address we add should be considered as a Proxy (always send traffic here) and VIP (allow optimization by keeping the traffic in the cluster). Previously Kubernetes considered all IPs as VIP, which caused issues when when the PROXY protocol was in use. We have previously recommended to use the annotation `load-balancer.hetzner.cloud/hostname` to workaround this problem. We now set the new field to `Proxy` if the PROXY protocol is active so the issue should no longer appear. If you only added the `load-balancer.hetzner.cloud/hostname` annotation for this problem, you can remove it after upgrading. Further information: - kubernetes/enhancements#1860 - #160 (comment) ``` ### Suffix / End This will be added to the end of the release notes. ```rp-suffix ### Kubernetes Support This version was tested with Kubernetes 1.29 - 1.31. Furthermore, we dropped v1.27 and v1.28 support. ``` </details> Co-authored-by: releaser-pleaser <>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
KEP-1860 introduced a new field
service.status.loadBalancer.ingress[].ipMode: VIP | Proxy to indicate the behavior of the Load Balancer.Currently users on IPVS-based networking setups can not access the Load Balancer IPs when they enable PROXY protocol.
Expected behavior
Users should always be able to access their services from inside of the cluster, even if they use IPVS and PROXY protocol.
IIUC we should set the IP Mode to Proxy for the IPs we return in the status.