| title | authors | owning-sig | participating-sigs | reviewers | approvers | editor | creation-date | last-updated | status | see-also | replaces | superseded-by | ||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
egress-source-ip-support |
|
sig-network |
|
|
@mkimuram |
2019-06-13 |
2019-06-18 |
provisional |
|
- Title
- kubernetes/enhancements issue in release milestone, which links to KEP (this should be a link to the KEP location in kubernetes/enhancements, not the initial KEP PR)
- KEP approvers have set the KEP status to
implementable - Design details are appropriately documented
- Test plan is in place, giving consideration to SIG Architecture and SIG Testing input
- Graduation criteria is in place
- "Implementation History" section is up-to-date for milestone
- User-facing documentation has been created in [kubernetes/website], for publication to [kubernetes.io]
- Supporting documentation e.g., additional design documents, links to mailing list discussions/SIG meetings, relevant PRs/issues, release notes
Egress source IP is a feature to assign a static egress source IP for packets from one or more pods to outside k8s cluster. The words "outside k8s cluster" here includes both (1) private network where k8s cluster is running and (2) internet that is outside the private network where k8s cluster is running.
In k8s, egress traffic has its source IP translated (SNAT) to appear as the node IP when it leaves the cluster. However, there are many devices and software that use IP based ACLs to restrict incoming traffic for security reasons and bandwidth limitations. As a result, this kind of ACLs outside k8s cluster will block packets from the pod, which causes a connectivity issue. To resolve this issue, we need a feature to assign a particular static egress source IP to one or more particular pods.
Related discussions are done in here and here.
Provide users with an official and common way to assign a static egress source IP for packets from one or more pods to outside k8s cluster.
TBD
Expose an egress API to users like below to allow users to assign a static egress source IP to specific pod(s).
With below API, users will be able to make source IPs of pods that have label app: MyApp to be 192.168.122.222.
apiVersion: v1
kind: Egress
metadata:
name: my-egress
spec:
selector:
app: MyApp
ip: 192.168.122.222
In "(1) private network where k8s cluster is running" case, the ip is restricted to the private network segment where k8s cluster is running. On the other hand, in "(2) internet that is outside the private network where k8s cluster is running" case, this restriction won't be applied.
Note that PoC implementation for (1) private network where k8s cluster is running is
In the PoC implementation, it uses k8s operator with below CRD. Then, daemonset reconcile the iptables and routing tables rules to SNAT packets from particluar PodIP according to the CRDs. Note that VIPs are assinged to one of the nodes by using keepalived-vip and the packets are SNATed from the node that has matching VIP.
apiVersion: egress.mapper.com/v1alpha1
kind: Egress
metadata:
name: my-egress
spec:
ip: 192.168.122.222
kind: pod
namespace: default
name: pod1
Design details for this KEP version will be discussed in Design Details, and it won't necessary be the same to the PoC one, as long as it meets the requirements above. Also, the PoC implementation doesn't cover "(2) internet that is outside the private network where k8s cluster is running" case, so we will need to find a way to achieve this.
As a user of Kubernetes, I have a pod which requires an access to a database that restricts access by source ip and exists outside the k8s cluster. So, a pod which requires database access needs a specific egress source IP when sending packets to the database.
As a user of Kubernetes, I have multiple pods which require an access to a database that restricts access by source ip and exists outside the k8s cluster. So, multiple pods which require database access need a specific egress source IP when sending packets to the database.
As a user of Kubernetes, I have some pods which require an access to different databases that restrict access by source ip and exists outside the k8s cluster. So, some pods which require database access need a specific egress source IP when sending packets to the database, and other pods need another specific egress source IP.
TBD
TBD
Below are design details that need to be discussed:
- Reconcile in k8s controller vs external tool (eg. k8s operator)
- As PoC implementation shows this feature could be implemented as an external tool. However, there are many functions that are duplicated with k8s, so it would be better to leverage existing mechanism, in order to avoid duplicate implementation and conflict between them. Functions that seem duplicated are PodIP tracking functions in service and forwarding rule update functions in kube-proxy.
- k8s core API vs CRD
- Both should work. It would depend on whether we choose to implement it as external tool or not.
- Label based approach vs resource name based approach
- Resource name is easily be changed. (It is easy to implement as a prototype.) --> Label based approach will be preferred.
- Fixed implementation vs Pluggable implementation
- There are already many CNI implementations and kube-proxy implementations that have different behavior inside. For example, some CNI plugin have their own features to achieve it (eg. Calico case) and some kube-proxy implmentations have another way to achieve forwarding packets that might conflict with egress rule. To make any combinations of them work well, we might need to choose pluggable implementation. --> Pluggable implementation will be preferred.
Note: Section not required until targeted at a release.
TBD
Note: Section not required until targeted at a release.
TBD
TBD
TBD
TBD
TBD
TBD
TBD
TBD
TBD
TBD
TBD