Update network doc for ip-per-pod reqs #12182

thockin · 2019-01-11T21:35:51Z

As per SIG-Network discussion, we can relax the requirement for all
nodes to be able to reach all pods on those platforms that do not
support hostNetwork.

The current thinking on Windows support is that hostNetwork is not a
requirement for Window users. As such, satisfying the old requirement
is very difficult and has no practical benefit.

This DOES NOT change the requirements for platforms that support
hostNetwork (e.g. Linux) nor does it make hostNetwork optional for Linux
nodes.

This PR also removes most comparisons to Docker, as they are stale and
no longer germane, and cleans up some of the text.

@dineshgovindasamy

dineshgovindasamy · 2019-01-11T21:38:09Z

Thanks @thockin

Looks good from my perspective.

k8sio-netlify-preview-bot · 2019-01-11T21:40:59Z

Deploy preview for kubernetes-io-master-staging ready!

Built with commit 55fe4e2

https://deploy-preview-12182--kubernetes-io-master-staging.netlify.com

chenopis · 2019-01-11T22:37:13Z

content/en/docs/concepts/cluster-administration/networking.md

-how to find each other, etc.  Rather than deal with this, Kubernetes takes a
-different approach.
+Kubernetes is all about sharing machines between applications.  Typically,
+sharing machines requires ensuring that two applications on the same do not try


Is there a word missing? i.e. "on the same..."? machine?

cleaned up a bit

bowei · 2019-01-12T00:32:41Z

added text about host port support lgtm

bowei · 2019-01-12T00:32:03Z

content/en/docs/concepts/cluster-administration/networking.md

+network (e.g. Linux supports this):
+
+   * pods in the host network of any node can communicate with all pods on all
+     nodes without NAT

 What this means in practice is that you can not just take two computers


do we want to remove "editorial" style text?

fcrisciani · 2019-01-11T23:34:52Z

content/en/docs/concepts/cluster-administration/networking.md

-   * all nodes can communicate with all containers (and vice-versa) without NAT
-   * the IP that a container sees itself as is the same IP that others see it as
+   * pods on any node can communicate with all pods on all nodes without NAT
+   * agents on a node can communicate with all pods on that node


is it worth to spend a one liner to define what we consider an agent? I did not see it mentioned in any other part of the doc. Maybe we can link to https://kubernetes.io/docs/concepts/overview/components/#kubelet like in the section above?

Yeah, we should clarify what they are. "infrastructure agents" or something, which include networking, logging, health checking, networking; eg anything required to get and keep the cluster functional

Will change to "agents on a node (e.g. system daemons, kubelet) can communicate with all pods on that node"

dcbw · 2019-01-14T15:32:44Z

@danwinship

dcbw · 2019-01-14T16:28:08Z

content/en/docs/concepts/cluster-administration/networking.md

+In addition to these, for those platforms that support `Pods` running in the host
+network (e.g. Linux supports this):
+
+   * pods in the host network of any node can communicate with all pods on all


@thockin hmm this is new; was this something that just came up or that should have been clarified from the start?

no, this was always intended, and was implied by the first rule (all pods <-> all pods). Otherwise some pods are not able to reach other pods.

I think it is in e2e, too.

As much as I dislike it, some apps run hostNetwork for performance reasons. Removing this constraint would potentially break them.

danwinship · 2019-01-14T16:34:35Z

content/en/docs/concepts/cluster-administration/networking.md

-insert dynamic port numbers into configuration blocks, services have to know
-how to find each other, etc.  Rather than deal with this, Kubernetes takes a
-different approach.
+Kubernetes is all about sharing machines between applications.  Typically,


I feel like even the rewritten version of this is still kind of explaining Kubernetes to former docker users who have been in a coma since 2015. :) No one these days is going to come into Kubernetes expecting to have to coordinate port numbers between unrelated apps, so it's just confusing to spend so much time explaining why we don't force them to.

Man, I get asked ALL THE TIME why we don't support dynamic port mapping.

danwinship · 2019-01-14T17:07:58Z

content/en/docs/concepts/cluster-administration/networking.md

+   * agents on a node can communicate with all pods on that node
+   * the IP that a pod sees itself as is the same IP that others see it as
+
+In addition to these, for those platforms that support `Pods` running in the host


This isn't "in addition"; we already required that "pods on any node can communicate with all pods on all nodes without NAT", so if host network pods exist, then they are already required to be able to communicate with all pods.

Maybe just something like "Note that for platforms that support Pods running in the host network, these requirements apply to those Pods as well."

(Except see my other comment about hostNetwork pods.)

reworded. Good point.

danwinship · 2019-01-14T17:11:58Z

content/en/docs/concepts/cluster-administration/networking.md

+How this is implemented is a detail of the particular container runtime in use.
+
+It is possible to request ports on the `Node` itself which forward to your `Pod`
+(called host-ports), but this is reduced to a very niche operation. How that


drop "reduced to" since that's part of the docker-vs-kubernetes comparison, which is now mostly gone

(also, I feel like the original "host ports" is better than "host-ports"?)

danwinship · 2019-01-14T17:26:05Z

content/en/docs/concepts/cluster-administration/networking.md

-   * the IP that a container sees itself as is the same IP that others see it as
+   * pods on any node can communicate with all pods on all nodes without NAT
+   * agents on a node can communicate with all pods on that node
+   * the IP that a pod sees itself as is the same IP that others see it as


Hm... what if a pod has multiple IPs? In particular, what about the case of a hostNetwork pod on a node with multiple IPs?

In openshift-sdn, every node has an IP address in the cluster network CIDR in addition to its external/primary/"eth0" IP. A hostNetwork pod would list the node's primary IP as its .spec.podIP, but when it communicates with other non-hostNetwork pods, the source IP would be the node's cluster network IP, not the primary IP. Is that legit?

(If so, we probably want to treat the hostNetwork case separately here, since I think we definitely want to say that for non-hostNetwork pods, the source address of pod-to-pod traffic will always be the pod's .spec.podIP.)

Do we even need this line? It's somewhat redundant with the "no NAT" line. I propose to remove it.

As per SIG-Network discussion, we can relax the requirement for all nodes to be able to reach all pods on those platforms that do not support hostNetwork. The current thinking on Windows support is that hostNetwork is not a requirement for Window users. As such, satisfying the old requirement is very difficult and has no practical benefit. This DOES NOT change the requirements for platforms that support hostNetwork (e.g. Linux) nor does it make hostNetwork optional for Linux nodes. This PR also removes most comparisons to Docker, as they are stale and no longer germane, and cleans up some of the text.

thockin

I missed that this was still open.

Pushed a new commit fixing comments.

thockin · 2019-02-08T00:35:38Z

content/en/docs/concepts/cluster-administration/networking.md

-insert dynamic port numbers into configuration blocks, services have to know
-how to find each other, etc.  Rather than deal with this, Kubernetes takes a
-different approach.
+Kubernetes is all about sharing machines between applications.  Typically,


Man, I get asked ALL THE TIME why we don't support dynamic port mapping.

thockin · 2019-02-08T00:35:47Z

content/en/docs/concepts/cluster-administration/networking.md

-how to find each other, etc.  Rather than deal with this, Kubernetes takes a
-different approach.
+Kubernetes is all about sharing machines between applications.  Typically,
+sharing machines requires ensuring that two applications on the same do not try


cleaned up a bit

thockin · 2019-02-08T00:39:32Z

content/en/docs/concepts/cluster-administration/networking.md

-   * all nodes can communicate with all containers (and vice-versa) without NAT
-   * the IP that a container sees itself as is the same IP that others see it as
+   * pods on any node can communicate with all pods on all nodes without NAT
+   * agents on a node can communicate with all pods on that node


Will change to "agents on a node (e.g. system daemons, kubelet) can communicate with all pods on that node"

thockin · 2019-02-08T00:42:29Z

content/en/docs/concepts/cluster-administration/networking.md

-   * the IP that a container sees itself as is the same IP that others see it as
+   * pods on any node can communicate with all pods on all nodes without NAT
+   * agents on a node can communicate with all pods on that node
+   * the IP that a pod sees itself as is the same IP that others see it as


Do we even need this line? It's somewhat redundant with the "no NAT" line. I propose to remove it.

thockin · 2019-02-08T00:45:17Z

content/en/docs/concepts/cluster-administration/networking.md

+   * agents on a node can communicate with all pods on that node
+   * the IP that a pod sees itself as is the same IP that others see it as
+
+In addition to these, for those platforms that support `Pods` running in the host


reworded. Good point.

thockin · 2019-02-08T00:47:24Z

content/en/docs/concepts/cluster-administration/networking.md

+In addition to these, for those platforms that support `Pods` running in the host
+network (e.g. Linux supports this):
+
+   * pods in the host network of any node can communicate with all pods on all


no, this was always intended, and was implied by the first rule (all pods <-> all pods). Otherwise some pods are not able to reach other pods.

I think it is in e2e, too.

As much as I dislike it, some apps run hostNetwork for performance reasons. Removing this constraint would potentially break them.

thockin · 2019-02-08T00:47:30Z

content/en/docs/concepts/cluster-administration/networking.md

+network (e.g. Linux supports this):
+
+   * pods in the host network of any node can communicate with all pods on all
+     nodes without NAT

 What this means in practice is that you can not just take two computers


thockin · 2019-02-08T00:48:22Z

content/en/docs/concepts/cluster-administration/networking.md

+How this is implemented is a detail of the particular container runtime in use.
+
+It is possible to request ports on the `Node` itself which forward to your `Pod`
+(called host-ports), but this is reduced to a very niche operation. How that


danwinship · 2019-02-08T13:22:28Z

/lgtm

fcrisciani · 2019-02-08T14:48:18Z

/lgtm

k8s-ci-robot · 2019-02-08T14:48:26Z

@fcrisciani: changing LGTM is restricted to assignees, and only kubernetes/website repo collaborators may be assigned issues.

In response to this:

/lgtm

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

thockin · 2019-02-11T20:36:10Z

@chenopis

Rajakavitha1 · 2019-02-17T12:59:19Z

@chenopis Could you please take a look at the updated content.

zparnold · 2019-02-18T01:30:49Z

/approve

k8s-ci-robot · 2019-02-18T01:30:59Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: zparnold

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~content/en/OWNERS~~ [zparnold]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

* Update network doc for ip-per-pod reqs As per SIG-Network discussion, we can relax the requirement for all nodes to be able to reach all pods on those platforms that do not support hostNetwork. The current thinking on Windows support is that hostNetwork is not a requirement for Window users. As such, satisfying the old requirement is very difficult and has no practical benefit. This DOES NOT change the requirements for platforms that support hostNetwork (e.g. Linux) nor does it make hostNetwork optional for Linux nodes. This PR also removes most comparisons to Docker, as they are stale and no longer germane, and cleans up some of the text. * Feedback on contents

popcornylu · 2019-03-03T00:21:50Z

content/en/docs/concepts/cluster-administration/networking.md

 Kubernetes imposes the following fundamental requirements on any networking
 implementation (barring any intentional network segmentation policies):

-   * all containers can communicate with all other containers without NAT
-   * all nodes can communicate with all containers (and vice-versa) without NAT
-   * the IP that a container sees itself as is the same IP that others see it as


The classic statements about kubernetes network are removed here
https://youtu.be/90kZRyPcRZw?t=829
https://youtu.be/6v_BDHIgOY8?t=95
https://www.digitalocean.com/community/tutorials/kubernetes-networking-under-the-hood

* Update network doc for ip-per-pod reqs As per SIG-Network discussion, we can relax the requirement for all nodes to be able to reach all pods on those platforms that do not support hostNetwork. The current thinking on Windows support is that hostNetwork is not a requirement for Window users. As such, satisfying the old requirement is very difficult and has no practical benefit. This DOES NOT change the requirements for platforms that support hostNetwork (e.g. Linux) nor does it make hostNetwork optional for Linux nodes. This PR also removes most comparisons to Docker, as they are stale and no longer germane, and cleans up some of the text. * Feedback on contents

thockin assigned bowei, dcbw and chenopis Jan 11, 2019

k8s-ci-robot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. labels Jan 11, 2019

k8s-ci-robot requested review from jimangel and stewart-yu January 11, 2019 21:36

k8s-ci-robot added the language/en Issues or PRs related to English language label Jan 11, 2019

chenopis reviewed Jan 11, 2019

View reviewed changes

bowei reviewed Jan 12, 2019

View reviewed changes

fcrisciani reviewed Jan 12, 2019

View reviewed changes

dcbw reviewed Jan 14, 2019

View reviewed changes

danwinship reviewed Jan 14, 2019

View reviewed changes

thockin added 2 commits February 7, 2019 16:32

Feedback on contents

55fe4e2

thockin force-pushed the relax-network-reqs-nodes-to-pods branch from 471e507 to 55fe4e2 Compare February 8, 2019 00:49

k8s-ci-robot added size/M Denotes a PR that changes 30-99 lines, ignoring generated files. and removed size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Feb 8, 2019

thockin commented Feb 8, 2019

View reviewed changes

k8s-ci-robot assigned danwinship Feb 8, 2019

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Feb 8, 2019

k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Feb 18, 2019

k8s-ci-robot merged commit 971d7af into kubernetes:master Feb 18, 2019

popcornylu reviewed Mar 3, 2019

View reviewed changes

jsturtevant mentioned this pull request May 19, 2021

tests: Spawn poststart / prestop pods on the same node as the http pod kubernetes/kubernetes#101063

Merged

Update network doc for ip-per-pod reqs #12182

Update network doc for ip-per-pod reqs #12182

Conversation

thockin commented Jan 11, 2019

dineshgovindasamy commented Jan 11, 2019

k8sio-netlify-preview-bot commented Jan 11, 2019 • edited by netlify bot Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bowei commented Jan 12, 2019

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dcbw commented Jan 14, 2019

dcbw Jan 14, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

thockin left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

danwinship commented Feb 8, 2019

fcrisciani commented Feb 8, 2019

k8s-ci-robot commented Feb 8, 2019

thockin commented Feb 11, 2019

Rajakavitha1 commented Feb 17, 2019

zparnold commented Feb 18, 2019

k8s-ci-robot commented Feb 18, 2019

Choose a reason for hiding this comment

k8sio-netlify-preview-bot commented Jan 11, 2019 •

edited by netlify bot

Loading

dcbw Jan 14, 2019 •

edited

Loading