-
Notifications
You must be signed in to change notification settings - Fork 7.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Updates for moby 28.0 networking #21612
base: main
Are you sure you want to change the base?
Changes from all commits
ce035f4
c3b09f7
c103e10
a056de8
42decd8
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change | ||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
@@ -63,6 +63,40 @@ | |||||||||||||
For more information about the different drivers, see [Network drivers | ||||||||||||||
overview](./drivers/_index.md). | ||||||||||||||
|
||||||||||||||
### Connecting to multiple networks | ||||||||||||||
|
||||||||||||||
A container can be connected to multiple networks. | ||||||||||||||
|
||||||||||||||
For example, a frontend container may be connected to a bridge network | ||||||||||||||
with external access, and a `--internal` network to communicate with | ||||||||||||||
containers running backend services that do not need external network | ||||||||||||||
access. | ||||||||||||||
|
||||||||||||||
A container may also be connected to different types of network. For example, | ||||||||||||||
an `ipvlan` network to provide internet access, and a `bridge` network for | ||||||||||||||
access to local services. | ||||||||||||||
|
||||||||||||||
When sending packets, if the destination is an address in a directly connected | ||||||||||||||
network, packets are sent to that network. Otherwise, packets are sent to | ||||||||||||||
a default gateway for routing to their destination. In the example above, | ||||||||||||||
Check warning on line 81 in content/manuals/engine/network/_index.md GitHub Actions / vale
|
||||||||||||||
the `ipvlan` network's gateway must be the default gateway. | ||||||||||||||
|
||||||||||||||
The default gateway is selected by docker, and may change each time a | ||||||||||||||
container's network connections change. | ||||||||||||||
To make docker choose a specific default gateway when creating the container | ||||||||||||||
Check failure on line 86 in content/manuals/engine/network/_index.md GitHub Actions / vale
|
||||||||||||||
Comment on lines
+84
to
+86
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||||||||||
or connecting a new network, set a gateway priority. See option `gw-priority` | ||||||||||||||
in [docker run](/reference/cli/docker/container/run.md) and | ||||||||||||||
[docker network connect](/reference/cli/docker/network/connect.md). | ||||||||||||||
Comment on lines
+88
to
+89
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||||||||||
|
||||||||||||||
The default `gw-priority` is `0` and the gateway in the network with the | ||||||||||||||
highest priority is the default gateway. So, when a network should always | ||||||||||||||
be the default gateway, it is enough to set its `gw-priority` to `1`. | ||||||||||||||
|
||||||||||||||
```console | ||||||||||||||
$ docker run --network name=gwnet,gw-priority=1 --network anet1 --name myctr myimage | ||||||||||||||
Check failure on line 96 in content/manuals/engine/network/_index.md GitHub Actions / vale
|
||||||||||||||
$ docker network connect anet2 myctr | ||||||||||||||
Check failure on line 97 in content/manuals/engine/network/_index.md GitHub Actions / vale
|
||||||||||||||
``` | ||||||||||||||
|
||||||||||||||
## Container networks | ||||||||||||||
|
||||||||||||||
In addition to user-defined networks, you can attach a container to another | ||||||||||||||
|
@@ -144,6 +178,14 @@ | |||||||||||||
|
||||||||||||||
## IP address and hostname | ||||||||||||||
|
||||||||||||||
When creating a network, IPv4 address allocation is enabled by default, it | ||||||||||||||
can be disabled using `--ipv4=false`. IPv6 address allocation can be enabled | ||||||||||||||
using `--ipv6`. | ||||||||||||||
|
||||||||||||||
```console | ||||||||||||||
$ docker network create --ipv6 --ipv4=false v6net | ||||||||||||||
``` | ||||||||||||||
|
||||||||||||||
By default, the container gets an IP address for every Docker network it attaches to. | ||||||||||||||
A container receives an IP address out of the IP subnet of the network. | ||||||||||||||
The Docker daemon performs dynamic subnetting and IP address allocation for containers. | ||||||||||||||
|
Original file line number | Diff line number | Diff line change | ||||
---|---|---|---|---|---|---|
|
@@ -105,16 +105,16 @@ | |||||
The following table describes the driver-specific options that you can pass to | ||||||
`--option` when creating a custom network using the `bridge` driver. | ||||||
|
||||||
| Option | Default | Description | | ||||||
|-------------------------------------------------------------------------------------------------|-----------------------------|------------------------------------------------------------------------------------------------| | ||||||
| `com.docker.network.bridge.name` | | Interface name to use when creating the Linux bridge. | | ||||||
| `com.docker.network.bridge.enable_ip_masquerade` | `true` | Enable IP masquerading. | | ||||||
| `com.docker.network.bridge.gateway_mode_ipv4`<br/>`com.docker.network.bridge.gateway_mode_ipv6` | `nat` | Enable NAT and masquerading (`nat`), or only allow direct routing to the container (`routed`). | | ||||||
| `com.docker.network.bridge.enable_icc` | `true` | Enable or Disable inter-container connectivity. | | ||||||
| `com.docker.network.bridge.host_binding_ipv4` | all IPv4 and IPv6 addresses | Default IP when binding container ports. | | ||||||
| `com.docker.network.driver.mtu` | `0` (no limit) | Set the containers network Maximum Transmission Unit (MTU). | | ||||||
| `com.docker.network.container_iface_prefix` | `eth` | Set a custom prefix for container interfaces. | | ||||||
| `com.docker.network.bridge.inhibit_ipv4` | `false` | Prevent Docker from [assigning an IP address](#skip-ip-address-configuration) to the network. | | ||||||
| Option | Default | Description | | ||||||
|-------------------------------------------------------------------------------------------------|-----------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------| | ||||||
| `com.docker.network.bridge.name` | | Interface name to use when creating the Linux bridge. | | ||||||
| `com.docker.network.bridge.enable_ip_masquerade` | `true` | Enable IP masquerading. | | ||||||
| `com.docker.network.bridge.gateway_mode_ipv4`<br/>`com.docker.network.bridge.gateway_mode_ipv6` | `nat` | Enable NAT and masquerading (`nat`), or only allow direct routing to the container (`routed`). See [Packet filtering and firewalls](packet-filtering-firewalls.md). | | ||||||
Check warning on line 112 in content/manuals/engine/network/drivers/bridge.md GitHub Actions / vale
|
||||||
| `com.docker.network.bridge.enable_icc` | `true` | Enable or Disable inter-container connectivity. | | ||||||
| `com.docker.network.bridge.host_binding_ipv4` | all IPv4 and IPv6 addresses | Default IP when binding container ports. | | ||||||
| `com.docker.network.driver.mtu` | `0` (no limit) | Set the containers network Maximum Transmission Unit (MTU). | | ||||||
| `com.docker.network.container_iface_prefix` | `eth` | Set a custom prefix for container interfaces. | | ||||||
| `com.docker.network.bridge.inhibit_ipv4` | `false` | Prevent Docker from [assigning an IP address](#skip-bridge-ip-address-configuration) to the bridge. | | ||||||
|
||||||
Some of these options are also available as flags to the `dockerd` CLI, and you | ||||||
can use them to configure the default `docker0` bridge when starting the Docker | ||||||
|
@@ -229,6 +229,20 @@ | |||||
$ docker network create --ipv6 --subnet 2001:db8:1234::/64 my-net | ||||||
``` | ||||||
|
||||||
If you do not provide a `--subnet` option, a unique-local address (ULA) prefix | ||||||
Check warning on line 232 in content/manuals/engine/network/drivers/bridge.md GitHub Actions / vale
|
||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||
will be chosen automatically. | ||||||
|
||||||
## IPv6-only bridge networks | ||||||
|
||||||
To skip IPv4 address configuration on the bridge and in its containers, create | ||||||
the network with option `--ipv4=false`, and enable IPv6 using `--ipv6`. | ||||||
|
||||||
```console | ||||||
$ docker network create --ipv6 --ipv4=false v6net | ||||||
``` | ||||||
|
||||||
IPv4 address configuration cannot be disabled in the default bridge network. | ||||||
|
||||||
## Use the default bridge network | ||||||
|
||||||
The default `bridge` network is considered a legacy detail of Docker and is not | ||||||
|
@@ -259,7 +273,12 @@ | |||||
} | ||||||
``` | ||||||
|
||||||
Restart Docker for the changes to take effect. | ||||||
In this example: | ||||||
|
||||||
- The bridge's address is "192.168.1.1/24" (from `bip`). | ||||||
- The bridge network's subnet is "192.168.1.0/24" (from `bip`). | ||||||
- Container addresses will be allocated from "192.168.1.0/25" (from `fixed-cidr`). | ||||||
|
||||||
|
||||||
### Use IPv6 with the default bridge network | ||||||
|
||||||
|
@@ -270,22 +289,34 @@ | |||||
user-defined networks. The addresses in below are examples from the | ||||||
IPv6 documentation range. | ||||||
|
||||||
- Option `ipv6` is required | ||||||
- Option `fixed-cidr-v6` is required, it specifies the network prefix to be used. | ||||||
- Option `ipv6` is required. | ||||||
- Option `bip6` is optional, it specifies the address of the default bridge, which | ||||||
will be used as the default gateway by containers. It also specifies the subnet | ||||||
for the bridge network. | ||||||
- Option `fixed-cidr-v6` is optional, it specifies the address range docker may | ||||||
Check failure on line 296 in content/manuals/engine/network/drivers/bridge.md GitHub Actions / vale
|
||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||
automatically allocate to containers. | ||||||
- The prefix should normally be `/64` or shorter. | ||||||
- For experimentation on a local network, it is better to use a Unique Local | ||||||
prefix (matching `fd00::/8`) than a Link Local prefix (matching `fe80::/10`). | ||||||
Address (ULA) prefix (matching `fd00::/8`) than a Link Local prefix (matching | ||||||
`fe80::/10`). | ||||||
- Option `default-gateway-v6` is optional. If unspecified, the default is the first | ||||||
address in the `fixed-cidr-v6` subnet. | ||||||
|
||||||
```json | ||||||
{ | ||||||
"ipv6": true, | ||||||
"bip6": "2001:db8::1111/64", | ||||||
"fixed-cidr-v6": "2001:db8::/64", | ||||||
"default-gateway-v6": "2001:db8:abcd::89" | ||||||
} | ||||||
``` | ||||||
|
||||||
If no `bip6` is specified, `fixed-cidr-v6` defines the subnet for the bridge | ||||||
network. If no `bip6` or `fixed-cidr-v6` is specified, a ULA prefix will be | ||||||
chosen. | ||||||
|
||||||
Restart Docker for changes to take effect. | ||||||
|
||||||
## Connection limit for bridge networks | ||||||
|
||||||
Due to limitations set by the Linux kernel, bridge networks become unstable and | ||||||
|
@@ -295,20 +326,22 @@ | |||||
For more information about this limitation, see | ||||||
[moby/moby#44973](https://github.com/moby/moby/issues/44973#issuecomment-1543747718). | ||||||
|
||||||
## Skip IP address configuration | ||||||
## Skip Bridge IP address configuration | ||||||
|
||||||
The bridge is normally assigned the network's `--gateway` address, which is | ||||||
used as the default route from the bridge network to other networks. | ||||||
|
||||||
The `com.docker.network.bridge.inhibit_ipv4` option lets you create a network | ||||||
that uses an existing bridge and have Docker skip configuring the IPv4 address | ||||||
on the bridge. This is useful if you want to configure the IP address for the | ||||||
bridge manually. For instance if you add a physical interface to your bridge, | ||||||
and need to move its IP address to the bridge interface. | ||||||
without the IPv4 gateway address being assigned to the bridge. This is useful | ||||||
if you want to configure the gateway IP address for the bridge manually. For | ||||||
instance if you add a physical interface to your bridge, and need it to have | ||||||
the gateway address. | ||||||
|
||||||
To use this option, you should first configure the Docker daemon to use a | ||||||
self-managed bridge, using the `bridge` option in the `daemon.json` or the | ||||||
`dockerd --bridge` flag. | ||||||
With this configuration, north-south traffic (to and from the bridge network) | ||||||
won't work unless you've manually configured the gateway address on the bridge, | ||||||
or a device attached to it. | ||||||
|
||||||
With this configuration, north-south traffic won't work unless you've manually | ||||||
configured the IP address for the bridge. | ||||||
This option can only be used with user-defined bridge networks. | ||||||
|
||||||
## Next steps | ||||||
|
||||||
|
Original file line number | Diff line number | Diff line change | ||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
@@ -129,31 +129,69 @@ | |||||||||||||
addresses that exist within a host. | ||||||||||||||
|
||||||||||||||
But, particularly with IPv6 you may prefer to avoid using NAT and instead | ||||||||||||||
arrange for external routing to container addresses. | ||||||||||||||
arrange for external routing to container addresses ("direct routing"). | ||||||||||||||
|
||||||||||||||
To access containers on a bridge network from outside the Docker host, | ||||||||||||||
you must set up routing to the bridge network via an address on the Docker | ||||||||||||||
host. This can be achieved using static routes, Border Gateway Protocol | ||||||||||||||
(BGP), or any other means appropriate for your network. | ||||||||||||||
|
||||||||||||||
The bridge network driver has options | ||||||||||||||
`com.docker.network.bridge.gateway_mode_ipv6=<nat|routed>` and | ||||||||||||||
`com.docker.network.bridge.gateway_mode_ipv4=<nat|routed>`. | ||||||||||||||
Within a local layer-2 network, any remote host can set up a static route | ||||||||||||||
to a container network via the docker host's address on the local network. | ||||||||||||||
Check failure on line 140 in content/manuals/engine/network/packet-filtering-firewalls.md GitHub Actions / vale
|
||||||||||||||
Comment on lines
+139
to
+140
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||||||||||
So, those local remote hosts can access containers directly. Remote hosts | ||||||||||||||
outside the local network will only have direct access to containers if | ||||||||||||||
routers are configured to enable it. | ||||||||||||||
Comment on lines
+141
to
+143
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Hmm, reading this again, "local remote hosts" are a bit weird! How about I change it to ...
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Sounds good to me! |
||||||||||||||
|
||||||||||||||
#### Gateway modes | ||||||||||||||
|
||||||||||||||
The bridge network driver has the following options: | ||||||||||||||
- `com.docker.network.bridge.gateway_mode_ipv6` | ||||||||||||||
- `com.docker.network.bridge.gateway_mode_ipv4` | ||||||||||||||
|
||||||||||||||
Each of these can be set to one of the gateway modes: | ||||||||||||||
- `nat` | ||||||||||||||
- `nat-unprotected` | ||||||||||||||
- `routed` | ||||||||||||||
|
||||||||||||||
The default is `nat`, NAT and masquerading rules are set up for each | ||||||||||||||
published container port. With mode `routed`, no NAT or masquerading rules | ||||||||||||||
are set up, but `iptables` are still set up so that only published container | ||||||||||||||
ports are accessible. | ||||||||||||||
published container port. Packets leaving the host will use a host address. | ||||||||||||||
|
||||||||||||||
With mode `routed`, no NAT or masquerading rules are set up, but `iptables` | ||||||||||||||
are still set up so that only published container ports are accessible. | ||||||||||||||
Outgoing packets from the container will use the container's address, | ||||||||||||||
not a host address. | ||||||||||||||
|
||||||||||||||
In `nat` mode, when a port is published to a specific host address, that | ||||||||||||||
port is only accessible via the host interface with that address. So, | ||||||||||||||
for example, publishing a port to an address on the loopback interface | ||||||||||||||
means remote hosts cannot access it. | ||||||||||||||
|
||||||||||||||
However, using direct routing, published container ports are always | ||||||||||||||
accessible from remote hosts, unless the Docker host's firewall has | ||||||||||||||
additional restrictions. Hosts on the local layer-2 network can set up | ||||||||||||||
direct routing without needing any additional network configuration. | ||||||||||||||
Hosts outside the local network can only use direct routing to the | ||||||||||||||
container if the network's routers are configured to enable it. | ||||||||||||||
|
||||||||||||||
In `nat-unprotected` mode, unpublished container ports are also | ||||||||||||||
accessible using direct routing, no port filtering rules are set up. | ||||||||||||||
This mode is included for compatibility with legacy default behaviour. | ||||||||||||||
|
||||||||||||||
The gateway mode also affects communication between containers that | ||||||||||||||
are connected to different docker networks on the same host. | ||||||||||||||
Check failure on line 181 in content/manuals/engine/network/packet-filtering-firewalls.md GitHub Actions / vale
|
||||||||||||||
- In `nat` and `nat-unprotected` modes, containers in other bridge | ||||||||||||||
networks can only access published ports via the host addresses they | ||||||||||||||
are published to. Direct routing from other networks is not allowed. | ||||||||||||||
- In `routed` mode containers in other networks can use direct | ||||||||||||||
routing to access ports, without going via a host address. | ||||||||||||||
|
||||||||||||||
In `routed` mode, a host port in a `-p` or `--publish` port mapping is | ||||||||||||||
not used, and the host address is only used to decide whether to apply | ||||||||||||||
the mapping to IPv4 or IPv6. So, when a mapping only applies to `routed` | ||||||||||||||
mode, only addresses `0.0.0.0` or `::1` are allowed, and a host port | ||||||||||||||
must not be given. | ||||||||||||||
|
||||||||||||||
Mapped container ports, in `nat` or `routed` mode, are accessible from | ||||||||||||||
any remote address, if routing is set up in the network, unless the | ||||||||||||||
Docker host's firewall has additional restrictions. | ||||||||||||||
mode, only addresses `0.0.0.0` or `::` should be used, and a host port | ||||||||||||||
should not be given. If a specific address or port is given, it will | ||||||||||||||
have no effect on the published port and a warning message will be | ||||||||||||||
logged. | ||||||||||||||
|
||||||||||||||
#### Example | ||||||||||||||
|
||||||||||||||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What is
--internal
, why is it written like a CLI flag?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good point - how about I make it a link to https://docs.docker.com/reference/cli/docker/network/create/#internal ?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes that works! I didn't recall this flag when reading 😅