Creating a new network "destroys" the previous one?

[dgouju]

Hello,
I'm running the following test scenario, and it seems that when creating a new network, it "destroys" the previous one in the plugin (but it is still present in docker network ls). I'm running docker 1.9.1 on ubuntu 15.10 with kernel 4.2.0-23-generic.

# ip link add name vlan100 link eth1 type vlan id 100
# ip link set vlan100 up
# ip link add name vlan101 link eth1 type vlan id 101
# ip link set vlan101 up

# docker network create -d ipvlan -o host_iface=vlan100 vlan100
b9027b3565d19682439bba1b6ca01ace1b0fa1b68447397827f10a197ee7fd0d
# docker run --net=vlan100 -it --rm ubuntu
root@8412224c6e59:/# exit
exit

# docker network create -d ipvlan -o host_iface=vlan101 vlan101
c1a7b21d984c606e15ee6491ccec5876febbe401837b35ae6bbd53346b3b6edf
root@Docker-dgouju:~# docker run --net=vlan101 -it --rm ubuntu
root@657c0fbc1fe8:/# exit
exit

So, far, so good. Now I create a container on the first network I created:

# docker run --net=vlan100 -it --rm ubuntu
Error response from daemon: Cannot start container 183946dbdf1a8e64f970e39066924776ece55331f90d094b1994048de0a5e253: failed to create endpoint dreamy_booth on network vlan100: remote: No such network b9027b3565d19682439bba1b6ca01ace1b0fa1b68447397827f10a197ee7fd0d

And I got the following error in the plugin:

WARN[0272] Network not found, [ b9027b3565d19682439bba1b6ca01ace1b0fa1b68447397827f10a197ee7fd0d ]

And in docker:

DEBU[0057] Calling POST /v1.21/containers/create
INFO[0057] POST /v1.21/containers/create
DEBU[0057] Calling POST /v1.21/containers/9fbe0659610f350719866da4e8415956f6f128a050d20922145664ba70349d4b/attach
INFO[0057] POST /v1.21/containers/9fbe0659610f350719866da4e8415956f6f128a050d20922145664ba70349d4b/attach?stderr=1&stdin=1&stdout=1&stream=1
DEBU[0057] attach: stdin: begin
DEBU[0057] attach: stdout: begin
DEBU[0057] attach: stderr: begin
DEBU[0057] Calling POST /v1.21/containers/9fbe0659610f350719866da4e8415956f6f128a050d20922145664ba70349d4b/start
INFO[0057] POST /v1.21/containers/9fbe0659610f350719866da4e8415956f6f128a050d20922145664ba70349d4b/start
DEBU[0057] Assigning addresses for endpoint small_mclean's interface on network vlan100
DEBU[0057] RequestAddress(LocalDefault/172.18.0.0/16, <nil>, map[])
DEBU[0057] Releasing addresses for endpoint small_mclean's interface on network vlan100
DEBU[0057] ReleaseAddress(LocalDefault/172.18.0.0/16, 172.18.0.2)
WARN[0057] failed to cleanup ipc mounts:
failed to umount /var/lib/docker/containers/9fbe0659610f350719866da4e8415956f6f128a050d20922145664ba70349d4b/shm: no such file or directory
failed to umount /var/lib/docker/containers/9fbe0659610f350719866da4e8415956f6f128a050d20922145664ba70349d4b/mqueue: no such file or directory
ERRO[0057] Handler for POST /v1.21/containers/9fbe0659610f350719866da4e8415956f6f128a050d20922145664ba70349d4b/start returned error: Cannot start container 9fbe0659610f350719866da4e8415956f6f128a050d20922145664ba70349d4b: failed to create endpoint small_mclean on network vlan100: remote: No such network 65ef47eca732aa15008ebc244f06df8c98fd0ebb2ef10254aa89d240644d291c
ERRO[0057] HTTP Error                                    err=Cannot start container 9fbe0659610f350719866da4e8415956f6f128a050d20922145664ba70349d4b: failed to create endpoint small_mclean on network vlan100: remote: No such network 65ef47eca732aa15008ebc244f06df8c98fd0ebb2ef10254aa89d240644d291c statusCode=500
DEBU[0057] Calling DELETE /v1.21/containers/9fbe0659610f350719866da4e8415956f6f128a050d20922145664ba70349d4b
INFO[0057] DELETE /v1.21/containers/9fbe0659610f350719866da4e8415956f6f128a050d20922145664ba70349d4b?v=1
DEBU[0057] Closing buffered stdin pipe
DEBU[0057] attach: stdin: end
DEBU[0057] attach: stdout: end
DEBU[0057] attach: stderr: end

But the network is still known in docker:

# docker network ls
NETWORK ID          NAME                DRIVER
5b407d8f4efa        bridge              bridge
b9027b3565d1        vlan100             ipvlan
c1a7b21d984c        vlan101             ipvlan
b60302697606        none                null
e122df25d512        host                host
# docker network inspect vlan100
[
    {
        "Name": "vlan100",
        "Id": "b9027b3565d19682439bba1b6ca01ace1b0fa1b68447397827f10a197ee7fd0d",
        "Scope": "local",
        "Driver": "ipvlan",
        "IPAM": {
            "Driver": "default",
            "Config": [
                {}
            ]
        },
        "Containers": {},
        "Options": {
            "host_iface": "vlan100"
        }
    }
]
# docker network inspect vlan101
[
    {
        "Name": "vlan101",
        "Id": "c1a7b21d984c606e15ee6491ccec5876febbe401837b35ae6bbd53346b3b6edf",
        "Scope": "local",
        "Driver": "ipvlan",
        "IPAM": {
            "Driver": "default",
            "Config": [
                {}
            ]
        },
        "Containers": {},
        "Options": {
            "host_iface": "vlan101"
        }
    }
]

Thank you for your help!

[YujiOshima]

@dgouju yeah, when create new network, the driver replace the "driver's" networkID not "network's" id.
https://github.com/gopher-net/ipvlan-docker-plugin/blob/master/plugin/ipvlan/driver.go#L253
So you cannot connect any containers to old network as driver forgot its networkID.
https://github.com/gopher-net/ipvlan-docker-plugin/blob/master/plugin/ipvlan/driver.go#L410
ping @nerdalert

[nerdalert]

Hey @dgouju sorry this took so long. Ive been working on getting this in natively instead of 3rd party drivers. If you want to give it a shot and 👍 if it works for you or feedback if you have issues. There is a binary in the PR notes in the patch https://github.com/docker/libnetwork/ the Pull request is 964. Closing this out. Thanks!

[dgouju]

@nerdalert Thank you no problem! I'm in a training this week, I will try to look at it next week!