New networking flake (root, compose) #11123

edsantiago · 2021-08-03T21:15:23Z

New flake. Looks identical to #10052, except this is root. It is also happening on my new PR, which sits on top of #11091.

[+0106s] ok 15 simple_port_map - up
[+0110s] curl: (7) Failed to connect to 127.0.0.1 port 5000: No route to host
[+0113s] curl: (7) Failed to connect to 127.0.0.1 port 5000: No route to host
[+0117s] curl: (7) Failed to connect to 127.0.0.1 port 5000: No route to host
[+0124s] curl: (7) Failed to connect to 127.0.0.1 port 5000: No route to host
         not ok 16 simple_port_map - curl (port 5000) failed with status 7
[rest of log is way too long]

The text was updated successfully, but these errors were encountered:

Luap99 · 2021-08-04T07:57:40Z

Just to be clear, this is not new. You have linked these flakes also in #10052, example log. My PR was intended to only fix the rootless issue.

This issue also happens in the rootful remote system tests, see #10052 (comment)

Luap99 · 2021-09-01T11:34:51Z

@edsantiago I will take a look today. It looks like the No route to host error happens in many different tests.

Luap99 · 2021-09-02T17:15:03Z

OK I was able to reproduce this once in a VM. The network configuration looked fine so I have no idea what is causing this. The only suspicious thing are these kernel messages:

Sep 02 11:01:15 pholzing-fedora-c6737534580424704 kernel: ll header: 00000000: ff ff ff ff ff ff aa 6e c1 86 28 69 08 06
Sep 02 11:01:15 pholzing-fedora-c6737534580424704 kernel: IPv4: martian source 10.89.0.1 from 10.89.0.3, on dev veth839e3188
Sep 02 11:01:14 pholzing-fedora-c6737534580424704 kernel: ll header: 00000000: ff ff ff ff ff ff aa 6e c1 86 28 69 08 06
Sep 02 11:01:14 pholzing-fedora-c6737534580424704 kernel: IPv4: martian source 10.89.0.1 from 10.89.0.3, on dev veth839e3188
Sep 02 11:01:12 pholzing-fedora-c6737534580424704 kernel: ll header: 00000000: ff ff ff ff ff ff aa 6e c1 86 28 69 08 06
Sep 02 11:01:12 pholzing-fedora-c6737534580424704 kernel: IPv4: martian source 10.89.0.1 from 10.89.0.3, on dev veth839e3188
Sep 02 11:01:11 pholzing-fedora-c6737534580424704 kernel: ll header: 00000000: ff ff ff ff ff ff aa 6e c1 86 28 69 08 06
Sep 02 11:01:11 pholzing-fedora-c6737534580424704 kernel: IPv4: martian source 10.89.0.1 from 10.89.0.3, on dev veth839e3188
Sep 02 11:01:10 pholzing-fedora-c6737534580424704 kernel: ll header: 00000000: ff ff ff ff ff ff aa 6e c1 86 28 69 08 06
Sep 02 11:01:10 pholzing-fedora-c6737534580424704 kernel: IPv4: martian source 10.89.0.1 from 10.89.0.3, on dev veth839e3188
Sep 02 11:01:09 pholzing-fedora-c6737534580424704 kernel: ll header: 00000000: ff ff ff ff ff ff aa 6e c1 86 28 69 08 06
Sep 02 11:01:09 pholzing-fedora-c6737534580424704 kernel: IPv4: martian source 10.89.0.1 from 10.89.0.3, on dev veth839e3188
Sep 02 11:01:08 pholzing-fedora-c6737534580424704 kernel: ll header: 00000000: ff ff ff ff ff ff aa 6e c1 86 28 69 08 06
Sep 02 11:01:08 pholzing-fedora-c6737534580424704 kernel: IPv4: martian source 10.89.0.1 from 10.89.0.3, on dev veth839e3188
Sep 02 11:01:07 pholzing-fedora-c6737534580424704 kernel: ll header: 00000000: ff ff ff ff ff ff aa 6e c1 86 28 69 08 06
Sep 02 11:01:07 pholzing-fedora-c6737534580424704 kernel: IPv4: martian source 10.89.0.1 from 10.89.0.3, on dev veth839e3188

mheon · 2021-09-02T17:52:37Z

Martian seems to imply the host doesn't know about the subnet. Maybe the interface on the host into the bridge network didn't properly pick up an IP address?

Luap99 · 2021-09-02T19:02:54Z

I checked the interface ips and iptables and everything looked normal.

mheon · 2021-09-02T19:49:24Z

Routing table, maybe? It shouldn't be required for directly-attached subnets, but I'm running out of ideas.

Luap99 · 2021-09-02T19:57:53Z

If I get it to reproduce again I can check. Somehow it runs for hours without failure now.

Luap99 · 2021-09-03T08:35:24Z

OK I got the root problem. The cni-podman interface is down because the veth interface is not attached to it.

# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1460 qdisc mq state UP group default qlen 1000
    link/ether 42:01:0a:9c:00:04 brd ff:ff:ff:ff:ff:ff
    altname enp0s4
    altname ens4
    inet 10.156.0.4/32 scope global dynamic noprefixroute eth0
       valid_lft 84993sec preferred_lft 84993sec
    inet6 fe80::4001:aff:fe9c:4/64 scope link 
       valid_lft forever preferred_lft forever
123: cni-podman1: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default qlen 1000
    link/ether e6:87:2b:1c:ea:c5 brd ff:ff:ff:ff:ff:ff
    inet 10.89.0.1/24 brd 10.89.0.255 scope global cni-podman1
       valid_lft forever preferred_lft forever
    inet6 fe80::e487:2bff:fe1c:eac5/64 scope link 
       valid_lft forever preferred_lft forever
124: veth92b50314@if3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default 
    link/ether 16:9b:5e:c1:ad:f3 brd ff:ff:ff:ff:ff:ff link-netns cni-d0086c0e-716b-5a14-1ad6-1301ba367082
    inet6 fe80::107e:a94a:1305:6b28/64 scope link noprefixroute 
       valid_lft forever preferred_lft forever
       
# ip route 
default via 10.156.0.1 dev eth0 proto dhcp metric 100 
10.89.0.0/24 dev cni-podman1 proto kernel scope link src 10.89.0.1 linkdown 
10.156.0.1 dev eth0 proto dhcp scope link metric 100

I don't know why but I think it has something to the with the NetworkManager. This is what I see in the journal the failed run:

Sep 02 16:12:16 pholzing-fedora-c6737534580424704 NetworkManager[621]: <info>  [1630617136.6387] policy: set-hostname: current hostname was changed outside NetworkManager: 'pholzing-fedora-c6737534580424704'
Sep 02 16:12:16 pholzing-fedora-c6737534580424704 NetworkManager[621]: <info>  [1630617136.6337] device (cni-podman1): state change: config -> unmanaged (reason 'connection-assumed', sys-iface-state: 'external')
Sep 02 16:12:16 pholzing-fedora-c6737534580424704 NetworkManager[621]: <info>  [1630617136.6331] device (cni-podman1): state change: prepare -> config (reason 'none', sys-iface-state: 'external')
Sep 02 16:12:16 pholzing-fedora-c6737534580424704 NetworkManager[621]: <info>  [1630617136.6324] device (cni-podman1): state change: disconnected -> prepare (reason 'none', sys-iface-state: 'external')
Sep 02 16:12:16 pholzing-fedora-c6737534580424704 NetworkManager[621]: <info>  [1630617136.6316] device (veth92b50314): state change: prepare -> config (reason 'none', sys-iface-state: 'managed')
Sep 02 16:12:16 pholzing-fedora-c6737534580424704 NetworkManager[621]: <info>  [1630617136.6315] device (cni-podman1): detached bridge port veth92b50314
Sep 02 16:12:16 pholzing-fedora-c6737534580424704 audit: ANOM_PROMISCUOUS dev=veth92b50314 prom=0 old_prom=256 auid=4294967295 uid=0 gid=0 ses=4294967295
Sep 02 16:12:16 pholzing-fedora-c6737534580424704 kernel: ll header: 00000000: ff ff ff ff ff ff de 89 76 f7 24 bd 08 06
Sep 02 16:12:16 pholzing-fedora-c6737534580424704 kernel: cni-podman1: port 1(veth92b50314) entered disabled state
Sep 02 16:12:16 pholzing-fedora-c6737534580424704 kernel: IPv4: martian source 10.89.0.9 from 10.89.0.9, on dev eth0
Sep 02 16:12:16 pholzing-fedora-c6737534580424704 kernel: device veth92b50314 left promiscuous mode
Sep 02 16:12:16 pholzing-fedora-c6737534580424704 kernel: ll header: 00000000: ff ff ff ff ff ff de 89 76 f7 24 bd 08 06
Sep 02 16:12:16 pholzing-fedora-c6737534580424704 kernel: IPv4: martian source 10.89.0.9 from 10.89.0.9, on dev cni-podman1
Sep 02 16:12:16 pholzing-fedora-c6737534580424704 NetworkManager[621]: <info>  [1630617136.6006] device (veth92b50314): state change: disconnected -> prepare (reason 'none', sys-iface-state: 'managed')
Sep 02 16:12:16 pholzing-fedora-c6737534580424704 NetworkManager[621]: <info>  [1630617136.5996] device (cni-podman1): Activation: starting connection 'cni-podman1' (34cb3444-73f8-400a-9793-55d20532450b)
Sep 02 16:12:16 pholzing-fedora-c6737534580424704 NetworkManager[621]: <info>  [1630617136.5969] device (cni-podman1): state change: unavailable -> disconnected (reason 'connection-assumed', sys-iface-state: 'external')
Sep 02 16:12:16 pholzing-fedora-c6737534580424704 NetworkManager[621]: <info>  [1630617136.5857] device (veth92b50314): Activation: starting connection 'Wired connection 1' (6908b35e-4c36-36fa-8b44-8ff5e2557e9e)
Sep 02 16:12:16 pholzing-fedora-c6737534580424704 NetworkManager[621]: <info>  [1630617136.5814] device (cni-podman1): state change: unmanaged -> unavailable (reason 'connection-assumed', sys-iface-state: 'external')
Sep 02 16:12:16 pholzing-fedora-c6737534580424704 NetworkManager[621]: <info>  [1630617136.5802] policy: auto-activating connection 'Wired connection 1' (6908b35e-4c36-36fa-8b44-8ff5e2557e9e)
Sep 02 16:12:16 pholzing-fedora-c6737534580424704 NetworkManager[621]: <info>  [1630617136.5703] device (veth92b50314): state change: unavailable -> disconnected (reason 'none', sys-iface-state: 'managed')
Sep 02 16:12:16 pholzing-fedora-c6737534580424704 NetworkManager[621]: <info>  [1630617136.5685] settings: (veth92b50314): created default wired connection 'Wired connection 1'
Sep 02 16:12:16 pholzing-fedora-c6737534580424704 NetworkManager[621]: <info>  [1630617136.5553] device (veth92b50314): state change: unmanaged -> unavailable (reason 'managed', sys-iface-state: 'external')
Sep 02 16:12:16 pholzing-fedora-c6737534580424704 NetworkManager[621]: <info>  [1630617136.5544] device (cni-podman1): carrier: link connected
Sep 02 16:12:16 pholzing-fedora-c6737534580424704 systemd-udevd[42556]: ethtool: autonegotiation is unset or enabled, the speed and duplex are not writable.
Sep 02 16:12:16 pholzing-fedora-c6737534580424704 kernel: cni-podman1: port 1(veth92b50314) entered forwarding state
Sep 02 16:12:16 pholzing-fedora-c6737534580424704 kernel: cni-podman1: port 1(veth92b50314) entered blocking state
Sep 02 16:12:16 pholzing-fedora-c6737534580424704 audit: ANOM_PROMISCUOUS dev=veth92b50314 prom=256 old_prom=0 auid=0 uid=0 gid=0 ses=8
Sep 02 16:12:16 pholzing-fedora-c6737534580424704 kernel: device veth92b50314 entered promiscuous mode
Sep 02 16:12:16 pholzing-fedora-c6737534580424704 kernel: cni-podman1: port 1(veth92b50314) entered disabled state
Sep 02 16:12:16 pholzing-fedora-c6737534580424704 kernel: cni-podman1: port 1(veth92b50314) entered blocking state
Sep 02 16:12:16 pholzing-fedora-c6737534580424704 systemd-udevd[42556]: Using default interface naming scheme 'v247'.
Sep 02 16:12:16 pholzing-fedora-c6737534580424704 systemd-udevd[42556]: ethtool: autonegotiation is unset or enabled, the speed and duplex are not writable.
Sep 02 16:12:16 pholzing-fedora-c6737534580424704 kernel: IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
Sep 02 16:12:16 pholzing-fedora-c6737534580424704 kernel: IPv6: ADDRCONF(NETDEV_CHANGE): veth92b50314: link becomes ready
Sep 02 16:12:16 pholzing-fedora-c6737534580424704 NetworkManager[621]: <info>  [1630617136.4981] device (veth92b50314): carrier: link connected
Sep 02 16:12:16 pholzing-fedora-c6737534580424704 NetworkManager[621]: <info>  [1630617136.4959] manager: (veth92b50314): new Veth device (/org/freedesktop/NetworkManager/Devices/131)
Sep 02 16:12:16 pholzing-fedora-c6737534580424704 systemd-udevd[42555]: ethtool: autonegotiation is unset or enabled, the speed and duplex are not writable.
Sep 02 16:12:16 pholzing-fedora-c6737534580424704 systemd-udevd[42555]: Using default interface naming scheme 'v247'.
Sep 02 16:12:16 pholzing-fedora-c6737534580424704 NetworkManager[621]: <info>  [1630617136.4806] manager: (cni-podman1): new Bridge device (/org/freedesktop/NetworkManager/Devices/130)
Sep 02 16:12:16 pholzing-fedora-c6737534580424704 podman[42273]: 2021-09-02 16:12:16.413299744 -0500 CDT m=+9.743713356 network connect baac746ddfec0c106f5ab771fdc85c3752db9001417fe6d6824726c84cc62301

This is what I see for a successful run:

Sep 02 16:11:53 pholzing-fedora-c6737534580424704 NetworkManager[621]: <info>  [1630617113.7885] device (cni-podman1): Activation: successful, device activated.
Sep 02 16:11:53 pholzing-fedora-c6737534580424704 NetworkManager[621]: <info>  [1630617113.7885] policy: set-hostname: current hostname was changed outside NetworkManager: 'pholzing-fedora-c6737534580424704'
Sep 02 16:11:53 pholzing-fedora-c6737534580424704 systemd-udevd[41986]: ethtool: autonegotiation is unset or enabled, the speed and duplex are not writable.
Sep 02 16:11:53 pholzing-fedora-c6737534580424704 NetworkManager[621]: <info>  [1630617113.7850] device (cni-podman1): state change: secondaries -> activated (reason 'none', sys-iface-state: 'external')
Sep 02 16:11:53 pholzing-fedora-c6737534580424704 NetworkManager[621]: <info>  [1630617113.7839] device (cni-podman1): state change: ip-check -> secondaries (reason 'none', sys-iface-state: 'external')
Sep 02 16:11:53 pholzing-fedora-c6737534580424704 NetworkManager[621]: <info>  [1630617113.7704] device (cni-podman1): state change: ip-config -> ip-check (reason 'none', sys-iface-state: 'external')
Sep 02 16:11:53 pholzing-fedora-c6737534580424704 NetworkManager[621]: <info>  [1630617113.7700] device (cni-podman1): state change: config -> ip-config (reason 'none', sys-iface-state: 'external')
Sep 02 16:11:53 pholzing-fedora-c6737534580424704 NetworkManager[621]: <info>  [1630617113.7696] device (cni-podman1): state change: prepare -> config (reason 'none', sys-iface-state: 'external')
Sep 02 16:11:53 pholzing-fedora-c6737534580424704 NetworkManager[621]: <info>  [1630617113.7691] device (cni-podman1): state change: disconnected -> prepare (reason 'none', sys-iface-state: 'external')
Sep 02 16:11:53 pholzing-fedora-c6737534580424704 NetworkManager[621]: <info>  [1630617113.7689] device (cni-podman1): Activation: starting connection 'cni-podman1' (15e7bd2b-cacd-43af-afc0-04d3f7213fe6)
Sep 02 16:11:53 pholzing-fedora-c6737534580424704 NetworkManager[621]: <info>  [1630617113.7678] device (cni-podman1): state change: unavailable -> disconnected (reason 'connection-assumed', sys-iface-state: 'external')
Sep 02 16:11:53 pholzing-fedora-c6737534580424704 NetworkManager[621]: <info>  [1630617113.7629] device (cni-podman1): state change: unmanaged -> unavailable (reason 'connection-assumed', sys-iface-state: 'external')
Sep 02 16:11:53 pholzing-fedora-c6737534580424704 NetworkManager[621]: <info>  [1630617113.7621] device (veth0d7ed49c): carrier: link connected
Sep 02 16:11:53 pholzing-fedora-c6737534580424704 kernel: ll header: 00000000: ff ff ff ff ff ff aa c6 45 d2 dd 3e 08 06
Sep 02 16:11:53 pholzing-fedora-c6737534580424704 kernel: IPv4: martian source 10.89.0.9 from 10.89.0.9, on dev eth0
Sep 02 16:11:53 pholzing-fedora-c6737534580424704 kernel: ll header: 00000000: ff ff ff ff ff ff aa c6 45 d2 dd 3e 08 06
Sep 02 16:11:53 pholzing-fedora-c6737534580424704 kernel: IPv4: martian source 10.89.0.9 from 10.89.0.9, on dev cni-podman1
Sep 02 16:11:53 pholzing-fedora-c6737534580424704 NetworkManager[621]: <info>  [1630617113.7529] device (cni-podman1): carrier: link connected
Sep 02 16:11:53 pholzing-fedora-c6737534580424704 NetworkManager[621]: <info>  [1630617113.7469] manager: (veth0d7ed49c): new Veth device (/org/freedesktop/NetworkManager/Devices/129)
Sep 02 16:11:53 pholzing-fedora-c6737534580424704 systemd-udevd[41985]: Using default interface naming scheme 'v247'.
Sep 02 16:11:53 pholzing-fedora-c6737534580424704 systemd-udevd[41985]: ethtool: autonegotiation is unset or enabled, the speed and duplex are not writable.
Sep 02 16:11:53 pholzing-fedora-c6737534580424704 kernel: IPv6: ADDRCONF(NETDEV_CHANGE): cni-podman1: link becomes ready
Sep 02 16:11:53 pholzing-fedora-c6737534580424704 kernel: cni-podman1: port 1(veth0d7ed49c) entered forwarding state
Sep 02 16:11:53 pholzing-fedora-c6737534580424704 kernel: cni-podman1: port 1(veth0d7ed49c) entered blocking state
Sep 02 16:11:53 pholzing-fedora-c6737534580424704 audit: ANOM_PROMISCUOUS dev=veth0d7ed49c prom=256 old_prom=0 auid=0 uid=0 gid=0 ses=8
Sep 02 16:11:53 pholzing-fedora-c6737534580424704 kernel: device veth0d7ed49c entered promiscuous mode
Sep 02 16:11:53 pholzing-fedora-c6737534580424704 kernel: cni-podman1: port 1(veth0d7ed49c) entered disabled state
Sep 02 16:11:53 pholzing-fedora-c6737534580424704 kernel: cni-podman1: port 1(veth0d7ed49c) entered blocking state
Sep 02 16:11:53 pholzing-fedora-c6737534580424704 kernel: IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
Sep 02 16:11:53 pholzing-fedora-c6737534580424704 systemd-udevd[41983]: ethtool: autonegotiation is unset or enabled, the speed and duplex are not writable.
Sep 02 16:11:53 pholzing-fedora-c6737534580424704 systemd-udevd[41983]: Using default interface naming scheme 'v247'.
Sep 02 16:11:53 pholzing-fedora-c6737534580424704 NetworkManager[621]: <info>  [1630617113.6687] manager: (cni-podman1): new Bridge device (/org/freedesktop/NetworkManager/Devices/128)
Sep 02 16:11:53 pholzing-fedora-c6737534580424704 podman[41704]: 2021-09-02 16:11:53.61021465 -0500 CDT m=+9.183835402 network connect 94e1f74479de15b780928b70bae397be785c13ec90b42b4554b4564b04cb4528

The offending line is NetworkManager[621]: <info> [1630617136.6315] device (cni-podman1): detached bridge port veth92b50314. I think NetworkManager wants to manage the veth device NetworkManager[621]: <info> [1630617136.5685] settings: (veth92b50314): created default wired connection 'Wired connection 1' and NetworkManager[621]: <info> [1630617136.5857] device (veth92b50314): Activation: starting connection 'Wired connection 1' (6908b35e-4c36-36fa-8b44-8ff5e2557e9e). In other successful runs this does not happen.

Here is the full journal in case someone wants to see this net-flake-journal.txt

Luap99 · 2021-09-03T12:53:29Z

I am now 100% sure that this is NetworkManager doing weird things, I added a NetworkManager config to exclude the cni/veth interfaces:

cat > /etc/NetworkManager/conf.d/podman-cni.conf << EOF
[keyfile]
unmanaged-devices=interface-name:cni-podman*;interface-name:veth*
EOF

It is now running over 4 hours without failure.

mheon · 2021-09-03T13:22:48Z

Alright... I wonder if this is something we have to worry about on user systems. We obviously can't be as general (excluding veth* on a user system won't fly)... Probably not worth it until we get a user report of this, though.

Luap99 · 2021-09-03T13:36:30Z

Maybe this is something special in the GCOULD VM setup. I was not able to reproduce this locally or 1minutetip.
I think we can add the workaround to our CI images and then not worry again until we see this elsewhere.

cevich · 2021-09-08T15:48:34Z

Opened containers/automation_images#88

These images contain a workaround for: containers/podman#11123 Ref: containers/podman#11070 containers/automation_images#88 Signed-off-by: Chris Evich <[email protected]>

These images contain a workaround for: containers/podman#11123 Prior-Ubuntu support is being dropped everywhere. Ref: containers/podman#11070 containers/automation_images#88 Signed-off-by: Chris Evich <[email protected]>

Mirror of changes from containers/skopeo#1444 These images contain a workaround for: containers/podman#11123 Prior-Ubuntu support is being dropped everywhere. Ref: containers/podman#11070 containers/automation_images#88 Signed-off-by: Chris Evich <[email protected]>

These images contain a workaround for: containers/podman#11123 Ref: containers/podman#11070 containers/automation_images#88 Signed-off-by: Chris Evich <[email protected]>

edsantiago added the flakes Flakes from Continuous Integration label Aug 3, 2021

edsantiago assigned Luap99 Aug 3, 2021

This was referenced Aug 11, 2021

systemtests: if registry times out, show container logs containers/skopeo#1413

Merged

Intermittent system-tests failures containers/skopeo#1354

Closed

Luap99 mentioned this issue Sep 10, 2021

Cirrus: NM/CNI workaround + Remove prior-Ubuntu #11512

Merged

openshift-merge-robot closed this as completed in #11512 Sep 14, 2021

cevich mentioned this issue Sep 14, 2021

Update VM Images + Drop prior-ubuntu testing containers/buildah#3519

Merged

cevich mentioned this issue Sep 14, 2021

Update VM Images + Drop prior-ubuntu references containers/skopeo#1444

Merged

cevich mentioned this issue Sep 14, 2021

Update VM Images + Drop prior-ubuntu references containers/image#1377

Merged

github-actions bot added the locked - please file new issue/PR Assist humans wanting to comment on an old issue or PR with locked comments. label Sep 21, 2023

github-actions bot locked as resolved and limited conversation to collaborators Sep 21, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

New networking flake (root, compose) #11123

New networking flake (root, compose) #11123

edsantiago commented Aug 3, 2021

Luap99 commented Aug 4, 2021

Luap99 commented Sep 1, 2021

Luap99 commented Sep 2, 2021

mheon commented Sep 2, 2021

Luap99 commented Sep 2, 2021

mheon commented Sep 2, 2021

Luap99 commented Sep 2, 2021

Luap99 commented Sep 3, 2021

Luap99 commented Sep 3, 2021

mheon commented Sep 3, 2021

Luap99 commented Sep 3, 2021

cevich commented Sep 8, 2021

New networking flake (root, compose) #11123

New networking flake (root, compose) #11123

Comments

edsantiago commented Aug 3, 2021

Luap99 commented Aug 4, 2021

Luap99 commented Sep 1, 2021

Luap99 commented Sep 2, 2021

mheon commented Sep 2, 2021

Luap99 commented Sep 2, 2021

mheon commented Sep 2, 2021

Luap99 commented Sep 2, 2021

Luap99 commented Sep 3, 2021

Luap99 commented Sep 3, 2021

mheon commented Sep 3, 2021

Luap99 commented Sep 3, 2021

cevich commented Sep 8, 2021