Creating network with CNI DHCP only outputs 'No route to host" #14352
Comments
|
Also, FWIW I have no firewall active. |
|
Please provide the output of |
|
@Luap99 Sure. Here it is... $ ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eno1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
link/ether 94:c6:91:30:b3:92 brd ff:ff:ff:ff:ff:ff
altname enp0s31f6
inet 192.168.8.8/24 brd 192.168.8.255 scope global dynamic noprefixroute eno1
valid_lft 604499sec preferred_lft 604499sec
inet6 fe80::ea0e:edd6:cd3a:250a/64 scope link noprefixroute
valid_lft forever preferred_lft forever
3: sit0@NONE: <NOARP> mtu 1480 qdisc noop state DOWN group default qlen 1000
link/sit 0.0.0.0 brd 0.0.0.0
5: wlan0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc pfifo_fast state DOWN group default qlen 1000
link/ether ac:bc:32:be:c4:2d brd ff:ff:ff:ff:ff:ff$ ip route
default via 192.168.8.1 dev eno1 proto dhcp metric 100
192.168.8.0/24 dev eno1 proto kernel scope link src 192.168.8.8 metric 100 |
|
weird, have you tried a different image or if you can connect from the container to the outside? |
|
Did you start the CNI DHCP forwarder daemon? |
|
@Luap99 I have, and I cannot. @mheon I have. (I have the $ sudo systemctl status cni-dhcp.service
● cni-dhcp.service - CNI DHCP service
Loaded: loaded (/lib/systemd/system/cni-dhcp.service; disabled; vendor preset: disabled)
Active: active (running) since Tue 2022-05-24 17:19:16 PDT; 13h ago
TriggeredBy: ● cni-dhcp.socket
Docs: https://github.com/containernetworking/plugins/tree/master/plugins/ipam/dhcp
Main PID: 38658 (dhcp)
Tasks: 8 (limit: 38356)
Memory: 2.8M
CPU: 22ms
CGroup: /system.slice/cni-dhcp.service
└─38658 /opt/cni/bin/dhcp daemon
May 24 18:04:44 n7 dhcp[38658]: 2022/05/24 18:04:44 Link "eth0" down. Attempting to set up
May 24 18:04:44 n7 dhcp[38658]: 2022/05/24 18:04:44 network is down
May 24 18:04:44 n7 dhcp[38658]: 2022/05/24 18:04:44 retrying in 1.313039 seconds
May 24 18:04:49 n7 dhcp[38658]: 2022/05/24 18:04:49 c2c35edcf7f9d91f09475d930d7af5f8aab828cffb164fb39c71539c9cdec762/home/eth>
May 24 18:11:08 n7 dhcp[38658]: 2022/05/24 18:11:08 c2c35edcf7f9d91f09475d930d7af5f8aab828cffb164fb39c71539c9cdec762/home/eth>
May 24 18:33:12 n7 dhcp[38658]: 2022/05/24 18:33:12 5b488c0d3cf0593ec2181f3c924aa8a88f5f7508afd225a1b4cecf2568e4d86e/home/eth>
May 24 18:33:12 n7 dhcp[38658]: 2022/05/24 18:33:12 Link "eth0" down. Attempting to set up
May 24 18:33:12 n7 dhcp[38658]: 2022/05/24 18:33:12 network is down
May 24 18:33:12 n7 dhcp[38658]: 2022/05/24 18:33:12 retrying in 1.193939 seconds
May 24 18:33:16 n7 dhcp[38658]: 2022/05/24 18:33:16 5b488c0d3cf0593ec2181f3c924aa8a88f5f7508afd225a1b4cecf2568e4d86e/home/eth>FWIW I noticed that the Gentoo packages didn't deploy the systemd files (for the That said, not a lot of people (that I know of) use containers on Gentoo, so perhaps the Gentoo ebuild is borked? I dunno. I was working with @ran-dall whom seems to do be doing a lot of the work on the |
|
One thing which looks odd is the |
|
@Luap99 Good question. I needed clarification as well. FMU, from the Gentoo folks, it automatically gets created and added when you enable IPv6 in the kernel. I'm told I shouldn't worry about it; as it really shouldn't affect anything. |
|
@Luap99 Maybe should I try |
|
I have no idea, sit is definitely new to me. As far as I can tell ipv6 works just fine without that, not sure why someone would want to tunnel ipv6 traffic via ipv4. I think Could you try to delete the sit interface. |
|
@Luap99 It doesn't want to delete... (I can't delete it cause it's a built-in module. Also, I've been told, I could disable it in the kernel or make it a module, but removing it will require disabling IPv6 entirely, not just the sit interface.) $ sudo ip link del sit0
$ ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eno1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
link/ether 94:c6:91:30:b3:92 brd ff:ff:ff:ff:ff:ff
altname enp0s31f6
inet 192.168.8.8/24 brd 192.168.8.255 scope global dynamic noprefixroute eno1
valid_lft 602478sec preferred_lft 602478sec
inet6 fe80::ea0e:edd6:cd3a:250a/64 scope link noprefixroute
valid_lft forever preferred_lft forever
3: sit0@NONE: <NOARP> mtu 1480 qdisc noop state DOWN group default qlen 1000
link/sit 0.0.0.0 brd 0.0.0.0
6: wlan0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc pfifo_fast state DOWN group default qlen 1000
link/ether ac:bc:32:be:c4:2d brd ff:ff:ff:ff:ff:ffAlso, FWIW, $ sudo podman network create -d macvlan -o parent=sit0 webnetwork
webnetwork
$ sudo podman run -dt --name webserver --network webnetwork quay.io/libpod/banner
Trying to pull quay.io/libpod/banner:latest...
Getting image source signatures
Copying blob c6f282ad3b41 done
Copying blob a0d0a0d46f8b done
Copying blob db9deeff4a88 done
Copying blob e3ecd43a8bc5 done
Copying config 4d50a7b41a done
Writing manifest to image destination
Storing signatures
WARN[0002] Failed to load cached network config: network webnetwork not found in CNI cache, falling back to loading network webnetwork from disk
Error: plugin type="macvlan" failed (add): cni plugin macvlan failed: failed to create macvlan: invalid argument |
|
Looks like the kernel cannot create macvlan devices with sit0, since this is not an actual device I think this is expected. Could try to set promisc mode on you eno1 interface. |
|
@Luap99 Yup. $ sudo ip link set eno1 promisc on
$ ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eno1: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
link/ether 94:c6:91:30:b3:92 brd ff:ff:ff:ff:ff:ff
altname enp0s31f6
inet 192.168.8.8/24 brd 192.168.8.255 scope global dynamic noprefixroute eno1
valid_lft 600772sec preferred_lft 600772sec
inet6 fe80::ea0e:edd6:cd3a:250a/64 scope link noprefixroute
valid_lft forever preferred_lft forever
3: sit0@NONE: <NOARP> mtu 1480 qdisc noop state DOWN group default qlen 1000
link/sit 0.0.0.0 brd 0.0.0.0
6: wlan0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc pfifo_fast state DOWN group default qlen 1000
link/ether ac:bc:32:be:c4:2d brd ff:ff:ff:ff:ff:ff
$ sudo podman container restart webserver
b094bb9c101a5926c4e81bc6100245b40478b4b88f12c644f20f07bfb41376c1
$ sudo podman exec webserver ip address show eth0
3: eth0@sit0: <BROADCAST,MULTICAST,UP,LOWER_UP,M-DOWN> mtu 1500 qdisc noqueue state UP
link/ether 66:9e:ae:f2:10:3e brd ff:ff:ff:ff:ff:ff
inet 192.168.8.114/24 brd 192.168.8.255 scope global eth0
valid_lft forever preferred_lft forever
inet6 fe80::649e:aeff:fef2:103e/64 scope link
valid_lft forever preferred_lft forever
$ curl http://192.168.8.114
curl: (7) Failed to connect to 192.168.8.114 port 80 after 3100 ms: No route to host |
|
@peterlobster You can safely turn |
|
@ran-dall Yup. $ ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eno1: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
link/ether 94:c6:91:30:b3:92 brd ff:ff:ff:ff:ff:ff
altname enp0s31f6
inet 192.168.8.8/24 brd 192.168.8.255 scope global dynamic noprefixroute eno1
valid_lft 603689sec preferred_lft 603689sec
inet6 fe80::ea0e:edd6:cd3a:250a/64 scope link noprefixroute
valid_lft forever preferred_lft forever
3: wlan0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc pfifo_fast state DOWN group default qlen 1000
link/ether ac:bc:32:be:c4:2d brd ff:ff:ff:ff:ff:ff@Luap99 Trying again, from the top. (Still using, $ sudo podman network create -d macvlan -o parent=eno1 webnetwork
webnetwork
$ sudo podman run -dt --name webserver --network webnetwork quay.io/libpod/banner
Trying to pull quay.io/libpod/banner:latest...
Getting image source signatures
Copying blob a0d0a0d46f8b done
Copying blob e3ecd43a8bc5 done
Copying blob db9deeff4a88 done
Copying blob c6f282ad3b41 done
Copying config 4d50a7b41a done
Writing manifest to image destination
Storing signatures
d93d29cf1dc249a969cedac27d6208a031ed4460dc5aaf05322d20bf4f283a52
$ sudo podman exec webserver ip address show eth0
2: eth0@eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP
link/ether 86:40:24:72:e2:50 brd ff:ff:ff:ff:ff:ff
inet 192.168.8.115/24 brd 192.168.8.255 scope global eth0
valid_lft forever preferred_lft forever
inet6 fe80::8440:24ff:fe72:e250/64 scope link
valid_lft forever preferred_lft forever
$ curl http://192.168.8.115
curl: (7) Failed to connect to 192.168.8.115 port 80 after 3109 ms: No route to host$ ip route
default via 192.168.8.1 dev eno1 proto dhcp metric 100
192.168.8.0/24 dev eno1 proto kernel scope link src 192.168.8.8 metric 100 Also, I don't know if this matter, but I am using |
|
A friendly reminder that this issue had no activity for 30 days. |
|
@Luap99 @ran-dall @peterlobster What is happening with this one? |
|
@rhatdan I dunno, it still doesn't work on my side. |
|
A friendly reminder that this issue had no activity for 30 days. |
|
A friendly reminder that this issue had no activity for 30 days. |
|
A friendly reminder that this issue had no activity for 30 days. |
|
I believe @baude is implementing this inf netavark-dhcp. |
|
The reproduction stage --ipam-driver=dhcp is really important. |
|
I'm seeing similar issues on RHEL8 and RHEL9 based podman environments, if they're running as kvm virtual machines using macvtap driver. The KVM hypervisor doesn't have a bridge device so i'm using macvtap-net virtio driver to connect the VM to the network. Containers started in host network mode work correctly and I know some broadcast packets are working because i'm running minidlna in a container. This required me to run the following in the hypervisor or set the following in the virtual machine definition Podman details for the RHEL9 environment |
|
Container host network |
|
Podman network was created via Details |
|
Trying a similar test with Output of /usr/libexec/cni/dhcp daemon |
|
Re-test on a physical environment with no macvtap in the way and the dhcp lease works. |
|
Retested on a vanilla RHEL 8.7 virtual host running on a bridge hypervisor network - no issues |
|
Confirmed switching the RHEL8.7 vm to macvtap now breaks DHCP requests, even if I run the following on the hypervisor hosts |
|
@steven-ellis This look of topic here. Macvlan or macvtap will always match packets on the parent based on the mac address, so when you VM already uses a macvtap device it cannot forward packets for other mac addresses. Therefore the DHCP lease fails. Reading back the original issue I think the problem is that the connection is only tested from the container host but it will only work from another host on the network. Macvlan does not allow communication between the parent and child interface. This is a kernel limitation. I found some workaround mentioned here with docker, I assume the same works with CNI or netavark. |
github-actions
bot
added
the
locked - please file new issue/PR
Is this a BUG REPORT or FEATURE REQUEST? (leave only one on its own line)
/kind bug
Description
I've been following the steps on here to create a macvlan setup, but they don't work.
After creating the container with
$ sudo podman run -dt --name webserver --network webnetwork quay.io/libpod/bannerand verify the ip withsudo podman exec webserver ip address show eth0I getcurl: (7) Failed to connect to 192.168.8.119 port 80 after 3068 ms: No route to hostDescribe the results you received:
$ sudo podman network inspect home [ { "name": "webnetwork", "id": "4ea140588150773ce3aace786aeef7f4049ce100fa649c94fbbddb960f1da942", "driver": "macvlan", "network_interface": "eno1", "created": "2022-05-24T18:31:27.889228163-07:00", "ipv6_enabled": false, "internal": false, "dns_enabled": false, "ipam_options": { "driver": "dhcp" } } ]Additional information you deem important (e.g. issue happens only occasionally):
Output of
podman version:Output of
podman info --debug:Package info (e.g. output of
rpm -q podmanorapt list podman):Have you tested with the latest version of Podman and have you checked the Podman Troubleshooting Guide? (https://github.com/containers/podman/blob/main/troubleshooting.md)
No
The text was updated successfully, but these errors were encountered: