failed to make veth pair: operation not supported

[Mingli-Yu]

# podman --version
podman version 3.3.1

# podman run -it --name mybusybox  docker.io/library/busybox
Trying to pull docker.io/library/busybox:latest...
Getting image source signatures
Copying blob 01c2cdc13739 done  
Copying config cabb9f684f done  
Writing manifest to image destination
Storing signatures
ERRO[0000] error loading cached network config: network "podman" not found in CNI cache 
WARN[0000] falling back to loading from existing plugins on disk 
Error: error configuring network namespace for container 670d4af03e7c805a9fd12b14cde25e473b89302710e316e7501ac150de4c4726: error adding pod mybusybox_mybusybox to CNI network "podman": failed to make veth pair: operation not supported

[giuseppe]

it looks like your kernel doesn't support veth pairs.

What kernel are you using?

[Mingli-Yu]

Thanks, have recompile the kernel with CONFIG_OVERLAY_FS=y and the error is missing and encounter a new issue:
# podman run -it --name test docker.io/library/busybox
Trying to pull docker.io/library/busybox:latest...
Getting image source signatures
Copying blob 01c2cdc13739 done  
Copying config cabb9f684f done  
Writing manifest to image destination
Storing signatures
[   76.482222] IPv6: ADDRCONF(NETDEV_CHANGE): vethbe376529: link becomes ready
[   76.483179] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
[   76.496738] cni-podman0: port 1(vethbe376529) entered blocking state
[   76.497365] cni-podman0: port 1(vethbe376529) entered disabled state
[   76.500882] device vethbe376529 entered promiscuous mode
[   76.505237] cni-podman0: port 1(vethbe376529) entered blocking state
[   76.505516] cni-podman0: port 1(vethbe376529) entered forwarding state
ERRO[0020] error loading cached network config: network "podman" not found in CNI cache 
WARN[0020] falling back to loading from existing plugins on disk 
[   79.831194] cni-podman0: port 1(vethbe376529) entered disabled state
[   79.842302] device vethbe376529 left promiscuous mode
[   79.842546] cni-podman0: port 1(vethbe376529) entered disabled state
Error: error configuring network namespace for container a519680ba7c0882990511d2196089b396c5ac7f439602e43e1272a2676d34287: error adding pod test_test to CNI network "podman": running [/usr/sbin/iptables -t filter -C CNI-FORWARD -d 10.88.0.3/32 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT --wait]: exit status 2: iptables v1.8.7 (legacy): Couldn't load match `conntrack':No such file or directory

Try `iptables -h' or 'iptables --help' for more information.

[giuseppe]

My suggestion is to start from a known working configuration, like the Fedora kernel configuration: https://src.fedoraproject.org/rpms/kernel/blob/rawhide/f/kernel-x86_64-fedora.config

Closing the issue since there is nothing we can do in Podman

[Mingli-Yu]

Thanks, install xt_conntrack.ko address the above error.

[Hergeirs]

I know this is closed, but I'm running a custom kernel. Does anyone by chance know which kernel modules would have to be added ? If not, just ignore this.

[giuseppe]

for xt_conntrack.ko you'd probably need CONFIG_NETFILTER_XT_MATCH_CONNTRACK

[Hergeirs]

It seems I needed quite a few modules.
The ones I didn't have already are listed below for anyone curious:

• bridge
• libcrc32c
• llc
• nf_conntrack
• nf_defrag_ipv4
• nf_defrag_ipv6
• nf_nat
• nfnetlink
• nf_tables
• nft_chain_nat
• nft_compat
• nft_counter
• stp
• veth
• xt_addrtype
• xt_comment
• xt_conntrack
• xt_mark
• xt_MASQUERADE
• xt_multiport
• xt_nat
• xt_tcpudp

I you deem this irrelevant or "noise" feel free to remove my message.

[giuseppe]

I you deem this irrelevant or "noise" feel free to remove my message.

I think it is useful for other users that end up with the same issue. Thanks for sharing it

[ctron]

On a Raspberry Pi with Ubuntu, the fix for this is:

sudo apt install linux-modules-extra-raspi

And a reboot.