Port Forwarding does not work on RHEL 8 with Firewalld running with FirewallBackend=nftables

[arkodg]

With RHEL8 and Firewalld with FirewallBackend=nftables enabled, docker port forwarding (e.g. docker run --name test-nginx -p 8080:80 -d nginx )does not work

Might need to revisit the logic in https://github.com/docker/libnetwork/blob/master/iptables/firewalld.go to get this to work

Workaround -

1. Set FirewallBackend in /etc/firewalld/firewalld.conf to iptables
2. or Include the interface
   firewall-cmd --permanent --zone=trusted --add-interface=docker0; firewall-cmd --reload

[yrro]

This happens because Docker configures rules via iptables. These are translated to nftables rules in the following chain:

table ip filter {
	chain FORWARD {
		type filter hook forward priority filter; policy accept;
		counter packets 0 bytes 0 jump LIBVIRT_FWX
		counter packets 0 bytes 0 jump LIBVIRT_FWI
		counter packets 0 bytes 0 jump LIBVIRT_FWO
		counter packets 4 bytes 336 jump DOCKER-USER
		counter packets 4 bytes 336 jump DOCKER-ISOLATION-STAGE-1
		oifname "docker0" ct state related,established counter packets 2 bytes 168 accept
		oifname "docker0" counter packets 0 bytes 0 jump DOCKER
		iifname "docker0" oifname != "docker0" counter packets 2 bytes 168 accept
		iifname "docker0" oifname "docker0" counter packets 0 bytes 0 accept
	}
}

Note the priority of this chain: filter. Firewalld puts its rules in its own chain with a higher priority value, filter + 10:

table inet firewalld {
	chain filter_FORWARD {
		type filter hook forward priority filter + 10; policy accept;
		ct state { established, related } accept
		ct status dnat accept
		iifname "lo" accept
		ip6 daddr { ::/96, ::ffff:0.0.0.0/96, 2002::/24, 2002:a00::/24, 2002:7f00::/24, 2002:a9fe::/32, 2002:ac10::/28, 2002:c0a8::/32, 2002:e000::/19 } log prefix "RFC3964_IPv4_REJECT: " reject with icmpv6 type addr-unreachable
		jump filter_FORWARD_IN_ZONES
		jump filter_FORWARD_OUT_ZONES
		ct state { invalid } meta pkttype host log prefix "STATE_INVALID_DROP: "
		ct state { invalid } drop
		meta pkttype host log prefix "FINAL_REJECT: "
		reject with icmpx type admin-prohibited
	}
}

Thus the iptables-managed chain runs first, and accepts packets. Then the firewalld-managed chain runs, which rejects them.

Libvirt also ran into this problem and solved it by creating its own zone. Docker could do the same, with something like:

# cat /etc/firewalld/zones/docker.xml
<?xml version="1.0" encoding="utf-8"?>
<zone target="ACCEPT">
  <short>libvirt</short>
  <description>
    The default policy of "ACCEPT" allows all packets to/from
    interfaces in the zone to be forwarded, while the (*low priority*)
    reject rule blocks any traffic destined for the host, except those
    services explicitly listed (that list can be modified as required
    by the local admin). This zone is intended to be used only by
    docker virtual networks.
  </description>
  <rule priority="32767">
    <reject/>
  </rule>
  <interface name="docker0"/>
  <protocol value="icmp"/>
  <protocol value="ipv6-icmp"/>
</zone>

Note that Libvirt tells firewalld to put its interfaces into the libvirt zone when it creates them, rather than defining the association statically in the zone definition like I did above. Docker could do something similar.

It's also worth noting that due to firewalld/firewalld#177, firewalld's filter_FWDI_docker and filter_FWDO_docker chains will default to accepting any traffic forwarded from or to the docker zone.