Can’t get Zerotier routing working

[javiernar1]

Hi, I have followed your instructions on the wiki but I can't get it fully working:

My setup:
WAN-Interface of the OpenWrt router : a public IP on the internet
LAN-Interface of the OpenWrt router : 192.168.3.1
IP-Range of the LAN switch on the OpenWrt router : 192.168.3.0/24
IP-Address of the zt interface on the OpenWrt router : 10.147.20.1
ZeroTier network route on the OpenWrt router : 10.147.20.1/24 via zt

“Allow Ethernet Bridging” is enabled on 10.147.20.1 on ZT.

I am using OpenWrt 23.05.5, and I have installed latest zerotier using the official repository (1.12.2-2).

At this moment, I can connect to the router via SSH from a remote computer connected to the ZT network using both Ips: 10.147.20.1 and 192.168.3.1, and I can ping, and get reply from other computers in 192.168.3.0/24, however I cannot access any TCP service (like web) there, I always get a timeout connecting.

My config:

root@OpenWrt:~# uci show zerotier
zerotier.openwrt_network=zerotier
zerotier.openwrt_network.join='xxxxxxxxxxx'
zerotier.openwrt_network.enabled='1'
zerotier.openwrt_network.port='9993'
zerotier.openwrt_network.secret=’xxxxxxxxxxxxxxxx'

root@OpenWrt:~# cat /etc/config/zerotier

config zerotier 'openwrt_network'
        list join 'xxxxxxxxxxxxx'
        option enabled '1'
        option port '9993'
        option secret ‘xxxxxxxxx'

root@OpenWrt:~# ip a
... (several networks)
11: br-lan: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    inet 192.168.3.1/24 brd 192.168.3.255 scope global br-lan
       valid_lft forever preferred_lft forever
    inet6 fd3d:9a67:9561::1/60 scope global noprefixroute
       valid_lft forever preferred_lft forever
    inet6 fe80::9683:c4ff:fea4:3426/64 scope link
       valid_lft forever preferred_lft forever
12: ztklhtwidz: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 2800 qdisc fq_codel state UNKNOWN group default qlen 1000
    link/ether 03:96:28:c8:dc:f5 brd ff:ff:ff:ff:ff:ff
    inet 10.147.20.1/24 brd 10.147.20.255 scope global ztklhtwidz
       valid_lft forever preferred_lft forever
    inet6 fe80::ecfc:a9ff:fede:7bd7/64 scope link
       valid_lft forever preferred_lft forever

root@OpenWrt:~# zerotier-cli info
200 info xxxxxxxx 1.12.2 ONLINE

root@OpenWrt:~# cat /etc/config/firewall

config defaults
        option input 'REJECT'
        option output 'ACCEPT'
        option forward 'REJECT'
        option synflood_protect '1'

config zone
        option name 'lan'
        list network 'lan'
        option input 'ACCEPT'
        option output 'ACCEPT'
        option forward 'ACCEPT'

config zone
        option name 'wan'
        list network 'wan'
        list network 'wan6'
        option input 'REJECT'
        option output 'ACCEPT'
        option forward 'REJECT'
        option masq '1'
        option mtu_fix '1'

config forwarding
        option src 'lan'
        option dest 'wan'

config rule
        option name 'Allow-DHCP-Renew'
        option src 'wan'
        option proto 'udp'
        option dest_port '68'
        option target 'ACCEPT'
        option family 'ipv4'

config rule
        option name 'Allow-Ping'
        option src 'wan'
        option proto 'icmp'
        option icmp_type 'echo-request'
        option family 'ipv4'
        option target 'ACCEPT'

config rule
        option name 'Allow-IGMP'
        option src 'wan'
        option proto 'igmp'
        option family 'ipv4'
        option target 'ACCEPT'

config rule
        option name 'Allow-DHCPv6'
        option src 'wan'
        option proto 'udp'
        option dest_port '546'
        option family 'ipv6'
        option target 'ACCEPT'

config rule
        option name 'Allow-MLD'
        option src 'wan'
        option proto 'icmp'
        option src_ip 'fe80::/10'
        list icmp_type '130/0'
        list icmp_type '131/0'
        list icmp_type '132/0'
        list icmp_type '143/0'
        option family 'ipv6'
        option target 'ACCEPT'

config rule
        option name 'Allow-ICMPv6-Input'
        option src 'wan'
        option proto 'icmp'
        list icmp_type 'echo-request'
        list icmp_type 'echo-reply'
        list icmp_type 'destination-unreachable'
        list icmp_type 'packet-too-big'
        list icmp_type 'time-exceeded'
        list icmp_type 'bad-header'
        list icmp_type 'unknown-header-type'
        list icmp_type 'router-solicitation'
        list icmp_type 'neighbour-solicitation'
        list icmp_type 'router-advertisement'
        list icmp_type 'neighbour-advertisement'
        option limit '1000/sec'
        option family 'ipv6'
        option target 'ACCEPT'

config rule
        option name 'Allow-ICMPv6-Forward'
        option src 'wan'
        option dest '*'
        option proto 'icmp'
        list icmp_type 'echo-request'
        list icmp_type 'echo-reply'
        list icmp_type 'destination-unreachable'
        list icmp_type 'packet-too-big'
        list icmp_type 'time-exceeded'
        list icmp_type 'bad-header'
        list icmp_type 'unknown-header-type'
        option limit '1000/sec'
        option family 'ipv6'
        option target 'ACCEPT'

config rule
        option name 'Allow-IPSec-ESP'
        option src 'wan'
        option dest 'lan'
        option proto 'esp'
        option target 'ACCEPT'

config rule
        option name 'Allow-ISAKMP'
        option src 'wan'
        option dest 'lan'
        option dest_port '500'
        option proto 'udp'
        option target 'ACCEPT'

config rule
        option name 'Allow-ZeroTier-Inbound'
        option src '*'
        option target 'ACCEPT'
        option proto 'udp'
        option dest_port '9993'

config zone
        option name 'vpn'
        option input 'ACCEPT'
        option output 'ACCEPT'
        option forward 'ACCEPT'
        option masq '1'
        list network 'ZeroTier'

config forwarding
        option src 'vpn'
        option dest 'lan'

config forwarding
        option src 'vpn'
        option dest 'wan'

config forwarding
        option src 'lan'
        option dest 'vpn'

Any idea? Thank you very much

[ogarcia]

Hi @javiernar1, sorry it took so long to reply, too much work and not enough time.

I understand that in the ZeroTier central you have configured the route for the 192.168.3.0/24 network through 10.147.20.1, right?

Are the computers on your network that you are trying to connect to Windows or do they have firewalls? Because it looks like that's what's cutting you off. Note that the origin of the requests to those computers is the 10.147.20.1/24 network and not the 192.168.3.0/24 network so if they only allow TCP connections from IPs on the 192.168.3.0/24 network then you are not going to reach them.

[sppmasterspp]

I have a similar issue.
Simply following the wiki https://github.com/mwarning/zerotier-openwrt/wiki/Configure-older-ZeroTier-on-OpenWrt
I cannot ping the host (OpenWrt router) from my PC although ip a shows that zerotier client was ONLINE and got an IP from zerotier network.
I use this setup for more than a year and I think that it worked in the past without the need of interface and firewall zone.
Now in order to get it working I have to setup the interface and firewall zone from the next guide
https://github.com/mwarning/zerotier-openwrt/wiki/Configure-ZeroTier-routing-in-OpenWrt
Then everything is OK but I cannot understand why I cannot ping the hosts when zerotier client is Online and connected.

[ogarcia]

I cannot ping the host (OpenWrt router) from my PC although ip a shows that zerotier client was ONLINE and got an IP from zerotier network.

This indicates that you have a firewall configuration on your client machine that prevents pinging the Zerotier network.

I use this setup for more than a year and I think that it worked in the past without the need of interface and firewall zone.

If you had neither interface nor firewall zone then communication was only possible on the Zerotier network, but in that case neither your clients connected to the router could communicate with your Zerotier network nor vice versa.

Please note that we have rewritten the guides and separated them into different pages to make it easier, maybe that is why you do not remember to have done that configuration (before the guide was a single page).

Now in order to get it working I have to setup the interface and firewall zone from the next guide https://github.com/mwarning/zerotier-openwrt/wiki/Configure-ZeroTier-routing-in-OpenWrt Then everything is OK but I cannot understand why I cannot ping the hosts when zerotier client is Online and connected.

Check what I have told you before. That you do not have any firewall that prevents the ICMP communication (it is very typical that Windows cuts this communication).

[sppmasterspp]

Thanks for your answers.
I have several routers connected to my ZT network.
I have a PC, a Laptop and a smartphone all connected to my ZT network and I want to be able to connect only to the routers to manage them. I don't want to connect to any LAN device connected to any of the routers.
That's why I followed the old guide up to the point where new interface and firewall are created.
But just stopping at it I cannot ping the routers from any other device. This used to work in the past.

[ogarcia]

I have a PC, a Laptop and a smartphone all connected to my ZT network and I want to be able to connect only to the routers to manage them. I don't want to connect to any LAN device connected to any of the routers.
That's why I followed the old guide up to the point where new interface and firewall are created.

In that case it is true, you only need to create the Zerotier network, configuring the routing (and the interface) is not necessary since you do not need that communication. In fact the guide itself indicates that once you create the network you should be able to ping between devices. However, the firewall configuration is necessary.

But just stopping at it I cannot ping the routers from any other device. This used to work in the past.

If the network and firewall configuration is done (I said, in your case the routing and interface is not necessary) and you are still not able to ping then check the firewalls that you may have in your devices because it is possible that you are cutting the communication in the Zerotier network.

[sppmasterspp]

If the network and firewall configuration is done (I said, in your case the routing and interface is not necessary) and you are still not able to ping then check the firewalls that you may have in your devices because it is possible that you are cutting the communication in the Zerotier network.

I understand your point very well and it used to work for my ZT network devices so far really well. I have the firewall rule configured since the beginning of ZT usage. I needed it because some of the routers are behind ISP CGNAT. I didn't have any issues pinging and connecting between all devices in my ZT.
I've just wasn't using it for some time but yesterday I just added a new OpenWrt router to my ZT network using zerotier version 1.14.0.
I configured the new router without interface and firewall and the zerotier-cli info said it was Online and had an IP address and I authorized it to my ZT network, but I couldn't connect to it nor ping it.
Then I checked another older device configured a few months ago and couldn't access it from all of my devices.
Today I added an interface rebooted the router but it was still inaccessible, then I added a firewall zone and only then the device became accessible and pingable. My OpenwWrt custom build is OpenWrt 23.05-SNAPSHOT (compiled a few days ago).
Just wonder what may be the reason for that behavior.
I don't have Allow Ethernet Bridging on any router as it's not needed for my purpose.

I've just tried another important test. I've just used OpenWrt diagnostic (from two different routers but with the same firmware) to ping my Desktop PC and smartphone ZT IPs.
First from the inaccessible router (it has only the FW rule and zerotier-cli is Online and it's authorized to my ZT network) - I cannot ping the Desktop PC nor my smartphone and vice versa. Zerotier-cli is Online with right ZT IP address.

Form the other router (with interface and firewall zone configured) no issues in any direction on any device.
If I delete the firewall zone it stops responding to ping and cannot ping other devices.

[ogarcia]

The only thing I can think of is that the default configuration of that device specifically prohibits this network traffic. If that is the case it is normal that the ping does not work until you create the firewall zone that allows this communication because there you are explicitly telling it to allow the incoming and outgoing communication of the Zerotier interface.

[sppmasterspp]

The only thing I can think of is that the default configuration of that device specifically prohibits this network traffic. If that is the case it is normal that the ping does not work until you create the firewall zone that allows this communication because there you are explicitly telling it to allow the incoming and outgoing communication of the Zerotier interface.

Thanks a lot for your help. As the 1.14.1 was added to the 23.05 snapshot branch I updated my OpenWrt build and installed the 1.14.1 version and configured it from scratch.
Now I am at the same situation with the exception that I can now ping any other ZT device from the router that doesn't respond to ping.
I use the same firewall rules for several routers that have the same configurations.
Maybe I should write in the ZT Community or OpenWrt forums.