docker-mac-net-connect stopped working with 4.16.1

[waterson]

I've had great success running docker-mac-net-connect with Docker Desktop 4.15 and below on an M1 mac.

However, that environment started to get a bit unstable after I upgraded to Ventura 13.1, possibly because of docker/for-mac#6530. Anyway, I recently upgraded to 4.16.1 in hopes of having things get a bit more stable, but I'm not able to get docker-mac-net-connect working anymore. Below is what I see in the debug log...

DEBUG: (utun0) 2023/01/13 12:57:59 Setting up Wireguard on Docker Desktop VM
Interface chip0 already exists. Removing.
Creating WireGuard interface chip0
Assigning IP to WireGuard interface
Configuring WireGuard device
Adding iptables NAT rule for host WireGuard IP
Setup container complete
Adding route for 192.168.58.0/24 -> utun0 (minikube)
Adding route for 172.17.0.0/16 -> utun0 (bridge)
DEBUG: (utun0) 2023/01/13 12:57:59 Watching Docker events
DEBUG: (utun0) 2023/01/13 12:58:32 peer(ek54…Crxg) - Sending handshake initiation
ERROR: (utun0) 2023/01/13 12:58:32 peer(ek54…Crxg) - Failed to send handshake initiation: no known endpoint for peer
DEBUG: (utun0) 2023/01/13 12:58:37 peer(ek54…Crxg) - Handshake did not complete after 5 seconds, retrying (try 2)
DEBUG: (utun0) 2023/01/13 12:58:37 peer(ek54…Crxg) - Sending handshake initiation
ERROR: (utun0) 2023/01/13 12:58:37 peer(ek54…Crxg) - Failed to send handshake initiation: no known endpoint for peer
DEBUG: (utun0) 2023/01/13 12:58:43 peer(ek54…Crxg) - Handshake did not complete after 5 seconds, retrying (try 3)
DEBUG: (utun0) 2023/01/13 12:58:43 peer(ek54…Crxg) - Sending handshake initiation
ERROR: (utun0) 2023/01/13 12:58:43 peer(ek54…Crxg) - Failed to send handshake initiation: no known endpoint for peer
DEBUG: (utun0) 2023/01/13 12:58:48 peer(ek54…Crxg) - Handshake did not complete after 5 seconds, retrying (try 4)
DEBUG: (utun0) 2023/01/13 12:58:48 peer(ek54…Crxg) - Sending handshake initiation

For now, I've downgraded back to 4.14.1 and things are working again (albeit with some instability):

...
Assigning IP to WireGuard interface
Configuring WireGuard device
DEBUG: (utun0) 2023/01/13 13:07:23 peer(ek54…Crxg) - Received handshake initiation
DEBUG: (utun0) 2023/01/13 13:07:23 peer(ek54…Crxg) - Sending handshake response
DEBUG: (utun0) 2023/01/13 13:07:23 peer(ek54…Crxg) - Receiving keepalive packet
Adding iptables NAT rule for host WireGuard IP
Setup container complete
Adding route for 172.17.0.0/16 -> utun0 (bridge)
Adding route for 192.168.58.0/24 -> utun0 (minikube)
DEBUG: (utun0) 2023/01/13 13:07:23 Watching Docker events
DEBUG: (utun0) 2023/01/13 13:07:48 peer(ek54…Crxg) - Receiving keepalive packet
DEBUG: (utun0) 2023/01/13 13:08:05 peer(ek54…Crxg) - Sending keepalive packet

Thanks in advance for any advice!

[obourgain]

I am affected by this issue. It was working on 4.15 and older versions, not on 4.16.

May I help diagnose the issue?

[tommy2d]

Same problem here, is this related to changed in the docker VM structure somehow?

[philbrookes]

I am also experiencing this issue, would love to know if there is any way around this. @waterson how are you downgrading to 4.14.1?

[obourgain]

I downgraded to an older version too, by following the uninstall instruction from the doc (change the tab to 'mac' as I can't link the tab directly).
Then you delete the docker.app from /Applications and install the old version.
You will lose every container, image etc

[vossmedien]

@obourgain I was able to downgrade to v15 simply by copying the older version in my application folder and overwrite the "newer" docker version. it was all working again then. containers are still there and working etc.

is this project simply ongoing? or is there an alternative?

[obourgain]

Thanks for the tip

[jamiefiedler]

The docker dev build linked in the issue below fixed this for me.
docker/for-mac#6699 (comment)

[skriss]

FYI Docker Desktop 4.17.0 is out and appears to have resolved this issue: https://docs.docker.com/desktop/release-notes/#4170

[Rodi26]

FYI docker/for-mac#6747

With 4.17.0, the application stopped working for me after a while.

[huangbaihua001]

FYI Docker Desktop For Mac 4.17.0 is OK.

[michelesr]

When it came out, Docker Desktop v4.17.0 was ok, but something has changed (not sure exactly what) and now docker-mac-net-connect fails to connect to the daemon socket in the Linux VM and set the bridge up.

[gregnr]

I've confirmed that 4.17.0 works from a fresh install (M2), but it's only been an hour so far.

Looks like a lot of people are seeing only a temporary fix with 4.17.0, then it reverts back to connection issues. Going to see if I can reproduce this on my end by leaving it running for awhile. Can anyone confirm if they were able to get 4.17.0 working permanently?

As @skriss pointed out, Docker Desktop for Mac did fix in 4.17.0 a UDP connection tracking bug when connecting to host.docker.internal. This bug would definitely have an impact on this tool (Wireguard uses UDP and connects via host.docker.internal, see below) - I assume this is what caused the original 4.16.1 problems.

In case it helps you debug, you can read about how the tunnels work in the README: https://github.com/chipmk/docker-mac-net-connect#how-does-it-work. In summary:

• Wireguard is used to tunnel packets
• Wireguard runs on UDP
• The macOS side (docker-mac-net-connect) acts as the Wireguard server, the Linux VM acts as the client (via Linux kernel module).
  • This is mainly because its simpler to address the macOS host from the VM than vice versa
  • macOS is addressed from VM via DNS @ host.docker.internal, so if this doesn't resolve, a connection won't establish

[gregnr]

I've been running Docker Desktop 4.17.0 with docker-mac-net-connect for the past week and done the following:

• Let the computer sleep and wake
• Restarted Docker Desktop
• Shut down Docker Desktop for a few hours and then restarted.
• Restarted macOS

After this, I am still able to connect to the containers directly as expected.

@waterson @obourgain @tommy2d @philbrookes @Rodi26 @michelesr looks like you all experienced issues. I'm hoping to resolve this if it is ongoing. Can you confirm if you are still unable to connect to containers after:

• Uninstall Docker Desktop
• Install the latest Docker Desktop 4.17.0 Edit: Latest is now 4.19.0
• Restart macOS

Thank you.

[DisasterCthulhu]

I'm seeing this issue still on 4.18.0 after following your steps with additionally purging all local vms/images/volumes after installing 4.18.0, rebooting, then rebuilding images. VMs were unable to do things such as apt update after a day.

[michelesr]

It's working again for me with Docker Desktop v4.19.0

Edit: tried again later and getting:

ERROR: (utun0) 2023/05/04 10:49:17 Failed to setup VM: failed to pull setup image: Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?

[tommy2d]

I cannot seem to reproduce the bug after upgrading to Docker Desktop v4.19.0. However, the bug always took some time (or some unknown trigger) to start manifesting itself in previous version. I will keep a close eye on it.

[chnliyong]

It's working again for me with Docker Desktop v4.19.0

Edit: tried again later and getting:

ERROR: (utun0) 2023/05/04 10:49:17 Failed to setup VM: failed to pull setup image: Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?

Ref: https://stackoverflow.com/questions/74173489/docker-socket-is-not-found-while-using-intellij-idea-and-docker-desktop-on-macos

EDITED 2023-07-17

If the option in "Settings > Advanced > Allow the default Docker socket to be used" is already enabled and the socket is not available try disabling it and re-enabling it.

[RafalSkolasinski]

Could we try to make it work with docker context instead of socket?
Just got some issues and indeed the error was related to the socket missing.

Probably should check the docker desktop options but as I was recently changing the socket between docker desktop and colima I had the command handy

sudo ln -sf ~/.docker/run/docker.sock /var/run/docker.sock

and that fixed it.

But it'd be great if it just picked up the active docker context and work with it, though not sure how it should behave if there are two docker servoces (desktop & colima) running at the same time...