Host name lookup failure or missing networks when using multiple networks

[milas]

Originally posted by @ToshY in #10668 (comment):

@milas While this seems to fix the not connected to network issue, it gave me a forward host lookup failed: Host name lookup failure in return when running waisbrot/wait with docker compose run.

Example

version: '3.9'
  
services:
  mysql:
    image: ${MYSQL_IMAGE_VERSION}
    ports:
      - "3308:3306"
    environment:
      MYSQL_USER: ${DATABASE_USER}
      MYSQL_DATABASE: ${DATABASE_NAME}
      MYSQL_ROOT_PASSWORD: ${DATABASE_ROOT_PASSWORD}
      MYSQL_PASSWORD: ${DATABASE_PASSWORD}
    volumes:
      - mysql_data:/var/lib/mysql
    networks:
      - webapp

  wait:
    image: waisbrot/wait
    environment:
      TARGETS: mysql:3306
      TIMEOUT: 300
    networks:
      - webapp

volumes:
  mysql_data:
    driver: local

networks:
  webapp:
    driver: bridge

$ docker compose up -d
[+] Running 4/4
 ✔ Container webapp-wait-1    Started                                                                                                                                                         3.0s
 ✔ Container webapp-mysql-1   Started                                                                                                                                                         2.9s

$ docker compose run --rm wait
Waiting for mysql:3306  .mysql: forward host lookup failed: Host name lookup failure
.mysql: forward host lookup failed: Host name lookup failure
.mysql: forward host lookup failed: Host name lookup failure

$ docker compose ps
NAME                            IMAGE                         COMMAND                  SERVICE             CREATED             STATUS              PORTS
webapp-mysql-1                  mysql:8.0                     "docker-entrypoint.s…"   mysql               8 minutes ago       Up 5 minutes        33060/tcp, 0.0.0.0:3308->3306/tcp, :::3308->3306/tcp

[jshufro]

Heyo. We just ran into this. The container is only added to one network, so dns doesn't resolve. Appears to be a regression

[johnthagen]

Wanted to point out that we hit this as well in our application when Docker Desktop updated to Docker Compose 2.19.1 in macOS.

Our application uses different networks to isolate our NGINX container from the database. Our backend sits on both networks (frontend and database) and could not resolve the database host name from the database network.

Our deployments on Ubuntu 22.04 using Docker Compose 2.18.1 seem to still function properly.

[opsec-infosec]

We have also faced this exact problem... nginx is using two networks api and app.. one of the docker networks fails to resolve the api host name. Doing a docker compose down then up seemed to resolve the failure, this is not ideal.

[jasonheffnerpsu]

Had to revert over the weekend to 2.18.1. One of the four containers replicated wouldn't join the postgres network, though all would connect to the main Traefik network fine.

[hugoghx]

+1 here, was wracking my brain for hours yesterday trying to figure out why my containers weren't in the networks I specified for them.

[oerp-odoo]

Got same problem. Postgres service on docker network fails to be recognized randomly.

[arazilsongweaver]

I ran into this issue quite a bit on the 2.19.1 official Debian Bookworm package (docker-compose-plugin_2.19.1-1~debian.12~bookworm_amd64.deb). Things appear to be back to normal now after reverting to the 2.18.1 package (docker-compose-plugin_2.18.1-1~debian.12~bookworm_amd64.deb) and using apt-mark hold docker-compose-plugin to temporarily avoid future upgrades.

[johnthagen]

@milas I'm curious, is there any ETA/plan on publishing a new release with this fixed, such as a 2.19.2?

One of the challenges we have is that it's hard (not possible?) to roll back Docker Desktop installs (which contain a bundled version of Compose), so our devs running on Mac can't easily roll back to 2.18.1.

Thanks.

[hamid-elaosta]

We've spent two days debugging this after our deployments suddenly broke. We realised that some containers were intermittently not connecting to one of the multiple networks they should be connected to; usually it was a our storage/db network, which fails pretty catastrophically. That led me here, to the compose Github, and this issue.

[milas]

A fix has been released for this: https://github.com/docker/compose/releases/tag/v2.20.0

@johnthagen It's possible to temporarily override the version of Compose in Docker Desktop. (It will be overwritten on the next Docker Desktop upgrade.)

curl -L -o /tmp/docker-compose 'https://github.com/docker/compose/releases/latest/download/docker-compose-darwin-aarch64' && \
  mv /tmp/docker-compose "${HOME}/.docker/cli-plugins/docker-compose" && \
  chmod +x "${HOME}/.docker/cli-plugins/docker-compose" && \
  docker compose version

[rfay]

Yay, docker-compose v2.20.0 solves

• build: bump docker-compose to 2.20.0 ddev/ddev#5006

as well.

[hubertgrzeskowiak]

This bug made me question my sanity for a few days.
Do we know when it was introduced, or does it apply to all prior Compose V2 releases?

[rfay]

IIRC it was introduced in 2.18.1 or 2.19.0. Nicely fixed in 2.20.0.

[johnthagen]

If you look at the Releases, you'll note that it was introduced in 2.19.0 and fixed in 2.20.0.

[jasonheffnerpsu]

I'm now doing yum/dnf versionlock on the docker-compose rpms due to decision for this known bug being released. It took several weeks for it to then be fixed and then filter down into the rpm repos. I had major concerns over "known" bugs being pushed to releases.

[hubertgrzeskowiak]

@johnthagen
I can see that it was being a "Known Issue" at the point 2.19.0 was released, but the note does not mention whether it was only then introduced or just discovered.

[johnthagen]

@hubertgrzeskowiak From my experience, I can tell you that we used 2.18.1 and didn't have this bug, and 2.19.0 introduced it, so we rolled back until 2.20.0 was released.

[trajano]

I'm on

Docker Compose version v2.20.2-desktop.1

I'm facing this bug. I'm on Windows.