[BUG] Network issues on 2.24.6
#11510
Comments
|
Hello @xtaixe |
|
@glours unfortunately, is not possible. |
|
beautiful, stuck on same issue since two days :) |
|
@glours I would install 25.0.3 but it is not available on archlinux extra-testing repo |
|
@glours so my machine now has 1022 network interfaces! It might be that docker-compose creates too many interfaces and hits a hard limit in the linux kernel or NetworkManager, and then it fails to create further networks! At the same time, docker-compose down does not properly delete created networks. Also Edit: A But it is weird that two interfaces |
|
@bf On Arch you can downgrade the package if it's still in the local cache (if not in your cache, see Arch Linux Archive): sudo pacman -U /var/cache/pacman/pkg/docker-compose-2.24.5-1-x86_64.pkg.tar.zstWhen upgrading the system, ignore the package until a fixed version is in the repository, e.g.: sudo pacman -Syu --ignore='docker-compose'But be aware that partial upgrades may cause trouble: https://wiki.archlinux.org/title/System_maintenance#Partial_upgrades_are_unsupported Obviously, just a workaround. |
|
I can't downgrade. Wasted now the second full day on this bullshit issue. Every couple of months docker-compose fucks up the networking and my user-defined bridge networks simply stop working. I'm using a bunch of vanilla bridge networks, and I wonder why this basic use case is not in a test suite somewhere? Are docker-compose devs using their own tool in a real-world project somewhere? 😭 I've isolated the bug to this: Request address PrefAddress:invalid IP - I don't even know |
|
this "PrefAddress:invalid IP" bug seems to be known since oct 2023 in moby/moby#46615 and a fix at moby/moby#46630 is also not active. FML |
|
I have fixed my issue. TLDR: Somewhere in a Dockerfile a VOLUME command is used but not connected in the docker-compose.yml. When commenting out the line with VOLUME in the respective Dockerfile, the random arp / hostname resolving errors of the user-defined bridge networks suddenly disappeared. It's extremely weird. So for some user-defined bridge network interfaces docker just forgets to set the hostname of the other container in the network. I have containers with 4 user-defined networks and 3 work fine, but 1 is connected but the counterparty is missing. The Dockerfile with the VOLUME command does not belong to the container(s) where the network issues appeared. It's like a race condition where at some point docker just seems to stop processing the arp announcements within the user-defined bridge networks (or it just stops connecting a container to the 3rd network out of 4). Docker then frantically tries to resolve the expected hostname via the host dns server which obviously fails due to internal-only networking, but that at least this shows up very prominently in However, the I hope this helps someone, I aged a year. |
|
@bf @xtaixe @ybc37 we bumped the version of compose-go yesterday into the main branch of Compose with a fix about networks order & priority, can you check if the latest binary of Compose potentially fixes your issue? |
|
Hello, |
I can confirm that while I hit the bug with 2.24.6, I do not hit it with this binary. Thanks for the heads up @glours ! |
|
Is it ok to close the issue? |
|
This fix has been included in Compose 2.24.7+. If you continue to have problems after upgrading, comment here and we can re-open the issue or create a new one as appropriate. Thanks for the report! |
This reverts commit b98310a. The bug with missing interfaces was not related to any race condition. It was a bug in docker compose: docker/compose#11510 The only affected version is: 2.24.6
Description
We are seeing test failures due to
UnknownHostExceptionandThe connection attempt failedafter upgrading from2.24.5to2.24.6without any other changes.This is happening both locally and on CI. The Docker Environment provided below is from Mac laptop with Docker Desktop where I installed version
2.24.6manually to reproduce the issue which was detected in CI.I don't have a reproducer I can share for now. Shooting in the dark here, but wondering if this could be related to #11429, since it seems to be the only change related to networking?
Steps To Reproduce
No response
Compose Version
Docker Environment
Anything else?
No response
The text was updated successfully, but these errors were encountered: