Bug: Running Container with_network makes get_exposed_port unreliable

[lhjnilsson]

Describe the bug

Hello!

I have a case where I want to run several containers inside a specific docker network environment.
However when attaching a container to a network it seems that port forwarding is not attached to the new network - making the overall experience unstable(not able to get postgres URL, few containers like Minio do healthcheck that hangs etc)

It seems like this is not always the case however. Roughly 30% of the times the ports are actually forwarded and it succeeded. However most of the time it fails: resulting in hanging operations etc.

To Reproduce

Given a case as below. Keep in mind that sometimes it does work. Most of the time it won't.
Result will be that get_exposed_port operation hangs

from testcontainers.postgres import PostgresContainer
from testcontainers.core.network import Network

with Network() as network:
    c = PostgresContainer("postgres:13.3-alpine")
    c.with_network(network)
    c.start()
    print("Port: ", c.get_exposed_port(5432))
    c.stop()

Runtime environment

Provide a summary of your runtime environment. Which operating system, python version, and docker version are you using? What is the version of testcontainers-python you are using? You can run the following commands to get the relevant information.

# Get the operating system information (on a unix os).
$ uname -a
Darwin Henriks-MBP-2.fritz.box 23.5.0 Darwin Kernel Version 23.5.0: Wed May  1 20:16:51 PDT 2024; root:xnu-10063.121.3~5/RELEASE_ARM64_T8103 arm64
# Get the python version.
$ python --version
Python 3.12.3
# Get the docker version and other docker information.
$ docker info
Client:
 Version:           26.1.4
 API version:       1.45
 Go version:        go1.21.11
 Git commit:        5650f9b
 Built:             Wed Jun  5 11:26:02 2024
 OS/Arch:           darwin/arm64
 Context:           default

Server: Docker Desktop 4.31.0 (153195)
 Engine:
  Version:          26.1.4
  API version:      1.45 (minimum version 1.24)
  Go version:       go1.21.11
  Git commit:       de5c9cf
  Built:            Wed Jun  5 11:29:12 2024
  OS/Arch:          linux/arm64
  Experimental:     true
 containerd:
  Version:          1.6.33
  GitCommit:        d2d58213f83a351ca8f528a95fbd145f5654e957
 runc:
  Version:          1.1.12
  GitCommit:        v1.1.12-0-g51d5e94
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0
# Get all python packages.
$ pip freeze
docker==7.1.0
testcontainers==4.6.0

[lhjnilsson]

Can achieve the desired result by passing network=network.name and desired hostname as kwargs when creating the container.

I am still unsure how and why it is so unstable when using the with_network, and also if this maybe should be removed. With reference to pass in on the initializer instead

[lhjnilsson]

@alexanderankin saw you added some labels.

I am happy to assist!
Shall i add depreciation- flag on with_network ? and add the network and hostname as explicit arguments for Container Initialiser?

[alexanderankin]

I have introduced this pattern for configurations which are experimental

testcontainers-python/core/testcontainers/core/config.py

Lines 65 to 66 in 41fbdd0

	if config and "DOCKER_AUTH_CONFIG" in _WARNINGS:
	warning(_WARNINGS.pop("DOCKER_AUTH_CONFIG"))

if we can add something like this which links to this exact issue that would be great.

what would be even better is identifying and resolving the issue, which i believe stems from faulty detection with regards to detecting whether or not we are inside of a nested containerized situation or not (or reacting to that correctly).

I would say it is certainly not deprecated for removal, I think we should try to fix it first before giving up entirely.

[alexanderankin]

lets not add container arguments - generally there is nothing magic going on here - maybe I can discover something after taking a closer look - if it works in the contstructor but not in the with_network should maybe even be an easy fix

[lhjnilsson]

Understood! Thanks
It is interesting that the with_network sometimes work, sometimes not.
I will see if i can look into it abit during the weekend

[g0di]

For people stumbling here and for which the workaround is not clear, here it is:

from testcontainers.core.container import DockerContainer
from testcontainers.core.network import Network

network = Network()
network_name = network.name
networking_config = {network_name: {"aliases": ["my-service-alias-1", "my-service-alias-2"]}
container = DockerContainer("your-image-name", network=docker_network.name, networking_config=networking_config)
container.start()
...
container.stop()

By the way, I'm wondering if the with_network and with_network_aliases should not just update _kwargs attribute on the container object. What is the reason to manually call connect function on the network instead of just relying on the container run command for attaching the container to a network?

[alexanderankin]

anyone care to weigh in if #678 fixes things? went out in 4.8.0 today

[amitlevy21]

Hi @alexanderankin , I'm using 4.8.1 and having the same issue. This is my code:

async def init_db(
    postgres_ro: PostgresContainer,
):
    print(postgres_ro.get_connection_url()) # outputs correctly
    print(postgres_ro.port) # prints the inner port 5321
    print(postgres_ro.ports) # prints {5321: None}
    print(postgres_ro.get_exposed_port(5321)) # hangs and crashes 

it looks like the only way to connect to the container would be to use the entire connection url or parse the exposed port from it like in my case where i need to explicitly give a port in my connect to db function