[Bug]: listChildContainers should be case insensitive - causes waitUntilServiceStarted to fail

[mccants-heb]

Module

Core

Testcontainers version

1.18.0

Using the latest Testcontainers version?

Yes

Host OS

Mac OS

Host Arch

ARM

Docker version

rush@RUSHs-MacBook-Pro dcs % docker compose version
Docker Compose version v2.17.2
rush@RUSHs-MacBook-Pro dcs % docker version
Client:
 Cloud integration: v1.0.31
 Version:           20.10.24
 API version:       1.41
 Go version:        go1.19.7
 Git commit:        297e128
 Built:             Tue Apr  4 18:21:21 2023
 OS/Arch:           darwin/arm64
 Context:           default
 Experimental:      true

Server: Docker Desktop 4.18.0 (104112)
 Engine:
  Version:          20.10.24
  API version:      1.41 (minimum version 1.12)
  Go version:       go1.19.7
  Git commit:       5d6db84
  Built:            Tue Apr  4 18:17:07 2023
  OS/Arch:          linux/arm64
  Experimental:     false
 containerd:
  Version:          1.6.18
  GitCommit:        2456e983eb9e37e47538f59ea18f2043c9a73640
 runc:
  Version:          1.1.4
  GitCommit:        v1.1.4-0-g5fd4c4d
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

What happened?

Running with DockerComposeContainer, I cannot properly wait for Services.

Example code that is failing:

@Container
private static DockerComposeContainer postGres = new DockerComposeContainer("OrderGroupProcessApiFunctionalTest_", new File("docker-compose.test.yml"))
    .withLocalCompose(false)
    .withServices("dcs-db")
    .waitingFor("dcs-db", Wait.defaultWaitStrategy());

Here is the error message:

Services named [flyway_1, dcs-db_1] do not exist, but wait conditions have been defined for them. This might mean that you misspelled the service name when defining the wait condition.
java.lang.IllegalStateException: Services named [dcs-db_1] do not exist, but wait conditions have been defined for them. This might mean that you misspelled the service name when defining the wait condition.
	at org.testcontainers.containers.DockerComposeContainer.waitUntilServiceStarted(DockerComposeContainer.java:280)
	at org.testcontainers.containers.DockerComposeContainer.start(DockerComposeContainer.java:191)
	at org.testcontainers.junit.jupiter.TestcontainersExtension$StoreAdapter.start(TestcontainersExtension.java:274)
	at org.testcontainers.junit.jupiter.TestcontainersExtension$StoreAdapter.access$200(TestcontainersExtension.java:261)
	at org.testcontainers.junit.jupiter.TestcontainersExtension.lambda$null$3(TestcontainersExtension.java:76)
	at org.junit.jupiter.engine.execution.ExtensionValuesStore.lambda$getOrComputeIfAbsent$4(ExtensionValuesStore.java:86)
	at org.junit.jupiter.engine.execution.ExtensionValuesStore$MemoizingSupplier.computeValue(ExtensionValuesStore.java:223)
	at org.junit.jupiter.engine.execution.ExtensionValuesStore$MemoizingSupplier.get(ExtensionValuesStore.java:211)
	at org.junit.jupiter.engine.execution.ExtensionValuesStore$StoredValue.evaluate(ExtensionValuesStore.java:191)
	at org.junit.jupiter.engine.execution.ExtensionValuesStore$StoredValue.access$100(ExtensionValuesStore.java:171)
	at org.junit.jupiter.engine.execution.ExtensionValuesStore.getOrComputeIfAbsent(ExtensionValuesStore.java:89)
	at org.junit.jupiter.engine.execution.NamespaceAwareStore.getOrComputeIfAbsent(NamespaceAwareStore.java:53)
	at org.testcontainers.junit.jupiter.TestcontainersExtension.lambda$startContainers$4(TestcontainersExtension.java:76)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
	at org.testcontainers.junit.jupiter.TestcontainersExtension.startContainers(TestcontainersExtension.java:76)
	at org.testcontainers.junit.jupiter.TestcontainersExtension.beforeAll(TestcontainersExtension.java:56)
	at ... 

This is caused because DockerComposeContainer.waitUntilServiceStarted() calls listChildContainers and it returns no child containers, even though it should. The reason it doesn't is because it fails to properly compare names on this line:

.filter(container -> Arrays.stream(container.getNames()).anyMatch(name -> name.startsWith("/" + project)))

In my case, the project name is OrderGroupProcessApiFunctionalTest_b8kvyt, but the name of the matching container is /ordergroupprocessapifunctionaltest_b8kvyt_flyway_1. The filter states filters out this container, because the filter statement is case sensitive.

This results in DockerComposeContainer.waitUntilServiceStarted() putting something in missingServiceInstances and throw an IllegalStateException resulting in the error message above.

Worse, this leaves orphaned Docker instances after the test completes.

Relevant log output

No response

Additional Information

Work around: lower case the project name when creating DockerComposeContainer.

[eddumelendez]

You are using compose v2 and it is not currently supported. See #5278

[mccants-heb]

That explains why running .withLocalCompose(true) fails, but since .withLocalCompose(false) is being used, doesn't that mean you are using whichever Docker Compose version you want? Seems like the bug should be reopened.

[eddumelendez]

There are differences between compose v1 and v2. The one is impacting Testcontainers is

Container names now use hyphens as separators instead of underscores.

See more here

withLocalCompose(true) it will be using your cli. Meanwhile, withLocalCompose(false) will use a default docker image which currently supports compose v1 too.

but since .withLocalCompose(false) is being used, doesn't that mean you are using whichever Docker Compose version you want?

There is an option in docker desktop which IIRC creates a symlink so executing docker-compose or docker compose would be the same in your cli.

[mccants-heb]

Thank you for your reply, @eddumelendez. Since I'm running using withLocalCompose(false), I'm still unsure why my installation of Docker Compose would be a problem. I included the locally installed Docker Compose version number above in case it was helpful, but maybe I just have created confusion.

Does withLocalCompose(false) still using the locally installed version of Docker Compose?

[eddumelendez]

Does withLocalCompose(false) still using the locally installed version of Docker Compose?

No, it doesn't. What happens in that case is a docker image is downloaded and the docker-compose.yml file is copied and executed in that container

[mccants-heb]

@eddumelendez , that is what I expected and matchs what I've seen.

Since this bug is seen running using withLocalCompose(false), I don't think having Docker Compose v2 installed should be a problem. Should the bug be reopened?

[eddumelendez]

I see, you are running ARM. And the existing docker image doesn't work on ARM. See #5524. So, we already have issues for those cases.

[eddumelendez]

Once #5608 is merged, both issues should be solved

[mccants-heb]

@eddumelendez, this does not seem to be an ARM related problem, sincemy work around listed in the original ticket makes TestContainer work. I think this is an entirely separate issue (and much smaller issue).

[eddumelendez]

I've reopen it and take a look closer later

[eddumelendez]

Sorry for the confusion @mccants-heb. I've submitted a PR adding a test case and fix for this issue. Thanks for the report.