Handle empty or invalid `$HOME/.docker/config.json` gracefully #4847

chibenwa · 2022-01-04T08:38:03Z

Reference: https://issues.apache.org/jira/browse/INFRA-22697

Apparently testcontainers requires $HOME/.docker/config.json to be a valid JSON file.

touch ~/.docker/config.json
// lauch some test containers tests will now fail

echo '{}' > ~/.docker/config.json
// lauch some test containers tests will now succeeds

rm ~/.docker/config.json
// lauch some test containers tests will now succeeds

The problem is that I have little control on the content of this file as provided on the ASF CI environments. I would like testcontainers to ignore it as missing when its content is missing...

The text was updated successfully, but these errors were encountered:

kiview · 2022-01-04T10:20:34Z

Hey @chibenwa, thanks for raising the issue. Can you please provide logs of the errors that occur in Testcontainers if config.json is empty?

chibenwa · 2022-01-04T11:54:40Z

I got for instance

java.lang.ExceptionInInitializerError
	at org.apache.james.backends.cassandra.DockerCassandraRule.start(DockerCassandraRule.java:46)
	at org.apache.james.backends.cassandra.DockerCassandraExtension.beforeAll(DockerCassandraExtension.java:48)
	at org.apache.james.backends.cassandra.CassandraClusterExtension.beforeAll(CassandraClusterExtension.java:54)
	at org.junit.jupiter.engine.descriptor.ClassBasedTestDescriptor.lambda$invokeBeforeAllCallbacks$8(ClassBasedTestDescriptor.java:368)
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at org.junit.jupiter.engine.descriptor.ClassBasedTestDescriptor.invokeBeforeAllCallbacks(ClassBasedTestDescriptor.java:368)
	at org.junit.jupiter.engine.descriptor.ClassBasedTestDescriptor.before(ClassBasedTestDescriptor.java:192)
	at org.junit.jupiter.engine.descriptor.ClassBasedTestDescriptor.before(ClassBasedTestDescriptor.java:78)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:136)
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:129)
	at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:127)
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:126)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:84)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1541)
	at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:38)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:143)
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:129)
	at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:127)
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:126)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:84)
	at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.submit(SameThreadHierarchicalTestExecutorService.java:32)
	at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor.execute(HierarchicalTestExecutor.java:57)
	at org.junit.platform.engine.support.hierarchical.HierarchicalTestEngine.execute(HierarchicalTestEngine.java:51)
	at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:108)
	at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:88)
	at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:54)
	at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:67)
	at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:52)
	at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:96)
	at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:75)
	at com.intellij.junit5.JUnit5IdeaTestRunner.startRunnerWithArgs(JUnit5IdeaTestRunner.java:71)
	at com.intellij.rt.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:33)
	at com.intellij.rt.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:220)
	at com.intellij.rt.junit.JUnitStarter.main(JUnitStarter.java:53)
	Suppressed: java.lang.NullPointerException
		at org.apache.james.backends.cassandra.CassandraClusterExtension.afterAll(CassandraClusterExtension.java:93)
		at org.junit.jupiter.engine.descriptor.ClassBasedTestDescriptor.lambda$invokeAfterAllCallbacks$14(ClassBasedTestDescriptor.java:434)
		at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
		at org.junit.jupiter.engine.descriptor.ClassBasedTestDescriptor.lambda$invokeAfterAllCallbacks$15(ClassBasedTestDescriptor.java:434)
		at java.base/java.util.ArrayList.forEach(ArrayList.java:1541)
		at org.junit.jupiter.engine.descriptor.ClassBasedTestDescriptor.invokeAfterAllCallbacks(ClassBasedTestDescriptor.java:434)
		at org.junit.jupiter.engine.descriptor.ClassBasedTestDescriptor.after(ClassBasedTestDescriptor.java:216)
		at org.junit.jupiter.engine.descriptor.ClassBasedTestDescriptor.after(ClassBasedTestDescriptor.java:78)
		at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:149)
		at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
		at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:149)
		... 29 more
Caused by: org.testcontainers.containers.ContainerLaunchException: Container startup failed
	at org.testcontainers.containers.GenericContainer.doStart(GenericContainer.java:330)
	at org.testcontainers.containers.GenericContainer.start(GenericContainer.java:311)
	at org.apache.james.backends.cassandra.DockerCassandra.start(DockerCassandra.java:198)
	at org.apache.james.backends.cassandra.DockerCassandraSingleton.<clinit>(DockerCassandraSingleton.java:40)
	... 40 more
Caused by: org.testcontainers.containers.ContainerFetchException: Can't get Docker image: RemoteDockerImage(imageName=<resolving>, imagePullPolicy=DefaultPullPolicy())
	at org.testcontainers.containers.GenericContainer.getDockerImageName(GenericContainer.java:1286)
	at org.testcontainers.containers.GenericContainer.logger(GenericContainer.java:615)
	at org.testcontainers.containers.GenericContainer.doStart(GenericContainer.java:320)
	... 43 more
Caused by: com.github.dockerjava.api.exception.DockerClientException: Failed to parse docker configuration file
	at org.testcontainers.shaded.com.github.dockerjava.core.DefaultDockerClientConfig.getDockerConfig(DefaultDockerClientConfig.java:265)
	at org.testcontainers.shaded.com.github.dockerjava.core.DefaultDockerClientConfig.getAuthConfigurations(DefaultDockerClientConfig.java:301)
	at org.testcontainers.dockerclient.AuthDelegatingDockerClientConfig.getAuthConfigurations(AuthDelegatingDockerClientConfig.java:23)
	at org.testcontainers.shaded.com.github.dockerjava.core.exec.AbstrDockerCmdExec.getBuildAuthConfigs(AbstrDockerCmdExec.java:40)
	at org.testcontainers.shaded.com.github.dockerjava.core.exec.BuildImageCmdExec.resourceWithOptionalAuthConfig(BuildImageCmdExec.java:29)
	at org.testcontainers.shaded.com.github.dockerjava.core.exec.BuildImageCmdExec.execute0(BuildImageCmdExec.java:126)
	at org.testcontainers.shaded.com.github.dockerjava.core.exec.BuildImageCmdExec.execute0(BuildImageCmdExec.java:20)
	at org.testcontainers.shaded.com.github.dockerjava.core.exec.AbstrAsyncDockerCmdExec.execute(AbstrAsyncDockerCmdExec.java:56)
	at org.testcontainers.shaded.com.github.dockerjava.core.exec.AbstrAsyncDockerCmdExec.exec(AbstrAsyncDockerCmdExec.java:21)
	at org.testcontainers.shaded.com.github.dockerjava.core.exec.AbstrAsyncDockerCmdExec.exec(AbstrAsyncDockerCmdExec.java:12)
	at org.testcontainers.shaded.com.github.dockerjava.core.command.AbstrAsyncDockerCmd.exec(AbstrAsyncDockerCmd.java:21)
	at org.testcontainers.images.builder.ImageFromDockerfile.resolve(ImageFromDockerfile.java:119)
	at org.testcontainers.images.builder.ImageFromDockerfile.resolve(ImageFromDockerfile.java:37)
	at org.testcontainers.utility.LazyFuture.getResolvedValue(LazyFuture.java:17)
	at org.testcontainers.utility.LazyFuture.get(LazyFuture.java:39)
	at org.testcontainers.shaded.com.google.common.util.concurrent.Futures$3.get(Futures.java:1332)
	at org.testcontainers.images.RemoteDockerImage.getImageName(RemoteDockerImage.java:104)
	at org.testcontainers.images.RemoteDockerImage.resolve(RemoteDockerImage.java:63)
	at org.testcontainers.images.RemoteDockerImage.resolve(RemoteDockerImage.java:27)
	at org.testcontainers.utility.LazyFuture.getResolvedValue(LazyFuture.java:17)
	at org.testcontainers.utility.LazyFuture.get(LazyFuture.java:39)
	at org.testcontainers.containers.GenericContainer.getDockerImageName(GenericContainer.java:1284)
	... 45 more
Caused by: java.io.IOException: Failed to parse docker config.json
	at org.testcontainers.shaded.com.github.dockerjava.core.DockerConfigFile.loadCurrentConfig(DockerConfigFile.java:164)
	at org.testcontainers.shaded.com.github.dockerjava.core.DockerConfigFile.loadConfig(DockerConfigFile.java:131)
	at org.testcontainers.shaded.com.github.dockerjava.core.DefaultDockerClientConfig.getDockerConfig(DefaultDockerClientConfig.java:263)
	... 66 more
Caused by: org.testcontainers.shaded.com.fasterxml.jackson.databind.exc.MismatchedInputException: No content to map due to end-of-input
 at [Source: (File); line: 1, column: 0]
	at org.testcontainers.shaded.com.fasterxml.jackson.databind.exc.MismatchedInputException.from(MismatchedInputException.java:59)
	at org.testcontainers.shaded.com.fasterxml.jackson.databind.ObjectMapper._initForReading(ObjectMapper.java:4360)
	at org.testcontainers.shaded.com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:4205)
	at org.testcontainers.shaded.com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3079)
	at org.testcontainers.shaded.com.github.dockerjava.core.DockerConfigFile.loadCurrentConfig(DockerConfigFile.java:162)
	... 68 more

kiview · 2022-01-05T11:58:31Z

That helps, thanks!

chibenwa · 2022-01-05T12:46:11Z

Handle empty $HOME/.docker/config.json gracefully

I wonder if it should not be rather Handle invalid $HOME/.docker/config.json gracefully.

As this config is apparently only used to discover private registry if invalid we could adopt a lenient beaviour and behave as if there was no such file (maybe with a log)

This is what docker CLI does:

$ echo "bad" > ~/.docker/config.json
$ cat ~/.docker/config.json
bad
$ docker ps
WARNING: Error loading config file: /home/interview1/.docker/config.json: invalid character 'b' looking for beginning of value
CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS               NAMES
...

You could see that even if it complains, the invalid file did not prevent the docker CLI from working normally...

I think such a behaviour would be less specific to my use case but benefit the whole community.

bsideup · 2022-01-05T13:06:09Z

BTW it looks like the exception is coming from docker-java, so a fix would be needed there first and then we just need to update it in Testcontainers.

dplavcic · 2022-06-20T19:59:18Z

Hi @chibenwa, can you still reproduce this if invalid config.json file is provided?

Reference: https://issues.apache.org/jira/browse/INFRA-22697

Apparently testcontainers requires $HOME/.docker/config.json to be a valid JSON file.
touch ~/.docker/config.json
// lauch some test containers tests will now fail

echo '{}' > ~/.docker/config.json
// lauch some test containers tests will now succeeds

rm ~/.docker/config.json
// lauch some test containers tests will now succeeds
The problem is that I have little control on the content of this file as provided on the ASF CI environments. I would like testcontainers to ignore it as missing when its content is missing...

I have tried to reproduce it locally (testcontainers version 1.17.2) by executing test provided here, but I can't (not sure if I'm doing something wrong, or if the "issue" is fixed) 🤔

touch ~/.docker/config.json
// lauch some test containers tests will now fail <-- this is no longer true

21:13:07.484 [main] DEBUG org.testcontainers.dockerclient.AuthDelegatingDockerClientConfig - Delegate call to effectiveAuthConfig failed with cause: 'Failed to parse docker configuration file'. Resolution of auth config will continue using RegistryAuthLocator.
21:13:07.486 [main] DEBUG org.testcontainers.utility.RegistryAuthLocator - Looking up auth config for image: testcontainers/ryuk:0.3.3 at registry: https://index.docker.io/v1/
21:13:07.487 [main] DEBUG org.testcontainers.utility.RegistryAuthLocator - RegistryAuthLocator has configFile: /qwe/user/.docker/config.json (exists) and commandPathPrefix:
21:13:07.487 [main] INFO org.testcontainers.utility.RegistryAuthLocator - Failure when attempting to lookup auth config. Please ignore if you don't have images in an authenticated registry. Details: (dockerImageName: testcontainers/ryuk:0.3.3, configFile: /qwe/user/.docker/config.json. Falling back to docker-java default behaviour. Exception message: No content to map due to end-of-input
 at [Source: /qwe/user/.docker/config.json; line: 1, column: 0]
21:13:07.489 [main] DEBUG org.testcontainers.utility.RegistryAuthLocator - No matching Auth Configs - falling back to defaultAuthConfig [AuthConfig{username=null, password=blank, auth=blank, email=null, registryAddress=https://index.docker.io/v1/, registryToken=blank}]
21:13:07.490 [main] DEBUG org.testcontainers.dockerclient.AuthDelegatingDockerClientConfig - Effective auth config [AuthConfig{username=null, password=blank, auth=blank, email=null, registryAddress=https://index.docker.io/v1/, registryToken=blank}]
2

echo '{}' > ~/.docker/config.json
// lauch some test containers tests will now succeeds <-- this is true

21:16:06.512 [main] DEBUG org.testcontainers.utility.RegistryAuthLocator - Looking up auth config for image: testcontainers/ryuk:0.3.3 at registry: https://index.docker.io/v1/
21:16:06.514 [main] DEBUG org.testcontainers.utility.RegistryAuthLocator - RegistryAuthLocator has configFile: /qwe/user/.docker/config.json (exists) and commandPathPrefix:
21:16:06.514 [main] DEBUG org.testcontainers.utility.RegistryAuthLocator - registryName [https://index.docker.io/v1/] for dockerImageName [testcontainers/ryuk:0.3.3]
21:16:06.515 [main] DEBUG org.testcontainers.utility.RegistryAuthLocator - No matching Auth Configs - falling back to defaultAuthConfig [null]
21:16:06.515 [main] DEBUG org.testcontainers.dockerclient.AuthDelegatingDockerClientConfig - Effective auth config [null]

rm ~/.docker/config.json
// lauch some test containers tests will now succeeds <-- this is true

21:17:47.531 [main] DEBUG org.testcontainers.utility.RegistryAuthLocator - Looking up auth config for image: testcontainers/ryuk:0.3.3 at registry: https://index.docker.io/v1/
21:17:47.532 [main] DEBUG org.testcontainers.utility.RegistryAuthLocator - RegistryAuthLocator has configFile: /qwe/user/.docker/config.json (does not exist) and commandPathPrefix:
21:17:47.532 [main] INFO org.testcontainers.utility.RegistryAuthLocator - Failure when attempting to lookup auth config. Please ignore if you don't have images in an authenticated registry. Details: (dockerImageName: testcontainers/ryuk:0.3.3, configFile: /qwe/user/.docker/config.json. Falling back to docker-java default behaviour. Exception message: /qwe/user/.docker/config.json (No such file or directory)
21:17:47.534 [main] DEBUG org.testcontainers.utility.RegistryAuthLocator - No matching Auth Configs - falling back to defaultAuthConfig [null]
21:17:47.534 [main] DEBUG org.testcontainers.dockerclient.AuthDelegatingDockerClientConfig - Effective auth config [null]

echo "bad" > ~/.docker/config.json
// lauch some test containers tests will now fail <-- this is no longer true

21:23:04.943 [main] INFO 🐳 [testcontainers/ryuk:0.3.3] - Creating container for image: testcontainers/ryuk:0.3.3
21:23:04.946 [main] DEBUG org.testcontainers.dockerclient.AuthDelegatingDockerClientConfig - Delegate call to effectiveAuthConfig failed with cause: 'Failed to parse docker configuration file'. Resolution of auth config will continue using RegistryAuthLocator.
21:23:04.947 [main] DEBUG org.testcontainers.utility.RegistryAuthLocator - Looking up auth config for image: testcontainers/ryuk:0.3.3 at registry: https://index.docker.io/v1/
21:23:04.948 [main] DEBUG org.testcontainers.utility.RegistryAuthLocator - RegistryAuthLocator has configFile: /qwe/user/.docker/config.json (exists) and commandPathPrefix:
21:23:04.949 [main] INFO org.testcontainers.utility.RegistryAuthLocator - Failure when attempting to lookup auth config. Please ignore if you don't have images in an authenticated registry. Details: (dockerImageName: testcontainers/ryuk:0.3.3, configFile: /qwe/user/.docker/config.json. Falling back to docker-java default behaviour. Exception message: Unrecognized token 'bad': was expecting ('true', 'false' or 'null')
 at [Source: /qwe/user/.docker/config.json; line: 1, column: 5]
21:23:04.950 [main] DEBUG org.testcontainers.utility.RegistryAuthLocator - No matching Auth Configs - falling back to defaultAuthConfig [AuthConfig{username=null, password=blank, auth=blank, email=null, registryAddress=https://index.docker.io/v1/, registryToken=blank}]
21:23:04.950 [main] DEBUG org.testcontainers.dockerclient.AuthDelegatingDockerClientConfig - Effective auth config [AuthConfig{username=null, password=blank, auth=blank, email=null, registryAddress=https://index.docker.io/v1/, registryToken=blank}]

axel-n · 2023-01-06T10:12:47Z

@chibenwa @kiview hi, let's close this issue
I also tried to reproduce the problem locally, but all works without errors

kiview · 2023-01-06T10:52:21Z

@chibenwa Any way this can be still reproduced?

WitekBaranowskiAllegro · 2023-04-07T10:44:25Z

It appeared on 1.18.0 (empty config.json file) (previous releases working fine)
I guess api platform('com.github.docker-java:docker-java-bom:3.2.13') shaded platform('com.github.docker-java:docker-java-bom:3.2.13') api platform('com.github.docker-java:docker-java-bom:3.3.0') shaded platform('com.github.docker-java:docker-java-bom:3.3.0')
cause it

eddumelendez · 2023-04-07T11:07:54Z

support for docker context was recently introduced in docker-java. So, that's what can cause the issue now.

the-sky7 · 2024-10-08T20:29:53Z

Hi @chibenwa, if this has been fixed could you please mark it done/close.

chibenwa · 2024-10-09T06:52:57Z

Sure!

kiview added resolution/waiting-for-info Waiting for more information of the issue author or another 3rd party. type/bug labels Jan 4, 2022

kiview added good first issue resolution/acknowledged and removed resolution/waiting-for-info Waiting for more information of the issue author or another 3rd party. labels Jan 5, 2022

bsideup changed the title ~~Apparently testcontainers fails when $HOME/.docker/config.json is empty~~ Handle empty $HOME/.docker/config.json gracefully Jan 5, 2022

kiview changed the title ~~Handle empty $HOME/.docker/config.json gracefully~~ Handle empty or invalid $HOME/.docker/config.json gracefully Jan 5, 2022

mmalenic mentioned this issue Sep 3, 2024

[Bug]: Consider handling non-existent ~/.docker/config.json testcontainers/testcontainers-go#2767

Closed

chibenwa closed this as completed Oct 9, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Handle empty or invalid `$HOME/.docker/config.json` gracefully #4847

Handle empty or invalid `$HOME/.docker/config.json` gracefully #4847

chibenwa commented Jan 4, 2022

kiview commented Jan 4, 2022

chibenwa commented Jan 4, 2022

kiview commented Jan 5, 2022

chibenwa commented Jan 5, 2022 •

edited

Loading

bsideup commented Jan 5, 2022

dplavcic commented Jun 20, 2022 •

edited

Loading

axel-n commented Jan 6, 2023

kiview commented Jan 6, 2023

WitekBaranowskiAllegro commented Apr 7, 2023 •

edited

Loading

eddumelendez commented Apr 7, 2023

the-sky7 commented Oct 8, 2024 •

edited

Loading

chibenwa commented Oct 9, 2024

Handle empty or invalid $HOME/.docker/config.json gracefully #4847

Handle empty or invalid $HOME/.docker/config.json gracefully #4847

Comments

chibenwa commented Jan 4, 2022

kiview commented Jan 4, 2022

chibenwa commented Jan 4, 2022

kiview commented Jan 5, 2022

chibenwa commented Jan 5, 2022 • edited Loading

bsideup commented Jan 5, 2022

dplavcic commented Jun 20, 2022 • edited Loading

axel-n commented Jan 6, 2023

kiview commented Jan 6, 2023

WitekBaranowskiAllegro commented Apr 7, 2023 • edited Loading

eddumelendez commented Apr 7, 2023

the-sky7 commented Oct 8, 2024 • edited Loading

chibenwa commented Oct 9, 2024

Handle empty or invalid `$HOME/.docker/config.json` gracefully #4847

Handle empty or invalid `$HOME/.docker/config.json` gracefully #4847

chibenwa commented Jan 5, 2022 •

edited

Loading

dplavcic commented Jun 20, 2022 •

edited

Loading

WitekBaranowskiAllegro commented Apr 7, 2023 •

edited

Loading

the-sky7 commented Oct 8, 2024 •

edited

Loading