[Bug]: Can't start Postgres Container on Ubuntu 22.04

[goutamvenkatCatalan]

Testcontainers version

3.2.0

Using the latest Testcontainers version?

Yes

Host OS

Ubuntu 22.04

Host arch

x86

.NET version

7.0

Docker version

Client: Docker Engine - Community
 Cloud integration: v1.0.33
 Version:           24.0.2
 API version:       1.43
 Go version:        go1.20.4
 Git commit:        cb74dfc
 Built:             Thu May 25 21:51:00 2023
 OS/Arch:           linux/amd64
 Context:           desktop-linux

Server: Docker Desktop 4.20.0 (109717)
 Engine:
  Version:          24.0.2
  API version:      1.43 (minimum version 1.12)
  Go version:       go1.20.4
  Git commit:       659604f
  Built:            Thu May 25 21:52:17 2023
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.6.21
  GitCommit:        3dce8eb055cbb6872793272b4f20ed16117344f8
 runc:
  Version:          1.1.7
  GitCommit:        v1.1.7-0-g860f061
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

Docker info

Client: Docker Engine - Community
 Version:    24.0.2
 Context:    desktop-linux
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.10.5
    Path:     /usr/lib/docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v2.18.1
    Path:     /usr/lib/docker/cli-plugins/docker-compose
  dev: Docker Dev Environments (Docker Inc.)
    Version:  v0.1.0
    Path:     /usr/lib/docker/cli-plugins/docker-dev
  extension: Manages Docker extensions (Docker Inc.)
    Version:  v0.2.19
    Path:     /usr/lib/docker/cli-plugins/docker-extension
  init: Creates Docker-related starter files for your project (Docker Inc.)
    Version:  v0.1.0-beta.4
    Path:     /usr/lib/docker/cli-plugins/docker-init
  sbom: View the packaged-based Software Bill Of Materials (SBOM) for an image (Anchore Inc.)
    Version:  0.6.0
    Path:     /usr/lib/docker/cli-plugins/docker-sbom
  scan: Docker Scan (Docker Inc.)
    Version:  v0.26.0
    Path:     /usr/lib/docker/cli-plugins/docker-scan
  scout: Command line tool for Docker Scout (Docker Inc.)
    Version:  v0.12.0
    Path:     /usr/lib/docker/cli-plugins/docker-scout

Server:
 Containers: 0
  Running: 0
  Paused: 0
  Stopped: 0
 Images: 3
 Server Version: 24.0.2
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 3dce8eb055cbb6872793272b4f20ed16117344f8
 runc version: v1.1.7-0-g860f061
 init version: de40ad0
 Security Options:
  seccomp
   Profile: builtin
  cgroupns
 Kernel Version: 5.15.49-linuxkit-pr
 Operating System: Docker Desktop
 OSType: linux
 Architecture: x86_64
 CPUs: 8
 Total Memory: 7.674GiB
 Name: docker-desktop
 ID: 0652941c-45fe-4342-b4c7-5193c59e6af1
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 HTTP Proxy: http.docker.internal:3128
 HTTPS Proxy: http.docker.internal:3128
 No Proxy: hubproxy.docker.internal
 Experimental: false
 Insecure Registries:
  hubproxy.docker.internal:5555
  127.0.0.0/8
 Live Restore Enabled: false

What happened?

public sealed class DbFixture : IAsyncLifetime
    {
        private readonly PostgreSqlContainer _dbContainer = new PostgreSqlBuilder()
                .WithCleanUp(true)
                .Build();
        private string _connectionString = string.Empty;

        public MyDbContext CreateContext()
        {
            var context = new(
                new DbContextOptionsBuilder<MyDbContext>()
                .UseSnakeCaseNamingConvention()
                .UseNpgsql(_connectionString, o => o
                    .UseNodaTime()
                    .CommandTimeout((int)TimeSpan.FromMinutes(5).TotalSeconds)
                )
            .Options);

            context.Database.EnsureDeleted();
            context.Database.EnsureCreated();
            return context;
        }

        public async Task DisposeAsync() => await _dbContainer.StopAsync();


        public async Task InitializeAsync()
        {
            await _dbContainer.StartAsync();
            _connectionString = _dbContainer.GetConnectionString();
        }
    }

I'm using the above DbFixture for my tests, however it errors out stating that the docker container is not found. The stack trace is attached below.

Relevant log output

Docker.DotNet.DockerContainerNotFoundException: Docker API responded with status code=NotFound, response={"message":"No such container: a105c6a4aa2a2a...

Docker.DotNet.DockerContainerNotFoundException
Docker API responded with status code=NotFound, response={"message":"No such container: a105c6a4aa2a2a65cedaacb8c09f864c3fee407dad410face73600175b2b9d7b"}

   at Docker.DotNet.ContainerOperations.<>c.<.cctor>b__30_0(HttpStatusCode statusCode, String responseBody)
   at Docker.DotNet.DockerClient.HandleIfErrorResponseAsync(HttpStatusCode statusCode, HttpResponseMessage response, IEnumerable`1 handlers)
   at Docker.DotNet.DockerClient.MakeRequestAsync(IEnumerable`1 errorHandlers, HttpMethod method, String path, IQueryString queryString, IRequestContent body, IDictionary`2 headers, TimeSpan timeout, CancellationToken token)
   at Docker.DotNet.ContainerOperations.InspectContainerAsync(String id, CancellationToken cancellationToken)
   at DotNet.Testcontainers.Containers.DockerContainer.<>c__DisplayClass57_0.<<UnsafeStartAsync>g__CheckPortBindingsAsync|0>d.MoveNext()
--- End of stack trace from previous location ---
   at DotNet.Testcontainers.Configurations.WaitStrategy.<>c__DisplayClass1_0.<<WaitUntilAsync>g__UntilAsync|0>d.MoveNext()
--- End of stack trace from previous location ---
   at DotNet.Testcontainers.Configurations.WaitStrategy.WaitUntilAsync(Func`1 wait, TimeSpan frequency, TimeSpan timeout, CancellationToken ct)
   at DotNet.Testcontainers.Containers.DockerContainer.UnsafeStartAsync(CancellationToken ct)
   at DotNet.Testcontainers.Containers.DockerContainer.StartAsync(CancellationToken ct)
   at DotNet.Testcontainers.Containers.ResourceReaper.GetAndStartNewAsync(Guid sessionId, IDockerEndpointAuthenticationConfiguration dockerEndpointAuthConfig, IImage resourceReaperImage, IMount dockerSocket, Boolean requiresPrivilegedMode, TimeSpan initTimeout, CancellationToken ct)
   at DotNet.Testcontainers.Containers.ResourceReaper.GetAndStartNewAsync(Guid sessionId, IDockerEndpointAuthenticationConfiguration dockerEndpointAuthConfig, IImage resourceReaperImage, IMount dockerSocket, Boolean requiresPrivilegedMode, TimeSpan initTimeout, CancellationToken ct)
   at DotNet.Testcontainers.Containers.ResourceReaper.GetAndStartDefaultAsync(IDockerEndpointAuthenticationConfiguration dockerEndpointAuthConfig, Boolean isWindowsEngineEnabled, CancellationToken ct)
   at DotNet.Testcontainers.Clients.TestcontainersClient.RunAsync(IContainerConfiguration configuration, CancellationToken ct)
   at DotNet.Testcontainers.Containers.DockerContainer.UnsafeCreateAsync(CancellationToken ct)
   at DotNet.Testcontainers.Containers.DockerContainer.StartAsync(CancellationToken ct)
   at UnitTests.DbFixture.InitializeAsync() in /home/irsoc/Documents/CatalanV2/service/Tests/DbFixture.cs:line 49
   at Xunit.Sdk.ExceptionAggregator.RunAsync(Func`1 code) in /_/src/xunit.core/Sdk/ExceptionAggregator.cs:line 90

Additional information

No response

[HofmeisterAn]

Can you please try to set the environment variable TESTCONTAINERS_DOCKER_SOCKET_OVERRIDE to /var/run/docker.sock and try it again? I think you run into #911.

[goutamvenkatCatalan]

I did add try adding this env variable, but the result was the same.

[HofmeisterAn]

The stack trace looks like Testcontainers cannot start moby-ryuk on your machine. Can you try to run Ryuk manual:

docker run -v /var/run/docker.sock:/var/run/docker.sock -e RYUK_PORT=8080 -p 8080:8080 testcontainers/ryuk:0.5.1

[xilapa]

I'm getting the same problem, testcontainers got stuck and doesn't start the postgres container when running github actions.

I've tried to start moby-ryuk as seen on the github action below, but it doesn't help. (I don't know if I've done it right, by the way)
https://github.com/xilapa/SiteWatcher/actions/runs/5271983930/jobs/9533630643

[HofmeisterAn]

You are having a different issue than the one we are discussing here. Based on your Workflow file (.github/workflows/dotnet-sonar.yml), it looks like that the problem relates in invoking the tests through the dotnet-coverage collect command. I would suggest trying to run dotnet test instead. Please note that our tests are running fine on Ubuntu (as you can see here). Furthermore, I have noticed the following lines in your code that do not make any sense at all, you can remove them.

[HofmeisterAn]

Since we have not received any response, I will close this issue. If you still struggle running Testcontainers please do not hesitate to reopen the issue again. Please note, the mentioned workaround in my first comment is no longer necessary with version 3.3.0.

[guillaumeagile]

Interesting topic though.
This kind of problem arises whatever the image you pull in a TestContainer.
I had the same issue while using MongoDb gateway and try to run it on my local Docker Engine (on MacOsX)
and the solution was to add 2 env variables to the CLI I use to start the test in the container

It goes like:

docker run --rm -v /var/run/docker.sock:/var/run/docker.sock -v /tmp/TestResults:/src/TestResults -e TESTCONTAINERS_HOST_OVERRIDE=host.docker.internal -e TESTCONTAINERS_DOCKER_SOCKET_OVERRIDE=/var/run/docker.sock myimageoftestproject

[max-kuzmin]

Interesting topic though. This kind of problem arises whatever the image you pull in a TestContainer. I had the same issue while using MongoDb gateway and try to run it on my local Docker Engine (on MacOsX) and the solution was to add 2 env variables to the CLI I use to start the test in the container

It goes like:

docker run --rm -v /var/run/docker.sock:/var/run/docker.sock -v /tmp/TestResults:/src/TestResults -e TESTCONTAINERS_HOST_OVERRIDE=host.docker.internal -e TESTCONTAINERS_DOCKER_SOCKET_OVERRIDE=/var/run/docker.sock myimageoftestproject

Setting this enviromental variables helped me to run tests in Docker using image docker:24.0.5

TESTCONTAINERS_HOST_OVERRIDE=host.docker.internal
TESTCONTAINERS_DOCKER_SOCKET_OVERRIDE=/var/run/docker.sock