[BUG] Regression: services security_opt array items[0,1] must be unique

[YtvwlD]

Description

I have a docker-compose.yaml:

services:
   traefik:
     image: traefik
     security_opt:
       - label:type:container_runtime_t

And I have a docker-compose.override.yaml which contains exactly the same content (yes, I know this is not a good idea).

With Docker Compose v2.26.1, this fails with "validating docker-compose.override.yaml: services.traefik.security_opt array items[0,1] must be unique".
With Docker Compose v2.20.2, this works.

Steps To Reproduce

1. create both docker-compose.yaml and docker-compose.override.yaml with the same content as above
2. run docker compose up
3. it fails on Docker Compose v.2.26.1

Compose Version

On the system where it doesn't work:

$ docker compose version
Docker Compose version v2.26.1
$ docker-compose version
Command 'docker-compose' not found, but can be installed with:
[…]

On the system where it does work:

$ docker compose version
Docker Compose version 2.20.2+ds1-0ubuntu1~22.04.1
$ docker-compose version
docker-compose version 1.29.2, build unknown
docker-py version: 5.0.3
CPython version: 3.10.12
OpenSSL version: OpenSSL 3.0.2 15 Mar 2022

Docker Environment

On the system where it doesn't work:

Client: Docker Engine - Community
 Version:    26.1.0
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version: v0.24.0
    Path: /usr/libexec/docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version: v2.26.1
    Path: /usr/libexec/docker/cli-plugins/docker-compose

Server:
  Containers: 10
    Running: 0
    Paused: 0
    Stopped: 10
  Images: 10
  Server Version: 26.1.0
  Storage Driver: overlay2
    Backing Filesystem: extfs
    Supports d_type: true
    Using metacopy: false
    Native Overlay Diff: true
    userxattr: false
  Logging Driver: json-file
  Cgroup Driver: systemd
  Cgroup Version: 2
  Plugins:
    Volume: local
    Network: bridge host ipvlan macvlan null overlay
    Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
  Swarm: inactive
  Runtimes: io.containerd.runc.v2 runc
  Default Runtime: runc
  Init Binary: docker-init
  containerd version: e377cd56a71523140ca6ae87e30244719194a521
  runc version: v1.1.12-0-g15d5e94
  init version: de40ad0
  Security Options:
    apparmor
    seccomp
      Profile: builtin
    cgroupns
  Kernel Version: 5.15.0-105-generic
  Operating System: Ubuntu 22.04.4 LTS
  OSType: linux
  Architecture: x86_64
  CPUs: 4
  Total Memory: 15.61GiB
  Name: <name>
  ID: <ID>
  Docker Root Dir: /var/lib/docker
  Debug Mode: false
  Experimental: false
  Insecure Registries:
    127.0.0.0/8
  Live Restore Enabled: false

On the system where it does work:

Client:
 Version:    24.0.5
 Context:    default
 Debug Mode: false
 Plugins:
  compose: Docker Compose (Docker Inc.)
    Version:  2.20.2+ds1-0ubuntu1~22.04.1
    Path:     /usr/libexec/docker/cli-plugins/docker-compose

Server:
 Containers: 8
  Running: 0
  Paused: 0
  Stopped: 8
 Images: 200
 Server Version: 24.0.5
 Storage Driver: btrfs
  Btrfs: 
 Logging Driver: json-file
 Cgroup Driver: systemd
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 
 runc version: 
 init version: 
 Security Options:
  apparmor
  seccomp
   Profile: builtin
  cgroupns
 Kernel Version: 5.15.0-105-generic
 Operating System: Ubuntu 22.04.4 LTS
 OSType: linux
 Architecture: x86_64
 CPUs: 4
 Total Memory: 7.487GiB
 Name: <name>
 ID: <ID>
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Username: ytvwld
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

Anything else?

It might be related to #11371.

[slt]

We were also getting this when including the same docker-compose file twice (our mistake). This had been working on prior docker compose versions but broke after upgrading to 2.26

A simplified example is:

docker-compose.our-service.yml

services:
   our-service:
     security_opt:
       - label:type:container_runtime_t

COMPOSE_FILE='docker/docker-compose.our-service.yml:docker/docker-compose.our-service.yml'

[kevbarns]

Hello,

I'm getting the same issue from volume_from on the latest version :

Client: Docker Engine - Community
 Version:    27.0.1
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.15.1
    Path:     /usr/libexec/docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v2.28.1
    Path:     /usr/libexec/docker/cli-plugins/docker-compose

[scnewma]

We are seeing this same problem upgrading from v2.29.2 -> v2.29.3.

Reproducible example:

# compose.yml
services:
  foo:
    image: foo:latest
    extra_hosts:
      - "host.docker.internal:host-gateway"

# compose.override.yml
services:
  foo:
    image: foo:1
    extra_hosts:
      - "host.docker.internal:host-gateway"

Execute:

$ export COMPOSE_FILE=compose.yml:compose.override.yml
$ docker compose config
validating compose.override.yml: services.foo.extra_hosts array items[0,1] must be unique

This only happens on v2.29.3 not on v2.29.2.

Edit: Maybe compose-spec/compose-go#678 is root cause?

[aaaahoang123]

@scnewma It's exactly the problem I've met after I updated the newest docker version on linux.

I must used a workaround by download the standalone docker-compose binary with v2.29.2 version and execute the compose comand by docker-compose instead of docker compose.

Hope for an update soon.

[ndeloof]

This has been fixed by compose-spec/compose-go@d3fd7d9