[BUG] using multiple yaml alias not working. mapping key "<<" already defined

[arulrajnet]

Description

docker-compose.yml

version: '3.9'

# yml aliases and anchors
x-mysql: &mysqlenv
  MYSQL_ROOT_PASSWORD: root
  MYSQL_DATABASE: test
  MYSQL_USER: test
  MYSQL_PASSWORD: test

x-mysql-extraenv: &mysqlextraenv
  MY_CUSTOM_VAR: mycustomvalue

services:
  mysql:
    image: mysql:5.7
    environment:
      <<: *mysqlenv
      <<: *mysqlextraenv
    volumes:
      - mysql-data:/var/lib/mysql

volumes:
  mysql-data:

Try docker-compose config. Then getting the following error

yaml: unmarshal errors:
  line 18: mapping key "<<" already defined at line 17

The version of my docker-compose is 2.17.2

The same is working in 2.15.1 , 2.16.0 and even release candidate version 2.17.0-rc.1.

This is broken from version 2.17.0

Steps To Reproduce

1. Windows 11
2. Docker compose v2.17.0
3. Copy the above docker-compose.yml
4. Then docker-compose config
5. You will see the error

Compose Version

Docker Compose version v2.17.0

Docker Environment

Client:
 Context:    default
 Debug Mode: false

Server:
 Containers: 9
  Running: 1
  Paused: 0
  Stopped: 8
 Images: 53
 Server Version: 23.0.0
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: systemd
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 31aa4358a36870b21a992d3ad2bef29e1d693bec
 runc version: v1.1.4-0-g5fd4c4d
 init version: de40ad0
 Security Options:
  apparmor
  seccomp
   Profile: builtin
  cgroupns
 Kernel Version: 5.15.0-67-generic
 Operating System: Ubuntu 22.04 LTS
 OSType: linux
 Architecture: x86_64
 CPUs: 4
 Total Memory: 8.736GiB
 Name: ubuntuserver2204
 ID: c8839b3e-9c74-4cb6-9444-e5c979394be0
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Username: arulrajnet
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

WARNING: API is accessible on http://0.0.0.0:2375 without encryption.
         Access to the remote API is equivalent to root access on the host. Refer
         to the 'Docker daemon attack surface' section in the documentation for
         more information: https://docs.docker.com/go/attack-surface/

Anything else?

Not Working in 2.17.0

arul@windows ~/Downloads/dc-yaml-alias-bug$~/Downloads/docker-compose-windows-x86_64-2.17.0.exe config
yaml: unmarshal errors:
  line 18: mapping key "<<" already defined at line 17

Working in 2.17.0-rc.1

arul@windows ~/Downloads/dc-yaml-alias-bug$~/Downloads/docker-compose-windows-x86_64-2.17.0-rc.1.exe config
name: dc-yaml-alias-bug
services:
  mysql:
    environment:
      MY_CUSTOM_VAR: mycustomvalue
      MYSQL_DATABASE: test
      MYSQL_PASSWORD: test
      MYSQL_ROOT_PASSWORD: root
      MYSQL_USER: test
    image: mysql:5.7
    networks:
      default: null
    volumes:
    - type: volume
      source: mysql-data
      target: /var/lib/mysql
      volume: {}
networks:
  default:
    name: dc-yaml-alias-bug_default
volumes:
  mysql-data:
    name: dc-yaml-alias-bug_mysql-data
x-mysql:
  MYSQL_DATABASE: test
  MYSQL_PASSWORD: test
  MYSQL_ROOT_PASSWORD: root
  MYSQL_USER: test
x-mysql-extraenv:
  MY_CUSTOM_VAR: mycustomvalue

[ndeloof]

Thanks for reporting this issue, I'll look into this today

[ndeloof]

This is an unexpected side-effect upgrading to go-yaml/v3

[ndeloof]

goyaml/v3 does not support repeated anchors (due to internal storage using << as key during parsing) but allows use of multi-values in anchors

    environment:
      <<:  [*mysqlenv, *mysqlextraenv]

According to https://yaml.org/type/merge.html this is the expected syntax

[arulrajnet]

Thanks, @ndeloof. This fix works. Will close this.

[gowrizrh]

One thing to note that it's not quite the same behaviour.

Value overrides don't work as before. If you previously had

x-mysql: &mysqlenv
  MYSQL_USER: test

x-mysql-extraenv: &mysqlextraenv
  MYSQL_USER: overridetest

    environment:
      <<: *mysqlenv
      <<: *mysqlextraenv

the value of the variable MYSQL_USER would have been overriden to the overridetest.

However with the new syntax, the value for MYSQL_USER is test.

    environment:
      <<: [*mysqlenv, *mysqlextraenv]

[gowrizrh]

Sorry for the noise, but it seems to merge in reverse?

Rewriting it as

    environment:
      <<: [*mysqlextraenv, *mysqlenv]

Does override values of mysqlenv with values from mysqlextraenv

[gowrizrh]

Ok, after swifting through https://yaml.org/type/merge.html it does mention this.

If the value associated with the merge key is a sequence, then this sequence is expected to contain mapping nodes and each of these nodes is merged in turn according to its order in the sequence. Keys in mapping nodes earlier in the sequence override keys specified in later mapping nodes.

I hope it's helpful to someone else.