'extra_hosts' not working anymore in 'docker stack deploy'

[membersound]

Description

So far, it was possible to pass docker compose config output to docker stack deploy without problems.

Today I upgraded to docker 25.0.2, and discovered loss of the extra_hosts mapping. The tag is completely ignored with the command, and nothing is written to /etc/hosts anymore!

This might be related to:
compose-spec/compose-go#499
compose-spec/compose-spec#156
docker/buildx#2201

Reproduce

cat docker-compose.yml

version: '3.8'

services:
  app:
    image: httpd:2.4
    extra_hosts:
      - "host.docker.internal:host-gateway"

Run:

docker compose config | (sed '/^name:/d') | docker stack deploy -c - httpd

Expected behavior

/etc/hosts should show the entry from extra_hosts.

docker version

Client: Docker Engine - Community
 Version:           25.0.2
 API version:       1.44
 Go version:        go1.21.6
 Git commit:        29cf629
 Built:             Thu Feb  1 00:23:19 2024
 OS/Arch:           linux/amd64
 Context:           default

Server: Docker Engine - Community
 Engine:
  Version:          25.0.2
  API version:      1.44 (minimum version 1.24)
  Go version:       go1.21.6
  Git commit:       fce6e0c
  Built:            Thu Feb  1 00:23:19 2024
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.6.28
  GitCommit:        ae07eda36dd25f8a1b98dfbf587313b99c0190bb
 runc:
  Version:          1.1.12
  GitCommit:        v1.1.12-0-g51d5e94
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

docker info

Client: Docker Engine - Community
 Version:    25.0.2
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.12.1
    Path:     /usr/libexec/docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v2.24.5
    Path:     /usr/libexec/docker/cli-plugins/docker-compose

Server:
 Containers: 1
  Running: 1
  Paused: 0
  Stopped: 0
 Images: 10
 Server Version: 25.0.2
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: systemd
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
 Swarm: active
  NodeID: j9wtq70drkn7uz09ve5cx7iru
  Is Manager: true
  ClusterID: 3kmynxgzfqluz4h5wv46zpu4j
  Managers: 1
  Nodes: 1
  Default Address Pool: 10.0.0.0/8  
  SubnetSize: 24
  Data Path Port: 4789
  Orchestration:
   Task History Retention Limit: 5
  Raft:
   Snapshot Interval: 10000
   Number of Old Snapshots to Retain: 0
   Heartbeat Tick: 1
   Election Tick: 10
  Dispatcher:
   Heartbeat Period: 5 seconds
  CA Configuration:
   Expiry Duration: 3 months
   Force Rotate: 0
  Autolock Managers: false
  Root Rotation In Progress: false
  Node Address: 127.0.0.1
  Manager Addresses:
   127.0.0.1:2377
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: ae07eda36dd25f8a1b98dfbf587313b99c0190bb
 runc version: v1.1.12-0-g51d5e94
 init version: de40ad0
 Security Options:
  apparmor
  seccomp
   Profile: builtin
  cgroupns
 Kernel Version: 6.2.0-39-generic
 Operating System: Ubuntu 23.04
 OSType: linux
 Architecture: x86_64
 CPUs: 16
 Total Memory: 62GiB
 Name:
 ID: 
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Username: 
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

Additional Info

No response

[thaJeztah]

It looks like docker compose config uses the = as default in the output (but also hard-codes the name for the default network - wondering if that was intentional :thinking_face:);

docker compose config
name: extra_hosts
services:
  app:
    extra_hosts:
      - host.docker.internal=host-gateway
    image: nginx:alpine
    networks:
      default: null
networks:
  default:
    name: extra_hosts_default

We should look indeed why the option from the compose-file doesn't work with =. In the meantime, does it work with docker stack config instead of docker compose config ?

docker stack config -c docker-compose.yaml
version: "3.8"
services:
  app:
    extra_hosts:
    - host.docker.internal:host-gateway
    image: nginx:alpine

docker stack config -c docker-compose.yaml || docker stack deploy -c - mystack

docker service inspect --format '{{json .Spec.TaskTemplate.ContainerSpec.Hosts }}' mystack_app
["host-gateway host.docker.internal"]

docker exec mystack_app.1.sz17bandwnt1mqefp8dk0ngis sh -c 'cat /etc/hosts'
127.0.0.1	localhost
::1	localhost ip6-localhost ip6-loopback
fe00::0	ip6-localnet
ff00::0	ip6-mcastprefix
ff02::1	ip6-allnodes
ff02::2	ip6-allrouters
10.0.2.3	8c560fd34766

[thaJeztah]

@robmry PTAL

[membersound]

In the meantime, does it work with docker stack config instead of docker compose config ?

The extra_hosts entry is working correctly with docker stack config (it keeps the : instead of replacing it with = which compose config does).
But is docker stack config relying on the old docker-compose (v1)? Or is it executing also a docker compose (v2) command under the hood?

Sidenote: in real world I'm using the command to merge multiple docker files, like:
docker compose config -f docker-compose.base.yml -f docker-compose.yml | docker stack deploy -c - appname

[thaJeztah]

The docker stack config parses the compose-files using the v3 schema (which is the schema that was created for swarm services / stacks, and currently used for docker stack deploy)

[robmry]

This change to the compose file format compose-spec/compose-go#499 means we need a matching change for docker stack deploy ...