docker-compose overrides values from .env with host value if present in host's environment variables

[Tim-Roy]

Description

docker-compose env_file overwrites values from .env with host value if present in host's environment variables. Otherwise correctly applies the value from .env file

Steps to reproduce the issue:

1. Create a variable that has a set value on the host system
2. Create a .env file with the same key but different value
3. include in a service in the docker-compose.yml the following
   env_file:
   • .env
4. Print container env variables after running docker-compose up --build -d

Describe the results you received:
The value for the env var is the same as the host

Describe the results you expected:
The value for the env var from the .env file

Additional information you deem important (e.g. issue happens only occasionally):
I downgraded docker-compose to v.2.5.1 and this issue is resolved

Output of docker compose version:

v2.6.0

Output of docker info:

Client:
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc., v0.8.2-docker)

Server:
 Containers: 0
  Running: 0
  Paused: 0
  Stopped: 0
 Images: 0
 Server Version: 20.10.16
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Native Overlay Diff: false
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: systemd
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 io.containerd.runtime.v1.linux runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 96df0994faabc1944fc614e52b0b3c6feb609a57.m
 runc version: 
 init version: de40ad0
 Security Options:
  apparmor
  seccomp
   Profile: default
  cgroupns
 Kernel Version: 5.17.15-1-MANJARO
 Operating System: Manjaro Linux
 OSType: linux
 Architecture: x86_64
 CPUs: 24
 Total Memory: 31.29GiB
 ID: RU7W:L7AD:6ZWZ:GP6M:YANE:BFLX:ZYHD:TVET:TW4V:7CKW:DODE:6FZL
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

Additional environment details:

[ZaxR]

I am facing this same issue - any word on a fix?

[grainnemcknight]

Related to #9593 and #9608?

[Jason-Crowe]

I don't think this is fixed by #9636
here is a one line test that shows the environment is still allowed to override the docker compose file specification:

$ printf 'name: test\nservices:\n test:\n  image: debian:buster-slim\n  command: sleep inf\n  init: true\n  env_file: [env-file]\n'>test-docker-compose.yml && echo TEST1=1>env-file && TEST1=2 TEST2=2 docker compose -f test-docker-compose.yml up -d &>/dev/null && docker inspect test-test-1 | jq -r .[0].Config.Env[] | grep ^TEST && docker compose -f test-docker-compose.yml down &>/dev/null && rm -f test-docker-compose.yml env-file 
TEST1=2

the expected output would be TEST1=1 as defined in the env-file. the issue again is that environment values are leaking into the docker environment and overriding the behavior specified by the compose file. this can lead to different behavior on different systems for the same invocation -- that would seem to violate the tenet of docker

here is an example running docker directly that demonstrates the correct behavior (ie. environment variables shouldn't affect a docker invocation):

$ echo TEST1=1>env-file && TEST1=2 TEST2=2 docker run --rm --env-file env-file debian:buster-slim bash -c set|grep ^TEST && rm -f env-file 
TEST1=1

the above tests were made with the latest docker desktop on mac os:

$ docker version -f json | jq -r .Server.Platform.Name
Docker Desktop 4.11.0 (83626)

$ docker compose version
Docker Compose version v2.7.0

[ulyssessouza]

Could you please test that with Docker Compose v2.9.0?
Your one-liner just works for me.

[xstast24]

Hi, I have the same problem. I want to use .env variables, but they get overridden by shell environment variables (docker compose v 2.10).
However, I am shocker that this is not a bug, but intended behavior according to the documentation:
"Values in the shell take precedence over those specified in the .env file."

Does anybody know how to force the .env variables over the host variables?

[ulyssessouza]

Hello @xstast24. The precedence mechanism is documented in https://docs.docker.com/compose/envvars-precedence/
If it's not working as documented, please open a new issue with a testcase so we can work this out.