env_file behavior break in Latest 4.10

[qrli]

• I have tried with the latest version of Docker Desktop
• I have tried disabling enabled experimental features
• I have uploaded Diagnostics
• Diagnostics ID:

Actual behavior

After upgraded to latest 4.10, the environment variable values brought in by env_file: in compose file are no longer honored. Instead, the environment variable with the same name on Windows system overrides the specified value in env file referenced by env_file:. This breaks compose file use cases as long as there is some Windows environment variable happen to have the same name.

Meanwhile, values specified by environment: work as expected, not affected by Windows environment variables.

Expected behavior

The values in env file referenced by env_file: should take effect. Environment variables from Windows system should not matter unless I reference them explicitly in compose file.

Information

• Windows Version: Windows 10 20H2
• Docker Desktop Version: 4.10.1
• WSL2 or Hyper-V backend? WSL2
• Are you running inside a virtualized Windows e.g. on a cloud server or a VM: Yes

Output of & "C:\Program Files\Docker\Docker\resources\com.docker.diagnose.exe" check

& "C:\Program Files\Docker\Docker\resources\com.docker.diagnose.exe" check
[2022-07-08T04:09:47.831668300Z][com.docker.diagnose.exe][I] set path configuration to OnHost
Starting diagnostics

[PASS] DD0027: is there available disk space on the host?
[PASS] DD0028: is there available VM disk space?
[PASS] DD0031: does the Docker API work?
[PASS] DD0004: is the Docker engine running?
[PASS] DD0011: are the LinuxKit services running?
[PASS] DD0016: is the LinuxKit VM running?
[PASS] DD0001: is the application running?
[SKIP] DD0018: does the host support virtualization?
[PASS] DD0002: does the bootloader have virtualization enabled?
[PASS] DD0017: can a VM be started?
[PASS] DD0024: is WSL installed?
[PASS] DD0021: is the WSL 2 Windows Feature enabled?
[PASS] DD0022: is the Virtual Machine Platform Windows Feature enabled?
[PASS] DD0025: are WSL distros installed?
[PASS] DD0026: is the WSL LxssManager service running?
[PASS] DD0029: is the WSL 2 Linux filesystem corrupt?
[PASS] DD0035: is the VM time synchronized?
[PASS] DD0015: are the binary symlinks installed?
[PASS] DD0003: is the Docker CLI working?
[PASS] DD0013: is the $PATH ok?
[FAIL] DD0005: is the user in the docker-users group? Access is denied.
[PASS] DD0007: is the backend responding?
[FAIL] DD0014: are the backend processes running? 1 error occurred:
* vpnkit.exe is not running

[PASS] DD0008: is the native API responding?
[PASS] DD0009: is the vpnkit API responding?
[PASS] DD0010: is the Docker API proxy responding?
[PASS] DD0006: is the Docker Desktop Service responding?
[PASS] DD0012: is the VM networking working?
[PASS] DD0032: do Docker networks overlap with host IPs?
[SKIP] DD0030: is the image access management authorized?
[PASS] DD0033: does the host have Internet access?

Please investigate the following 2 issues:

1 : The test: are the backend processes running?
Failed with: 1 error occurred:
* vpnkit.exe is not running

Not all of the backend processes are running.

2 : The test: is the user in the docker-users group?
Failed with: Access is denied.

The current user must be member of the docker-users group. Press the Win + R keys to open Run, type lusrmgr.msc into Run, followed by Enter to open Local Users and Groups.

Steps to reproduce the behavior

file appshared.env

ASPNETCORE_ENVIRONMENT=Production

file compose.yml

version: '3.8'

services:
ubuntu:
image: ubuntu:22.04
env_file: appshared.env
command: echo $ASPNETCORE_ENVIRONMENT

test

PS> $env:ASPNETCORE_ENVIRONMENT="Development"
PS> docker compose up
It should output Production but instead output Development

[thaJeztah]

Looks like the issue described in #9521

[qrli]

@thaJeztah Thanks. That issue does seem to be closely related.
But I'm not sure they are exact dup, as that one is about unexpected override by .env but mine is about unexpected override by Windows environment variables.

[thaJeztah]

IIUC, that one seems to be related due to a combination of #8108 and #9512, where the latter (#9512) loaded .env environment variables to be added to the current environment, and the older ticket (#8108) reported an issue that env-vars from the current environment would overwrite env-vars specified in a service's env_file

[digaoddc]

It looks like the PR #9512 added the bug. The issues #9608 and #9521 are describing this issue.

[nicks]

I'm going to transfer this issue to the compose repo. I'll leave it up to the compose team to figure out whether it's better to track as separate issues, or to centralize the discussion on one issue.