Parameter expansion when using docker run --env-file ....

[MarvinGravert]

Description

Hi,
when running docker run --env-file env.list --rm ubuntu, parameter expansion is not applied to elements in env.list. The entry VAR=$USER is not expanded thus the env VAR will be assigned the literal string "$USER" in the container. This is inconsistant with the behavior of environment files when used with docker compose.

I hope this the correct repo and label. Sorry for any inconvenience.

Reproduce

1. echo VAR=\$USER > env.list
2. docker run --env-file env.list --rm -it ubuntu env | grep VAR
3. => VAR=$USER

Expected behavior

The expected outcome should be the expanded value. In my case VAR=marvin. This would be in line with the behavior of docker compose.

services:
  test:
    image: "ubuntu"
    env_file:
      - ./env.list
    command: "bash -c 'env | grep VAR'"

docker compose -f docker-compose.yml up results in VAR=marvin

docker version

Client:
 Version:           24.0.2
 API version:       1.43
 Go version:        go1.20.4
 Git commit:        cb74dfc
 Built:             Thu May 25 21:51:00 2023
 OS/Arch:           linux/amd64
 Context:           default

docker info

Client:
 Version:    24.0.2
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.10.5
    Path:     /usr/libexec/docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v2.18.1
    Path:     /usr/libexec/docker/cli-plugins/docker-compose

Additional Info

No response

[thaJeztah]

Hm, ISTR this was by design, and --env-file would not do variable expansion, and only straight NAME=value (literal) with the exception of NAME (without =) which would be propagated with the corresponding env-var from the CLI's environment (if present);

• support for docker run environment variables file (--env-file) moby/moby#4174

If compose is expanding environment variables for these, that looks like a bug (or at least incompatible behavior)

/cc @glours @tianon (if you recall more context on this)

[thaJeztah]

Oh! I guess compose's --env-file is to specify the custom path for the .env file (and not the equivalent of docker run --env-file); those are separate things (and not equivalents), but (unfortunately) named confusingly.

edit: Hm.. but env_file is the equivalent, so if that expands env-vars, that sounds like a potential bug

[sply88]

Came accross something related. Seems like the parameter expansion capabilities of compose were extended relatively recently.
Building on the orignal example consider:

docker compose version

echo VAR=\${USER_VAR:-default} > env.list

cat <<EOF > docker-compose.yml
services:
  test:
    image: "ubuntu"
    env_file:
      - ./env.list
    command: "bash -c 'env | grep VAR'"
EOF

docker compose up
# v2.6.0: VAR=:-default}
# v.2.10.2: VAR=default

USER_VAR=42 docker compose up
# v2.6.0: VAR=42:-default}
# v.2.10.2: VAR=42

Did install the different compose versions via

sudo apt-get install docker-compose-plugin=2.6.0~ubuntu-focal

and

sudo apt-get install docker-compose-plugin=2.10.2~ubuntu-focal

,respectively.

With docker run --env-file env.list --rm -it ubuntu env | grep VAR there is no parameter expansion in any case.

Looks like the default value handling for parameter expansion was added with this PR which ended up in v2.7.0

So docker run --env-file and compose file env_file are inconsistent, but compose behavior seems to be intended.

[pryhodkin]

I think I just duplicated these issue here

So I'm begging you to put here as much attension as you can, because it confuses very much and I think these feature can be extremely relevant and useful!

[pryhodkin]

Hey @thaJeztah,

What do you think, is there a possibility to implement such functionallity?
Maybe you have some feedbask from developers team?