Escape sequences cannot be used inside quoted strings

[milas]

Description of the issue

Docker Compose v2 (compose-go) is stripping out escape characters when inside of a quoted string in a command (might happen in other circumstances).

If it's not in a quoted string, they are left as-is. For example, command: printf 'Hello\tworld\n' works while command: sh -c "printf 'Hello\tworld\n'" does not.

Context information (for bug reports)

• Using Compose V2 docker compose ...
• Using Compose V1 docker-compose ...

Output of docker(-)compose version

Docker Compose version v2.0.0-rc.2

Output of docker version

Client:
 Cloud integration: 1.0.17
 Version:           20.10.8
 API version:       1.41
 Go version:        go1.16.6
 Git commit:        3967b7d
 Built:             Fri Jul 30 19:55:20 2021
 OS/Arch:           darwin/amd64
 Context:           default
 Experimental:      true

Server: Docker Engine - Community
 Engine:
  Version:          20.10.8
  API version:      1.41 (minimum version 1.12)
  Go version:       go1.16.6
  Git commit:       75249d8
  Built:            Fri Jul 30 19:52:10 2021
  OS/Arch:          linux/amd64
  Experimental:     true
 containerd:
  Version:          1.4.9
  GitCommit:        e25210fe30a0a703442421b0f60afac609f950a3
 runc:
  Version:          1.0.1
  GitCommit:        v1.0.1-0-g4144b63
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

Output of docker-compose config
w/ v2 (v2.0.0-rc.2):

services:
  hello-world:
    command:
    - sh
    - -c
    - printf 'Hellotworldn'
    image: busybox
    networks:
      default: null
networks:
  default:
    name: escape-repro_default

w/ v1 (1.29.2, build 5becea4):

services:
  hello-world:
    command: sh -c "printf 'Hello\tworld\n'"
    image: busybox
version: '3.9'

Steps to reproduce the issue

1. Use an escape sequence (e.g. \t or \n) inside of a quoted string in command

Observed result

• Docker Compose v2 (compose-go) strips out the \ characters

Expected result

• Docker Compose leaves the \ characters as-is

Stacktrace / full error message

n/a

Additional information

n/a

[ndeloof]

compose v2 config do render the fully parsed compose model, as it is used to invoke on engine API and create container. Doing so, command is parsed into a slice of command and arguments.

but getting \t converted into unescaped character t is definitively a bug.

[milas]

Yeah - the conversion from string -> list isn't surprising, just the stripping out of \ characters.

Output of running under both versions:

v1:

$ docker-compose -p escape-repro -f escape-repro.yml up
Creating network "escape-repro_default" with the default driver
Creating escape-repro_hello-world_1 ... done
Attaching to escape-repro_hello-world_1
hello-world_1  | Hello	world
escape-repro_hello-world_1 exited with code 0

v2:

$ docker-compose -p escape-repro -f escape-repro.yml up
[+] Running 2/2
 ⠿ Network escape-repro_default          Created                                                                                                                                          0.1s
 ⠿ Container escape-repro_hello-world_1  Created                                                                                                                                          0.1s
Attaching to hello-world_1
hello-world_1 exited with code 0

(FWIW There's also a subtle difference here in v2 which is that if the last log line doesn't end in \n it is never printed - in v1 it would get printed although it kinda messes with the output since then the "exited with code" line from compose is on the same line as it but 🤷)

[ulyssessouza]

It actually works with double backslash.
Ex. \\t and \\n

Looks like the backslash is escaped too.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[milas]

I can confirm this is still the case as of v2.3.3

[stale]

This issue has been automatically marked as not stale anymore due to the recent activity.

[paoliniluis]

seems that all my bash scripts that were passed in the command to bash containers broke because of this

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[stale]

This issue has been automatically closed because it had not recent activity during the stale period.

[stale]

This issue has been automatically marked as not stale anymore due to the recent activity.

[ndeloof]

this bug is caused by shellwords.Parse converting the configured string into a ShellCommand

[milas]

Oh, nice find! Ironically, I just added a use of that function today (but in e2e code so not likely to be affected)

It'd be nice to publish a Go utility lib (or augment one of our existing ones) that handled a lot of the POSIX-isms our tools rely one on of these days

[mmunz]

I recently also stumbled upon that issue and it is currently present in many test setups for TYPO3 and it's extensions.

E.g. here: https://github.com/TYPO3/styleguide/blob/3baee32b70d60fe13741435597053c240ccc3463/Build/testing-docker/docker-compose.yml#L294

The lines

  XDEBUG_MODE=\"off\" \
  bin/phpunit -c Web/typo3conf/ext/styleguide/Build/FunctionalTests.xml ${EXTRA_TEST_OPTIONS} ${TEST_FILE};

are executed not as one, but two lines and therefore the XDEBUG_MODE variable is not used by the phpunit command afterwards.

I'm using Debian 12 and docker-compose v2.17.0, another user i chatted with who also had problems with that used Ubuntu 23.04 and docker-compose v2.17.2.

Is this something that can and should be fixed in docker-compose or should we rewrite our docker-compose.yml files?

[ndeloof]

@mmunz command should better not be used with such a long shell script. This will make maintenance painful and subject to bugs like this one. Better move this into a script file you mount as a volume (on copy in a Dockerfile) and pass environment to the container so value get resolved during execution. This also make it easier to run this exact same script standalone outside a Docker container for testing purpose

[mmunz]

@ndeloof while i tend to agree, it is imho still a problem that docker-compose v1 and v2 parse these scripts differently.

[ndeloof]

this is for sure an issue, but we don't have any simple solution here.
docker-compose v1 is coded in python, and relies on python's shlex to parse the command attribute. There's no direct equivalent in Go.

The root issue here is that command has been introduce with the assumption we can strictly reproduce shell script parsing in compose codebase. That makes the tool more convenient to configure for users, but some corner cases exists.