"docker-compose" and "docker compose"

[gramian]

BurmillaOS Version: v2.0.0-rc1

Where are you running BurmillaOS? Virtual Box

Which processor architecture you are using? Apple M2

Do you use some extra hardware? No

Which console you use? default

Do you use some service(s) which are not enabled by default? docker-compose

Have you installed some extra tools to console? No

Do you use some other customizations? No

Please share copy of your cloud-init. None

I enabled/downloaded "docker-compose" by running docker-compose. However, the meanwhile more common docker compose does not work, as it results in: docker: 'compose' is not a docker command.

Could an alias be added for docker compose (not an actual alias due to the whitespace)?

[olljanat]

docker compose is newer, not more common to be exact.

We use static binaries published by Docker to https://download.docker.com/linux/static/stable/x86_64/
No idea why they don't include cli plugins to those packages. Also don't know if it is same binary or different as very rarely using those.

Content of that package actually is very simple script nowadays so it can be easily improved if you figure out solution to this https://github.com/burmilla/os-services/blob/master/images/20-dockercompose/download.sh

[gramian]

So maybe something like the following would work (based on this) as a function inside the shell script:

docker() {
    if [[ $1 == "compose" ]]; then
        command docker compose "$@"
    else
        command docker "$@"
    fi
}
export -f docker

WDYT? Maybe a little overkill wrapping all docker calls with this function?

[olljanat]

Check documentation about Docker Plugins. There should be way without custom wrapper.

[gramian]

Sorry, could provide a link? Thank you

[olljanat]

I think that it is called for "Docker CLI plugins". Haven't studied how they works exactly but check example docker/cli#1534 or https://github.com/docker/buildx

Alternatively you can check from https://github.com/docker/docker-ce-packaging how they create Docker CE packages for different platforms.

[gramian]

Got it, I thought you meant in the BurmillaOS docs.

[olljanat]

Btw, I understood that you are running BurmillaOS as virtual machine on MacOS? On that case you probably get better user experience by having Docker CLI directly on OSX. Look example https://blog.programster.org/use-remote-docker-host-with-ssh

[gramian]

MacOS is just for experimenting, I sure have a Docker directly running, too. Ultimately I want to test BurmillaOS in an OpenStack.

[gramian]

OK, thanks for the tip with the CLI plugins. Here is a working solution based on this:

mkdir -p ~/.docker/cli-plugins
cp /usr/bin/docker-compose ~/.docker/cli-plugins
chmod +x ~/.docker/cli-plugins/docker-compose

Then

docker compose

seems to work (just a symlink does not work). Very cool!

[netsandbox]

docker-compose is version 1, a python script
docker compose is version 2, a Docker CLI Plugin written in Go

@gramian copying /usr/bin/docker-compose to ~/.docker/cli-plugins is not the right way, because you copy the version 1 script to the location where the version 2 cli plugin should be.

@olljanat it would be great if the docker compose version 2 cli plugin would be included in BurmillaOS 2.
See https://docs.docker.com/compose/install/linux/#install-the-plugin-manually

The correct directory for the plugin is /usr/local/lib/docker/cli-plugins, so if someone create another user beside the rancher user, the plugin is also available for this one.

[olljanat]

On 2.x versions of BurmillaOS, docker-compose command download latest 2.x version of it on first run.

You can see logic in

• os/cmd/control/console_init.go

  Lines 186 to 192 in 8a9e14f

  	// create placeholder for docker-compose binary
  	const ComposePlaceholder = `
  	#!/bin/bash
  	echo 'INFO: System service "docker-compose" is not yet enabled'
  	sudo ros service enable docker-compose
  	sudo ros service up docker-compose
  	`

• https://github.com/burmilla/os-services/blob/master/images/20-dockercompose/download.sh

Pull request to add support for docker compose are welcome but for backward compatibility both commands should be supported. Because cli plugin does not works symlink it probably would easier to have binary on that name and add symlimk for old name.

[netsandbox]

But then this is still wrong, if you run docker-compose, you would expect to run version 1.

[netsandbox]

Also this doesn't seem to be working in rc2:

rancher@burmillaos-dev02:~$ sudo ros service list
disabled amazon-ecs-agent
disabled container-cron
disabled zfs
disabled kernel-extras
disabled kernel-headers
disabled kernel-headers-system-docker
enabled  open-vm-tools
disabled hyperv-vm-tools
disabled qemu-guest-agent
disabled amazon-metadata
disabled volume-cifs
disabled volume-efs
disabled volume-nfs
disabled modem-manager
disabled waagent
enabled  docker-compose

rancher@burmillaos-dev02:~$ docker-compose ps
INFO: System service "docker-compose" is not yet enabled
INFO[0000] Project [os]: Starting project               
INFO[0000] [0/20] [docker-compose]: Starting            
INFO[0000] Rebuilding docker-compose                    
INFO[0000] [1/20] [docker-compose]: Started             
INFO[0000] Project [os]: Project started                

rancher@burmillaos-dev02:~$ docker-compose ps
INFO: System service "docker-compose" is not yet enabled
INFO[0000] Project [os]: Starting project               
INFO[0000] [0/20] [docker-compose]: Starting            
INFO[0001] Rebuilding docker-compose                    
INFO[0001] [1/20] [docker-compose]: Started 

[olljanat]

But then this is still wrong, if you run docker-compose, you would expect to run version 1.

Nope. v1.9.x versions are using 1.x versions of docker-compose but as those are not supported anymore BurmillaOS 2.x must use 2.x versions of docker-compose.

To make sure backward compatibility is responsibility of BurmillaOS to make sure that docker-compose command still works on new versions and it is responsibility of https://github.com/docker/compose project to make sure that compose files created for old versions still works.

Also this doesn't seem to be working in rc2

That is most likely result of burmilla/os-services@5114838 where I started to implement cli plugin support but didn't had time to fully test/finalize it yet.

[olljanat]

It is now in way in v2.0.0 that running docker-compose adds support for docker compose command.

Not perfect but looks to be working and be improved by contributing to os-services repo without needs to release new version.

[netsandbox]

@olljanat you sad:

To make sure backward compatibility is responsibility of BurmillaOS to make sure that docker-compose command still works on new versions ...

which is not the case for new v2 installs (tested on a fresh install of v2.0.1).

The docker-compose command is no longer working (same result on repeated runs of the command):

$ sudo ros os version
v2.0.1

$ docker-compose version
INFO: System service "docker-compose" is not yet enabled
INFO[0000] Project [os]: Starting project               
INFO[0000] [0/20] [docker-compose]: Starting            
INFO[0000] Rebuilding docker-compose                    
INFO[0000] [1/20] [docker-compose]: Started             
INFO[0000] Project [os]: Project started

This is actually a difference to upgraded installations (upgraded v1.9.6 to v2.0.1) where the docker-compose command still works:

$ sudo ros os version
v2.0.1

$ docker-compose version
docker-compose version 1.29.2, build 5becea4c
docker-py version: 5.0.0
CPython version: 3.7.10
OpenSSL version: OpenSSL 1.1.0l  10 Sep 2019

[olljanat]

which is not the case for new v2 installs (tested on a fresh install of v2.0.1).

That is true and that why this issue is open. Someone need to take action and update logic in https://github.com/burmilla/os-services/tree/v2.0.1/images/20-dockercompose so it gets fixed to all the existing and new installations.

But as this low priority bug, I didn't wanted to delay v2.0.0 release more because of it.

[netsandbox]

As fresh installation and upgraded installation behave differently, I would not call this a low priority bug.

Also at least the release notes should mention this problem.

[olljanat]

As fresh installation and upgraded installation behave differently, I would not call this a low priority bug.

You are of course allowed to disagree but I call it low priority bug because it does not prevent anyone from using BurmillaOS, docker-compose is not even part of core functionalities as it is extra service and because it is very easy to download docker-compose from internet. Annoying bug for sure but still low priority.

Also at least the release notes should mention this problem.

Version number changed from v1.9.x to v2.0.x to make it clear for everyone that there have been massive changes under the hood and that there might be bugs. Bugs is not something what you list in release notes. It is role of this issue tracker.

PS. I would like to have this issue fixed longtime ago but unless more people who are ready contributing to code appears here I'm unfortunately forced to prioritize these things.

[benok]

I didn't follow this thread's discussion (sorry), but I use the following script to install the latest docker-compose & docker-buildx.

- content: |+
    #!/bin/sh
    # https://gist.github.com/lukechilds/a83e1d7127b78fef38c2914c4ececc3c
    get_latest_release() {
       curl --silent "https://api.github.com/repos/$1/releases/latest" | # Get latest release from GitHub api
       grep '"tag_name":' |                                            # Get tag line
       sed -E 's/.*"([^"]+)".*/\1/'                                    # Pluck JSON value
    }

    # install docker compose plugin (https://kazuhira-r.hatenablog.com/entry/2022/04/30/002702)
    sudo mkdir -p /usr/local/lib/docker/cli-plugins
    ver=$(get_latest_release docker/compose)
    url=https://github.com/docker/compose/releases/download/$ver/docker-compose-linux-x86_64
    echo "Downloading $url..."
    sudo curl -SL $url -o /usr/local/lib/docker/cli-plugins/docker-compose
    sudo chmod a+x /usr/local/lib/docker/cli-plugins/docker-compose
    # support both "docker compose" and "docker-compose" for backward comatibility
    sudo ln -sf /usr/local/lib/docker/cli-plugins/docker-compose /usr/local/bin/docker-compose

    # install docker buildx
    ver=$(get_latest_release docker/buildx)
    url=https://github.com/docker/buildx/releases/download/$ver/buildx-$ver.linux-amd64
    echo "Downloading $url..."
    sudo curl -SL $url -o /usr/local/lib/docker/cli-plugins/docker-buildx
    sudo chmod a+x /usr/local/lib/docker/cli-plugins/docker-buildx
  owner: root
  path: /etc/rc.install-docker-plugins
  permissions: "0755"

I feel docker-compose as a service container (from RancherOS) seems too tricky and harder to update, so I've not used that since a few years before.

p.s.
I also uploaded the whole of my cloud-init.yml to my gist page.

[olljanat]

I feel docker-compose as a service container (from RancherOS) seems too tricky and harder to update

It was added because of request in #18 but I have been also thinking that it probably was bad idea. Docker compose have been updating too often.

Personally I don't ever use docker-compose or even docker component directly from console but instead of have separate deployment container running which is based on CLI version of this and which have CI agent installed on it.

[olljanat]

I finally had time to look this. Added now small wrapper which redirects docker compose and docker-compose commands to container which is running docker:cli and it is always using same version selected user docker engine is.

Can be tested on v2.0.2-rc2