[RFE] Include Docker packages and dependencies in portage-stable

[t-lo]

Current situation

Docker is packaged and shipped via Torcx. Docker ebuilds and dependencies reside in custom directories app-emulation in coreos-overlay instead of the upstream app-containers.

Impact

Staying up to date causes maintenance overhead as we are not re-using Gentoo's ebuilds. Switching from Torcx to sysext is soft-blocked until the docker packages are available via portage-stable (because we don't want to re-use the custom ebuilds structure Torcx uses today).

Ideal future situation

Packages are available in portage-stable and emerge-[amd64|arm64]-usr docker emerges the latest Gentoo docker version. All integration tests pass.

Additional information

@krishjainx wanted to look into this.

[tormath1]

Note: This is a good opportunity to pull at the same time selinux-docker to have /usr/bin/docker and friends correctly labelled.

[pothos]

The issue now would mean to drop the downstream patches because in coreos-overlay we also moved to app-containers, and they get taken instead of portage-stable packages.
We have these downstream changes and I think we can try to make the package usable (maybe through flags that the sysext emerge script could set when building) and reduce them where possible:
flatcar/scripts@9d6af12
First I would remove the kernel check and hardcode the 6.1 case.

Can we rename this issue to not have two different docker packages but instead one that works for both cases?

[krishjainx]

@pothos @t-lo In order to maintain the downstream changes, which are there for a reason, and avoid having multiple packages, I keep it as it is and remove the kernel dependency. I also hardcode the kernel 6.1 case. Currently, I am testing the installation of Docker in the SDK container. I will make a pull request for this once the testing is complete.

[t-lo]

@pothos do you think it would be worthwhile to review the downstream changes we apply? We could save us a bit of maintenance work if we used the upstream ebuilds directly. We could also import more features from upstream w/o increasing our maintenance load, e.g. docker-compose.

Also, we'd be able to introduce new versions to upstream (e.g. docker 24, runc 1.1.7) since we test docker quite thoroughly in our test suite.

[pothos]

I hope that we can get rid of the downstream changes. One change is for controlling the default use flags but maybe we could do this in the emerge profile. One other difference I see could be the Go cross compilation that we do with coreos-go-utils.eclass but maybe everything works without this now? We can switch to the upstream systemd units and if we find differences, use drop-ins for them - we also have networkd units and I'm not sure if they are upstream but since I reworked the exclusion logic in Flatcar I think we could already drop them.

[krishjainx]

With this patch (adding acct-group/docker to portage-stable) I am able to emerge app-containers/docker

diff --git a/sdk_container/src/third_party/portage-stable/acct-group/docker/docker-0-r1.ebuild b/sdk_container/src/third_party/portage-stable/acct-group/docker/docker-0-r1.ebuild
new file mode 100644
index 0000000000..a448b78e40
--- /dev/null
+++ b/sdk_container/src/third_party/portage-stable/acct-group/docker/docker-0-r1.ebuild
@@ -0,0 +1,8 @@
+# Copyright 2020 Gentoo Authors
+# Distributed under the terms of the GNU General Public License v2
+
+EAPI=7
+
+inherit acct-group
+
+ACCT_GROUP_ID=48
diff --git a/sdk_container/src/third_party/portage-stable/acct-group/docker/metadata.xml b/sdk_container/src/third_party/portage-stable/acct-group/docker/metadata.xml
new file mode 100644
index 0000000000..db463f3eeb
--- /dev/null
+++ b/sdk_container/src/third_party/portage-stable/acct-group/docker/metadata.xml
@@ -0,0 +1,8 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!DOCTYPE pkgmetadata SYSTEM "https://www.gentoo.org/dtd/metadata.dtd">
+<pkgmetadata>
+       <maintainer type="person">
+               <email>[email protected]</email>
+               <name>William Hubbs</name>
+       </maintainer>
+</pkgmetadata>

Should I send a pull request with this? @pothos @t-lo @tormath1

[krishjainx]

My PR flatcar/scripts#951 which has now been merged includes the docker dependency needed to be able to emerge it. flatcar/scripts#958 includes some needed changes for the recently added acct-group/docker package.

Once the build_sysext script PR is merged could we build and ship the Docker sysext already? @t-lo

[t-lo]

Once the build_sysext script PR is merged could we build and ship the Docker sysext already? @t-lo

Yes, that's the next step. Then rip out torcx for good (in the build scripts as well as deprecating the actual torcx ebuild), and we're done.