Consider dropping moby-engine from the base image

[jshuffle]

Describe the enhancement

This isn't a complaint and I hope it doesn't get taken that way ☺️. I know how thankless package maintainership is, and that the moby-engine packages aren't trivial. Two relevant threads on Fedora Discussion here and here.

The problem

I think the moby-engine packages have these two problems:

1. There's a lack of activity from current maintainers. For example, there were CVEs fixed in upstream 24.0.7 (6 months old), but the CoreOS package is still on 24.0.5 (9 months old).
2. Users don't have control over version, which is made more noticeable by the lack of maintenance. Major version updates (like 20.10.x to 24.0.x in F39) happen intermittently and unpredictably; on regular Fedora you have a grace period to stay on current Fedora until EOL, but you can't on CoreOS.

If a user needs upstream features, bug fixes or security fixes (eg, for compliance), the only option is:

rpm-ostree override remove containerd moby-engine runc
# Then install packages from upstream docker-ce.repo.

But removing base packages is a bit hacky, and isn't recommended or supported.

My proposed solution

I propose: don't ship moby-engine, containerd or runc in the base image.

It's fine that moby-engine isn't a priority, especially since podman is pretty great for people that can choose it. But a "container optimized" OS shipping a poorly maintained Docker is (in my opinion) actually worse than not shipping Docker at all.

Dropping these packages helps to mitigate the two problems above:

1. Lack of maintenance is not such a problem, as users can choose to install from upstream instead. If maintenance picks up, users can still take advantage by rpm-ostree install moby-engine.
2. Users that need more control over the version of Docker can install from Docker's upstream repo, but now without having to do hacky base overrides.

System details

No response

Additional information

No response

[dustymabe]

We discussed this at the community meeting today.

There is some background here that makes moby-engine not just like any other package in FCOS.

From me today in the meeting:

When we started building Fedora CoreOS one of the things we wanted to do was keep Container Linux users happy. Users who wanted to continue to use docker could do so without issue. We've held to that principle for a long time. I would like to continue to ship it because I know there are good number of people who do use it.

Now that doesn't mean we will ship it forever if it goes unmaintained, but we'll probably take several actions before we'd remove it.

It just so happens the current maintainer showed up to our meeting and started taking part in the discussion. There is a re-architecture happening that will make it easier to maintain in Fedora. It's currently blocked on a few package reviews to go through:

• https://bugzilla.redhat.com/show_bug.cgi?id=2274561
• https://bugzilla.redhat.com/show_bug.cgi?id=2274654
• https://bugzilla.redhat.com/show_bug.cgi?id=2274656

With all that being said we did decide:

gursewak
!agreed : Add documentation on how to install upstream docker.

So that we can document how to remove and replace the installed docker with the one from upstream if a user has those needs. Follow in coreos/fedora-coreos-docs#639

[jshuffle]

@dustymabe Amazing, thanks so much. I love that I can read the minutes from the meeting. And also fortuitous that the current maintainer turned up (who, if you are reading, I hope I didn't offend you!).

Thanks for the helpful links.

And thanks to everyone for taking this into consideration and coming up with a reasonable plan. 🚀

Not sure if you want to keep this ticket open. Close if desired ☺️

[travier]

Ideally we would offer an additional variant of Fedora CoreOS that has no container engine included by default so that you can pick and choose the one you want, be it the latest podman or the Docker version that you prefer.

See: coreos/fedora-coreos-config#2877

Unfortunately this is costly in terms of CI, maintenance, testing and release engineering efforts right now as we should likely not drop what we have currently, so that would be an additional image.

[cgwalters]

@jshuffle you may be interested in the new https://docs.fedoraproject.org/en-US/bootc/ project btw - and the https://gitlab.com/fedora/bootc/examples/-/tree/main/docker example shows installing docker-ce as part of a container build.

[dustymabe]

With all that being said we did decide:

gursewak
!agreed : Add documentation on how to install upstream docker.

docs added in coreos/fedora-coreos-docs#641