design for yum/dnf wrapper

[cgwalters]

Design for yum/dnf frontend to rpm-ostree

Extracted from #2844

Goal: Support existing UX in Dockerfile

The Ostree Native Container change is leading to a fundamental change in how we think about rpm-ostree. In particular, we now support running rpm-ostree install foo inside a Dockerfile. But this is the same thing that people have been doing for ages with yum install foo inside a Dockerfile.

In the end, I think it makes sense to...have our container images support yum install (and dnf install). To the user, there isn't a hugely important difference between what we do and what yum|dnf install are doing.

There are however some important implementation things going on that we will want to maintain. For example, eventually we want to move things like tmpfiles.d translation to happen at build time.

There's a lot of stuff in rpm-ostree around things like the rpmdb handling, dropping the dnf sqlite database that we don't need or want, etc.

So the suggested implementation here is that rpm-ostree learns to respond to a subset of yum|dnf commands.

Goal: Be friendlier to admins coming from existing Fedora-derivative ecosystem in general

Once we take this step, it makes sense also to support this on the client side in general.

There are some things that really should work.
Let's take as an example: https://tailscale.com/download/linux/fedora
This is exactly an operating system extension; it doesn't
make sense in toolbox or flatpak. (It could be containerized
but...let's leave that aside)

The basic goal here is not that dnf install tailscale actually works
directly, but it should explain to people what to do. More on
this below.

Implementation

Note that the implementation of this will be that rpm-ostree gains
a new partial compatible yum CLI written in Rust. There are no
plans to require Python for example, or change the existing yum/dnf
code.

(Note: The terms yum and dnf are used synonymously here because
in practice in current Fedora and RHEL8 the former is just a symlink to the latter.)

Longer term, we may share more CLI code with "dnf 5" as it migrates to C++,
though this clashes some with rpm-ostree's increasing migration to Rust.

Non-goal: Reverting emphasisis on containerization

We want people to migrate application code, debugging and development
tools etc. into containers as opposed to layering everything on the host.
So to emphasize: we explicitly do not want yum install gcc or dnf install tailscale
to silently Just Work out of the box. More on this below.

High level goal: A new yum image subcommand

This will be a front-end to the existing ostree container flow:

# Equivalent to rpm-ostree rebase --experimental ostree-image-signed:quay.io/myuser/customos:latest
$ yum image rebase quay.io/myuser/customos:latest

After that, yum update will pull that container!

Demonstration CLI examples:

upgrades

$ yum update
Note: This system is image (rpm-ostree) based.  Upgrades queue in the background
by default and will not affect the running system.
...

On Fedora CoreOS with automatic upgrades on by default, this would of course error out with:

error: Updates and deployments are driven by Zincati (zincati.service)
See Zincati's documentation at https://github.com/coreos/zincati
Use --bypass-driver to bypass Zincati and perform the operation anyways

So one would then use e.g.:

$ yum update --bypass-driver

On current Fedora IoT/Silverblue which do not have automatic upgrades on by default:

$ yum update
Receiving objects: ...
Upgraded:
  kernel 5.12.6-300.fc34 -> 5.12.8-300.fc34
  openssh 8.5p1-2.fc34 -> 8.6p1-3.fc34
Completing upgrade requires a reboot.
Partial upgrade can be initiated via `yum apply-live`.

Note this proposed text includes analysis of whether the change requires a reboot or not, as well as stabilization of apply-live.

$ yum apply-live openssh
Applying: openssh 8.5p1-2.fc34 -> 8.6p1-3.fc34
Reloaded: openssh.service

dnf needs-restarting

We should support the dnf needs-restarting [-r] [-s] CLI, which might sensibly be used by automation after dnf upgrade.

Layering

$ yum install gcc
Note: This system is image (rpm-ostree) based.
Before installing packages to the host root filesystem, consider other options:

 - `toolbox`: For command-line development and debugging tools in a privileged container
 - `podman`: General purpose, isolated containers
 - `flatpak`: For desktop (GUI) applications
 - `rpm-ostree install`: For packages which need to be directly on the host. Treat these as "OS extensions". Add `--apply-live` to immediately start using the layered packages.

(Exit code 1)

$ sudo dnf config-manager --add-repo https://pkgs.tailscale.com/stable/fedora/tailscale.repo
Note: This system is image (rpm-ostree) based.
Assuming use of rpm-md repository for operating system extensions.
$ sudo dnf install tailscale
(error text as above)
$ rpm-ostree install -A tailscale
$ sudo systemctl enable --now tailscaled
$ sudo tailscale up

Replacing base components

Background: User is trying to downgrade the kernel on a machine to bisect an issue.

$ yum install ./kernel-5.12.6-300.fc34.x86_64.rpm
Note: This system is image (rpm-ostree) based.
The following packages are part of the base image and require
explicit action to override:

 - kernel

To complete this action, use:
  rpm-ostree override replace ./kernel-5.12.6-300.fc34.x86_64.rpm
$ rpm-ostree override replace ./kernel-5.12.6-300.fc34.x86_64.rpm
Downgraded:
  kernel 5.12.8-300.fc34 -> 5.12.6-300.fc34
...
Completing changes requires a reboot.
$

Upgrading just individual packages

$ yum update openssh

The flow for this will encourage the user to download the full base image
update, and cherry pick just the openssh change live if desired via
apply-live.

There was a lot of debate in #2844

[cgwalters]

OK a fairly important detail here is the lack of -y in rpm-ostree. The omission was intentional originally, precisely because we only queue updates for the next boot, there's no need to have it be rpm-ostree -y update because we print the diff and you can undo.

But I recently realized we lost this whole aspect with rpm-ostree install -A becoming stable. For that case...I think I have a sub-proposal here to add a -y option, and a --confirm argument, and deprecate not having either being present.

[jlebon]

The doc looks really good to me! I like the yum install ./kernel.rpm text a lot. The only bit I'm not sure on is related to #2844 (comment). I think I'm cool with not printing a redirection warning. Though in helper commands we print out, I think we should consistently use rpm-ostree and not dynamically print based on what $0?

E.g. for rpm-ostree override replace kernel..., I think we should print out at the end something like:

Use rpm-ostree override reset kernel to return to the base OSTree version.

which is consistent with yum install ./kernel.rpm printing the actual rpm-ostree override replace ... command to use.

And probably in some of the compat text we should talk about rpm-ostree status. We can tweak that stuff later on though!

[jlebon]

Cross-posting some comments from #2844:

General idea makes a lot of sense to me! One thing we talked about is that we shouldn't hide rpm-ostree too much either, because a lot of the value becomes apparent when you understand some of the basic concepts. What you have here is in line with that.

I think also we should make rpm-ostree more discoverable by being more helpful in our command outputs. E.g. rpm-ostree install should reference rpm-ostree uninstall. And rpm-ostree override replace/remove should reference rpm-ostree override reset, etc... And almost everything should probably reference rpm-ostree status. Etc... Much like how porcelain git commands are.

I guess at a high-level, I see this endeavor more as an opportunity to gently guide new users towards learning base rpm-ostree concepts and commands (and even allow them to punt on learning anything at first to still be functional and e.g. update and install things). IMO trying to wrap rpm-ostree commands into new yum commands would be more confusing than helpful and may do them a disservice in the long term by letting them keep punting on learning a new way of managing their machines. So it's more like we're "meeting them halfway". WDYT of that framing?

[sinnykumari]

I like the overall idea of guiding yum/dnf based users to smoothly transition to rpm-ostree based system. I was wondering, instead of adding wrapper that would help users to directly run some of rpm-ostree command using yum/dnf like dnf update, why not we just keep this wrapper even simpler? What I mean is when user executes any yum/dnf command, just print equivalent rpm-ostree command with relevant messaging on what they should run. And, like Jonathan said, keep rpm-ostree more discoverable for users. This allows to do least maintenance work on yum/dnf wrapper which can be easily deprecated in future since user's script would never be relying on a yum/dnf command. Also, it allows us to put majority of effort in making rpm-ostree usage more user friendly for existing/new users.

[cgwalters]

What I mean is when user executes any yum/dnf command, just print equivalent rpm-ostree command with relevant messaging on what they should run.

I think the core question though here is: is there a lot of value in telling a user to run rpm-ostree install -y -A tailscale instead of dnf -y install tailscale?

[cjao]

Since users are already accustomed to managing software using dnf/yum, and in fact will have to continue using that tool in their toolbox environments, wouldn't it make sense for rpm-ostree to present itself as a special "mode" of dnf/yum on ostree systems?

[cgwalters]

Tangentially related to this, we should also wrap rpm -Uvh for similar reasons, and when we do that we should be careful to try to preserve the aspect that using rpm does not trigger or require rpm-md - xref #3153

[cgwalters]

Update: this is shipping now by default in Fedora CoreOS rawhide.

[vwbusguy]

This needs clarification for groups (eg, dnf groupinstall) since rpm-ostree doesn't currently support groups. For example, dnf install @virtualization would work on Workstations but rpm-ostree install @virtualization does not. Ideally, rpm-ostree should support groups (minimally by @id), but perhaps dnf breaking out the group packages to install via rpm-ostree might be a good in-between solution?