Inherit from fedora-bootc's tier-x on Fedora 42+ #3177

jlebon · 2024-09-23T19:01:16Z

There is a new tier-x in the fedora-bootc project whose goal is to
provide a common base that all variants (including tier-1) can share.

Move FCOS over to use this new tier, but only starting from Fedora 42.

This is a profound change and the start of an exciting new future! This
formalizes our relationship to other image-mode variants, encouraging us
to innovate and solve problems together in a more direct way.

Put more practically, e.g. bug fixes, new features, or temporary
workarounds that concern all/most tier-x derivatives should probably be
carried out at the tier-x level rather than the CoreOS level.

Eventually, this inheritance will be made even more explicit by having
FCOS be built FROM the tier-x image. For now, we share at the manifest
level, which is a stepping stone towards that goal.

Patches to actually dedupe our manifests with tier-x will follow. Though
note there is no change in the resulting package set here.

jlebon · 2024-09-23T19:01:39Z

Targeting rawhide for now to exercise it in CI. Will retarget testing-devel once this is ready to go.

Other outstanding items:

verify configbot handling of git submodules
prepare openshift/os changes in advance

jlebon · 2024-09-23T19:12:46Z

Some diff stats:

testing-devel

No package diff
No initramfs diff

OSTree diff:

$ ostree diff 40.20240922.dev.0 40.20240922.dev.1
M    /usr/etc/pki/ca-trust/extracted/java/cacerts
M    /usr/lib/os-release
M    /usr/lib/modules/6.10.10-200.fc40.x86_64/initramfs.img
M    /usr/lib/sysimage/rpm-ostree-base-db/rpmdb.sqlite
M    /usr/share/rpm/rpmdb.sqlite
M    /usr/share/rpm-ostree/treefile.json
A    /usr/bin/systemd-firstboot
A    /usr/lib/.build-id/07/052139ba8f88405bf6bef78a8f51b67894a761
A    /usr/lib/systemd/system/systemd-firstboot.service

These are all expected. See related commit for the systemd-firstboot-related one. The others churn naturally.

rawhide

No package diff
Initramfs diff
```
-rw-r--r--   1 root     root           27 Dec 31  1969 usr/lib/ostree/prepare-root.conf
-rw-r--r--   1 root     root           52 Dec 31  1969 usr/lib/ostree/prepare-root.conf
```
This is from prepare-root.conf now also including the sysroot.readonly, which should also squash this warning from ostree-prepare-root:

ostree-prepare-root[1319]: Found legacy sysroot.readonly flag, not configured in ostree/prepare-root.conf

(Compare what we ship to what's in tier-0.)

OSTree diff:

$ ostree diff 42.20240922.dev.0 42.20240922.dev.1
M    /usr/etc/pki/ca-trust/extracted/java/cacerts
M    /usr/lib/os-release
M    /usr/lib/modules/6.12.0-0.rc0.20240920gitbaeb9a7d8b60.7.fc42.x86_64/initramfs.img
M    /usr/lib/ostree/prepare-root.conf
M    /usr/lib/sysimage/rpm-ostree-base-db/rpmdb.sqlite
M    /usr/lib/tmpfiles.d/provision.conf
M    /usr/share/rpm/rpmdb.sqlite
M    /usr/share/rpm-ostree/treefile.json
A    /usr/bin/systemd-firstboot
A    /usr/lib/.build-id/df/db1101b9d05b622ff69b938d29783028f69ad5
A    /usr/lib/dracut/dracut.conf.d/20-bootc-base.conf
A    /usr/lib/dracut/dracut.conf.d/22-bootc-generic.conf
A    /usr/lib/dracut/dracut.conf.d/49-bootc-tpm2-tss.conf
A    /usr/lib/dracut/dracut.conf.d/59-altfiles.conf
A    /usr/lib/systemd/system/systemd-firstboot.service

These are all expected. See related commit for the systemd-firstboot-related one. The dracut dropins are redundant with our current settings, so no change there. The prepare-root.conf change is explained above. The others churn naturally.

jlebon · 2024-09-23T19:20:04Z

Huh, looks like the submodule isn't getting initialized in CI. Investigating.
Edit: coreos/coreos-ci-lib#160

dustymabe

Thanks for working on this.

dustymabe · 2024-09-27T20:20:10Z

manifests/networking-tools.yaml

  # Firewall manipulation
-  - iptables nftables
+  - nftables


I'm surprised iptables is included in bootc tier-x, but nftables isn't.

nftables actually is in tier-0 (and so also tier-x), as a dependency of the container stack. We could make it explicit there to make that clearer?

dustymabe · 2024-09-27T20:21:56Z

manifests/user-experience-tier-x-dupes.yaml

+# This manifest can go away in Fedora 42. It duplicates tier-x.
+
+# Default to `bash` in our container, the same as other containers we ship.
+# Note this changes to /sbin/init in f42 as inherited by tier-x.


does this make the generated container not podman run able without specifying more on the command line?

Though more precisely it is still podman run able it just runs systemd and not a shell...

dustymabe · 2024-09-27T20:24:53Z

manifests/ignition-and-ostree.yaml

@@ -5,9 +5,6 @@
 # One good model is to add fedora-coreos-config as a git submodule.  See:
 # https://github.com/coreos/coreos-assembler/pull/639

-# Include rpm-ostree + kernel + bootloader
-include: bootable-rpm-ostree.yaml
-


notable that fedora-coreos-base.yaml isn't included in RHCOS, but ignition-and-ostree.yaml is. So there will be some fixups to that repo needed when we bump the f-c-c git submodule there.

Yeah, there's a bunch of tweaks needed in openshift/os to adapt to this. I'm working on that in parallel with this PR but haven't published it yet.

manifests/ignition-and-ostree.yaml

dustymabe · 2024-09-27T20:32:37Z

manifests/ignition-and-ostree-tier-x-dupes.yaml

+tmp-is-dir: true
+
+# Required by Ignition, and makes the system not compatible with Anaconda
+machineid-compat: false


This is also in manifests/tier-x.yaml

Yes, this is by design. We need to keep this manifest until we only have tier-x. It's only included in the < 42 path.

dustymabe · 2024-09-27T20:39:48Z

.gitmodules

I'm not the biggest fan of a submodule here but I guess that's really the only way to share at this stage. I really don't know of anything better, but a lot of questions to come to mind.

since we sync around things in this repo using config bot, how are updates to the submodule going to work?

we sometimes have our "streams" on different versions of Fedora. Right now stable testing and testing-devel are on F40, next and next-devel are on F41, and rawhide F42. How do we account for this with a submodule. https://gitlab.com/fedora/bootc/base-images.git isn't a linear definition, it has branches like the rest of Fedora.

Filed https://gitlab.com/fedora/bootc/tracker/-/issues/39 related to this.

since we sync around things in this repo using config bot, how are updates to the submodule going to work?

We should be able to propagate submodule bumps just like any other file. I need to make sure configbot handles this well.

gitlab.com/fedora/bootc/base-images.git isn't a linear definition, it has branches like the rest of Fedora

This is fixed now! The main branch of base-images tracks all active releases (and we can use releasever conditionals there like we do here, but don't yet).

As of recently I started using subtree merges a bit...it has a huge set of advantages here but some disadvantages in that it's way easier to accidentally fork. But the plus side is it is easy to fork and we don't need hacks like "push downstream branch to upstream".

The other way of course we should share is a variant of https://gitlab.com/fedora/bootc/tracker/-/issues/32 where we publish the manifests as a container image and it's required to COPY --from that image when doing a build.

This has its own advantages and disadvantages.

cgwalters · 2024-10-30T18:31:28Z

Sounds like this one isn't blocked, just needs a rebase and socializing?

jlebon · 2024-10-30T18:45:25Z

This was unblocked by https://gitlab.com/fedora/bootc/base-images/-/merge_requests/63. I'm planning to get back to it soon.

There is a new tier-x in the fedora-bootc project whose goal is to provide a common base that all variants (including tier-1) can share. Move FCOS over to use this new tier, but only starting from Fedora 42. This is a profound change and the start of an exciting new future! This formalizes our relationship to other image-mode variants, encouraging us to innovate and solve problems together in a more direct way. Put more practically, e.g. bug fixes, new features, or temporary workarounds that concern all/most tier-x derivatives should probably be carried out at the tier-x level rather than the CoreOS level. Eventually, this inheritance will be made even more explicit by having FCOS be built `FROM` the tier-x image. For now, we share at the manifest level, which is a stepping stone towards that goal. Patches to actually dedupe our manifests with tier-x will follow. Though note there is no change in the resulting package set here.

All these manifests are already part of tier-x, so conditionalize them on <f42.

Split all the things redundant with tier-x into a separate manifest that only gets inherited on <f42.

No functional change. Prep for future patch.

All the requests in this manifest are part of tier-x, so conditionalize on <f42.

In the spirit of abusing `remove-from-packages` less and being more closely aligned with tier-x, stop literally nuking the systemd-firstboot binary and systemd unit. Instead, just disable the systemd unit. Note it's also disabled in the presets (in `40-coreos-systemd.preset`) and as mentioned in the comment, via a runtime dropin. So that's triple disablement already. But to be sure, also add a non-exclusive test to ensure it doesn't come back.

Split all the things redundant with tier-x into a separate manifest that only gets inherited on <f42.

No functional change. This just makes it easier to share the list of dupes with RHCOS downstream.

This workflow bumps the fedora-bootc submodule. It's better than just using Dependabot because it only checks for changes in tier-0 and tier-x. AFAICT, there isn't a way to tell Dependabot to do this.

jlebon · 2024-11-07T20:23:04Z

Rebased this now that https://gitlab.com/fedora/bootc/tracker/-/issues/39 is resolved and updated for comments!

Also started tracking some of the outstanding items in #3177 (comment).

This was referenced Sep 25, 2024

Roadmap to Fedora Bootable Containers coreos/fedora-coreos-tracker#1726

Open

wrong SELinux context on /var/opt/cni openshift/os#1624

Open

dustymabe reviewed Sep 27, 2024

View reviewed changes

jlebon added 12 commits November 7, 2024 14:37

manifests: conditionalize duplicate manifests

135c139

All these manifests are already part of tier-x, so conditionalize them on <f42.

manifests: split system-configuration.yaml

2454606

Split all the things redundant with tier-x into a separate manifest that only gets inherited on <f42.

manifests: split networking-tools.yaml

8860030

Split all the things redundant with tier-x into a separate manifest that only gets inherited on <f42.

manifests: split user-experience.yaml

0feeb1d

Split all the things redundant with tier-x into a separate manifest that only gets inherited on <f42.

manifests: hoist up bootable-rpm-ostree.yaml include

0b138ce

No functional change. Prep for future patch.

manifests: move veritysetup request to ignition-and-ostree.yaml

429ceed

No functional change. Prep for future patch.

manifests: conditionalize bootable-rpm-ostree.yaml

01b9126

All the requests in this manifest are part of tier-x, so conditionalize on <f42.

manifests: split ignition-and-ostree.yaml

9e87548

Split all the things redundant with tier-x into a separate manifest that only gets inherited on <f42.

manifests: split tier-x dupes into separate manifest

a66d63f

No functional change. This just makes it easier to share the list of dupes with RHCOS downstream.

workflows: add bump-fedora-bootc workflow

0009012

This workflow bumps the fedora-bootc submodule. It's better than just using Dependabot because it only checks for changes in tier-0 and tier-x. AFAICT, there isn't a way to tell Dependabot to do this.

jlebon force-pushed the pr/inherit-tier-x-rawhide branch from d1dd8ef to 0009012 Compare November 7, 2024 20:20

jlebon mentioned this pull request Nov 8, 2024

manifest: Add bootc #3246

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Inherit from fedora-bootc's tier-x on Fedora 42+ #3177

Inherit from fedora-bootc's tier-x on Fedora 42+ #3177

jlebon commented Sep 23, 2024

jlebon commented Sep 23, 2024 •

edited

Loading

jlebon commented Sep 23, 2024

jlebon commented Sep 23, 2024 •

edited

Loading

dustymabe left a comment

dustymabe Sep 27, 2024

jlebon Nov 7, 2024

dustymabe Sep 27, 2024

cgwalters Oct 22, 2024

cgwalters Oct 22, 2024

dustymabe Sep 27, 2024

jlebon Nov 7, 2024

dustymabe Sep 27, 2024

jlebon Nov 7, 2024

dustymabe Sep 27, 2024

jlebon Sep 30, 2024

jlebon Nov 7, 2024

cgwalters Nov 7, 2024

cgwalters Nov 7, 2024

cgwalters commented Oct 30, 2024

jlebon commented Oct 30, 2024

jlebon commented Nov 7, 2024

Inherit from fedora-bootc's tier-x on Fedora 42+ #3177

Are you sure you want to change the base?

Inherit from fedora-bootc's tier-x on Fedora 42+ #3177

Conversation

jlebon commented Sep 23, 2024

jlebon commented Sep 23, 2024 • edited Loading

jlebon commented Sep 23, 2024

testing-devel

rawhide

jlebon commented Sep 23, 2024 • edited Loading

dustymabe left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

cgwalters commented Oct 30, 2024

jlebon commented Oct 30, 2024

jlebon commented Nov 7, 2024

jlebon commented Sep 23, 2024 •

edited

Loading

jlebon commented Sep 23, 2024 •

edited

Loading