payload/rpmostree: Add support for bootupd #5298
Conversation
|
Hello @cgwalters! Thanks for updating this PR. We checked the lines you've touched for PEP 8 issues, and found: There are currently no PEP 8 issues detected in this Pull Request. Cheers! 🍻 Comment last updated at 2023-11-02 21:03:24 UTC |
|
This is totally untested as of right now; putting up for early feedback. I'll need some help in spinning up an Anaconda development environment (I did it long ago in the past). For reference, a goal here is it should work with this to do e.g. |
cgwalters
force-pushed
the
ostree-bootupctl-auto
branch
from
48eb5c5 to
6cfe7b3
Compare
The https://github.com/coreos/bootupd project was created to fill the gap in bootloader management for ostree-based systems. When it was created, it was just integrated into Fedora CoreOS and derivatives; this left the Atomic Desktops (Silverblue etc.) as unfixed, and it was never used by RHEL for Edge. This PR is aiming to circle back and close that gap. We detect if bootupd is in the target root; if it is, then we should act as if `bootloader --location=none` had been specified, and just run bootupd to perform the installation. The other hacks we have around the grub config are no longer necessary in this mode.
cgwalters
force-pushed
the
ostree-bootupctl-auto
branch
from
6cfe7b3 to
e78aa0d
Compare
KKoukiou
requested review from
jkonecny12,
poncovka and
VladimirSlavik
and removed request for
jkonecny12 and
poncovka
| """ | ||
| rc = execWithRedirect( | ||
| "bootupctl", | ||
| ["backend", "install", "--auto", "--with-static-configs", |
There was a problem hiding this comment.
It's worth noting another big difference here vs how Anaconda works by default is that we use a static grub config. This means we don't run grub2-mkconfig for every kernel update - which in turns we don't run os-prober. And IMO os-prober is one of the worst chunks of code we ship that runs as root, scanning all your block devices to see if there's e.g. a BeOS or Windows partition there every time you just want to update your kernel...
|
/build-image |
|
Any feedback on this? I see some unit tests started falling over; it may be slightly tricky to mock this. I can look, though I ran into other unrelated problems running the unit tests locally around locales I need to figure out. |
| bootloader = STORAGE.get_proxy(BOOTLOADER) | ||
| bootloader.set_use_bootupd() |
There was a problem hiding this comment.
I don't understand the current anaconda architecture really, and I was wondering if it really works to make "cross actor" dynamic calls or not like this. Basically here the bootloader state can only be fully computed once we've written the payload today. Hopefully this can work.
There was a problem hiding this comment.
That won't work. The place doing return [SomeTask()] is called before calling the run() that checks if there is bootupd installed. The whole installation task queue is gathered before starting the tasks.
Is there some way of detecting this from the payload before installation?
There was a problem hiding this comment.
@VladimirSlavik That's what I though as well, but the installation queue still uses workarounds from the time when the payload wasn't fully migrated, so these tasks are actually collected on demand during the installation. That makes me sad, because it means that this will work until we remove the workarounds.
On the other side, we can hook the bootloader tasks to the bootloader mode change signal and update their states on change. Alternatively, we can wrap them into another task. It won't be so difficult to support this use case.
There was a problem hiding this comment.
Is there some way of detecting this from the payload before installation?
Not today. we could add something to the container image metadata for it, and then have a flow that does:
- fetch metadata (i.e. container image manifest)
- set up kickstart state
- perform installation
This topic intersects with #5197 a bit...
What I think would be really neat is generalized support for embedding kickstart fragments in the container image. This seems most viable then if it's done in the metadata instead of the payload, because otherwise in the general case we'd need to buffer the whole payload into RAM before an install.
If we did that, then we could add e.g. inst.bootc=quay.io/examplecorp/someos:latest on the kernel commandline which could then entirely replace inst.ks - the ergonomic improvement there would be huge in the general case.
(But this general case again is for "using pre-generated os/distro ISO with custom payload", like Fedora today, but not like what we want for a custom Image Builder flow)
There was a problem hiding this comment.
Ah, actually, not having a kickstart option and state would be preferable. If there's any way to read this from the payload once it's "known", before being "installed", that would be best.
There was a problem hiding this comment.
Yep agree that's preferable! In this stubbed out code we detect this state by noticing that /usr/bin/bootupctl is in the target root...that's after "install" but should still be before the bootloader stage, right?
Failing that, I think we can add a metadata property (LABEL bootupd.enabled=true) or so in the container image manifest. But implementing a separate "discovery" phase would be some new code.
(Not difficult code, just need to fork off skopeo inspect <container image reference>)
| @property | ||
| def use_bootupd(self): | ||
| """Whether bootupd is enabled""" | ||
| return self._use_bootupd | ||
|
|
||
| def set_use_bootupd(self): | ||
| """Install the bootloader using https://github.com/coreos/bootupd""" | ||
| self._use_bootupd = True | ||
| self.set_bootloader_mode(BootloaderMode.SKIPPED) | ||
|
|
There was a problem hiding this comment.
This is the "implementation", in the bootloader "module", but it needs to be added to the "interface", too. In the file bootlader_interface.py, add the same thing, CamelCase-named, touching self.implementation.<name-from-module>. Also, it's a D-Bus property, so in the interface should be a property, not a function. Check bootloader_mode and BootloaderMode to see an example how this is done.
(The interface members are translated to D-Bus automagically)
| bootloader = STORAGE.get_proxy(BOOTLOADER) | ||
| if not bootloader.use_bootupd: |
There was a problem hiding this comment.
Here you're calling the interface, so CamelCase UseBootupd once added as explained above.
| @@ -509,7 +513,7 @@ def _move_grub_config(self): | |||
| os.rename(boot_grub2_cfg, target_grub_cfg) | |||
| os.symlink('../loader/grub.cfg', boot_grub2_cfg) | |||
|
|
|||
| def _set_kargs(self): | |||
| def _set_kargs(self, bootloader): | |||
There was a problem hiding this comment.
If you add this option, please add it to docstring, it's a "proxy to bootloader". However, as it's just a Pythonified wrapper for D-Bus, it's equal to grabbing it anew as it was below, so just not doing this change is an option too.
There was a problem hiding this comment.
At a procedural level please feel free to just force-push fixups. I haven't yet spun up an Anaconda dev/test environment.
|
I think...
|
| @@ -456,6 +469,11 @@ def install_bootloader_with_tasks(self, payload_type, kernel_versions): | |||
| :param kernel_versions: a list of kernel versions | |||
| :return: a list of tasks | |||
| """ | |||
| if self._use_bootupd: | |||
There was a problem hiding this comment.
The thing is that setting the bootloader mode to SKIPPED is enough to disable the tasks in the else branch (there are conditions in their run methods that will stop the tasks before any action). So the only effect of the use_bootupd flag on this module is running the bootupctl tool.
I am wondering if the bootupd code shouldn't stay in the rpm ostree module and the bootupctl tool shoudn't be called from there. This functionality is specific to ostree installations, right? Or is there a possibility of expansion to other types of payloads in the future?
There was a problem hiding this comment.
This functionality is specific to ostree installations, right?
Today yes.
Or is there a possibility of expansion to other types of payloads in the future?
In theory it's possible, in practice it'd be a giant change and not really worth it.
|
Thanks for all the review so far! Thinking about this a bit more, since our initial primary goal here will be generating "self-installing" ISOs via Image Builder with the content embedded, perhaps it's far simpler to add a new flag like |
|
If it satisfies the goal, then yes, that would be probably easier. Given how the sdboot stuff looks in kickstart, it would be |
This SGTM. Though to be clear right now I'm juggling about 10 things related to this, and happy to assist/pair program/meet on this, just probably can't drive it. |
|
Hi @cgwalters! We discovered some issues with the current approach. The idea is to act the same way as I was testing the current workaround using https://centos.github.io/centos-boot/example.ks and virt-install: There are some notes from my investigation:
|
Yes, this is fair. Hmm....so the primary problems here are around the BIOS/MBR handling? Perhaps we can keep Anaconda doing that part...i.e. something like
Hmm, what is the default Anaconda semantics of doing this with this kickstart? Is it trying to install an MBR on both disks? Will look. I don't actually know the semantics of the existing |
|
OK I just tried tweaking the kickstart to do basically: and dropping the bootupd stuff...i.e. just checking "what does anaconda do by default here", and the result seems quite bizarre to me: Are partitions just...randomly chosen here? Why put the bootloader bits on the second disk but the OS install on the first? I am totally aware that this example kickstart does not explicitly request specific disks, and hence there's a bit of a GIGO effect here but... For what we're targeting honestly I am not sure we really need to support use cases where the primary OS root is on a separate drive than the bootloader. |
|
Hi @cgwalters ! I will get to your questions. Just a quick update, @VladimirSlavik is currently working on an alternative minimal solution that he and I agreed on. The idea is to start with something small and simple that works in most cases and build on that.
|
SGTM!
Also agreed.
Yes definitely! |
|
BTW when I was playing with this I now understand that also we definitely don't want |
Anaconda needs to be able to create hybrid boot disks. For example: clearpart --all --initlabel --disklabel=gpt part prepboot --size=4 --fstype=prepboot part biosboot --size=1 --fstype=biosboot part /boot/efi --size=100 --fstype=efi part /boot --size=1000 --fstype=ext4 --label=boot part / --grow --fstype xfs However, this kickstart snippet is not working with two or more disks. The bootloader-related partitions should be all created on the disk the computer will boot from, but Blivet does that only for platform -specific partitions. The rest of them are created on any disk with enough space. It looks like this can be easily fixed by setting the same weight to all of these partitions regardless of the current platform. See: rhinstaller/anaconda#5298
|
@cgwalters About the problematic scheduling of bootloader partitions on multiple disks, long story short, I think we are able to fix that (storaged-project/blivet#1174). It doesn't solve all issues, but it should help with kickstart partitionings like this one: https://github.com/CentOS/centos-bootc/blob/main/docs/example.ks.
About the |
Anaconda needs to be able to create hybrid boot disks. For example: clearpart --all --initlabel --disklabel=gpt part prepboot --size=4 --fstype=prepboot part biosboot --size=1 --fstype=biosboot part /boot/efi --size=100 --fstype=efi part /boot --size=1000 --fstype=ext4 --label=boot part / --grow --fstype xfs However, this kickstart snippet is not working with two or more disks. The bootloader-related partitions should be all created on the disk the computer will boot from, but Blivet does that only for platform -specific partitions. The rest of them are created on any disk with enough space. It looks like this can be easily fixed by setting the same weight to all of these partitions regardless of the current platform. See: rhinstaller/anaconda#5298
Anaconda needs to be able to create hybrid boot disks. For example: clearpart --all --initlabel --disklabel=gpt part prepboot --size=4 --fstype=prepboot part biosboot --size=1 --fstype=biosboot part /boot/efi --size=100 --fstype=efi part /boot --size=1000 --fstype=ext4 --label=boot part / --grow --fstype xfs However, this kickstart snippet is not working with two or more disks. The bootloader-related partitions should be all created on the disk the computer will boot from, but Blivet does that only for platform -specific partitions. The rest of them are created on any disk with enough space. It looks like this can be easily fixed by setting the same weight to all of these partitions regardless of the current platform. See: rhinstaller/anaconda#5298
Anaconda needs to be able to create hybrid boot disks. For example: clearpart --all --initlabel --disklabel=gpt part prepboot --size=4 --fstype=prepboot part biosboot --size=1 --fstype=biosboot part /boot/efi --size=100 --fstype=efi part /boot --size=1000 --fstype=ext4 --label=boot part / --grow --fstype xfs However, this kickstart snippet is not working with two or more disks. The bootloader-related partitions should be all created on the disk the computer will boot from, but Blivet does that only for platform -specific partitions. The rest of them are created on any disk with enough space. It looks like this can be easily fixed by setting the same weight to all of these partitions regardless of the current platform. See: rhinstaller/anaconda#5298
|
The "hybrid boot" thing is probably a bit of a distraction. I forget why I ended up doing that in the kickstart; I don't think there was a good reason. |
The installer supports bootupd now, so we can drop the workaround. See: rhinstaller/anaconda#5298
The partitioning defined in the example kickstart file suggests that the installer supports hybrid boot. That's misleading and not true. Let's use the `reqpart` kickstart command to automatically create partitions required by the detected platform instead of creating all of them for all platforms. Note: The `reqpart` command doesn't work with `bootloader --location=none` or `bootloader --disabled`, so this commit depends on the installer's support for bootupd: rhinstaller/anaconda#5298
The partitioning defined in the example kickstart file suggests that the installer supports hybrid boot. That's misleading and not true. Let's use the `reqpart` kickstart command to automatically create partitions required by the detected platform instead of creating all of them for all platforms. Note: The `reqpart` command doesn't work with `bootloader --location=none` or `bootloader --disabled`, so this commit depends on the installer's support for bootupd: rhinstaller/anaconda#5298
The installer supports bootupd now, so we can drop the workaround. See: rhinstaller/anaconda#5298
The partitioning defined in the example kickstart file suggests that the installer supports hybrid boot. That's misleading and not true. Let's use the `reqpart` kickstart command to automatically create partitions required by the detected platform instead of creating all of them for all platforms. Note: The `reqpart` command doesn't work with `bootloader --location=none` or `bootloader --disabled`, so this commit depends on the installer's support for bootupd: rhinstaller/anaconda#5298
I think I know why. The I have opened a draft with these changes, because we need to know what we are aiming for: CentOS/centos-bootc#72 |
|
The PR from us #5342 is merged, so this can be closed as a code contribution. Feel free to continue discussion if needed... |
The installer supports bootupd now, so we can drop the workaround. See: rhinstaller/anaconda#5298
The partitioning defined in the example kickstart file suggests that the installer supports hybrid boot. That's misleading and not true. Let's use the `reqpart` kickstart command to automatically create partitions required by the detected platform instead of creating all of them for all platforms. Note: The `reqpart` command doesn't work with `bootloader --location=none` or `bootloader --disabled`, so this commit depends on the installer's support for bootupd: rhinstaller/anaconda#5298
The https://github.com/coreos/bootupd project was created to fill the gap in bootloader management for ostree-based systems.
When it was created, it was just integrated into Fedora CoreOS and derivatives; this left the Atomic Desktops (Silverblue etc.) as unfixed, and it was never used by RHEL for Edge.
This PR is aiming to circle back and close that gap. We detect if bootupd is in the target root; if it is, then we should act as if
bootloader --location=nonehad been specified, and just run bootupd to perform the installation.The other hacks we have around the grub config are no longer necessary in this mode.