Support OEM systemd-sysext images and Flatcar extensions #24
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
pothos
force-pushed
the
kai/postinst-sysext
branch
3 times, most recently
from
e613ee6 to
677d950
Compare
pothos
changed the title
pothos
force-pushed
the
kai/postinst-sysext
branch
6 times, most recently
from
6c8255b to
5ebd18b
Compare
pothos
force-pushed
the
kai/postinst-sysext
branch
2 times, most recently
from
8f55ebc to
6f81277
Compare
krnowak
reviewed
Jul 31, 2023
| # Download official Flatcar extensions | ||
| # The enabled-sysext.conf file is read from /etc and /usr and contains one name per line, | ||
| # and when the name is prefixed with a "-" it means that the extension should be disabled if enabled by default in the file from /usr. | ||
| # It may contain comments starting with "#" at the beginning of a line or after a name. |
There was a problem hiding this comment.
Again, I'd avoid having comments after a name, no point in complicating it like that.
| # It may contain comments starting with "#" at the beginning of a line or after a name. | ||
| # The file is also used in bootengine to know which extensions to enable. | ||
| # Note that we don't need "{ grep || true ; }" to suppress the match return code because in for _ in $(grep...) return codes are ignored | ||
| for NAME in $(grep -h -o '^[^#]*' /etc/flatcar/enabled-sysext.conf /usr/share/flatcar/enabled-sysext.conf | grep -v -x -f <(grep '^-' /etc/flatcar/enabled-sysext.conf | cut -d - -f 2-) | grep -v -P '^(-).*'); do |
There was a problem hiding this comment.
Probably could simplify this if we make the file format simpler, see my comment in the init PR.
pothos
force-pushed
the
kai/postinst-sysext
branch
from
17f1c54 to
91f4176
Compare
In the past it was tried to support old installations by copying the libs over that will be missing. We stopped doing that and instead recommended reprovisioning or a helper update script. In the next weeks we'll officially support an update path for OEM software. Remove the old workaround that at the moment isn't used because it wasn't updated to work with newer versions.
The GRUB-patching logic worked for a certain set of old binaries and only when the /boot/coreos folder was present. Flatcar doesn't use the /boot/coreos folder anyway and this logic isn't required for newer GRUB versions as they don't suffer from the memory corruption reported in coreos/bugs#2400 Therefore, completely remove the GRUB patching logic.
The Docker 1.12 version required by old Tectonic clusters is long gone on Flatcar and the workaround to switch the node to the old Docker won't work anymore. Remove the Tectonic workaround.
The VMware OEM units shipped already have the right settings for long time and don't require the drop-in to be created. Remove the VMware OEM drop-in workaround.
The workaround for instances migrated from CoreOS shouldn't be needed anymore, especially if we are now migrating the OEM partition contents to the new setup with the OEM systemd-sysext image. Remove the old Azure workaround.
The old workaround to restart locksmithd isn't needed since a long time. Remove the old locksmithd workaround.
Flatcar doesn't ship the kernel image on /usr since 2021 and this anyway didn't support dm-verity setups. The old bootloader handling is anyway not needed for a long time now and also doesn't work without the kernel. Remove the broken kernel and bootloader handling.
We move the OEM mount to /oem but for old instances this isn't used at the time they process this update. Add a comment that we should keep it as is.
The deadline attribute was used for the Chrome/Chromium browser and is not used in Flatcar (directly or through the deadline file), nor is this properly documented in Omaha. Remove the parsing of the attribute and the creation of the deadline flag file in /tmp.
The moreinfo attribute is not used by Flatcar and is Mac-specific (see https://github.com/google/omaha/blob/main/doc/ServerProtocolV2.md). Remove the unused moreinfo attribute (for the web view URL shown by Google Software Update Agent on the Mac).
The prompt attribute is like MoreInfo Mac-specific and not used by Flatcar (see https://github.com/google/omaha/blob/main/doc/ServerProtocolV2.md). Remove the unused prompt attribute (on Mac it was there to display a prompt UI before installing an update).
The needsadmin attribute is not used by Flatcar and it also wouldn't have a use case in the future. Remove it from the code base to focus on what makes sense.
pothos
force-pushed
the
kai/postinst-sysext
branch
from
91f4176 to
8b8fc04
Compare
4 tasks
|
For testing it makes sense to also use bincache as fallback download location, I'll add that. |
pothos
force-pushed
the
kai/postinst-sysext
branch
3 times, most recently
from
84bdf5f to
bc708b4
Compare
pothos
commented
Sep 1, 2023
pothos
force-pushed
the
kai/postinst-sysext
branch
from
a16a941 to
ea6a398
Compare
The OEM software and future official Flatcar extensions will be shipped as systemd-sysext images, coupled to the Flatcar version and A/B updated. Instead of adding support for this in the update-engine C++ code the plan was to use a new helper binary in the post-inst action for downloading of the payload. Since this is not ready we use curl and a small script to decode the payload. The A/B update mechanism includes a migration path for old instances that first need a fallback download, because the old update-engine client doesn't pass the XML dump, then they need to go through another update cycle to have systemd-sysext images for both partitions, and then the migration can take place in the initrd where the old OEM contents get cleaned up.
pothos
force-pushed
the
kai/postinst-sysext
branch
from
ea6a398 to
7b59d69
Compare
Since there can be multiple signatures, we should look at all and not only the first two.
2 tasks
krnowak
approved these changes
Sep 8, 2023
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The OEM software and future official Flatcar extensions will be shipped
as systemd-sysext images, coupled to the Flatcar version and A/B
updated. Instead of adding support for this in the update-engine C++
code the plan was to use a new helper binary in the post-inst action
for downloading of the payload. Since this is not ready we use curl and
a small script to decode the payload. The A/B update mechanism includes
a migration path for old instances that first need a fallback download,
because the old update-engine client doesn't pass the XML dump, then
they need to go through another update cycle to have systemd-sysext
images for both partitions, and then the migration can take place in
the initrd where the old OEM contents get cleaned up.
The PR also contains cleanups to remove unused stuff.
How to use
Testing done
Manual: With the image built from the scripts PR I ran an update from
initialsysext to versioned sysext. I also had removed theactive-sysextflag to simulate a migration from an old instance. Theinitialsysext got removed when both partitions had the version of the versioned sysext.Command to update:
sudo ./flatcar-update -V 1.2.3 -P flatcar_test_update.gz -E flatcar_test_update-oem-qemu.gzThis can be done from a Stable image as well as a new image from the scripts PR. In the first case the fallback download from bincache is done because the old update-engines wont't pass the XML to the postinstall action.
Also tested with mantle kola: flatcar/mantle#448