Skip to content

Commit

Permalink
Migrate disk image building to systemd-repart
Browse files Browse the repository at this point in the history
Let's switch to systemd-repart to build disk images. systemd-repart
allows building disk images declaratively. repart will assemble a
disk image according to a set of partition definition files, which
it will search for in a set of predefined locations or in the locations
provided via the --definitions option. Partition definition files are
defined in the usual systemd ini style and define a single partition
each. Many aspects of the partition can be configured, including its
size, type, and how it should be populated.

Of note should be the CopyFiles= setting that allows providing a set
of files/directories from which the partition should be populated. The
CopyFiles= setting takes a list of path pairs, where the first path in
a pair defines the source location, and the second path the destination
location in the partition. If combined with repart's --root or --image
options, the source path is resolved relative to the given root directory.

Along with switching to systemd-repart, we also rework how we build
images. Previously, we first provisioned the disk image with all its
partitions, then mounted the disk image and finally populated it with
contents. Now, we provision to a regular directory on the host filesystem
first, followed by creating the disk image with systemd-repart. The partition
definition files can be supplied via the --repart-directory option or
in the mkosi.repart/ directory. If not provided, a default configuration
consisting of an automatically sized root partition and an ESP partition of
256M is used.

repart's --root switch is used so that the partition definition files
don't need to encode the full path to the root directory of the image
we're building. For example, to copy the root directory to a root
partition, it's sufficient to specify "CopyFiles=/:/" in the definition
file.

If we're building non-bootable images, any ESP partition definitions
are excluded. To allow building UKIs with an embedded roothash cmdline
parameter, we run repart twice, first to populate all partitions except
the ESP/XBOOTLDR partitions, and again to populate the ESP/XBOOTLDR
partitions including any UKIs with embedded roothash cmdline parameter
(which we determined when running repart the first time).

Because we don't know up-front anymore where the ESP partition will be
mounted, all boot loader files are installed to /boot. So to populate
an ESP partition, you'd use "CopyFiles=/boot:/" in the partition
definition file of the ESP partition.

Also, since we don't know up-front anymore which filesystem we'll be
building for, we stop installing filesystem related packages by default.
Users will be required to add the necessary filesystem packages themselves.
Similarly, we also stop installing cryptsetup and device-mapper by default
since we don't know upfront anymore whether partitions will be verity
protected or encrypted.

By switching to systemd-repart, we also set ourselves up for building
images without needing root privileges or loop devices. systemd-repart
is fully capable (from v253 onwards) of building disk images without
needing root privileges or loop devices. After we switch to
systemd-repart, we'll only need root privileges to be able to run
systemd-nspawn (which should not be necessary in the future anymore).

This PR also removes all the option related to disk image building that
can now be specified in repart definition files instead. This includes:

- Format=gpt_xxx options are replaced with a single "disk" options.
Filesystem can now be specified with repart's Format= option
- Format=plain_squashfs (Can be reproduced by a single repart squashfs
root partition combined with SplitArtifacts=yes)
- Verity= (Replaced by repart's Verity= options)
- Encrypt= (Replaced by repart's Encrypt= option)
- RootSize=, HomeSize=, VarSize=, TmpSize=, ESPSize=, SwapSize=, SrvSize=
(Replaced by repart's size options)
- UsrOnly= (replaced with `CopyFiles=/:/usr` in a usr partition definition)
- OutputSplitRoot=, OutputSplitVerity=, (Replaced by repart's SplitName= option)
- OutputSplitKernel= (UKI is now always written to its own output file)
- GPTFirstLBA (Removed, no equivalent in repart)
- ReadOnly= (Replaced by repart's ReadOnly= option per partition)
- Minimize= (Replaced by repart's Minimize= option per partition)
- CompressFs= (No equivalent in repart, can be replicated by replacing mkfs.<fs>
in $PATH with a script that adds the necessary command line option)
- MkSquashfs= (Can be replaced with a script in $PATH that invokes
the correct binary)

We also remove the WithoutUnifiedKernelImages= switch as building unified
kernel images is trivial and fast these days.
  • Loading branch information
DaanDeMeyer committed Dec 21, 2022
1 parent 4ccef88 commit 5901cbb
Show file tree
Hide file tree
Showing 21 changed files with 750 additions and 3,761 deletions.
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
#!/bin/bash -eux
#!/bin/bash
# SPDX-License-Identifier: LGPL-2.1-or-later
set -eux

systemctl --failed --no-legend | tee /failed-services

Expand Down
34 changes: 22 additions & 12 deletions NEWS.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,28 @@
current behaviour.
- Add `--idmap` option to run `--systemd-nspawn` with ID mapping support. Defaults
to `True`. `--idmap=no` can be used to prevent usage of ID mapping.
- Migrated to systemd-repart. Many options are dropped in favor of specifying them directly
in repart partition definition files:
- Format=gpt_xxx options are replaced with a single "disk" options. Filesystem to use can now be specified with repart's Format= option
- Format=plain_squashfs (Can be reproduced by a single repart squashfs
root partition combined with SplitArtifacts=yes)
- Verity= (Replaced by repart's Verity= options)
- Encrypt= (Replaced by repart's Encrypt= option)
- RootSize=, HomeSize=, VarSize=, TmpSize=, ESPSize=, SwapSize=, SrvSize=
(Replaced by repart's size options)
- UsrOnly= (replaced with `CopyFiles=/:/usr` in a usr partition definition)
- OutputSplitRoot=, OutputSplitVerity=, (Replaced by repart's SplitName= option)
- OutputSplitKernel= (UKI is now always written to its own output file)
- GPTFirstLBA (Removed, no equivalent in repart)
- ReadOnly= (Replaced by repart's ReadOnly= option per partition)
- Minimize= (Replaced by repart's Minimize= option per partition)
- CompressFs= (No equivalent in repart, can be replicated by replacing mkfs.<fs>
in $PATH with a script that adds the necessary command line option)
- MkSquashfs= (Can be replaced with a script in $PATH that invokes
the correct binary)

We also remove the WithoutUnifiedKernelImages= switch as building unified
kernel images is trivial and fast these days.

## v14

Expand Down Expand Up @@ -72,18 +94,6 @@
you weren't relying on this file to configure any network interfaces other
than the tun/tap virtio-net interface created by mkosi when booting the image
in QEMU with the `--netdev` option. If you were relying on this config file
to configure other interfaces, you'll have to re-create it with the correct
match and a lower initial number in the filename to make sure
`systemd-networkd` will keep configuring your interface, e.g. via the
`mkosi.skeleton` or `mkosi.extra` trees or a `mkosi.postinst` script.
- The `kernel-install` script for building unified kernel images has been
removed. From v13 onwards, on systems using `kernel-install`, `mkosi` won't
automatically build new unified kernel images when a kernel is updated or
installed. To keep the old behavior, you can install the `kernel-install`
script manually via a skeleton tree; a copy can be found
[here](https://github.com/systemd/mkosi/blob/3798eb0c2ebcdf7dac207a559a3cb5a65cdb77b0/mkosi/resources/dracut_unified_kernel_install.sh).
- New `QemuKvm` option configures whether to use KVM when running `mkosi qemu`.
- `mkosi` will not default to the same OS release as the host system anymore
when the host system uses the same distribution as the image that's being
built. Instead, when no release is specified, mkosi will now always default
to the default version embedded in mkosi itself.
Expand Down
22 changes: 14 additions & 8 deletions action.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -24,25 +24,31 @@ runs:
e2fsprogs \
xfsprogs \
squashfs-tools \
btrfs-progs
btrfs-progs \
mtools
sudo pacman-key --init
sudo pacman-key --populate archlinux
# Try to eliminate "Failed to dissect image: Connection timed out" errors from nspawn by compiling
# systemd-nspawn from v251 from source.
- name: Update systemd-nspawn
- name: Update systemd
shell: bash
working-directory: ${{ github.action_path }}
run: |
echo "deb-src http://archive.ubuntu.com/ubuntu/ $(lsb_release -cs) main restricted universe multiverse" | sudo tee -a /etc/apt/sources.list
sudo apt-get update
sudo apt-get build-dep systemd
git clone https://github.com/systemd/systemd-stable --branch v251.2 --depth=1
meson systemd-stable/build systemd-stable
ninja -C systemd-stable/build systemd-nspawn
sudo ln -svf $PWD/systemd-stable/build/systemd-nspawn $(which systemd-nspawn)
sudo apt-get install libfdisk-dev
git clone https://github.com/systemd/systemd --depth=1
meson systemd/build systemd -Drepart=true -Defi=true
ninja -C systemd/build systemd-nspawn systemd-dissect systemd-repart bootctl
sudo ln -svf $PWD/systemd/build/systemd-nspawn /usr/bin/systemd-nspawn
sudo ln -svf $PWD/systemd/build/systemd-dissect /usr/bin/systemd-dissect
sudo ln -svf $PWD/systemd/build/systemd-repart /usr/bin/systemd-repart
sudo ln -svf $PWD/systemd/build/bootctl /usr/bin/bootctl
systemd-nspawn --version
systemd-dissect --version
systemd-repart --version
bootctl --version
- name: Install
shell: bash
Expand Down
Loading

0 comments on commit 5901cbb

Please sign in to comment.