qemu: Facilitate testing Heads with persistent VMs

Create a persistent VM with the proper configuration using virsh
Provide instructions for bootstrapping a complete working system in qemu

Signed-off-by: Jonathon Hall <[email protected]>

README.md

@@ -70,12 +70,83 @@ Notes:
 * Building coreboot's cross compilers can take a while.  Luckily this is only done once.
 * Builds are finally reproducible! The [reproduciblebuilds tag](https://github.com/osresearch/heads/issues?q=is%3Aopen+is%3Aissue+milestone%3Areproduciblebuilds) tracks any regressions.
 * Currently only tested in QEMU, the Thinkpad x230, Librem series and the Chell Chromebook.
-** Xen and the TPM do not work in QEMU, so it is only for testing the `initrd` image.
+** Xen does not work in QEMU.  Signing, HOTP, and TOTP do work; see below.
 * Building for the Lenovo X220 requires binary blobs to be placed in the blobs/x220/ folder.
 See the readme.md file in that folder
 * Building for the Librem 13 v2/v3 or Librem 15 v3/v4 requires binary blobs to be placed in
 the blobs/librem_skl folder. See the readme.md file in that folder
 
+QEMU:
+---
+
+OS booting can be tested in QEMU using a software TPM.  HOTP can be tested by forwarding a USB token from the host to the guest.
+
+GPG keys can't be added within Heads since it cannot reflash the firmware under QEMU.  Instead, a public key can be injected during the build by setting `PUBKEY_ASC` to the path to a public key file.
+
+The `provision` target builds Heads and creates (or updates) a libvirt VM configured to use the built firmware and a software TPM.  A USB token can be forwarded from the host, and a disk can be attached to boot an OS - use virt-manager or another libvirt tool, then boot the VM normally.
+
+Bootstrapping a fully functioning VM from scratch involves several steps:
+
+1. Install QEMU, KVM, libvirt, virsh, virt-manager, swtpm, and xmlstarlet.
+   * Many distributions already package swtpm, but Debian Bullseye does not.  (Bookworm does.)  On Bullseye you will have to build and install libtpms and swtpm from source, and you will need to create AppArmor rules manually - see below.
+     * https://github.com/stefanberger/libtpms
+     * https://github.com/stefanberger/swtpm
+2. Build Heads and provision the VM:
+   * `make BOARD=qemu-coreboot-fbwhiptail provision`
+3. Open virt-manager and find the new VM, `heads-qemu-coreboot-fbwhiptail`:
+   * Add a SATA hard disk (not virtio), create a new image
+   * Attach your OS installation image as a USB disk (not CD-ROM)
+   * To use HOTP, forward a USB token to the guest as a USB host device
+     * We will replace any GPG key / HOTP secret that's already on the token, do not use one that you need for a physical device
+4. Boot the VM and install your OS (select Boot from USB, then select the installer), then shut down
+5. Create an image file on the host to act as a virtual USB flash drive:
+   * `sudo dd if=/dev/zero of=/var/lib/libvirt/images/fd_heads_gpg.raw bs=1M count=128`
+   * `sudo kpartx -av /var/lib/libvirt/images/fd_heads_gpg.raw` (use resulting `loop#p1` below)
+   * `sudo mkfs.vfat /dev/mapper/loop#p1`
+   * `sudo kpartx -d /var/lib/libvirt/images/fd_heads_gpg.raw`
+6. Remove the installer image, then attach the virtual USB flash drive image as a USB disk to the VM.
+7. Boot Heads, and proceed with the OEM reset (it will fail when flashing the firmware, which we will handle)
+   * You _do_ need to export the GPG key to a USB disk, otherwise defaults are fine.
+   * Head will show an error saying it can't flash the firmware, continue
+   * Then Heads will indicate that there is no TOTP code yet, at this point shut down (Continue to main menu -> Power off)
+8. Get the public key that was saved to the virtual USB flash drive
+   * `sudo kpartx -av /var/lib/libvirt/images/fd_heads_gpg.raw` (use resulting `loop#p1` below)
+   * `sudo mkdir /media/fd_heads_gpg`
+   * `sudo mount /dev/mapper/loop#p1 /media/fd_heads_gpg`
+   * Look in `/media/fd_heads_gpg` and copy the most recent public key
+   * `sudo umount /media/fd_heads_gpg`
+   * `sudo kpartx -d /var/lib/libvirt/images/fd_heads_gpg.raw`
+9. Inject the GPG key into the Heads image and update the VM
+   * `make BOARD=qemu-coreboot-fbwhiptail PUBKEY_ASC=<key>.asc provision`
+10. Boot the VM again and initialize the TPM - select "Reset the TPM" at the TOTP error prompt
+11. Sign files in /boot for the first time - go to "Options -> Update checksums and sign all files in /boot"
+12. Finally, select "Default boot" and boot the OS
+
+swtpm on Debian Bullseye
+---
+
+libtpms and swtpm must be built and installed from source on Debian Bullseye, and AppArmor must be tweaked to allow libvirtd to execute them.
+
+1. Install dependencies
+   * `sudo apt install automake autoconf libtool make gcc libc-dev libssl-dev dh-autoreconf libssl-dev libtasn1-6-dev pkg-config libtpms-dev net-tools iproute2 libjson-glib-dev libgnutls28-dev expect gawk socat gnutls-bin libseccomp-dev libfuse-dev python3-twisted selinux-policy-dev trousers devscripts equivs`
+2. Build libtpms
+   * `git clone https://github.com/stefanberger/libtmps`
+   * `cd libtpms; git checkout v0.9.4` (latest release as of this writing)
+   * `./autogen.sh --with-tpm2 --with-openssl`
+   * `make all`
+   * `sudo make install`
+3. Build swtpm
+   * `git clone https://github.com/stefanberger/swtpm`
+   * `cd swtpm; git checkout v0.7.3` (latest release as of this writing)
+   * `./autogen.sh --with-openssl`
+   * `make -j#` (specify number of CPUs for #)
+   * `make -j# check`
+   * `sudo make install`
+4. Permit execution from /usr/local/bin in AppArmor
+   * `echo '/usr/local/bin/* PUx,' | sudo tee -a /etc/apparmor.d/local/usr.sbin.libvirtd`
+   * `sudo apparmor_parser -r /etc/apparmor.d/usr.sbin.libvirtd` (reload the profile that included the above override)
+   * `echo '/usr/local/bin/* PUx,' | sudo tee -a /etc/apparmor.d/local/abstractions/libvirt-qemu` (this will be included by the VM's profile, so there is nothing to reload if you haven't created the VM yet)
+
 coreboot console messages
 ---
 The coreboot console messages are stored in the CBMEM region

boards/qemu-coreboot-fbwhiptail/heads-qemu-coreboot-fbwhiptail.xml

@@ -0,0 +1,38 @@
+<domain type='kvm'>
+  <name>heads-qemu-coreboot-fbwhiptail</name>
+  <memory unit='KiB'>6291456</memory>
+  <currentMemory unit='KiB'>6291456</currentMemory>
+  <vcpu placement='static'>2</vcpu>
+  <os>
+    <type arch='x86_64' machine='q35'>hvm</type>
+    <loader readonly='no' type='pflash'>{{HEADS_ROM_PATH}}</loader>
+  </os>
+  <features>
+    <acpi/>
+    <apic/>
+    <smm state="on"/>
+  </features>
+  <devices>
+    <emulator>/usr/bin/qemu-system-x86_64</emulator>
+    <interface type='network'>
+      <source network='default'/>
+      <model type='virtio'/>
+    </interface>
+    <tpm model='tpm-tis'>
+      <backend type='emulator' version='1.2'/>
+      <alias name='tpm0'/>
+    </tpm>
+    <graphics type='spice' autoport='yes'>
+      <image compression='off'/>
+    </graphics>
+    <video>
+      <model type='bochs' heads='1' primary='yes'/>
+    </video>
+    <rng model='virtio'>
+      <backend model='random'>/dev/urandom</backend>
+      <alias name='rng0'/>
+      <address type='pci' domain='0x0000' bus='0x05' slot='0x00' function='0x0'/>
+    </rng>
+  </devices>
+</domain>
+

boards/qemu-coreboot-fbwhiptail/qemu-coreboot-fbwhiptail.config

@@ -67,3 +67,33 @@ run:
 		-netdev user,id=u1 -device e1000,netdev=u1 \
 		-serial stdio \
 	; stty sane
+
+# Provision a persistent VM using this firmware and a software TPM - create the
+# VM, or if it already exists, update the ROM configuration.  The machine can
+# then have a disk attached in order to boot an OS, and a USB token can be
+# forward into the guest for HOTP support.
+#
+# This VM is configured with libvirt rather than invoked ad-hoc, so that
+# libvirt will manage the swtpm invocation and persistent disks can be
+# configured easily.
+
+# Default for virsh is qemu:///session, but default for virt-manager is
+# qemu:///system
+LIBVIRT_CONNECTION ?= qemu:///system
+
+# Use the GPG-injected ROM if a key was given, since we can't reflash a GPG
+# keyring in QEMU.  Otherwise use the plain ROM, some things can still be tested
+# that way without a GPG key.
+ifneq "$(PUBKEY_ASC)" ""
+QEMU_BOOT_ROM := $(build)/$(BOARD)/$(CB_OUTPUT_FILE_GPG_INJ)
+else
+QEMU_BOOT_ROM := $(build)/$(BOARD)/$(CB_OUTPUT_FILE)
+endif
+
+provision: $(QEMU_BOOT_ROM)
+	if ! virsh -c "$(LIBVIRT_CONNECTION)" domuuid heads-qemu-coreboot-fbwhiptail 2>/dev/null; then \
+		virsh -c "$(LIBVIRT_CONNECTION)" define "boards/$(BOARD)/heads-qemu-coreboot-fbwhiptail.xml"; \
+	fi
+	virsh -c "$(LIBVIRT_CONNECTION)" dumpxml heads-qemu-coreboot-fbwhiptail \
+		| xmlstarlet ed --update /domain/os/loader --value "$(QEMU_BOOT_ROM)" \
+		| virsh -c "$(LIBVIRT_CONNECTION)" define /dev/stdin