Add support for replacing otacerts.zip in the system image

Previously, overriding otacerts.zip in the system partition required the
user to flash a Magisk/KernelSU module that would bind mount over the
file during boot. While this worked well enough, it's insufficient for
unrooted setups, which has become more important since unrooting is the
only safe way to use the new OEM repair mode feature. With the stock
otacerts.zip, the OEM's default OTA updater app could run and install an
OS upgrade that's not signed by the user's key.

With this key, the raw otacerts.zip bytes in the system partition are
directly replaced with a new zip that contains the user's certificate.
This method was inspired by @pascallj's comment in #216 suggestiing
intentionally corrupting the otacerts.zip data in the filesystem.

Because avbroot does not have filesystem parsers for ext4/f2fs/erofs, we
rely on a heuristic-based search on the raw filesystem image. The file
is always smaller than one block (which is at least 4096 bytes on all
known devices), so the file data is stored continguously on disk and in
the case of erofs, won't be compressed. None of the three filesystems
are copy-on-write and thus, have no filesystem-level data checksums. For
the dm-verity layer one level up, avbroot already knows how to recompute
the hash tree and FEC data.

Since the new approach is doing a raw search and replace, the old and
new files must have the same size. When the new zip is smaller, null
bytes are added to the zip archive comment field to pad to the correct
size. When the new zip is larger, avbroot will attempt the following to
try and make the file size smaller:

1. Enable zip deflate compression
2. Strip the X.509 signature from the certificate
3. Clear out the issuer RDN sequence from the certificate
4. Clear out the subject RDN sequence from the certificate

The latter three changes work because Android never performs any PKI
operations with the certificate. There is no CA certificate chain. The
X.509 certificate file is nothing more than a way to transport an RSA
public key.

avbroot requires the user's key to be RSA 4096. If the original zip had
the same key size, then none of these shrinking methods are needed. If
it contained an RSA 2048 key, then the first two modifications are
usually sufficient. The latter two modifications should only be needed
if the user picked a really long subject value when generating the
certificate.

With these new changes, the OTA patching time will approximately double
on a system with an SSD and modern CPU. This is dominated by the time it
takes to XZ-compress the system partition image. The compression is
already parallelized and scales linearly with the number of cores.
There's likely not much more that can be done to further speed this up.

Finally, these new changes are currently excluded from the e2e tests
because including the system partition in the stripped OTAs would
increase the file size by an order of magnitude. This could potentially
be solved in the future by generating our own small OTAs to use for
testing instead of running against real device OTAs.

Fixes: #225

Signed-off-by: Andrew Gunnerson <[email protected]>

README.md

@@ -19,12 +19,14 @@ Having a good understanding of how AVB and A/B OTAs work is recommended prior to
 
 ## Patches
 
-avbroot applies two patches to the boot images:
+avbroot applies the following patches to the partition images:
 
 * The `boot` or `init_boot` image, depending on device, is patched to enable root access. For Magisk, the patch is equivalent to what would be normally done by the Magisk app.
 
 * The `boot`, `recovery`, or `vendor_boot` image, depending on device, is patched to replace the OTA signature verification certificates with the custom OTA signing certificate. This allows future patched OTAs to be sideloaded from recovery mode after the bootloader has been locked. It also prevents accidental flashing of the original unpatched OTA.
 
+* The `system` or `my_engineering` image, depending on device, is also patched to replace the OTA signature verification certificates. This prevents the OS' system updater app from installing an unpatched OTA and also allows the use of custom OTA updater apps.
+
 ## Warnings and Caveats
 
 * **Always leave the `OEM unlocking` checkbox enabled when using a locked bootloader with root.** This is critically important. Root access allows the boot partition to potentially be overwritten, either accidentally or intentionally, with an image that is not properly signed. In this scenario, if the checkbox is turned off, both the OS and recovery mode will be made unbootable and `fastboot flashing unlock` will not be allowed. This effectively renders the device **_hard bricked_**.
@@ -218,22 +220,22 @@ To stop using avbroot and revert to the stock firmware:
 
 4. That's it! There are no other remnants to clean up.
 
-## avbroot modules
-
-avbroot's Magisk/KernelSU modules can be downloaded from the [releases page](https://github.com/chenxiaolong/avbroot/releases).
-
-### `clearotacerts`: Block OTA Updates from default updater app
+## OTA updates
 
-Unpatched OTA updates are already blocked when booted into recovery mode because the original OTA certificate has been replaced with the custom certificate. However, this doesn't prevent the Android's system updater app from attempting to install an unpatched OTA update.
+avbroot replaces `/system/etc/security/otacerts.zip` in both the system and recovery partitions with a new zip that contains the custom OTA signing certificate. This prevents an unpatched OTA from inadvertently being installed both when booted into Android and when sideloading from recovery.
 
-Disabling the system updater app is recommended. To do so:
+Disabling the system updater app is recommended to prevent it from even attempting to install an unpatched OTA. To do so:
 
 * Stock OS: Turn off `Automatic system updates` in Android's Developer Options.
 * Custom OS: Disable the system updater app (or block its network access) from Settings -> Apps -> See all apps -> (three-dot menu) -> Show system -> (find updater app).
 
-As an extra safety measure, flashing the `clearotacerts` module will intentionally make OTAs fail to install while booted into Android. It does so by overriding `/system/etc/security/otacerts.zip` with an empty zip containing no certificates so that even if an OTA is downloaded, signature verification will fail. This may cause some custom OS' system updater app to get stuck in an infinite loop downloading an OTA update and then retrying when signature verification fails, so make sure the system updater app is disabled.
+This is especially important for some custom OS's because their system updater app may get stuck in an infinite loop downloading an OTA update and then retrying when signature verification fails.
 
-As an alternative to this module, see [Custota](https://github.com/chenxiaolong/Custota) for a custom OTA updater app that installs updates from a self-hosted OTA server.
+To self-host a custom OTA server, see [Custota](https://github.com/chenxiaolong/Custota).
+
+## avbroot modules
+
+avbroot's Magisk/KernelSU modules can be downloaded from the [releases page](https://github.com/chenxiaolong/avbroot/releases).
 
 ### `oemunlockonboot`: Enable OEM unlocking on every boot
 

avbroot/Cargo.toml

@@ -11,6 +11,7 @@ publish = false
 [dependencies]
 anyhow = "1.0.75"
 base64 = "0.21.3"
+bitflags = "2.4.1"
 bstr = "1.6.2"
 byteorder = "1.4.3"
 cap-std = "2.0.0"

avbroot/src/cli/avb.rs

@@ -195,7 +195,7 @@ fn write_raw_and_update(
         AppendedDescriptorMut::HashTree(d) => {
             d.hash_algorithm = promote_insecure_hash_algorithm(&d.hash_algorithm).to_owned();
             d.image_size = image_size;
-            d.update(&raw_file, &raw_file, cancel_signal)
+            d.update(&raw_file, &raw_file, None, cancel_signal)
                 .context("Failed to update hash tree descriptor")?;
         }
         AppendedDescriptorMut::Hash(d) => {
@@ -642,7 +642,7 @@ fn repack_subcommand(cli: &RepackCli, cancel_signal: &AtomicBool) -> Result<()>
         // There could have been errors in the original FEC data itself.
         if let AppendedDescriptorMut::HashTree(d) = info.header.appended_descriptor_mut()? {
             d.hash_algorithm = promote_insecure_hash_algorithm(&d.hash_algorithm).to_owned();
-            d.update(&file, &file, cancel_signal)?;
+            d.update(&file, &file, None, cancel_signal)?;
         }
 
         update_dm_verity_cmdline(&mut info)?;