Use fastboot flashall to flash partitions initially

[chenxiaolong]

fastboot flashall is identical to the fastboot update command used by the Pixel factory images, except it reads from a directory instead of a zip file. It knows how to flip between the fastboot and fastbootd modes without user intervention.

Fixes: #252

[chenxiaolong]

I'm pretty certain this works, but it'll probably be some time before I get the chance to test on my own device.

[AgentOak]

2. When setting things up for the first time, the device must already be running the correct OS.

Is this still the case when flashing all partitions?

[chenxiaolong]

Oh, good catch. Fixed!

[chenxiaolong]

So there are some complications:

1. The android-info.txt file is required. avbroot can easily generate a minimal file like:

   require board=<device codename>
   

   The device codename can be parsed from META-INF/com/android/metadata.pb.

2. fastboot flashall requires super_empty.img by default. On A/B devices, one or more super partitions are combined to form a big logical partition, and then that is split up into dynamic partitions, like system, product, and vendor. Every device I've seen only has a single super partition named super, but Android supports having multiple.

   The structure of these super partitions are defined in a super_empty.img file, which is usually shipped with the device's factory images (at least with Google Pixels). It contains all of the dynamic partitions, except that they all have size 0. This is because the actual contents of the dynamic partitions are stored in separate files, like system.img.

   For better efficiency, if the device only has a single super partition, fastboot flashall will take super_empty.img, combine it with all the individual partition images locally, and then flash the full super partition in one go.

   The problem is that avbroot cannot generate a proper super_empty.img. There simply isn't enough information inside an OTA to recreate it in a way that exactly matches the factory image. An OTA does not contain the list of super partitions and their exact sizes, which is required to generate super_empty.img.

   This can be worked around by using a fastboot-info.txt file, which overrides fastboot flashall's default behavior. We want to avoid the optimization of combining and flashing the full super image and instead, flash each individual dynamic partition individually. This can be done by creating a fastboot-info.txt file like this:

   version 1
   # Flash regular, non-dynamic partitions in bootloader's fastboot mode
   flash boot
   flash init_boot
   flash dtbo
   flash vendor_kernel_boot
   flash pvmfw
   flash vendor_boot
   flash --apply-vbmeta vbmeta
   flash vbmeta_system
   flash vbmeta_vendor
   # Reboot to fastbootd (recovery's fastboot mode)
   reboot fastboot
   # Flash dynamic partitions
   flash system
   flash system_dlkm
   flash system_ext
   flash product
   flash vendor
   flash vendor_dlkm
   # Wipe if fastboot's `-w` option was used
   if-wipe erase userdata
   if-wipe erase metadata

   This is a minimally documented fastboot feature, but it is well supported. In fact, starting with Android 15, the build system generates these files, so any Android OS developer flashing their test devices will be using this. I don't foresee this feature getting removed any time soon.

   This approach also fixes an additional problem: fastboot flashall only flashes well-known partitions by default (ie. those that AOSP/Google knows about). It normally silently skips weird OEM partitions, like OnePlus' my_bigball.img. When we generate a custom fastboot-info.txt, we fully control what will be flashed, avoiding this problem entirely.

[chenxiaolong]

This has been implemented, though I still don't have a device I can use to test this right now.

[AgentOak]

Does not appear to work on Pixel 4a (sunfish) with CalyxOS 5.3.1
avbroot binary from: https://github.com/chenxiaolong/avbroot/actions/runs/7735203244

avbroot ota patch + extract + generated info files

$ ll
total 1.5G
-rwxrwxrwx 1 user user 8.1M Feb  1 02:07 avbroot*
-rwxrwxrwx 1 user user  12M Feb  1 07:51 Magisk-v26.4.apk*
-rwxrwxrwx 1 user user 1.5G Feb  1 07:53 sunfish-ota_update-24503010.zip*
$ ./avbroot ota patch --input sunfish-ota_update-24503010.zip --key-avb ~/avbroot/avb.key --key-ota ~/avbroot/ota.key --cert-ota ~/avbroot/ota.crt --magisk Magisk-v26.4.apk --magisk-preinit-device metadata
[*] Replacing zip entry: META-INF/com/android/otacert
[*] Copying zip entry: apex_info.pb
[*] Copying zip entry: care_map.pb
[*] Patching zip entry: payload.bin
[*] Extracting from original payload: vbmeta
[*] Extracting from original payload: system
[*] Extracting from original payload: boot
[*] Extracting from original payload: vbmeta_system
[*] Patching boot images: boot
[*] Patching system image: system
[*] Patched otacerts.zip offsets in system: [323682304..323683938]
[*] Patching vbmeta images: vbmeta_system, vbmeta
[*] Compressing full image: vbmeta_system
[*] Compressing full image: vbmeta
[*] Compressing partial image: system: [323682304..323683938, 893005824..907166528, 907407296..907407360]
[*] Compressing full image: boot
[*] Generating new OTA payload
[*] Patching zip entry: payload_properties.txt
[*] Generating new OTA metadata
[*] Verifying metadata offsets
[*] Completed after 15.8s
$ ./avbroot ota extract --input sunfish-ota_update-24503010.zip.patched --directory extracted --all --fastboot
[*] Extracting from the payload: abl, aop, boot, devcfg, dtbo, hyp, keymaster, modem, product, qupfw, system, system_ext, tz, uefisecapp, vbmeta, vbmeta_system, vendor, xbl, xbl_config
$ ll extracted/
total 3.2G
-rwxrwxrwx 1 user user 1.0M Feb  1 08:09 abl.img*
-rwxrwxrwx 1 user user   22 Feb  1 08:09 android-info.txt*
-rwxrwxrwx 1 user user 164K Feb  1 08:09 aop.img*
-rwxrwxrwx 1 user user  64M Feb  1 08:09 boot.img*
-rwxrwxrwx 1 user user  44K Feb  1 08:09 devcfg.img*
-rwxrwxrwx 1 user user 8.0M Feb  1 08:09 dtbo.img*
-rwxrwxrwx 1 user user  471 Feb  1 08:09 fastboot-info.txt*
-rwxrwxrwx 1 user user 388K Feb  1 08:09 hyp.img*
-rwxrwxrwx 1 user user 248K Feb  1 08:09 keymaster.img*
-rwxrwxrwx 1 user user  71M Feb  1 08:09 modem.img*
-rwxrwxrwx 1 user user 1.3G Feb  1 08:09 product.img*
-rwxrwxrwx 1 user user  52K Feb  1 08:09 qupfw.img*
-rwxrwxrwx 1 user user 307M Feb  1 08:09 system_ext.img*
-rwxrwxrwx 1 user user 866M Feb  1 08:09 system.img*
-rwxrwxrwx 1 user user 2.0M Feb  1 08:09 tz.img*
-rwxrwxrwx 1 user user 124K Feb  1 08:09 uefisecapp.img*
-rwxrwxrwx 1 user user 8.0K Feb  1 08:09 vbmeta.img*
-rwxrwxrwx 1 user user 4.0K Feb  1 08:09 vbmeta_system.img*
-rwxrwxrwx 1 user user 623M Feb  1 08:09 vendor.img*
-rwxrwxrwx 1 user user  88K Feb  1 08:09 xbl_config.img*
-rwxrwxrwx 1 user user 3.3M Feb  1 08:09 xbl.img*
$ cat extracted/android-info.txt
require board=sunfish
$ cat extracted/fastboot-info.txt
# Generated by avbroot
version 1
# Flash non-dynamic partitions
flash abl
flash aop
flash boot
flash devcfg
flash dtbo
flash hyp
flash keymaster
flash modem
flash qupfw
flash tz
flash uefisecapp
flash --apply-vbmeta vbmeta
flash vbmeta_system
flash xbl
flash xbl_config
# Reboot to fastbootd
reboot fastboot
# Flash dynamic partitions
flash product
flash system
flash system_ext
flash vendor
# Wipe data when -w flag is used
if-wipe erase userdata
if-wipe erase metadata

$ fastboot --version
fastboot version 34.0.5-10900879
Installed as C:\Users\User\Programs\platform-tools\fastboot.exe
$ fastboot flashing unlock
OKAY [  0.231s]
Finished. Total time: 0.231s
$ cmd.exe
Microsoft Windows [Version 10.0.19045.3930]
(c) Microsoft Corporation. All rights reserved.

C:\Users\User\Workdir\Phone>set ANDROID_PRODUCT_OUT=extracted

C:\Users\User\Workdir\Phone>fastboot flashall
--------------------------------------------
Bootloader Version...: s5-0.5-10252351
Baseband Version.....: g7150-00112-230505-B-10075601
Serial Number........: xxxxxxxxxxxxxx
--------------------------------------------
Checking 'product'                                 OKAY [  0.069s]
Setting current slot to 'b'                        OKAY [  0.293s]
Sending 'abl_b' (1024 KB)                          OKAY [  0.147s]
Writing 'abl_b'                                    FAILED (remote: 'Not allowed to flash (abl_b)')
fastboot: error: Command failed

(Had to use cmd.exe because environment variables don't get passed to fastboot.exe from my WSL environment)

Even tried an additional reboot-bootloader before the flashall. fastboot screen reads:

Fastboot Mode
Product revision: sunfish MP1.0(ROW)
Bootloader Version: s5-0.5-10252351
Baseband Version: g7150-00112-230505-B-10075601
Serial Number: xxxxxxxxxxxxxx
Secure boot: PRODUCTION
[...]
Device state: unlocked
Boot slot: b
Enter reason: reboot bootloader

The stock factory ROMs flash the bootloader partitions like this.

fastboot flash bootloader bootloader-sunfish-s5-0.5-10252351.img
fastboot reboot-bootloader
sleep 5
fastboot flash radio radio-sunfish-g7150-00112-230505-b-10075601.img
fastboot reboot-bootloader
sleep 5

Perhaps fastboot never allows these partitions to be flashed individually?

[chenxiaolong]

Thank you very much for testing!

Perhaps fastboot never allows these partitions to be flashed individually?

Yeah, it appears so. The OTA update process flashes them separately, but that's subject to the A/B scheme and it will revert back to the old/good slot if needed. With fastboot, I suppose Google wants to prevent someone from flashing only part of the bootloader partitions to both slots and permanently bricking the device (eg. with fastboot flash --slot all abl abl.img).

I took a quick look at the bootloader-<device>-<version>.img file from the factory zip. It's unfortunately OEM-specific and the OTAs don't contain everything needed to recreate it.

---

I suppose there are a couple approaches we can take:

1. Skip adding bootloader partitions to fastboot-info.txt. If the user is using avbroot for the first time and is coming from an older Android build, then the bootloader would be out of date. The workaround would be to reboot to recovery and sideload the OTA after fastboot flashall.
2. Go back to the old/current approach of requiring that the user is already running the correct Android build. Then we only flash the changed partitions with fastboot flashall.

(For number 2, the only change to this PR would be adding back the "When setting things up for the first time, the device must already be running the correct OS." step and removing the --all parameter from the avbroot ota extract step.)

[AgentOak]

requiring that the user is already running the correct Android build

Can we reliably identify which partitions belong to bootloader/radio (maybe all those not listed in any vbmeta?) Perhaps we could flash all the ROM-partitions and only tell the user he has to make sure he has the bootloader and radio ("if in doubt, flash/install the entire ROM" as it is now) - pixel factory images already include a separate flash-base.sh script that specifically flashes only bootloader and radio.

[chenxiaolong]

Yeah, I think that is reasonable. I've updated the PR to implement that and update the README. Only the OS partitions will be added to fastboot-info.txt now. To get the list of OS partitions, avbroot will loop through all the extracted .img files and find the ones that contain an AVB header.

(Scanning all the vbmeta images would work everywhere aside from OnePlus devices. Half of their partitions use the "alternative" AVB method where they are verifed against a public key stored on disk instead of a vbmeta partition.)

[chenxiaolong]

Finally got a chance to give it a try on my device. This should be good to go now.