build(d1): automate building D1 oreboot SD card images

[hawkw]

there are some instructions in https://github.com/tosc-rs/mnemos/blob/main/platforms/allwinner-d1/README.md explaining how to make a bootable SD card for the D1 using MnemOS + Oreboot. currently, this is accomplished by using dd to write the oreboot and mnemOS binaries directly to the SD device.

it would be cool if we could add some automation for producing a bootable ISO image or similar that can then be written to a SD card, similar to how our build process can produce x86_64 bootable images. ideally, this would:

• either be an xtask-style tool for building images, or a separate crate that outputs the image as a build target (the way we build x86 images (
• depend on Oreboot as a cargo artifact dep (if possible?)

[hawkw]

started on this; i was hoping we could do something similar to the mnemos-x86_64-bootloader crate and have a bindep on the actual kernel binary, but this unfortunately runs into some cargo bugs, so we might need to go with an xtask approach instead.

[hawkw]

also, we probably need to download prebuilt oreboot artifacts from GitHub Releases or something — we can't depend on oreboot as a bin dep, because it needs post-build processing, which I believe oreboot does with an xtask.

[hawkw]

cc @jspngh, you might be interested in this as well?

[jspngh]

Yes, indeed!
I think the easiest way of getting the bootloader is downloading a prebuilt binary from somewhere, for the reasons you mentioned.
I should be able to add releases to my oreboot fork to make it available there.
Or maybe it's possible to create a pipeline that builds on every push and to always take the latest artifact?

I'm not too far away from reading data from SD cards in mnemos itself, so it would be nice to have a mnemos-d1-bootlader.
But the first stage bootloader needs to fit in 32K, which may not be feasible looking at the current size of the D1 binaries.

Anyway, having a self-contained iso file that can be used like usual (just dd to SD card without extra argument) would be awesome.
What I would put in there is:

• FAT or other FS header
• bootloader
• mnemos
• FS partition(s)

Then when a user plugs in the SD card into the computer, they can work with the filesystem (e.g., add forth scripts?).

[hawkw]

@jspngh oh, that's fantastic, i didn't realize you were that far along on reading from the SD card!

the eventual goal is definitely to be able to build an image with a filesystem and put some user mode programs on there as well (maybe a forth program launcher/task browser, and/or an init system type thing). so, a way to provide other files you want on the fs is probably the next step once we have a nice way to build a bootable ISO!

[hawkw]

I'm not too far away from reading data from SD cards in mnemos itself, so it would be nice to have a mnemos-d1-bootlader.
But the first stage bootloader needs to fit in 32K, which may not be feasible looking at the current size of the D1 binaries.

i think we could probably produce a significantly smaller mnemos binary, with a little work. the D1 kernel binaries aren't built with opt-level='s', for one thing, and they include a bunch of code we wouldn't need in a first stage, like the TWI, i2c_puppet, and keyboard drivers. we could probably statically disable the more verbose tracing diagnostics in a first stage build as well.

if we can't jam a size-optimized kernel with most of that stuff disabled into 32k, though, we could also consider writing a separate bootloader that doesn't use the actual mnemos kernel. the async runtime probably results in generating a lot of code. since a stage 0 wouldn't be running arbitrary user tasks (it's just doing DRAM initialization, loading the kernel from SD, and maybe printing stuff to the UART for debugging), we could maybe use some of the existing platform HAL stuff we've written but leave behind the async executor, timer wheel, and other big stuff like that.

but, for now, oreboot works fine too, so my first pass at a boot image builder will just use your oreboot fork.

[jspngh]

I've added a release, we should be able to start using this URL
https://github.com/jspngh/oreboot/releases/download/v0.1.0-mnemos/oreboot-nezha-bt0.bin

If you have any suggestions for improvement, I'm happy to hear it.

if we can't jam a size-optimized kernel with most of that stuff disabled into 32k, though, we could also consider writing a separate bootloader that doesn't use the actual mnemos kernel. the async runtime probably results in generating a lot of code. since a stage 0 wouldn't be running arbitrary user tasks (it's just doing DRAM initialization, loading the kernel from SD, and maybe printing stuff to the UART for debugging), we could maybe use some of the existing platform HAL stuff we've written but leave behind the async executor, timer wheel, and other big stuff like that.

That would be neat, especially if we could use the existing drivers. I don't know how it would work without an async executor though, since the drivers also contain async functions?

but, for now, oreboot works fine too, so my first pass at a boot image builder will just use your oreboot fork.

Yep, that sounds like the easiest way forward.

[jspngh]

I've added a release, we should be able to start using this URL
https://github.com/jspngh/oreboot/releases/download/v0.1.0-mnemos/oreboot-nezha-bt0.bin

Now we can also get rid of the oreboot-bt0.tar.gz in the repo root,
I think you may have accidentally committed it with some other files 😁

[hawkw]

I've added a release, we should be able to start using this URL
https://github.com/jspngh/oreboot/releases/download/v0.1.0-mnemos/oreboot-nezha-bt0.bin

lovely, thank you! i haven't had much time to spend on the actual build script yet, but this will be very helpful!