-
Notifications
You must be signed in to change notification settings - Fork 6.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
cmake creates too many build artifacts #5009
Comments
Just a clarification: ESP32 takes the striped ELF and turns it into a format that's specific to ESP32 (it's the format that the first stage bootloader, part of the SoC mask ROM, understands). The .bin file generated in the output directory is not used by the flasher tool; the flashing script will overwrite the BIN format with this proprietary format prior to flashing. |
Modifying the build scripts is pretty straightforward. Creating a user-friendly Kconfig is a bit more tricky, I'll give it a go later on unless someone has any ideas themselves. I imagine we might want a menu for the configuring the build system. Perhaps the menu "Compiler Options" could be sub-menu in this "Build system options" menu. |
@SebastianBoe can you submit this as a PR, I am looking into making this configurable using the board yaml files which will be able to specify which output binaries they need and will support. This will be done in a layered approach, so we can specify this for an architecture or a group of boards as well. |
Done: #5112 |
Not all boards require the various binary formats zephyr generates. So be selective based on the arch, SoC or board and only geenrate the binaries actually needed. Fixes zephyrproject-rtos#5009 Signed-off-by: Anas Nashif <[email protected]>
Not all boards require the various binary formats zephyr generates. So be selective based on the arch, SoC or board and only geenrate the binaries actually needed. Fixes #5009 Signed-off-by: Anas Nashif <[email protected]>
Temporary bugs, corner cases and obsolete toolchains aside, the Zephyr build is reproducible most of the time: zephyrproject-rtos#50205 and zephyrproject-rtos#14593 This means two different build machines using the same toolchain will always produce the same binary output. The previous, one-line commit made it trivial to verify that binary outputs are indeed the same by adding this single line in the buid logs: ``` [16/16] Linking C executable zephyr/zephyr.elf Memory region Used Size Region Size %age Used RAM: 53280 B 3 MB 1.69% IDT_LIST: 0 GB 2 KB 0.00% fdd2ddf2ad7d5da5bbd79b41cef8d7...1a896b989a8281111d8e205 zephyr.strip ``` This commit enables that feature by default because build reproducibility matters for (at least) two important reasons: - Security / supply chain attacks, see https://www.cisa.gov/sbom, zephyrproject-rtos#50205, https://reproducible-builds.org/ and many others. - Making sure build configurations are strictly identical when trying to reproduce elusive issues or when issuing releases. It was of course already possible to _manually_ make this Kconfig change and manually compute this checksum. However this can be impossible when dealing with an automated build system that does not archive all _intermediate_ (zephyrproject-rtos#5009) files like `zephyr.elf`. Tweaking the build configuration can also be difficult and error-prone for people who are not Zephyr developers. Most automated CI systems preserve build logs by default. Displaying the reproducible checksum by default accelerates the discovery of reproducibility bugs like zephyrproject-rtos#48195. When measured with `west build -p -b qemu_x86 samples/hello_world/`, the additional `build/zephyr/zephyr.strip` disk space required is 43 kilobytes compared to a total of 11 Megabytes. Measuring a more realistic SOF example, `zephyr.strip` weighed 690 kb which was about 0.1% of a total `build/` directory weighing 65M. To measure the build time cost I ran `west build -p -b qemu_x86 samples/hello_world/` many times in a loop with and without this PR on my Linux workstation. Stripping and checksumming made literally no time difference compared to the "noise" observed when building the same configuration. This is not surprising considering how small `zephyr.strip`: so the extra cost is most likely dominated by process creation and the total number of processes created during a Zephyr build dwarfs the few extra processes required by this feature. More surprisingly, I measured incremental builds by running `touch kernel/timer.c; west build ...` in a loop and I could not observe any visible time difference either. Signed-off-by: Marc Herbert <[email protected]>
As a baseline, any Zephyr build needs to produce a .elf file for the kernel.
Depending on the target hardware requirements, this .elf file may need to be additionally converted into another binary type to support device provisioning.
Examples:
What CMake seems to be doing is generating all these different output file types unnecessarily for all board targets.
What we need is at a board or SOC level, have some Kconfig which tells the build system what output artifact types are necessary for that board.
For targets like QEMU targets, no additional artifacts need to be created at all; the ELF is the final build product.
The scripts for "make flash" for our boards should provide clues on what particular build artifacts are needed for any given board.
The text was updated successfully, but these errors were encountered: