From 17b866582aff3f70f8b402ee7d49e595d8d47119 Mon Sep 17 00:00:00 2001 From: Marc Herbert Date: Wed, 2 Nov 2022 23:29:36 +0000 Subject: [PATCH 1/2] cmake: when stripping zephyr.elf, checksum the reproducible output Temporary bugs, corner cases and obsolete toolchains aside, the Zephyr build is most of the time reproducible: #50205 and #14593. This means two different build machines using the same toolchain will always produce the same binary output. The one-line addition in this commit makes it trivial to verify that binary outputs are indeed the same by adding a single checksum line in the build logs: ``` [16/16] Linking C executable zephyr/zephyr.elf Memory region Used Size Region Size %age Used RAM: 53280 B 3 MB 1.69% IDT_LIST: 0 GB 2 KB 0.00% fdd2ddf2ad7d5da5bbd79b41cef...7b16ef549a8281111d8e205 zephyr.strip ``` This commit makes a non-measurable build time difference. Build reproducibility matters for (at least) two important reasons: - Security / supply chain attacks, see https://www.cisa.gov/sbom, #50205, https://reproducible-builds.org/ and many others. - Making sure build configurations are strictly identical when trying to reproduce elusive issues or when issuing releases. Displaying a reproducible checksum accelerates the investigation of temporary reproducibility issues like #48195. Signed-off-by: Marc Herbert --- CMakeLists.txt | 1 + 1 file changed, 1 insertion(+) diff --git a/CMakeLists.txt b/CMakeLists.txt index d31ad89ba66f45..9014b4fdc586a6 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -1679,6 +1679,7 @@ if(CONFIG_BUILD_OUTPUT_STRIPPED) $${KERNEL_ELF_NAME} $${KERNEL_STRIP_NAME} $ + COMMAND ${CMAKE_COMMAND} -E sha256sum ${KERNEL_STRIP_NAME} ) list(APPEND post_build_byproducts From 52a4154cb1a97b4b0bcd99412ce830ac8d84a7e1 Mon Sep 17 00:00:00 2001 From: Marc Herbert Date: Thu, 27 Oct 2022 21:17:08 +0000 Subject: [PATCH 2/2] cmake: compute and display the reproducible checksum by default Temporary bugs, corner cases and obsolete toolchains aside, the Zephyr build is reproducible most of the time: #50205 and #14593 This means two different build machines using the same toolchain will always produce the same binary output. The previous, one-line commit made it trivial to verify that binary outputs are indeed the same by adding this single line in the buid logs: ``` [16/16] Linking C executable zephyr/zephyr.elf Memory region Used Size Region Size %age Used RAM: 53280 B 3 MB 1.69% IDT_LIST: 0 GB 2 KB 0.00% fdd2ddf2ad7d5da5bbd79b41cef8d7...1a896b989a8281111d8e205 zephyr.strip ``` This commit enables that feature by default because build reproducibility matters for (at least) two important reasons: - Security / supply chain attacks, see https://www.cisa.gov/sbom, #50205, https://reproducible-builds.org/ and many others. - Making sure build configurations are strictly identical when trying to reproduce elusive issues or when issuing releases. It was of course already possible to _manually_ make this Kconfig change and manually compute this checksum. However this can be impossible when dealing with an automated build system that does not archive all _intermediate_ (#5009) files like `zephyr.elf`. Tweaking the build configuration can also be difficult and error-prone for people who are not Zephyr developers. Most automated CI systems preserve build logs by default. Displaying the reproducible checksum by default accelerates the discovery of reproducibility bugs like #48195. When measured with `west build -p -b qemu_x86 samples/hello_world/`, the additional `build/zephyr/zephyr.strip` disk space required is 43 kilobytes compared to a total of 11 Megabytes. Measuring a more realistic SOF example, `zephyr.strip` weighed 690 kb which was about 0.1% of a total `build/` directory weighing 65M. To measure the build time cost I ran `west build -p -b qemu_x86 samples/hello_world/` many times in a loop with and without this PR on my Linux workstation. Stripping and checksumming made literally no time difference compared to the "noise" observed when building the same configuration. This is not surprising considering how small `zephyr.strip`: so the extra cost is most likely dominated by process creation and the total number of processes created during a Zephyr build dwarfs the few extra processes required by this feature. More surprisingly, I measured incremental builds by running `touch kernel/timer.c; west build ...` in a loop and I could not observe any visible time difference either. Signed-off-by: Marc Herbert --- Kconfig.zephyr | 1 + 1 file changed, 1 insertion(+) diff --git a/Kconfig.zephyr b/Kconfig.zephyr index e92db23d1fb230..f7acb131483d1d 100644 --- a/Kconfig.zephyr +++ b/Kconfig.zephyr @@ -575,6 +575,7 @@ endif # BUILD_OUTPUT_UF2 config BUILD_OUTPUT_STRIPPED bool "Build a stripped binary" + default y help Build a stripped binary zephyr/zephyr.strip in the build directory. The name of this file can be customized with CONFIG_KERNEL_BIN_NAME.