secure boot: lockdown, grub fallback, CI

[jepio]

secure boot: lockdown, grub fallback, CI

• import secureboot lockdown kernel patches. These are required by the upstream shim community. The alternative is to either always force lockdown (bad - no unsigned kernel modules) or do so through a kernel cmdline parameter in grub (frowned upon and harder to fix later). I took and rebased the patches from debian: https://sources.debian.org/src/linux/6.6.13-1~bpo12%2B1/debian/patches/features/all/lockdown/.
• enable grub fallback. this can be triggered if the kernel signature fails verification, and will go through gptprio selection and eventually fallback to the previous slot. fallback only supports numerical menuentry ids (source: https://github.com/coreos/grub/blob/2.02-coreos/grub-core/normal/menu.c#L350).
• CI: changes from @sayanchowdhury for use with support testing secureboot in Qemu mantle#556

How to use

[ describe what reviewers need to do in order to validate this PR ]

Testing done

Tested here:
amd64: http://jenkins.infra.kinvolk.io:8080/job/container/job/test/26834/cldsv/
arm64: http://jenkins.infra.kinvolk.io:8080/job/container/job/test/26780/cldsv/

• Changelog entries added in the respective changelog/ directory (user-facing change, bug fix, security fix, update)
• Inspected CI output for image differences: /boot and /usr size, packages, list files for any missing binaries, kernel modules, config files, kernel modules, etc.

[chewi]

LGTM besides concerns already raised. I'm surprised these patches aren't mainline already.

[ader1990]

As there is a work in progress and if the desire is to move to a new kernel soon-ish (#2300), I would suggest to have these patches only for the 6.10 or the new LTS 6.12, and to not have to maintain them for 6.6 too (less work if we maintain only for very up-to-date Linux versions). And maybe in the meantime find a way to upstream / gather some support for mainline.

[github-actions]

Build action triggered: https://github.com/flatcar/scripts/actions/runs/10902608332

[jepio]

LGTM besides concerns already raised. I'm surprised these patches aren't mainline already.

They will never be mainline. You can see the full discussion here: https://lore.kernel.org/lkml/[email protected]/.

As there is a work in progress and if the desire is to move to a new kernel soon-ish (#2300), I would suggest to have these patches only for the 6.10 or the new LTS 6.12, and to not have to maintain them for 6.6 too (less work if we maintain only for very up-to-date Linux versions). And maybe in the meantime find a way to upstream / gather some support for mainline.

The aim is to submit a shim for signing in october so we can't risk it by coupling the required changes to other ones like the kernel upgrade. If we have to maintain patches on two kernel branches then so be it.

[tormath1]

The indentation seems off?

[jepio]

you're right, fixed it

[jepio]

Jenkins CI ran successfully here: http://jenkins.infra.kinvolk.io:8080/job/container/job/packages_all_arches/4670/cldsv/
GHA CI failed, pushed a fix and rerunning.

[sayanchowdhury]

LGTM

[jepio]

@ader1990 are you OK with merging?

[ader1990]

@ader1990 are you OK with merging?

I suppose I am okay with merging, but I still think it is not an optimal way forward.

Because having this unreviewed (and soon rebased, conflicts need to be taken care of, etc) patches affect Flatcar images that will never be used in a secure boot environment, thus adding more possible security vulnerabilities. I suggested to have a more one-step at a time approach, activate this patch just for the OEMs that we know need the secure boot implementation now, see how it goes for a while, and then activate the patches for all the images.

I am saying this because some of the code from the patches executes independently of the kernel config option is set or not, which opens the possibility for unwanted behaviours.
If the patches were just an if check_kernel_config: do something, it would have been great. But it is not the case.

[jepio]

@ader1990 are you OK with merging?

I suppose I am okay with merging, but I still think it is not an optimal way forward.

Because having this unreviewed (and soon rebased, conflicts need to be taken care of, etc) patches affect Flatcar images that will never be used in a secure boot environment, thus adding more possible security vulnerabilities. I suggested to have a more one-step at a time approach, activate this patch just for the OEMs that we know need the secure boot implementation now, see how it goes for a while, and then activate the patches for all the images.

I am saying this because some of the code from the patches executes independently of the kernel config option is set or not, which opens the possibility for unwanted behaviours. If the patches were just an if check_kernel_config: do something, it would have been great. But it is not the case.

The patches don't do anything unless secure boot is enabled at runtime (the switch checks whether firmware set SECUREBOOT=1)

 		case efi_secureboot_mode_enabled:
 			set_bit(EFI_SECURE_BOOT, &efi.flags);
+#ifdef CONFIG_LOCK_DOWN_IN_EFI_SECURE_BOOT
+			lock_kernel_down("EFI Secure Boot",
+					 LOCKDOWN_INTEGRITY_MAX);
+#endif

So lockdown is only triggered in a secure boot environment.

We can't ship a different kernel configuration for different OEMs because that would require preparing separate update payloads and nebraska can't handle that. These patches are part of standard Ubuntu, Debian, Fedora, RHEL, SuSe kernels so they have been well tested.

I would also rather not have them.

[ader1990]

@ader1990 are you OK with merging?

I suppose I am okay with merging, but I still think it is not an optimal way forward.
Because having this unreviewed (and soon rebased, conflicts need to be taken care of, etc) patches affect Flatcar images that will never be used in a secure boot environment, thus adding more possible security vulnerabilities. I suggested to have a more one-step at a time approach, activate this patch just for the OEMs that we know need the secure boot implementation now, see how it goes for a while, and then activate the patches for all the images.
I am saying this because some of the code from the patches executes independently of the kernel config option is set or not, which opens the possibility for unwanted behaviours. If the patches were just an if check_kernel_config: do something, it would have been great. But it is not the case.

The patches don't do anything unless secure boot is enabled at runtime (the switch checks whether firmware set SECUREBOOT=1)

 		case efi_secureboot_mode_enabled:
 			set_bit(EFI_SECURE_BOOT, &efi.flags);
+#ifdef CONFIG_LOCK_DOWN_IN_EFI_SECURE_BOOT
+			lock_kernel_down("EFI Secure Boot",
+					 LOCKDOWN_INTEGRITY_MAX);
+#endif

So lockdown is only triggered in a secure boot environment.

We can't ship a different kernel configuration for different OEMs because that would require preparing separate update payloads and nebraska can't handle that. These patches are part of standard Ubuntu, Debian, Fedora, RHEL, SuSe kernels so they have been well tested.

I would also rather not have them.

efi_get_fdt_params is changed, also some small other places in the process. This code change runs irrespective of the secureboot kernel config.

If other operating systems use the same byte-wise patches, I suppose it is a smaller risk. Let's hope for the best.

I see, for example, that 6.10 also has the same level of support for the patches: https://sources.debian.org/data/main/l/linux/6.10.6-1~bpo12+1/debian/patches/features/all/lockdown/, which is great, as we can copy those over byte-wise. Maybe we could add a small .github ci automation to get these patches automatically on a kernel change, like we do with the gentoo upstream - to have the same source of truth Flatcar and Debian.

[ader1990]

Leaving this here for future usage and reference:

https://sources.debian.org/data/main/l/linux/6.6.13-1~bpo12%2B1/debian/patches/features/all/lockdown/efi-lock-down-the-kernel-if-booted-in-secure-boot-mo.patch vs https://sources.debian.org/data/main/l/linux/6.10.6-1~bpo12+1/debian/patches/features/all/lockdown/efi-lock-down-the-kernel-if-booted-in-secure-boot-mo.patch

Seems that the rebasing work is already done, although it lags on a few patch releases.

For: #2300