Flatcar transient issues on older hardware

[ader1990]

Description

Opening this issue thread to gather community experience with Flatcar on older hardware.

Impact

Flatcar is not usable out of the box on older hardware, especially when Kubernetes with systemd-sysext is used.
Known old hardware where this happens: Dell servers from 2011-2012.

Environment and steps to reproduce

• Deploy Flatcar on older hardware, deploy Kubernetes using the systemd-sysext.
• Reboot machine

Issues seen:

• sometimes, the systemd-sysext refresh is not working (tbd: logs to be shared), in the sense that the kubectl/containerd is not showing up, because the k8s sysext was not applied.
• sometimes, the kubelet starts after the systemd-hostnamed/systemd-resolved sets the hostname, and kubelet cannot register the hostname localhost to its own node.

Expected behavior

Flatcar to be usable on older hardware.

Next steps

I propose adding Mantle tests for k8s and systemd-sysexts that have reboots, to make sure that there is no issue from a fundamental workflow / systemd units workflow / ordering perspective.
TBD: Hyper-V has a nice feature that I might be able to use to properly reproduce these scenarios, called disk IOPS, where you can limit the disk IO to reproduce old slower hardware.