Add support for e.g. /usr/lib/bootc/kargs.d or equivalent

[oglok]

(Edited by @cgwalters )

This issue is basically the bootc equivalent of ostreedev/ostree#2217
which has a lot of history.

I think it makes sense to revisit this in a container-native flow. Having a drop-in dir for kernel arguments that can be applied alongside kernel updates would make sense to me.

---

Original issue text:

rpm-ostree kargs --append='audit=0' is not working:

podman build -t quay.io/flightctl/flightctl-agent:latest -f packaging/Containerfile.fedora ./
STEP 1/13: FROM quay.io/centos-bootc/fedora-bootc:eln
STEP 2/13: COPY rpmbuild/RPMS/x86_64/flightctl-agent-0.0.1-1.el9.x86_64.rpm /tmp/
--> Using cache d6c77d8351a0e8777c7786cf4218fd16f120ec240b778a00bfa69eb93106db99
--> d6c77d8351a0
STEP 3/13: COPY packaging/flightctl-custom-assets/flightctl_rsa.pub /usr/etc-system/root.keys
--> Using cache e9d47ae7f9756a2a45a3a803f5817599f0a4c8f8d166fade365bfd5e06d3669f
--> e9d47ae7f975
STEP 4/13: RUN touch /etc/ssh/sshd_config.d/30-auth-system.conf;     mkdir -p /usr/etc-system/;     echo 'AuthorizedKeysFile /usr/etc-system/%u.keys' >> /etc/ssh/sshd_config.d/30-auth-system.conf;     chmod 0600 /usr/etc-system/root.keys
--> Using cache f094fb7d73fbc90d709f1d43dffc5b9b44f4739d6fae3b7f44018a48dbcb1cb7
--> f094fb7d73fb
STEP 5/13: VOLUME /var/roothome
--> Using cache 5c199e0456b55e4feb83f4bbd7afa08ef3bb8e360b141f6e203c793b1f15403b
--> 5c199e0456b5
STEP 6/13: ADD packaging/flightctl-custom-assets/config.yaml /etc/flightctl/
--> Using cache 08068ad64e01a17b5deaf1f17a5b111441b081994aa19341c1430f70c44b1935
--> 08068ad64e01
STEP 7/13: ADD packaging/flightctl-custom-assets/ca.crt /etc/flightctl
--> Using cache 6cf50e3e940d43a151d8df1efa69adb4f1ab78be5f278bba851b7bc4791ae076
--> 6cf50e3e940d
STEP 8/13: ADD packaging/flightctl-custom-assets/client-enrollment.* /etc/flightctl/
--> Using cache 467f88db436db8c62bf93775f271861275679c2e5ec8bdbed7f4e10eaf9cbb44
--> 467f88db436d
STEP 9/13: RUN rpm-ostree install -y /tmp/flightctl-agent-0.0.1-1.el9.x86_64.rpm
--> Using cache 51a5903e3b71a19a548282e35e933c153d03470071b02525b622877e3e5b2c69
--> 51a5903e3b71
STEP 10/13: RUN ln -s /usr/lib/systemd/system/podman.socket /usr/lib/systemd/system/multi-user.target.wants/
--> Using cache 886a0421c6f532094be9c8298f451e63e241c59316ad61907e06e6c6187650b6
--> 886a0421c6f5
STEP 11/13: RUN ln -s /usr/lib/systemd/system/flightctl-agent.service /usr/lib/systemd/system/multi-user.target.wants/
--> Using cache e83c1f9965b89846251ea2b1e4df3c969a52c9984e7f42c50e855035ee39fa79
--> e83c1f9965b8
STEP 12/13: RUN rpm-ostree kargs --append=audit=0
error: This system was not booted via libostree.
Currently, most rpm-ostree commands only work on ostree-based host systems.

Error: building at STEP "RUN rpm-ostree kargs --append=audit=0": while running runtime: exit status 1

This is the containerfile:

FROM quay.io/centos-bootc/fedora-bootc:eln

COPY rpmbuild/RPMS/x86_64/flightctl-agent-0.0.1-1.el9.x86_64.rpm /tmp/

COPY packaging/flightctl-custom-assets/flightctl_rsa.pub /usr/etc-system/root.keys
RUN touch /etc/ssh/sshd_config.d/30-auth-system.conf; \
    mkdir -p /usr/etc-system/; \
    echo 'AuthorizedKeysFile /usr/etc-system/%u.keys' >> /etc/ssh/sshd_config.d/30-auth-system.conf; \
    chmod 0600 /usr/etc-system/root.keys
VOLUME /var/roothome

ADD packaging/flightctl-custom-assets/config.yaml /etc/flightctl/
ADD packaging/flightctl-custom-assets/ca.crt /etc/flightctl
ADD packaging/flightctl-custom-assets/client-enrollment.* /etc/flightctl/

RUN rpm-ostree install -y /tmp/flightctl-agent-0.0.1-1.el9.x86_64.rpm
RUN ln -s /usr/lib/systemd/system/podman.socket /usr/lib/systemd/system/multi-user.target.wants/
RUN ln -s /usr/lib/systemd/system/flightctl-agent.service /usr/lib/systemd/system/multi-user.target.wants/
RUN rpm-ostree kargs --append='audit=0'
RUN ostree container commit

kargs support needs to be added. Current workaround (thanks @miabbott ) is here: https://containers.github.io/bootc/install/#operating-system-install-configuration-required

[cgwalters]

Anything around having rpm-ostree emit errors should probably start as an issue in that repository. However, it's fine...I reworded and edited this issue.

Ultimately when executed in a container image, rpm-ostree args and tools like grubby would basically write files here.

Note a large difference versus the bootc install configs is that those kargs are only applied at bootc install time today, there's no "day 2 changes".

[jlebon]

I know bootc wants to leave the door open for non-ostree backends, but given that kargs are intimately linked to the thing that updates the bootloader, isn't the right place to implement this at the ostree level? (I.e. revive ostreedev/ostree#2217 and then users can put things in those directories in their derived builds.)

I can certainly imagine /usr/lib/bootc/kargs.d being a symlink to /usr/lib/ostree/kargs.d so that users don't have to type ostree.

[cgwalters]

Yeah, it's a fair point. Mainly for things like this I just want to ideally do new features in Rust in this project, having it more of a mono-repo style.

[miabbott]

As part of whatever kargs support is implemented, we should consider how to support per-architecture kargs.

For example, console settings; see the discussion on the MR from @trgeiger - https://gitlab.com/bootc-org/examples/-/merge_requests/11#note_1783996244

[mvo5]

This is also relevant for bootc-image-builder where it is nice to support customizing the kernel commandline