Bake k0s as a systemd sysext image. #99

mnbro · 2024-11-21T10:37:18Z

Bake k0s as a systemd sysext image.

This PR aims to bake k0s as a systemd sysext and also provide a systemd unit file to manage k0s.

How to use

[ describe what reviewers need to do in order to validate this PR ]

Testing done

[Describe the testing you have done before submitting this PR. Please include both the commands you issued as well as the output you got.]

Changelog entries added in the respective changelog/ directory (user-facing change, bug fix, security fix, update)
Inspected CI output for image differences: /boot and /usr size, packages, list files for any missing binaries, kernel modules, config files, kernel modules, etc.

mnbro · 2024-11-25T10:47:45Z

@tormath1 Can you please take a look and see if we can merge this?

After this is merged it would be nice to align k0s with the latest releases in the same way it was implemented for k3s in #98

tormath1

Thanks for the PR, some feedback:

do you want some units to be started when the sysext is loaded? If yes, you might need to add this kind of command:

sysext-bakery/create_containerd_sysext.sh

Lines 69 to 70 in 045a4f9

    
           mkdir -p "${SYSEXTNAME}/usr/lib/systemd/system/multi-user.target.d" 
        
           { echo "[Unit]"; echo "Upholds=containerd.service"; } > "${SYSEXTNAME}/usr/lib/systemd/system/multi-user.target.d/10-containerd-service.conf"

could you write some documentation on how to use this sysext (e.g does it need provisioning with Ignition to create files / enable units / etc.)

tormath1 · 2024-11-22T11:58:34Z

create_k0s_sysext.sh

+# The github release uses different arch identifiers, we map them here
+# and rely on bake.sh to map them back to what systemd expects
+if [ "${ARCH}" = "amd64" ] || [ "${ARCH}" = "x86-64" ]; then
+  URL="https://github.com/k0sproject/k0s/releases/download/${VERSION}/k0s-${VERSION}-amd64"
+elif [ "${ARCH}" = "arm64" ] || [ "${ARCH}" = "aarch64" ]; then
+  URL="https://github.com/k0sproject/k0s/releases/download/${VERSION}/k0s-${VERSION}-arm64"
+fi


I think we can just set the correct ARCH and only assign one URL. Something like that:

Suggested change

# The github release uses different arch identifiers, we map them here

# and rely on bake.sh to map them back to what systemd expects

if [ "${ARCH}" = "amd64" ] || [ "${ARCH}" = "x86-64" ]; then

URL="https://github.com/k0sproject/k0s/releases/download/${VERSION}/k0s-${VERSION}-amd64"

elif [ "${ARCH}" = "arm64" ] || [ "${ARCH}" = "aarch64" ]; then

URL="https://github.com/k0sproject/k0s/releases/download/${VERSION}/k0s-${VERSION}-arm64"

fi

# The github release uses different arch identifiers, we map them here

# and rely on bake.sh to map them back to what systemd expects

if [ "${ARCH}" = "amd64" ] || [ "${ARCH}" = "x86-64" ]; then

ARCH="amd64"

fi

if [ "${ARCH}" = "arm64" ] || [ "${ARCH}" = "aarch64" ]; then

ARCH="arm64"

fi

URL="https://github.com/k0sproject/k0s/releases/download/${VERSION}/k0s-${VERSION}-${ARCH}"

tormath1 · 2024-11-22T11:59:30Z

create_k0s_sysext.sh

+popd
+
+mkdir -p "${SYSEXTNAME}"/usr/local/lib/systemd/system/
+cat > "${SYSEXTNAME}"/etc/systemd/system/k0s.service << EOF


Suggested change

cat > "${SYSEXTNAME}"/etc/systemd/system/k0s.service << EOF

cat > "${SYSEXTNAME}"/usr/local/lib/systemd/system/k0s.service << EOF

/etc content is not applied when the sysext is merged to the OS (only /usr and /opt).

tormath1 · 2024-11-22T12:00:06Z

create_k0s_sysext.sh

+WantedBy=multi-user.target
+EOF
+
+cat > "${SYSEXTNAME}"/etc/systemd/system/k0scontroller.service << EOF


Suggested change

cat > "${SYSEXTNAME}"/etc/systemd/system/k0scontroller.service << EOF

cat > "${SYSEXTNAME}"/usr/local/lib/systemd/system/k0scontroller.service << EOF

Same as above.

tormath1 · 2024-11-22T12:00:43Z

create_k0s_sysext.sh

+WantedBy=multi-user.target
+EOF
+
+cat > "${SYSEXTNAME}"/etc/systemd/system/k0sworker.service << EOF


Same as above.

create_k0s_sysext.sh

mnbro · 2024-11-25T16:22:47Z

Thanks for the PR, some feedback:

do you want some units to be started when the sysext is loaded? If yes, you might need to add this kind of command:

sysext-bakery/create_containerd_sysext.sh

Lines 69 to 70 in 045a4f9

mkdir -p "${SYSEXTNAME}/usr/lib/systemd/system/multi-user.target.d"

{ echo "[Unit]"; echo "Upholds=containerd.service"; } > "${SYSEXTNAME}/usr/lib/systemd/system/multi-user.target.d/10-containerd-service.conf"

There will be no systemd units automatically enabled. As will be written in the documentation:
i. choosing the embedded components of k0s will lead to some systemd units being enabled by the user(via Ignition) -> (k0s.service for controllers and k0sworker.service for workers )

ii. not opting for k0s embedded componets will lead to other systemd units being enabled by the user(via Ignition) ->(k0scontroller.service for controllers and k0sworker.service for workers)

could you write some documentation on how to use this sysext (e.g does it need provisioning with Ignition to create files / enable units / etc.)

Ok

mnbro · 2024-11-26T08:13:44Z

@tormath1 I've committed the requested changes. Please take a look.

tormath1 · 2024-11-28T08:56:31Z

README.md

+1. the sysext plus the configuration files /etc/k0s/k0s.yaml and/or /etc/default/k0s
+2. the systemd units
+
+After you enable k0s syext following the guide deacribed on Consuming the published images, follow the next steps:


Suggested change

After you enable k0s syext following the guide deacribed on Consuming the published images, follow the next steps:

After you enabled k0s sysext following the guide described on consuming the published images, follow the next steps:

tormath1 · 2024-11-28T08:59:51Z

create_k0s_sysext.sh

This file needs to be pushed with executable mode:

diff --git a/create_k0s_sysext.sh b/create_k0s_sysext.sh old mode 100644 new mode 100755

tormath1 · 2024-11-28T09:05:38Z

create_k0s_sysext.sh

+EnvironmentFile=-/etc/default/k0s
+StartLimitInterval=5
+StartLimitBurst=10
+ExecStart=/bin/sh -c '[ -n "${CRI_SOCKET}" ] && exec /usr/local/bin/k0s controller --config=/etc/k0s/k0s.yaml --cri-socket=${CRI_SOCKET} || exec /usr/local/bin/k0s controller --config=/etc/k0s/k0s.yaml'


This won't work:

./create_k0s_sysext.sh: line 41: CRI_SOCKET: unbound variable

You need to escape those values (individually or globally with \EOF)

tormath1 · 2024-11-28T09:05:52Z

create_k0s_sysext.sh

+EnvironmentFile=-/etc/default/k0s
+StartLimitInterval=5
+StartLimitBurst=10
+ExecStart=/bin/sh -c '[ -n "${CRI_SOCKET}" ] && exec /usr/local/bin/k0s controller --config=/etc/k0s/k0s.yaml --cri-socket=${CRI_SOCKET} || exec /usr/local/bin/k0s controller --config=/etc/k0s/k0s.yaml'


Suggested change

ExecStart=/bin/sh -c '[ -n "${CRI_SOCKET}" ] && exec /usr/local/bin/k0s controller --config=/etc/k0s/k0s.yaml --cri-socket=${CRI_SOCKET} || exec /usr/local/bin/k0s controller --config=/etc/k0s/k0s.yaml'

ExecStart=/bin/sh -c '[ -n "${CRI_SOCKET}" ] && exec /usr/local/bin/k0s controller --config=/etc/k0s/k0s.yaml --cri-socket="${CRI_SOCKET}" || exec /usr/local/bin/k0s controller --config=/etc/k0s/k0s.yaml'

(Same for the other units)

tormath1 · 2024-11-28T09:06:13Z

create_k0s_sysext.sh

+EnvironmentFile=-/etc/default/k0s
+StartLimitInterval=5
+StartLimitBurst=10
+ExecStart=/usr/local/bin/k0s worker --cri-socket=$CRI_SOCKET --token-file=/etc/k0s/worker-token


Suggested change

ExecStart=/usr/local/bin/k0s worker --cri-socket=$CRI_SOCKET --token-file=/etc/k0s/worker-token

ExecStart=/usr/local/bin/k0s worker --cri-socket="${CRI_SOCKET}" --token-file=/etc/k0s/worker-token

tormath1 · 2024-11-28T09:18:26Z

README.md

+
+After you enable k0s syext following the guide deacribed on Consuming the published images, follow the next steps:
+
+1. 


Is this mandatory? One might not know by advance the private IP address.

tormath1 · 2024-11-28T09:19:53Z

create_k0s_sysext.sh

+EnvironmentFile=-/etc/default/k0s
+StartLimitInterval=5
+StartLimitBurst=10
+ExecStart=/bin/sh -c '[ -n "${CRI_SOCKET}" ] && exec /usr/local/bin/k0s controller --config=/etc/k0s/k0s.yaml --cri-socket=${CRI_SOCKET} --token-file=/etc/k0s/controller-token|| exec /usr/local/bin/k0s controller --config=/etc/k0s/k0s.yaml --token-file=/etc/k0s/controller-token'


Who's supposed to generate this /etc/k0s/controller-token ?

Should we start the k0s.service before ? If yes, it should be documented.

mnbro · 2024-12-04T08:15:25Z

Adding k0s as a systemdsysext will brake k0sctl.

Add create_k0s_sysext.sh

daa5dd6

mnbro mentioned this pull request Nov 21, 2024

[RFE] Auto-detect latest release versions in release builds #42

Open

github-actions bot mentioned this pull request Nov 22, 2024

Monthly contributions report 2024-10-22 - 2024-11-21 flatcar/Flatcar#1585

Open

tormath1 reviewed Nov 25, 2024

View reviewed changes

mnbro added 2 commits November 25, 2024 18:25

Add create_k0s_sysext.sh, fix1

e0a5e34

Add documentation

18b991e

tormath1 reviewed Nov 28, 2024

View reviewed changes

mnbro closed this Dec 4, 2024

mnbro deleted the bake-k0s branch December 4, 2024 08:15

github-actions bot mentioned this pull request Dec 22, 2024

Monthly contributions report 2024-11-22 - 2024-12-21 flatcar/Flatcar#1603

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Bake k0s as a systemd sysext image. #99

Bake k0s as a systemd sysext image. #99

mnbro commented Nov 21, 2024

mnbro commented Nov 25, 2024

tormath1 left a comment

tormath1 Nov 22, 2024

tormath1 Nov 22, 2024

tormath1 Nov 22, 2024

tormath1 Nov 22, 2024

mnbro commented Nov 25, 2024

mnbro commented Nov 26, 2024

tormath1 Nov 28, 2024

tormath1 Nov 28, 2024

tormath1 Nov 28, 2024

tormath1 Nov 28, 2024

tormath1 Nov 28, 2024

tormath1 Nov 28, 2024

tormath1 Nov 28, 2024

mnbro commented Dec 4, 2024

	mkdir -p "${SYSEXTNAME}/usr/lib/systemd/system/multi-user.target.d"
	{ echo "[Unit]"; echo "Upholds=containerd.service"; } > "${SYSEXTNAME}/usr/lib/systemd/system/multi-user.target.d/10-containerd-service.conf"

	cat > "${SYSEXTNAME}"/etc/systemd/system/k0s.service << EOF
	cat > "${SYSEXTNAME}"/usr/local/lib/systemd/system/k0s.service << EOF

	After you enable k0s syext following the guide deacribed on Consuming the published images, follow the next steps:
	After you enabled k0s sysext following the guide described on consuming the published images, follow the next steps:

	ExecStart=/bin/sh -c '[ -n "${CRI_SOCKET}" ] && exec /usr/local/bin/k0s controller --config=/etc/k0s/k0s.yaml --cri-socket=${CRI_SOCKET} \|\| exec /usr/local/bin/k0s controller --config=/etc/k0s/k0s.yaml'
	ExecStart=/bin/sh -c '[ -n "${CRI_SOCKET}" ] && exec /usr/local/bin/k0s controller --config=/etc/k0s/k0s.yaml --cri-socket="${CRI_SOCKET}" \|\| exec /usr/local/bin/k0s controller --config=/etc/k0s/k0s.yaml'

	ExecStart=/usr/local/bin/k0s worker --cri-socket=$CRI_SOCKET --token-file=/etc/k0s/worker-token
	ExecStart=/usr/local/bin/k0s worker --cri-socket="${CRI_SOCKET}" --token-file=/etc/k0s/worker-token

Bake k0s as a systemd sysext image. #99

Bake k0s as a systemd sysext image. #99

Conversation

mnbro commented Nov 21, 2024

Bake k0s as a systemd sysext image.

How to use

Testing done

mnbro commented Nov 25, 2024

tormath1 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mnbro commented Nov 25, 2024

mnbro commented Nov 26, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mnbro commented Dec 4, 2024