Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

📖 Add documentation on how to gate upgrades with kyverno #1135

Merged
merged 2 commits into from
Mar 15, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 15 additions & 3 deletions docs/content/en/docs/Architecture/meta.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,9 +22,21 @@ Every component is extensible and modular such as it can be customized and repla

### Internal components

Kairos encompasses several components, some externally, most notably:

Kairos encompasses several components, external and internal.

Internal:
- [kairos](https://github.com/kairos-io/kairos) is the main repository, building the `kairos-agent` and containing the image definitions which runs on our CI pipelines.
- [immucore](https://github.com/kairos-io/immucore) is the immutability management interface.
- [AuroraBoot](https://github.com/kairos-io/AuroraBoot) is the Kairos Node bootstrapper
- [elemental-cli](https://github.com/kairos-io/elemental-cli) manages the installation, reset, and upgrade of the Kairos node.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We ened to rename this asap :D

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yep indeed... we have #711 in the sprint :) , I would have waited for #711, but this list in the docs was not matching the truth since a long time already, so thought to updated it

- [system packages](https://github.com/kairos-io/packages) contains additional packages, cross-distro, partly used in framework images
- [kcrypt](https://github.com/kairos-io/kcrypt) is the component responsible for encryption and decryption of data at rest
- [kcrypt-challenger](https://github.com/kairos-io/kcrypt-challenger) is the `kairos` plugin that works with the TPM chip to unlock LUKS partitions
- [osbuilder](https://github.com/kairos-io/osbuilder) is used to build bootable artifacts from container images
- [entangle](https://github.com/kairos-io/entangle) a CRD to interconnect Kubernetes clusters
- [entangle-proxy](https://github.com/kairos-io/entangle-proxy) a CRD to control interconnetted clusters

External:
- [K3s](https://k3s.io) as a Kubernetes distribution
- [edgevpn](https://mudler.github.io/edgevpn) (optional) as fabric for the distributed network, node coordination and bootstrap. Provides also embedded DNS capabilities for the cluster. Internally uses [libp2p](https://github.com/libp2p/go-libp2p) for the P2P mesh capabilities.
- [elemental-toolkit](https://rancher.github.io/elemental-toolkit/docs/) as a fundament to build the Linux derivative. Indeed, any `Elemental` docs applies to `Kairos` as well.
- [nohang](https://github.com/hakavlad/nohang) A sophisticated low memory handler for Linux.
4 changes: 0 additions & 4 deletions docs/content/en/docs/Reference/cli.md
Original file line number Diff line number Diff line change
Expand Up @@ -61,8 +61,6 @@ kairos:
device: ""
poweroff: false

# Cloud init syntax to setup users.
# See https://rancher.github.io/elemental-toolkit/docs/reference/cloud_init/
stages:
network:
- name: "Setup users"
Expand Down Expand Up @@ -90,8 +88,6 @@ kairos:
device: ""
poweroff: false

# Cloud init syntax to setup users.
# See https://rancher.github.io/elemental-toolkit/docs/reference/cloud_init/
stages:
network:
- name: "Setup users"
Expand Down
2 changes: 1 addition & 1 deletion docs/content/en/docs/Reference/recovery_mode.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ description: >
The Kairos recovery mode can be used to recover a damaged system or to regain access remotely (with assistance) to a machine which has been lost access to. The recovery mode is accessible only from the GRUB menu, from both the LiveCD, and an installed system.

{{% alert title="Note" %}}
On installed system, there are two recovery modes available during boot. Below describes only how the Kairos remote recovery works. The manual recovery entry has nothing special from the standard Elemental-toolkit recovery mode. It can be used to reset the A/B partitions (with the user/pass used during setup) and perform any other operation without remote access.
On installed system, there are two recovery modes available during boot. Below describes only how the Kairos remote recovery works. It can be used to reset the A/B partitions (with the user/pass used during setup) and perform any other operation without remote access.
{{% /alert %}}

## Boot into recovery mode
Expand Down
1 change: 0 additions & 1 deletion docs/content/en/docs/Reference/troubleshooting.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,4 +42,3 @@ On all nodes, which are deployed with the P2P full-mesh feature of the cluster,
## See also

- [Dracut debug docs](https://fedoraproject.org/wiki/How_to_debug_Dracut_problems)
- [Elemental troubleshooting docs](https://rancher.github.io/elemental-toolkit/docs/reference/troubleshooting/)
64 changes: 64 additions & 0 deletions docs/content/en/docs/Upgrade/kubernetes.md
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,70 @@ system-upgrade apply-os-upgrade-on-kairos-with-1a1a24bcf897bd275730bdd8548-h7f

Done! We should have all the basics to get our first cluster rolling, but there is much more we can do.

## Verify images attestation during upgrades

Container images can be signed during the build phase of a CI/CD pipeline using [Cosign](https://github.com/sigstore/cosign), Kairos signs every artifact as part of the release process.

To ensure that the images used during upgrades match the expected signatures, [Kyverno](https://kyverno.io/) can be used to set up policies. This is done by checking if the signature is present in the OCI registry and if the image was signed using the specified key. The policy rule check fails if either of these conditions is not met.

To learn more about this specific Kyverno feature, you can refer to the [documentation](https://kyverno.io/docs/writing-policies/verify-images/). This allows for the verification of image authenticity directly at the node level prior to upgrading.

A Kyverno policy for `provider-kairos` images might look like the following:

```yaml
apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
name: check-image
spec:
validationFailureAction: Enforce
background: false
webhookTimeoutSeconds: 30
failurePolicy: Fail
rules:
- name: check-image
match:
any:
- resources:
kinds:
- Pod
verifyImages:
- imageReferences:
- "quay.io/kairos/kairos-*"
attestors:
- entries:
# See: https://kyverno.io/docs/writing-policies/verify-images/#keyless-signing-and-verification
- keyless:
subject: "https://github.com/kairos-io/provider-kairos/.github/workflows/release.yaml@refs/tags/*"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is kind of icky.

Hear me out, this is really good until cosign or the CI breaks and we need to sign the already published artifacts in a different way, maybe by a different workflows like we used to do in that other project. Then this doesnt match anymore so you either drop a release completely as it wont be signed, resign it and break this gating for users or run a new release and left the unsigned release there.

does this subject allows for regex? or for value.in so we can provide a list?

Copy link
Member Author

@mudler mudler Mar 15, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

that's a valid and good point the subject is a regex already indeed: https://github.com/kairos-io/provider-kairos/.github/workflows/release.yaml@refs/tags/* I think it's fine as a baseline example and nothing more, for instance, a policy can be applied also per-releases prior applying a plan, and so on so forth, it's up to you to which kinda of policy you want to set - this one is very greedy and assumes all provider-kairos repository are from that CI pipeline.

issuer: "https://token.actions.githubusercontent.com"
rekor:
url: https://rekor.sigstore.dev
```

To install Kyverno in a Kairos cluster, you can simply use the community [bundles](/docs/advanced/bundles). For example, you can use the following installation cloud config file:

```yaml
#cloud-config

hostname: kyverno-{{ trunc 4 .MachineID }}

# Specify the bundle to use
bundles:
- targets:
- run://quay.io/kairos/community-bundles:system-upgrade-controller_latest
- run://quay.io/kairos/community-bundles:cert-manager_latest
- run://quay.io/kairos/community-bundles:kyverno_latest

users:
- name: kairos
passwd: kairos

k3s:
enabled: true
```

This configuration file prepare the system with the `cert-manager`, `system-upgrade-controller` and the `kyverno` bundle, enabling `k3s`.

## Customize the upgrade plan

It is possible to run additional commands before the upgrade takes place into the node, consider the following example:
Expand Down
1 change: 0 additions & 1 deletion tests/assets/autoinstall.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,6 @@ install:
auto: true
reboot: true
device: /dev/vda
# https://rancher.github.io/elemental-toolkit/docs/customizing/configure_grub/#grub-environment-variables
grub_options:
extra_cmdline: "foobarzz"

Expand Down