add PodSecurityStandard to quickstar topology and book

* use parametrizable patches via ClusterClass instead of statically adding the Pod Security Standard * book: introduce security guidelines section * book: add security guidelines section about Pod Security Standard * book: add section about limitation of cluster class json patches Signed-off-by: Christian Schlotter <[email protected]>
kubernetes-sigs · Apr 27, 2022 · 9d3927f · 9d3927f
1 parent ac24a5b
commit 9d3927f
Show file tree

Hide file tree

Showing 7 changed files with 396 additions and 1 deletion.
diff --git a/docs/book/src/SUMMARY.md b/docs/book/src/SUMMARY.md
@@ -23,6 +23,8 @@
             - [Changing a ClusterClass](./tasks/experimental-features/cluster-class/change-clusterclass.md)
             - [Operating a managed Cluster](./tasks/experimental-features/cluster-class/operate-cluster.md)
         - [Ignition Bootstrap configuration](./tasks/experimental-features/ignition.md)
+- [Security Guidelines](./security/index.md)
+    - [Pod Security Standards](./security/pod-security-standards.md)
 - [clusterctl CLI](./clusterctl/overview.md)
     - [clusterctl Commands](clusterctl/commands/commands.md)
         - [init](clusterctl/commands/init.md)

diff --git a/docs/book/src/security/index.md b/docs/book/src/security/index.md
@@ -0,0 +1,6 @@
+# Security Guidelines
+
+This section provides security guidelines useful to provision clusters which are
+_secure by default_ to follow the [secure defaults guidelines for cloud native apps].
+
+[secure defaults guidelines for cloud native apps]: https://github.com/cncf/tag-security/blob/main/security-whitepaper/secure-defaults-cloud-native-8.md
diff --git a/docs/book/src/security/pod-security-standards.md b/docs/book/src/security/pod-security-standards.md
@@ -0,0 +1,212 @@
+# Pod Security Standards
+
+Pod Security Admission allows applying [Pod Security Standards] during creation of pods at the cluster level.
+
+The flavor `development-topology` for the docker provider used in [Quick Start](../user/quick-start.md) already includes a basic Pod Security Standard configuration.
+It is using ClusterClass variables and patches to inject the configuration.
+
+## Adding a basic Pod Security Standards configuration to a ClusterClass
+
+By adding the following variables and patches Pod Security Standards can be added to every ClusterClass which references a [Kubeadm based control plane](../tasks/kubeadm-control-plane.md).
+
+### Adding the variables to a ClusterClass
+
+```yaml
+apiVersion: cluster.x-k8s.io/v1beta1
+kind: ClusterClass
+spec:
+  variables:
+  - name: podSecurityStandard
+    required: false
+    schema:
+      openAPIV3Schema:
+        type: object
+        properties: 
+          enabled: 
+            type: boolean
+            default: true
+            description: "enabled enables the patches to enable Pod Security Standard via AdmissionConfiguration."
+          enforce:
+            type: string
+            default: "baseline"
+            description: "enforce sets the level for the enforce PodSecurityConfiguration mode. One of privileged, baseline, restricted."
+            pattern: "privileged|baseline|restricted"
+          audit:
+            type: string
+            default: "restricted"
+            description: "audit sets the level for the audit PodSecurityConfiguration mode. One of privileged, baseline, restricted."
+            pattern: "privileged|baseline|restricted"
+          warn:
+            type: string
+            default: "restricted"
+            description: "warn sets the level for the warn PodSecurityConfiguration mode. One of privileged, baseline, restricted."
+            pattern: "privileged|baseline|restricted"
+  ...
+```
+
+* The version field in Pod Security Admission Config defaults to `latest`.
+* The `kube-system` namespace is exempt from Pod Security Standards enforcement, because it runs control-plane pods that need higher privileges.
+
+### Adding the patches to a ClusterClass
+
+The following snippet contains the patch to be added to the ClusterClass.
+
+Due to [limitations of ClusterClass with patches](../tasks/experimental-features/cluster-class/write-clusterclass.md#json-patches-tips--tricks) there are two versions for this patch.
+
+{{#tabs name:"tab-configuration-patches" tabs:"Add to existing slice,Create slice"}}
+{{#tab Append}}
+
+Use this patch if the following keys **already exist** inside the `KubeadmControlPlaneTemplate` referred by the ClusterClass:
+
+- `.spec.template.spec.kubeadmConfigSpec.clusterConfiguration.apiServer.extraVolumes`
+- `.spec.template.spec.kubeadmConfigSpec.files`
+
+```yaml
+apiVersion: cluster.x-k8s.io/v1beta1
+kind: ClusterClass
+spec:
+  ...
+  patches:
+  - name: podSecurityStandard
+    description: "Adds an admission configuration for PodSecurity to the kube-apiserver."
+    definitions:
+    - selector:
+        apiVersion: controlplane.cluster.x-k8s.io/v1beta1
+        kind: KubeadmControlPlaneTemplate
+        matchResources:
+          controlPlane: true
+      jsonPatches:
+      - op: add
+        path: "/spec/template/spec/kubeadmConfigSpec/clusterConfiguration/apiServer/extraArgs"
+        value:
+          admission-control-config-file: "/etc/kubernetes/kube-apiserver-admission-pss.yaml"
+      - op: add
+        path: "/spec/template/spec/kubeadmConfigSpec/clusterConfiguration/apiServer/extraVolumes/-"
+        value:
+          name: admission-pss
+          hostPath: /etc/kubernetes/kube-apiserver-admission-pss.yaml
+          mountPath: /etc/kubernetes/kube-apiserver-admission-pss.yaml
+          readOnly: true
+          pathType: "File"
+      - op: add
+        path: "/spec/template/spec/kubeadmConfigSpec/files/-"
+        valueFrom:
+          template: |
+            content: |
+              apiVersion: apiserver.config.k8s.io/v1
+              kind: AdmissionConfiguration
+              plugins:
+              - name: PodSecurity
+                configuration:
+                  apiVersion: pod-security.admission.config.k8s.io/v1beta1
+                  kind: PodSecurityConfiguration
+                  defaults:
+                    enforce: "{{ .podSecurity.enforce }}"
+                    enforce-version: "latest"
+                    audit: "{{ .podSecurity.audit }}"
+                    audit-version: "latest"
+                    warn: "{{ .podSecurity.warn }}"
+                    warn-version: "latest"
+                  exemptions:
+                    usernames: []
+                    runtimeClasses: []
+                    namespaces: [kube-system]
+            path: /etc/kubernetes/kube-apiserver-admission-pss.yaml
+    enabledIf: "{{ .podSecurityStandard.enabled }}"
+...
+```
+
+{{#/tab }}
+{{#tab Create}}
+
+
+Use this patches if the following keys **do not** exist inside the `KubeadmControlPlaneTemplate` referred by the ClusterClass:
+
+- `.spec.template.spec.kubeadmConfigSpec.clusterConfiguration.apiServer.extraVolumes`
+- `.spec.template.spec.kubeadmConfigSpec.files`
+
+> **Attention:** Existing values inside the `KubeadmControlPlaneTemplate` at the mentioned keys will be replaced by this patch.
+
+```yaml
+apiVersion: cluster.x-k8s.io/v1beta1
+kind: ClusterClass
+spec:
+  ...
+  patches:
+  - name: podSecurityStandard
+    description: "Adds an admission configuration for PodSecurity to the kube-apiserver."
+    definitions:
+    - selector:
+        apiVersion: controlplane.cluster.x-k8s.io/v1beta1
+        kind: KubeadmControlPlaneTemplate
+        matchResources:
+          controlPlane: true
+      jsonPatches:
+      - op: add
+        path: "/spec/template/spec/kubeadmConfigSpec/clusterConfiguration/apiServer/extraArgs"
+        value:
+          admission-control-config-file: "/etc/kubernetes/kube-apiserver-admission-pss.yaml"
+      - op: add
+        path: "/spec/template/spec/kubeadmConfigSpec/clusterConfiguration/apiServer/extraVolumes"
+        value:
+        - name: admission-pss
+          hostPath: /etc/kubernetes/kube-apiserver-admission-pss.yaml
+          mountPath: /etc/kubernetes/kube-apiserver-admission-pss.yaml
+          readOnly: true
+          pathType: "File"
+      - op: add
+        path: "/spec/template/spec/kubeadmConfigSpec/files"
+        valueFrom:
+          template: |
+            - content: |
+                apiVersion: apiserver.config.k8s.io/v1
+                kind: AdmissionConfiguration
+                plugins:
+                - name: PodSecurity
+                  configuration:
+                    apiVersion: pod-security.admission.config.k8s.io/v1beta1
+                    kind: PodSecurityConfiguration
+                    defaults:
+                      enforce: "{{ .podSecurity.enforce }}"
+                      enforce-version: "latest"
+                      audit: "{{ .podSecurity.audit }}"
+                      audit-version: "latest"
+                      warn: "{{ .podSecurity.warn }}"
+                      warn-version: "latest"
+                    exemptions:
+                      usernames: []
+                      runtimeClasses: []
+                      namespaces: [kube-system]
+              path: /etc/kubernetes/kube-apiserver-admission-pss.yaml
+    enabledIf: "{{ .podSecurityStandard.enabled }}"
+...
+```
+
+{{#/tab }}
+{{#/tabs }}
+
+
+[Pod Security Standards]: https://kubernetes.io/docs/concepts/security/pod-security-standards
+
+### Create a secure Cluster using the ClusterClass
+
+After adding the variables and patches the Pod Security Standards would be applied by default.
+It is also possible to disable this patch or configure different levels for the configuration 
+using variables.
+
+```yaml
+apiVersion: cluster.x-k8s.io/v1beta1
+kind: Cluster
+metadata:
+  name: "my-cluster"
+spec:
+  ...
+  topology:
+    ...
+    class: my-secure-cluster-class
+    variables:
+    - name: podSecurityStandard
+      value: 
+        enabled: true
+        enforce: "restricted"
+```
diff --git a/docs/book/src/tasks/experimental-features/cluster-class/write-clusterclass.md b/docs/book/src/tasks/experimental-features/cluster-class/write-clusterclass.md
@@ -3,7 +3,7 @@
 
 A ClusterClass becomes more useful and valuable when it can be used to create many Cluster of a similar 
 shape. The goal of this document is to explain how ClusterClasses can be written in a way that they are 
-flexible enough to be used in as many Cluster as possible by supporting variants of the same base Cluster shape.
+flexible enough to be used in as many Clusters as possible by supporting variants of the same base Cluster shape.
 
 **Table of Contents**
 
@@ -16,6 +16,7 @@ flexible enough to be used in as many Cluster as possible by supporting variants
     * [Complex variable types](#complex-variable-types)
     * [Using variable values in JSON patches](#using-variable-values-in-json-patches)
     * [Optional patches](#optional-patches)
+* [JSON patches tips &amp; tricks](#json-patches-tips--tricks)
 
 ## Basic ClusterClass
 
@@ -676,6 +677,101 @@ individually.
 
 </aside>
 
+## JSON patches tips & tricks
+
+JSON patches specification [RFC6902] requires that the target of
+add operation must exist.
+
+As a consequence ClusterClass authors should pay special attention when the following
+conditions apply in order to prevent errors when a patch is applied:
+
+* the patch tries to `add` a value to an **array** (which is a **slice** in the corresponding go struct)
+* the slice was defined with `omitempty`
+* the slice currently does not exist
+
+A workaround in this particular case is to create the array in the patch instead of adding to the non-existing one.
+When creating the slice, existing values would be overwritten so this should only be used when it does not exist.
+
+The following example shows both cases to consider while writing a patch for adding a value to a slice.
+This patch targets to add a file to the `files` slice of a `KubeadmConfigTemplate` which has [omitempty](https://github.com/kubernetes-sigs/cluster-api/blob/main/bootstrap/kubeadm/api/v1beta1/kubeadmconfig_types.go#L54) set.
+
+{{#tabs name:"tab-configuration-patches" tabs:"Add to existing slice,Create slice"}}
+{{#tab Add to existing slice}}
+
+This patch **requires** the key `.spec.template.spec.files` to exist to succeed.
+
+```yaml
+apiVersion: cluster.x-k8s.io/v1beta1
+kind: ClusterClass
+metadata:
+  name: my-clusterclass
+spec:
+  ...
+  patches:
+  - name: add file
+    definitions:
+    - selector:
+        apiVersion: bootstrap.cluster.x-k8s.io/v1beta1
+        kind: KubeadmConfigTemplate
+      jsonPatches:
+      - op: add
+        path: /spec/template/spec/files/-
+        value:
+          content: Some content.
+          path: /some/file
+---
+apiVersion: bootstrap.cluster.x-k8s.io/v1beta1
+kind: KubeadmConfigTemplate
+metadata:
+  name: "quick-start-default-worker-bootstraptemplate"
+spec:
+  template:
+    spec:
+      ...
+      files:
+      - content: Some other content
+        path: /some/other/file
+```
+
+{{#/tab }}
+{{#tab Create slice}}
+
+This patch would **overwrite** an existing slice at `.spec.template.spec.files`.
+
+```yaml
+apiVersion: cluster.x-k8s.io/v1beta1
+kind: ClusterClass
+metadata:
+  name: my-clusterclass
+spec:
+  ...
+  patches:
+  - name: add file
+    definitions:
+    - selector:
+        apiVersion: bootstrap.cluster.x-k8s.io/v1beta1
+        kind: KubeadmConfigTemplate
+      jsonPatches:
+      - op: add
+        path: /spec/template/spec/files
+        value:
+        - content: Some content.
+          path: /some/file
+---
+apiVersion: bootstrap.cluster.x-k8s.io/v1beta1
+kind: KubeadmConfigTemplate
+metadata:
+  name: "quick-start-default-worker-bootstraptemplate"
+spec:
+  template:
+    spec:
+      ...
+```
+
+{{#/tab }}
+{{#/tabs }}
+
 <!-- links -->
 [Changing a ClusterClass]: ./change-clusterclass.md
 [clusterctl alpha topology plan]: ../../../clusterctl/commands/alpha-topology-plan.md
+[RFC6902]: https://datatracker.ietf.org/doc/html/rfc6902#appendix-A.12
diff --git a/docs/book/src/user/quick-start.md b/docs/book/src/user/quick-start.md
@@ -511,6 +511,11 @@ export POD_CIDR=["192.168.0.0/16"]
 export SERVICE_DOMAIN="k8s.test"
 ```
 
+It is also possible but **not recommended** to disable the per-default enabled [Pod Security Standard](../security/pod-security-standards.md):
+```bash
+export ENABLE_POD_SECURITY_STANDARD="false"
+```
+
 {{#/tab }}
 {{#tab Equinix Metal}}
 

diff --git a/test/infrastructure/docker/templates/cluster-template-development-topology.yaml b/test/infrastructure/docker/templates/cluster-template-development-topology.yaml
@@ -22,6 +22,12 @@ spec:
       value: ""
     - name: coreDNSImageTag
       value: ""
+    - name: podSecurityStandard
+      value:
+        enabled: ${POD_SECURITY_STANDARD_ENABLED:=true}
+        enforce: "baseline"
+        audit: "restricted"
+        warn: "restricted"
     version: ${KUBERNETES_VERSION}
     workers:
       machineDeployments: