Add CIS Benchmark plugin blog post (#1002)

Signed-off-by: Jonas Rosland <[email protected]>

site/_posts/2019-11-18-cis-benchmark-plugin.md

@@ -0,0 +1,186 @@
+---
+title: A Sonobuoy Plugin to Check Cluster Security with the CIS Kubernetes Benchmark
+excerpt: Check your cluster's security configuration with the new CIS Kubernetes Benchmark plugin.
+author_name: John Schnake
+author_url: https://github.com/johnschnake
+author_avatar: /img/contributors/john-schnake.png
+categories: [kubernetes, sonobuoy, conformance]
+# Tag should match author to drive author pages
+tags: ['Sonobuoy Team']
+---
+
+Sonobuoy was always designed to facilitate third-party plugins in order to accommodate custom testing requirements but, until recently, the design of Sonobuoy made some advanced plugins impossible to create.
+
+One of the most requested plugins is for the [CIS Kubernetes Benchmark](https://www.cisecurity.org/benchmark/kubernetes/) from the Center for Internet Security (CIS). These benchmarks are prescriptive tests for establishing a secure configuration posture for Kubernetes. However, we had difficulty implementing them as a Sonobuoy plugin until recently due to their numerous customization requirements, which were not supported. Over numerous releases, we've chipped away at these problems and we now have a CIS Kubernetes Benchmark plugin which utilizes the implementation provided by [kube-bench](https://github.com/aquasecurity/kube-bench). kube-bench is a Go application that runs the tests documented in the CIS Kubernetes Benchmark.
+
+Now that we’ve implemented these benchmarks as a Sonobuoy plugin, you can easily spot security concerns in your own clusters. In this article, we will:
+
+- Demonstrate how to run the new CIS benchmarks plugin
+- Review the new Sonobuoy features that make the plugin possible
+- Describe how the plugin was created so that you can understand how it works and how to implement your own custom plugin
+
+## Running the Plugin
+
+The CIS Benchmark plugin is actually two separate plugins: one for master nodes and one for worker nodes. The default YAML files for the plugins have been published in our new [Sonobuoy plugins repo](https://github.com/vmware-tanzu/sonobuoy-plugins). You can run the new plugin by using the following command:
+
+```
+$ sonobuoy run \
+  --plugin https://raw.githubusercontent.com/vmware-tanzu/sonobuoy-plugins/cis-benchmarks/cis-benchmarks/kube-bench-plugin.yaml \
+  --plugin https://raw.githubusercontent.com/vmware-tanzu/sonobuoy-plugins/cis-benchmarks/cis-benchmarks/kube-bench-master-plugin.yaml \
+  --wait
+```
+
+Once the process has completed, you can see the results by running the following command:
+
+```
+$ outfile=$(sonobuoy retrieve) && sonobuoy results $outfile
+```
+
+You can also list each test by using the `--mode detailed` option and pipe the results through other tools like `jq` for further analysis. Kube-bench even serializes the entire test object into JSON and places it into the “system-out” field for your inspection. The command below prints the JSON serialization for each of the tests that failed:
+
+```
+$ sonobuoy results $outfile --plugin kube-bench-master --mode detailed | jq 'select(.status=="failed")|.details|.["system-out"]' -r | jq
+{
+  "test_number": "1.1.6",
+  "test_desc": "Ensure that the --insecure-port argument is set to 0 (Scored)",
+  "audit": "/bin/ps -ef | grep kube-apiserver | grep -v grep",
+  "AuditConfig": "",
+  "type": "",
+  "remediation": "Edit the API server pod specification file /etc/kubernetes/manifests/kube-apiserver.yaml\napiserver.yaml on the master node and set the below parameter.\n--insecure-port=0\n",
+  "test_info": [
+    "Edit the API server pod specification file /etc/kubernetes/manifests/kube-apiserver.yaml\napiserver.yaml on the master node and set the below parameter.\n--insecure-port=0\n"
+  ],
+  "status": "FAIL",
+  "actual_value": "",
+  "scored": true,
+  "expected_result": ""
+}
+...
+```
+
+And that's it! If you want to learn more about how we created the plugin, keep reading below. Otherwise, enjoy the plugin and let us know how it works for you.
+
+## Creating the CIS Benchmark Plugin
+
+There are two steps to make a plugin:
+
+- Creating or choosing the image to run in the container
+- Creating the plugin definition file for Sonobuoy
+
+For a refresher on how Sonobuoy plugins work, check out our earlier blog post, [Fast and Easy Sonobuoy Plugins for Custom Testing of Almost Anything](https://blogs.vmware.com/cloudnative/2019/04/30/sonobuoy-plugins-custom-testing/).
+
+### Choosing the Image
+
+For our starting line, we are targeting the CIS benchmarks implemented in the [kube-bench](https://github.com/aquasecurity/kube-bench) repo. These benchmarks implement version 1.4.0 of the benchmarks and were written for Kubernetes 1.13.
+
+Luckily for us, the Aqua Security provides an image ready to run the benchmarks. We are able to leverage this image and use our custom plugin specification to redirect the results to Sonobuoy.
+
+_Note: Currently we're using a custom image temporarily made from the current master branch the kube-bench repo. This is only temporary until v0.2.0 is released. The reason for this is so that we can leverage the new `--junit` flag so that Sonobuoy can more easily process the output._
+
+### Creating the Plugin Definition
+
+Now we need to tell Sonobuoy how to run the image. The plugin for the CIS benchmarks provides a bit of a challenge compared with simpler plugins, because the benchmarks require privileged access to the host file system, including `hostPID: true` and numerous volume mounts. Up until recently, there was no way for a user to change this information but a recent [pull request for Sonobuoy](https://github.com/vmware-tanzu/sonobuoy/pull/837) enables you to set any pod spec options.
+
+Our steps will be:
+
+- Utilize the `sonobuoy gen plugin` command with the new `--show-default-podspec` option to generate a plugin definition
+- Modify the plugin definition with our custom `podSpec` options
+
+First, we need to decide on the command that the plugin should run. The `kube-bench` image defaults to printing results to stdout. Instead, we want to redirect them to a file (kube-bench has a flag for this) and then notify Sonobuoy when complete. 
+
+We can chain these desired commands together as a single bash command and then save the plugin definition to a file:
+
+```
+$ sonobuoy gen plugin \
+    --name kube-bench-worker \
+    --image=schnake/kube-bench:v0.2.0-demo \
+    --cmd=/bin/sh
+    --args=-c
+    --args=”kube-bench --version 1.13 --outputfile /tmp/results/output.xml --junit ; echo -n /tmp/results/output.xml > /tmp/results/done” \
+    --show-default-podspec > kube-bench-worker.yaml
+```
+
+Now open up `kube-bench-worker.yaml` and modify the `podSpec` to include the extra required volumes and `hostPID: true`:
+
+```
+podSpec:
+  containers: []
+  dnsPolicy: ClusterFirstWithHostNet
+  hostIPC: true
+  hostNetwork: true
+  hostPID: true
+  serviceAccountName: sonobuoy-serviceaccount
+  tolerations:
+  - operator: Exists
+  volumes:
+  - name: var-lib-etcd
+    hostPath:
+      path: "/var/lib/etcd"
+  - name: var-lib-kubelet
+    hostPath:
+      path: "/var/lib/kubelet"
+  - name: etc-systemd
+    hostPath:
+      path: "/etc/systemd"
+  - name: etc-kubernetes
+    hostPath:
+      path: "/etc/kubernetes"
+  - name: usr-bin
+    hostPath:
+      path: "/usr/bin"
+```
+
+_Note: All these values came from reading the [sample code](https://github.com/aquasecurity/kube-bench/blob/master/job.yaml) in the kube-bench repo._
+
+Lastly, we need to understand that the CIS benchmarks come in two different flavors: one for worker nodes and one for master nodes. Therefore, we need to run different versions of the plugin on the different nodes. To accomplish this, we will:
+
+- Add the appropriate [node affinity](https://kubernetes.io/docs/concepts/configuration/assign-pod-node/#affinity-and-anti-affinity) to the worker node plugin so it runs on the right nodes.
+- Copy the plugin definition to a new file and transform the necessary values for the master plugin name, command, and node affinity.
+
+The affinity section for the worker node plugin should look like this:
+
+```
+affinity:
+    nodeAffinity: 
+      requiredDuringSchedulingIgnoredDuringExecution: 
+        nodeSelectorTerms:
+        - matchExpressions:
+          - key: node-role.kubernetes.io/master
+            operator: DoesNotExist
+```
+
+For the master nodes, we need to do three things:
+
+- Change `DoesNotExist` to `Exists` to target the correct nodes
+- Change the plugin name so the plugins have unique names, which is a Sonobuoy requirement
+- Tweak the containers command so that the correct set of checks is run
+
+Each of these things can be done manually, or with the following command:
+
+```
+$ cat kub-bench-worker.yaml | \
+  sed ‘/kube-bench-worker/kube-bench-master/’ | \
+  sed ‘/- kube-bench/- kube-bench master/’ | \
+  sed ‘/Exists/DoesNotExist/’ > kube-bench-master.yaml
+```
+
+Now you can run the plugins with the following command:
+
+```
+$ sonobuoy run \
+  --plugin kube-bench-worker.yaml \
+  --plugin kube-bench-master.yaml \
+  --wait
+```
+
+## Summary
+
+Now that a Sonobuoy plugin exists for the CIS Kubernetes Benchmark, you can easily integrate security tests into your workflows and feel more confident in your Kubernetes deployment configuration. Sonobuoy has made great improvements to unblock advanced use cases like this, and we hope to continue providing more valuable feedback about your clusters in an increasingly simple format.
+
+
+
+Join the Sonobuoy community:
+
+- Get updates on Twitter ([@projectsonobuoy](https://twitter.com/projectsonobuoy))
+- Chat with us on the Kubernetes Slack ([#sonobuoy](https://kubernetes.slack.com/messages/sonobuoy))
+- Join the [Kubernetes Software Conformance Working Group](https://github.com/cncf/k8s-conformance)