There is an initial framework for the cluster helm test

[izgeri]

Overview

After a Kubernetes administrator has run helm install ... or helm upgrade ... using the cluster prep Helm chart, we would like to provide a way for the admin to validate the Helm release, i.e. to prove that the Kubernetes objects that have been deployed can be trusted to support the deployment of applications that will use the associated authn-k8s authenticator.

This will require the implementation of a Helm test (see https://helm.sh/docs/topics/chart_tests/) that can be run on-demand by the Kubernetes administrator after helm install ... and helm upgrade ....

Helm tests typically use the deployment of Kubernetes Pods or Jobs to run custom test applications to exercise the functionality of the Release that they're designed to test.

For the cluster prep Helm chart, the Helm test will refer to the data in the Golden ConfigMap (using a Pod volume mount), and will use curl and openssl to access the configured Conjur instance (based on Conjur URL) to validate that the configuration is correct for this Conjur instance.

For an example of a Helm test as a reference, see:
https://github.com/cyberark/conjur-oss-helm-chart/tree/master/conjur-oss/templates/tests

Tests to be Performed

NOTE: This user story covers creating just the basic framework for the cluster prep Helm chart.
More tests will be added subsequently via Issue #230, and Issue #231.

For the initial implementation of the Helm test for the Kubernetes cluster prep Helm chart,
we'll include only a couple of simple tests (the Golden ConfigMap will be volume-mounted to the Helm test Pod/Job):

• curl -k <Conjur URL>
  This verifies that the Conjur server can be reached (i.e. logical ping) using the Conjur URL,
  regardless of whether the Conjur SSL certificate is accurate or not.
• Use the openssl utility to retrieve the Conjur SSL certificate using the configured Conjur URL,
  and verify that the retrieved SSL certificate matches the configured Conjur SSL certificate.
• OPTIONAL: Add more SSL certificate checking using openssl???

Required Components for Helm Test Framework

Dockerfile

The Helm test will require a custom image that contains:

• curl
• openssl
  Here's a rough idea of what's needed:

FROM alpine:3.12

RUN mkdir -p /tests
WORKDIR /tests

# Install Docker client
RUN apk add --no-cache curl openssl bash

ENTRYPOINT [ "/tests/test" ]

build script

We'll need a build script to build a Docker image via the Dockerfile described above

Manifest for test Pod/Job

A Kubernetes manifest for a Pod or Job to run the test scripts is needed.
Here is a rough idea of what's needed, based on an early P.O.C.:

apiVersion: v1
kind: Pod
metadata:
  name: {{ .Release.Name }}-cluster-prep-test
  labels:
  annotations:
    "helm.sh/hook": test-success
spec:
  initContainers:
  - name: {{ .Release.Name }}-bats-init
    image: bats/bats:v1.1
    command:
    - "bash"
    - "-exc"
    - |
      # copy bats to tools dir
      cp -R /opt/bats/libexec/bats-core/ /tools/bats/
    volumeMounts:
    - mountPath: /tools
      name: tools
  containers:
  - name: {{ .Release.Name }}-test
    image: diverdane/conjur-cluster-prep-test:latest
    workingDir: "/tools/bats"
    command: ["/tools/bats/bats", "-t", "/tests/run.sh"]
    env:
    - name: PATH
      value: "/tools/bats:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
    envFrom: 
    - configMapRef: 
        name: authn-k8s-config-map
    volumeMounts:
    - mountPath: /tests
      name: tests
      readOnly: true
    - mountPath: /tools
      name: tools
  volumes:
  - name: tests
    configMap:
      name: {{ .Release.Name }}-tests-configmap
  - name: tools
    emptyDir: {}
  restartPolicy: Never

OPTIONAL: Manifest for test ConfigMap containing 'bats' test scripts

Optionally, the tests scripts can be provided to the test Pod/Job as a ConfigMap,
similar to what's done for the Conjur OSS Helm chart Helm test.

Here's a rough idea of what's needed for the first curl -k ... test, based on an early P.O.C.:

apiVersion: v1
kind: ConfigMap
metadata:
  name: {{ .Release.Name }}-tests-configmap
data:
  run.sh: |-
    @test "Testing basic connectivity to Conjur" {
        curl -k "$conjurApplianceUrl:443"
    }

Test Results Visiblity

The test results for failed test cases must be visible on the helm test command line.
When a failure occurs, it should be clear to the person running the test what the specific failure was.
It might be sufficient to dump the test Pod/Job logs when failure occurs.

DoD

• Helm test components created:
  • Dockerfile
  • Build/push script
  • Test Pod/Job manifest
  • Optional: Test ConfigMap
• Helm test happy path works
• Helm test fails for invalid Conjur config
• Failures are clearly described in helm test ... command output
• Test image is pushed for builds on master branch using version as a tag.

[izgeri]

1 point rolling into May 3 sprint