Kubernetes test automation and demo materials are revised

[izgeri]

The problem

At current, Kubernetes end-to-end test automation primarily happens in the Kubernetes Conjur Demo project, which was not designed to be an end-to-end test project and whose experience as a user-facing demo for our Kubernetes integrations suffers from the mixed concerns.

The test automation has additional challenges, including:

• it does not do a good job of covering our test matrix (more details in the test matrix section below)
• it deploys the Conjur leader cluster to Kubernetes, which is not recommended for production use cases and is not a realistic setup
• it mainly uses old logic for policy loading, and requires that hosts be in a specific policy branch and that the hosts are defined in a way that is no longer documented
• it does a poor job of separating the logic into actions needed to be completed by specific personas, and is thus not well aligned with our Kubernetes best practices (draft private docs here)
• our test matrix does not include flows that use different Kubernetes resources for authentication to Conjur

The plan

We would like to revise our automated e2e test strategy for Kubernetes / OpenShift, and to take the opportunity to improve on the challenges listed in the problem section.

As a first pass, we've created a helm chart for deploying the sample pet store app in #238. We've migrated the main Kubernetes-Conjur Demo scripts into this project in #239. We're working on building an e2e flow running in this project that runs the following flow:

• Conjur OSS
• Kubernetes (KinD)
• Authn-K8s client (?? init container or sidecar?)
• Service account-based Conjur authentication (??)
• Pet store app with DB config via env vars injected by Summon
• PostgreSQL DB for test app

The remaining cards in this epic reflect the work to add support for running these test flows with:

• Conjur Editions
  • Conjur Open Source
  • Conjur Enterprise / Follower in cluster
  • Conjur Enterprise / Follower outside cluster
• Sidecars
  • Secrets Provider init container
  • Authn-K8s client (init container / sidecar)
  • Secretless
• DB backends (for Secretless flows only)
  • MySQL
  • MSSQL
• Kubernetes editions
  • GKE
  • OpenShift (multiple versions)
  • EKS (future?)
• Conjur K8s auth resources
  • Namespace
  • Deployment
  • DeploymentConfig (OpenShift only)
  • StatefulSet
  • ServiceAccount
  • Pod

In addition, we will build a quick start flow based on our test automation scripts that enables technical consumers of our content to trial our features.

We will also build a simple Katacoda tutorial to enable users to explore our Conjur Kubernetes authentication.

Once this is all complete, the Kubernetes Conjur Demo project can be deprecated and archived.

Test matrix

We outlined above the set of scenarios we'd like to cover in our test automation. The first step is ensuring our test automation is flexible enough to support all of the possible configurations outlined above. Given that, however, we can leverage web PICT with the following input to generate a reasonable matrix of test flows that we should be running.

Input:

Conjur Edition: Open Source, Enterprise w/Internal Follower, Enterprise w/External Follower
K8s client: Secretless, Authn-K8s Client Init Container, Authn-K8s Client Sidecar, Secrets Provider Init Container, Secrets Provider Job
Database: PostgreSQL, MySQL, MSSQL
Kubernetes version: KinD, GKE, Openshift 3.11, OpenShift Oldest, OpenShift Current
Auth resource: Namespace, Deployment, DeploymentConfig, StatefulSet, ServiceAccount, Pod

IF [K8s client] <> "Secretless" THEN [Database] = "PostgreSQL";
IF [Conjur Edition] = "Open Source" THEN [Kubernetes version] = "KinD";
IF [Conjur Edition] <> "Open Source" THEN [Kubernetes version] <> "KinD";

Conjur Edition	K8s client	Database	Kubernetes version	Auth resource
Enterprise w/Internal Follower	Secrets Provider Job	PostgreSQL	Openshift 3.11	Deployment
Enterprise w/External Follower	Secretless	MSSQL	OpenShift Current	Deployment
Enterprise w/External Follower	Authn-K8s Client Init Container	PostgreSQL	OpenShift Oldest	Namespace
Enterprise w/External Follower	Secrets Provider Job	PostgreSQL	GKE	DeploymentConfig
Enterprise w/Internal Follower	Secretless	MySQL	OpenShift Oldest	DeploymentConfig
Open Source	Authn-K8s Client Sidecar	PostgreSQL	KinD	StatefulSet
Enterprise w/Internal Follower	Secretless	MySQL	GKE	Namespace
Enterprise w/External Follower	Secrets Provider Init Container	PostgreSQL	Openshift 3.11	StatefulSet
Enterprise w/Internal Follower	Secretless	MSSQL	GKE	Pod
Enterprise w/External Follower	Secretless	MSSQL	Openshift 3.11	Namespace
Enterprise w/Internal Follower	Authn-K8s Client Sidecar	PostgreSQL	OpenShift Current	Namespace
Open Source	Secrets Provider Init Container	PostgreSQL	KinD	DeploymentConfig
Open Source	Secrets Provider Job	PostgreSQL	KinD	ServiceAccount
Enterprise w/Internal Follower	Secrets Provider Init Container	PostgreSQL	OpenShift Oldest	ServiceAccount
Open Source	Secrets Provider Init Container	PostgreSQL	KinD	Namespace
Enterprise w/External Follower	Secrets Provider Init Container	PostgreSQL	OpenShift Current	Pod
Enterprise w/Internal Follower	Authn-K8s Client Init Container	PostgreSQL	OpenShift Current	StatefulSet
Enterprise w/External Follower	Authn-K8s Client Init Container	PostgreSQL	Openshift 3.11	DeploymentConfig
Open Source	Authn-K8s Client Init Container	PostgreSQL	KinD	Pod
Enterprise w/External Follower	Authn-K8s Client Sidecar	PostgreSQL	Openshift 3.11	DeploymentConfig
Enterprise w/External Follower	Secretless	MySQL	GKE	StatefulSet
Enterprise w/External Follower	Secrets Provider Job	PostgreSQL	OpenShift Oldest	StatefulSet
Enterprise w/External Follower	Authn-K8s Client Sidecar	PostgreSQL	OpenShift Oldest	Pod
Enterprise w/Internal Follower	Authn-K8s Client Init Container	PostgreSQL	GKE	Deployment
Open Source	Secretless	MySQL	KinD	ServiceAccount
Enterprise w/External Follower	Secretless	MSSQL	OpenShift Current	ServiceAccount
Enterprise w/Internal Follower	Secretless	MSSQL	OpenShift Current	DeploymentConfig
Enterprise w/Internal Follower	Authn-K8s Client Sidecar	PostgreSQL	GKE	ServiceAccount
Enterprise w/Internal Follower	Secrets Provider Job	PostgreSQL	OpenShift Current	Namespace
Enterprise w/External Follower	Secretless	MySQL	Openshift 3.11	Pod
Enterprise w/External Follower	Secrets Provider Init Container	PostgreSQL	OpenShift Oldest	Deployment
Enterprise w/Internal Follower	Secretless	MSSQL	OpenShift Oldest	StatefulSet
Open Source	Secrets Provider Job	PostgreSQL	KinD	Pod
Open Source	Authn-K8s Client Sidecar	PostgreSQL	KinD	Deployment
Open Source	Secretless	MSSQL	KinD	ServiceAccount
Enterprise w/External Follower	Authn-K8s Client Init Container	PostgreSQL	Openshift 3.11	ServiceAccount
Open Source	Secretless	PostgreSQL	KinD	ServiceAccount
Enterprise w/External Follower	Secretless	MySQL	OpenShift Current	Deployment
Enterprise w/External Follower	Secrets Provider Init Container	PostgreSQL	GKE	StatefulSet