linkerd check should check for PodSecurityPolicy admission controller

[siggy]

Background

Linkerd user @jamessthompson reported a failing linkerd check --pre, due to PodSecurityPolicy:

$ linkerd check --pre --verbose
...
pre-kubernetes-capability
-------------------------
× has NET_ADMIN capability
    found 7 PodSecurityPolicies, but none provide NET_ADMIN
    see https://linkerd.io/checks/#pre-k8s-cluster-net-admin for hints

Despite the failed check, Linkerd successfully installed. This is likely due to the PodSecurityPolicy admission controller not running.

Reproduction

gcloud beta container clusters create psp-test --enable-pod-security-policy

Proposal

Modify linkerd check --pre to validate whether the PodSecurityPolicy is running, in addition to check the actual PodSecurityPolicy's.

If the admission controller is not running, but PodSecurityPolicy's still exist that would disallow NET_ADMIN, issue a warning.

Additionally, if the admission controller is running, and no PodSecurityPolicy's are found, this check should fail. This is not the case in the current implementation.

Relevant code:

linkerd2/pkg/healthcheck/healthcheck.go

Lines 985 to 988 in bd4c278

	// if PodSecurityPolicies are found, validate one exists that:
	// 1) permits usage
	// AND
	// 2) provides NET_ADMIN

[ihcsim]

@jamessthompson Can you take a look at the Linkerd deployments to see whether the kubernetes.io/psp: annotation has been added. If the PSP admission controller is running, and the service account has the right RBAC to use the policy, this annotation will show you the policy that is in-effect.

Also speaking of RBAC, I think you need to make sure all the Linkerd service accounts have the correct RBAC to use the PSP policy(ies), because when it comes to non-pod workloads like Deployment, the service accounts are the ones creating the pods. See example here.

[jamessthompson]

That annotation does not exist, which matches what we determined yesterday. This ticket is more about providing clarity during installs for people like me that have never used the PSP admissions controller but have 3rd party charts installed that added PSPs. I thought I had a broken install and reached out on slack looking for an example of a valid linkerd PSP.

[alpeb]

When #2920 merges, we'll provide a PSP for Linkerd's control plane that will grant the NET_ADMIN capability for non-CNI installs, which will probably render this particular check moot.

[siggy]

Closed via #2958.