Kubernetes scanning from remote

[mirisbowring]

Hi,

currently, there are 2 methods on scanning a kubernetes cluster available and I am not sure which one to choose or whether any of them are actually appropriate...

We have some dedicated Kubernetes Clusters and some shared ones (User get only access to specific namespaces). The shared ones also have strict policies like disabling hostPath option.

What would be the correct way of scanning?

Should the cluster operator execute the scans via the kubeconfig and make the results available to the corresponding applications?
Should one scan from "outside" and export the kubeconfig to the scanning host (so all kubeconfigs on one machine with a prowler instance)?

Somehow none of the options feel really good. Other solutions are using kubernetes operators to be installed on the master node.

What would be the correct way of scanning all clusters (ideally from remote) to make the results available to the individuals?

[MrCloudSec]

Hi @mirisbowring,

Great questions! Here’s a quick breakdown:

Cluster Operator Scanning via Kubeconfig

This is a valid approach, especially for dedicated clusters. Operators can scan using Prowler and share results with users. For shared clusters, you can limit scans to specific namespaces with the --namespace flag to comply with security policies.

External Scanning with Exported Kubeconfigs

This works well for managing multiple clusters. You can centralize kubeconfigs on one machine and run scans using Prowler’s --kubeconfig-file and --context flags. Just ensure the scanning host is secure since it holds access to multiple clusters.

Kubernetes Operators on Master Node

While this automates scanning, strict policies (like disabling hostPath) in shared clusters may conflict with operator permissions, so this might not be ideal for all environments.

Recommended Approach:

For dedicated clusters, either method works. For shared clusters, external scanning is likely the safest and most compliant.

Let me know if you need further clarification!

[mirisbowring]

Thank you very much for the detailed response! Will take this into our considerations!

[mirisbowring]

Hi, I am coming back with a more technical question since we tried multiple ways.
In detail, we have "problems" with the Shared Clusters.

Permissions for remote scanning

Since we also need to somehow map the results to specific applications and export them, we tried the (in theory) easy approach:

1. A Cluster User (can manage namespaces and resources within) hands over a valid kubeconfig and enters all neccessary mapping details.
2. The central instance starts the periodic scan on the specified namespace

Problem is, that the scan from remote fails because the kube provider wants to list all clusterrolebindings. Unfortunately, the users don't have permissions to do so.
We also tried to specify a single specific namespace for the scan and reduce the scanned categories to container-security only.

The question is: Is it expected that the prowler kubernetes provider is not able to setup if he has no clusterrolebinding persmissions? In the example above the check below (that fails the scan) is not neccessary:

prowler/prowler/providers/kubernetes/kubernetes_provider.py

Line 367 in d997ebb

rbac_api.list_cluster_role_binding().items,

"Realtime" Scanning

How would one achieve a (near-)"realtime" scan? So instead of scheduling it x-times a day, the scan would be executed on a specific namespace, as soon as something changes within.

I think this is only (rational) possible with a kubernetes operator right? so basically we would need to create an operator that would listen to changes and then call the scan on this namespace. Do you have a different Idea?

[MrCloudSec]

Good catch, I lowered the error level and Prowler will not raise a error anymore if it does not have permissions to list the roles (#5630).

About the realtime scanning, you would need to create an infrastructure to trigger Prowler when a change is made since Prowler does not have the ability to scan in real time yet.

Thanks for the feedback @mirisbowring !