Permissions required by kube-monkey?

[KITSDanMassie]

What permissions does kube-monkey require to run? I am trying to run it in a cluster with RBAC enabled and have assigned a service account with the below cluster role to the container:

kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: kube-monkey
rules:
- apiGroups: ["", "extensions", "apps"]
  resources: ["deployments"]
  verbs:  ["get", "list", "watch"]
- apiGroups: [""]
  resources: ["namespaces"]
  verbs: ["get","list","watch"]
- apiGroups: [""]
  resources: ["pods"]
  verbs:  ["get", "list", "watch", "delete"]

kube-monkey runs, but fails to delete any pods as it is unable to see the running pods for any deployment:

I1204 12:21:18.610160       1 config.go:71] Successfully validated configs
I1204 12:21:18.610604       1 main.go:52] Starting kube-monkey with v logging level 5 and local log directory /var/log/kube-monkey
I1204 12:21:18.635426       1 kubemonkey.go:26] Generating next schedule at 2017-12-05 08:00:00 +0000 UTC
I1205 08:00:00.000198       1 schedule.go:43] Generating schedule for terminations
I1205 08:00:00.010612       1 schedule.go:28] 	********** Today's schedule **********
I1205 08:00:00.010625       1 schedule.go:32] 	Deployment		Termination time
I1205 08:00:00.010628       1 schedule.go:33] 	----------		----------------
I1205 08:00:00.010634       1 schedule.go:35] 	my-api		2017-12-05 11:26:00 +0000 UTC
I1205 08:00:00.010649       1 schedule.go:39] 	********** End of schedule **********
I1205 08:00:00.010664       1 kubemonkey.go:64] Waiting for terminations to run
E1205 11:26:00.014078       1 kubemonkey.go:70] Failed to execute termination for deployment my-api Error: Deployment my-api has no running pods at the moment
I1205 11:26:00.014440       1 kubemonkey.go:77] All terminations done
I1205 11:26:00.014478       1 kubemonkey.go:26] Generating next schedule at 2017-12-06 08:00:00 +0000 UTC

This is in a 1.7.11 cluster, could it be due to incompatibilities with the client lib?

[asobti]

E1205 11:26:00.014078 1 kubemonkey.go:70] Failed to execute termination for deployment my-api Error: Deployment my-api has no running pods at the moment

Have you confirmed that my-api has Pods in the Running state? If yes, then it seems to be a permissions issue fetching the list of pods.

I unfortunately do not have an RBAC cluster at my disposal to test/repro this. Is there a way for you to give kube-monkey ALL permissions to verify if that resolves the issue?

This is in a 1.7.11 cluster, could it be due to incompatibilities with the client lib?

This is quite possible too. The version of client-lib used by kube-monkey does not support 1.7, although I am not aware of any breaking changes to the get pods or delete pods API that should affect us.

[KITSDanMassie]

Figured it out. It seems its due to the monkey-victim example missing labels. The RBAC permissions above are sufficient.

The deployment needs labels in addition to the spec template.

---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: monkey-victim
  namespace: app-namespace
  labels:
    kube-monkey/enabled: enabled
    kube-monkey/identifier: monkey-victim
    kube-monkey/mtbf: '2'
spec:
  template:
    metadata:
      labels:
        kube-monkey/enabled: enabled
        kube-monkey/identifier: monkey-victim
[... omitted ...]