Simplified Merge-All #33

Aergonus · 2018-01-03T00:54:17Z

Aggregates #18, #19, #24, #25, #27, #28, #29, #30, #31, #34

added some extra guiding instructions

Technically shouldn't force the user to use the kube-monkey namespace

asobti#22

Had to run `glide cache-clear` and then `glide update --strip-vendor`

Allow people to copy pasta ;) asobti#23

Only relevant after asobti#24

CreateClient involves creating a clientset that accesses the k8 api It does not belong in chaos but rather kubernetes

New sublocation for all kinds

run with make gofmt

standardized variables to loc added timezone for debugging printing specified format printout for schedule addresses asobti#21 and asobti#23

asobti#25

Possible after templatizing in asobti#27 and reference issue #10

Allow victims to do the heavylifting for common methods

Whitelists and Blacklists both operate. Blacklist overrides a whitelist.

See asobti#29

Log improvements

K8 upgrade

Templatize

Print time fixes

Statefulsets

Whitelist

Newbie

With all these changes it's time for a version bump

version++

glide up --v && v0.2.0

Aergonus · 2018-01-24T00:10:16Z

deploy-test.yaml

---
apiVersion: v1
kind: Namespace
metadata:
  name: test-system
---
  apiVersion: extensions/v1beta1
  kind: Deployment
  metadata:
    name: counter
    namespace: test-system
  spec:
    replicas: 2
    template:
      metadata:
        labels:
          kube-monkey/enabled: enabled
          kube-monkey/identifier: monkey-victim-counter
          kube-monkey/mtbf: "1"
          kube-monkey/kill-mode: "kill-all"
      spec:
        containers:
        - args:
          - /bin/sh
          - -c
          - 'i=0; while true; do echo "$i: $(date)"; i=$((i+1)); sleep 1; done'
          image: busybox:latest
          imagePullPolicy: Always
          name: test-counter
        restartPolicy: Always
---
  apiVersion: extensions/v1beta1
  kind: Deployment
  metadata:
    name: counter-delete-all-fail-test
    namespace: test-system
  spec:
    replicas: 2
    template:
      metadata:
        labels:
          kube-monkey/enabled: enabled
          kube-monkey/identifier: monkey-victim-delete-all-fail-test
          kube-monkey/mtbf: "1"
          kube-monkey/kill-mode: "kill-all"
      spec:
        containers:
        - args:
          - /bin/sh
          - -c
          - 'i=0; while true; do echo "$i: $(date)"; i=$((i+1)); sleep 1; done'
          image: DOESNOTEXIST:latest
          imagePullPolicy: Always
          name: test-counter-delete-all-fail-test
        restartPolicy: Always
---
  apiVersion: extensions/v1beta1
  kind: Deployment
  metadata:
    name: singlecounter
    namespace: test-system
  spec:
    replicas: 1
    template:
      metadata:
        labels:
          kube-monkey/enabled: enabled
          kube-monkey/identifier: monkey-victim-single-counter
          kube-monkey/mtbf: "2"
          kube-monkey/kill-mode: "fixed"
          kube-monkey/kill-value: "2"
      spec:
        containers:
        - args:
          - /bin/sh
          - -c
          - 'i=0; while true; do echo "$i: $(date)"; i=$((i+1)); sleep 1; done'
          image: busybox:latest
          imagePullPolicy: Always
          name: test-single-counter
        restartPolicy: Always
---
  apiVersion: extensions/v1beta1
  kind: Deployment
  metadata:
    name: opt-out-counter
    namespace: test-system
  spec:
    replicas: 1
    template:
      metadata:
        labels:
          kube-monkey/enabled: disabled
          kube-monkey/identifier: not-a-counter-victim
          kube-monkey/mtbf: "1"
          kube-monkey/kill-mode: "kill-all"
      spec:
        containers:
        - args:
          - /bin/sh
          - -c
          - 'i=0; while true; do echo "$i: $(date)"; i=$((i+1)); sleep 86400; done'
          image: busybox:latest
          imagePullPolicy: Always
          name: not-a-counter-victim
        restartPolicy: Always
---
  apiVersion: extensions/v1beta1
  kind: Deployment
  metadata:
    name: fail-deploy
    namespace: test-system
  spec:
    replicas: 2
    template:
      metadata:
        labels:
          kube-monkey/enabled: enabled
          kube-monkey/identifier: failure-test
          kube-monkey/mtbf: "1"
          kube-monkey/kill-mode: "fixed"
          kube-monkey/kill-value: "2"
      spec:
        containers:
        - args:
          - /bin/sh
          - -c
          - 'i=0; while true; do echo "$i: $(date)"; i=$((i+1)); sleep 300; done'
          image: invalidimage
          imagePullPolicy: Always
          name: fail-deploy
        restartPolicy: Always
---
# missing-secret.yaml
# a failed deployment
  apiVersion: extensions/v1beta1
  kind: Deployment
  metadata:
    name: secret-pod
    namespace: test-system
  spec:
    replicas: 2
    template:
      metadata:
        labels:
          kube-monkey/enabled: enabled
          kube-monkey/identifier: missing-secret-test
          kube-monkey/mtbf: "1"
          kube-monkey/kill-mode: "random-max-percent"
          kube-monkey/kill-value: "2"
      spec:
        containers:
          - name: missing-secret-test
            image: busybox:latest
            command: [ "/bin/sh", "-c", "env" ]
            volumeMounts:
              - mountPath: /etc/secret/
                name: myothersecret
        volumes:
          - name: myothersecret
            secret:
              secretName: myothersecret
        restartPolicy: Always

ss-test.yaml

---
apiVersion: v1
kind: Namespace
metadata:
  name: test-system
---
apiVersion: v1
kind: Service
metadata:
  labels:
    app: counter-ss
  name: counter-ss
  namespace: test-system
spec:
  clusterIP: None
  ports:
  - port: 9001
  selector:
    app: counter-ss
---
  apiVersion: apps/v1beta1
  kind: StatefulSet
  metadata:
    name: counter-ss
    namespace: test-system
  spec:
    serviceName: counter-ss
    replicas: 3
    template:
      metadata:
        labels:
          kube-monkey/enabled: enabled
          kube-monkey/identifier: monkey-victim-counter-ss
          kube-monkey/mtbf: "1"
          kube-monkey/kill-mode: "kill-all"
      spec:
        containers:
        - args:
          - /bin/sh
          - -c
          - 'i=0; while true; do echo "$i: $(date)"; i=$((i+1)); sleep 1; done'
          image: busybox:latest
          imagePullPolicy: Always
          name: test-counter-ss
        restartPolicy: Always
---
apiVersion: v1
kind: Service
metadata:
  labels:
    app: counter-delete-all-fail-test-ss
  name: counter-delete-all-fail-test-ss
  namespace: test-system
spec:
  clusterIP: None
  ports:
  - port: 9002
  selector:
    app: counter-delete-all-fail-test-ss
---
  apiVersion: apps/v1beta1
  kind: StatefulSet
  metadata:
    name: counter-delete-all-fail-test-ss
    namespace: test-system
  spec:
    serviceName: counter-delete-all-fail-test-ss
    replicas: 3
    template:
      metadata:
        labels:
          kube-monkey/enabled: enabled
          kube-monkey/identifier: monkey-victim-delete-all-fail-test-ss
          kube-monkey/mtbf: "1"
          kube-monkey/kill-mode: "kill-all"
      spec:
        containers:
        - args:
          - /bin/sh
          - -c
          - 'i=0; while true; do echo "$i: $(date)"; i=$((i+1)); sleep 1; done'
          image: DOESNOTEXIST:latest
          imagePullPolicy: Always
          name: test-counter-delete-all-fail-test-ss
        restartPolicy: Always
---
apiVersion: v1
kind: Service
metadata:
  labels:
    app: singlecounter-ss
  name: singlecounter-ss
  namespace: test-system
spec:
  clusterIP: None
  ports:
  - port: 9003
  selector:
    app: singlecounter-ss
---
  apiVersion: apps/v1beta1
  kind: StatefulSet
  metadata:
    name: singlecounter-ss
    namespace: test-system
  spec:
    serviceName: singlecounter-ss
    replicas: 1
    template:
      metadata:
        labels:
          kube-monkey/enabled: enabled
          kube-monkey/identifier: monkey-victim-single-counter-ss
          kube-monkey/mtbf: "2"
          kube-monkey/kill-mode: "fixed"
          kube-monkey/kill-value: "2"
      spec:
        containers:
        - args:
          - /bin/sh
          - -c
          - 'i=0; while true; do echo "$i: $(date)"; i=$((i+1)); sleep 1; done'
          image: busybox:latest
          imagePullPolicy: Always
          name: test-single-counter-ss
        restartPolicy: Always
---
apiVersion: v1
kind: Service
metadata:
  labels:
    app: opt-out-counter-ss
  name: opt-out-counter-ss
  namespace: test-system
spec:
  clusterIP: None
  ports:
  - port: 9004
  selector:
    app: opt-out-counter-ss
---
  apiVersion: apps/v1beta1
  kind: StatefulSet
  metadata:
    name: opt-out-counter-ss
    namespace: test-system
  spec:
    serviceName: opt-out-counter-ss
    replicas: 1
    template:
      metadata:
        labels:
          kube-monkey/enabled: disabled
          kube-monkey/identifier: not-a-counter-victim-ss
          kube-monkey/mtbf: "1"
          kube-monkey/kill-mode: "kill-all"
      spec:
        containers:
        - args:
          - /bin/sh
          - -c
          - 'i=0; while true; do echo "$i: $(date)"; i=$((i+1)); sleep 86400; done'
          image: busybox:latest
          imagePullPolicy: Always
          name: not-a-counter-victim-ss
        restartPolicy: Always
---
apiVersion: v1
kind: Service
metadata:
  labels:
    app: fail-deploy-ss
  name: fail-deploy-ss
  namespace: test-system
spec:
  clusterIP: None
  ports:
  - port: 9005
  selector:
    app: fail-deploy-ss
---
  apiVersion: apps/v1beta1
  kind: StatefulSet
  metadata:
    name: fail-deploy-ss
    namespace: test-system
  spec:
    serviceName: fail-deploy-ss
    replicas: 2
    template:
      metadata:
        labels:
          kube-monkey/enabled: enabled
          kube-monkey/identifier: failure-test-ss
          kube-monkey/mtbf: "1"
          kube-monkey/kill-mode: "random-max-percent"
          kube-monkey/kill-value: "75"
      spec:
        containers:
        - args:
          - /bin/sh
          - -c
          - 'i=0; while true; do echo "$i: $(date)"; i=$((i+1)); sleep 300; done'
          image: invalidimage
          imagePullPolicy: Always
          name: fail-deploy-ss
        restartPolicy: Always
---
apiVersion: v1
kind: Service
metadata:
  labels:
    app: secret-pod-ss
  name: secret-pod-ss
  namespace: test-system
spec:
  clusterIP: None
  ports:
  - port: 9006
  selector:
    app: secret-pod-ss
---
# missing-secret.yaml
# a failed deployment
  apiVersion: apps/v1beta1
  kind: StatefulSet
  metadata:
    name: secret-pod-ss
    namespace: test-system
  spec:
    serviceName: secret-pod-ss
    replicas: 2
    template:
      metadata:
        labels:
          kube-monkey/enabled: enabled
          kube-monkey/identifier: missing-secret-test-ss
          kube-monkey/mtbf: "1"
          kube-monkey/kill-mode: "random-max-percent"
          kube-monkey/kill-value: "75"
      spec:
        containers:
          - name: missing-secret-test-ss
            image: busybox:latest
            command: [ "/bin/sh", "-c", "env" ]
            volumeMounts:
              - mountPath: /etc/secret/
                name: myothersecret
        volumes:
          - name: myothersecret
            secret:
              secretName: myothersecret
        restartPolicy: Always

config.toml

  config.toml: |
    [kubemonkey]
    dry_run = false
    time_zone = "America/New_York"
    run_hour = 8
    start_hour = 10
    end_hour = 16
    graceperiod_sec = 10
    blacklisted_namespaces = ["kube-system"]
    whitelisted_namespaces = ["default", "test-system"]
    [debug]
    enabled = false
    schedule_delay = 10
    force_should_kill = false
    schedule_immediate_kill = true

Aergonus · 2018-01-24T00:31:31Z

I just updated the glide.lock by rewriting it (y)

asobti · 2018-01-24T01:04:55Z

chaos/chaos.go

-	return c.DeletePod(client, targetPod)
-}
+	// Has the victim been removed from the whitelist since scheduling?
+	if !c.Victim().IsWhitelisted() {


@Spellchaser Does this mean that each victim has to specifically be listed in the whitelist for kube-monkey to act on it?

Yes, but the default for whitelisting is all namespaces. So if you don't enable a whitelist, it doesn't affect you.

I suppose a better comment would be something like
// Does the namespace whitelist still include the victim's namespace now at killtime?

It looks like the default is NamespaceDefault

viper.SetDefault(param.WhitelistedNamespaces, []string{metav1.NamespaceDefault})

Is that the same as NamespaceAll?

Nada, NamespaceDefault is "default" NamespaceAll is ""
I figured it'd be safer, abliet more confusing for users

So, if I don't specify the whitelisted_namespaces in my config file, my victims will not be killed (assuming they are in their own namespace) ?

Ok, ignore that. I saw the following commit where you update docs for the whitelisting stuff and that clears it up.

Aergonus · 2018-01-24T02:01:34Z

Not entirely... If you didn't specify whitelist in the configs it would default to whitelisting the `default` namespace. Your deployments in `default` still have to be opted in. If you whitelisted only `test-namespace`, only deployments in `test-namespace` could opt in. On Jan 23, 2018 8:55 PM, "Ayush Sobti" <[email protected]> wrote: *@asobti* commented on this pull request. ------------------------------ In chaos/chaos.go <#33 (comment)>:

- glog.V(2).Infof("Terminating pod %s for deployment %s", targetPod, c.deployment.Name()) - return c.DeletePod(client, targetPod) -} + // Has the victim been removed from the whitelist since scheduling? + if !c.Victim().IsWhitelisted() { So, if I don't specify the whitelisted_namespaces in my config file, my victims will not be killed (assuming they are in their own namespace) ? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#33 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ACV_i6AiQDcJWnIo64dow2N8rfy21F4Kks5tNo14gaJpZM4RRPx6> .

Aergonus added 30 commits October 29, 2017 17:22

Support copy pasta

855b60f

KISS

1e958a1

added some extra guiding instructions

Clarifying CM Namespace

ed47ef1

Technically shouldn't force the user to use the kube-monkey namespace

Including apiserver overload info

46c90e5

Todo --> Contribute

0390bc1

Updated after logging merge

73d6e3e

Spelling Fix

bdebdee

Deployment identifier example

d824e55

asobti#22

Client-go update

c7976f9

Glide and vendor update

970ed42

Had to run `glide cache-clear` and then `glide update --strip-vendor`

Add timezone to example config

dcfff06

Allow people to copy pasta ;) asobti#23

Merge branch 'master' into newbie

05412e0

Update Logging Conventions

a75c85d

Only relevant after asobti#24

Refactor CreateClient

655399e

CreateClient involves creating a clientset that accesses the k8 api It does not belong in chaos but rather kubernetes

Templatize for victims

ac12218

Move deployments to victims factory

84ca6da

New sublocation for all kinds

Convert deployment to victim template

166a83d

go fmt codebase

b768751

run with make gofmt

Print time fixes

37b4e42

standardized variables to loc added timezone for debugging printing specified format printout for schedule addresses asobti#21 and asobti#23

Upgraded client-go

801e7b0

asobti#25

Add new contribution types

0d348f0

Possible after templatizing in asobti#27 and reference issue #10

Refactor deployments --> victims

e43f906

Allow victims to do the heavylifting for common methods

Copy deployments --> statefulsets

dc6ab73

Add statefulsets

9ffa53c

Add Whitelisting Namespaces

fa00c5e

Whitelists and Blacklists both operate. Blacklist overrides a whitelist.

Update docs for whitelist

a112f81

Added statefulsets

c70db49

See asobti#29

Merge pull request #5 from Spellchaser/log_improvements

a1ec881

Log improvements

Merge branch 'master' into k8_upgrade

1f7cc77

Merge branch 'master' into k8_upgrade

251dcd0

Aergonus added 12 commits January 2, 2018 19:19

Merge pull request #6 from Spellchaser/k8_upgrade

d85c41b

K8 upgrade

Merge branch 'master' into templatize

6bb53b2

Merge pull request #7 from Spellchaser/templatize

973d618

Templatize

Merge pull request #8 from Spellchaser/timezone-print

f04edfe

Print time fixes

Merge pull request #9 from Spellchaser/statefulsets

6475c6b

Statefulsets

Merge pull request #10 from Spellchaser/whitelist

99d1de8

Whitelist

Merge branch 'master' into newbie

95a06f0

Merge pull request #11 from Spellchaser/newbie

d94a5fe

Newbie

version++

ac6db8f

With all these changes it's time for a version bump

Merge pull request #12 from Spellchaser/version++

811c3ca

version++

glide up --v && v0.2.0

c3b6f52

Merge pull request #13 from Spellchaser/version++

e48eecd

glide up --v && v0.2.0

Aergonus force-pushed the master branch from 8b12416 to e48eecd Compare January 3, 2018 21:16

Aergonus force-pushed the master branch from 24c8286 to 39f4e79 Compare January 24, 2018 00:22

asobti mentioned this pull request Jan 24, 2018

Update Documentation #18

Merged

asobti reviewed Jan 24, 2018

View reviewed changes

Merge branch 'master' of https://github.com/asobti/kube-monkey

eeb2aae

Aergonus force-pushed the master branch from fb39c8c to eeb2aae Compare January 24, 2018 01:11

asobti approved these changes Jan 24, 2018

View reviewed changes

asobti merged commit aa8fc3a into asobti:master Jan 24, 2018

This was referenced Jan 24, 2018

Version++ #31

Closed

Add option to delete % of victim #34

Merged

Aergonus deleted the master branch January 24, 2018 14:44

Aergonus restored the master branch January 24, 2018 14:45

Aergonus deleted the master branch January 24, 2018 16:51

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Simplified Merge-All #33

Simplified Merge-All #33

Aergonus commented Jan 3, 2018 •

edited

Loading

Aergonus commented Jan 24, 2018

Aergonus commented Jan 24, 2018

asobti Jan 24, 2018

Aergonus Jan 24, 2018

Aergonus Jan 24, 2018 •

edited

Loading

asobti Jan 24, 2018

Aergonus Jan 24, 2018

asobti Jan 24, 2018

asobti Jan 24, 2018

Aergonus commented Jan 24, 2018 via email

Simplified Merge-All #33

Simplified Merge-All #33

Conversation

Aergonus commented Jan 3, 2018 • edited Loading

Aergonus commented Jan 24, 2018

Aergonus commented Jan 24, 2018

asobti Jan 24, 2018

Choose a reason for hiding this comment

Aergonus Jan 24, 2018

Choose a reason for hiding this comment

Aergonus Jan 24, 2018 • edited Loading

Choose a reason for hiding this comment

asobti Jan 24, 2018

Choose a reason for hiding this comment

Aergonus Jan 24, 2018

Choose a reason for hiding this comment

asobti Jan 24, 2018

Choose a reason for hiding this comment

asobti Jan 24, 2018

Choose a reason for hiding this comment

Aergonus commented Jan 24, 2018 via email

Aergonus commented Jan 3, 2018 •

edited

Loading

Aergonus Jan 24, 2018 •

edited

Loading