Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Timezone print #28

Merged
merged 12 commits into from
Jan 24, 2018
Merged

Timezone print #28

merged 12 commits into from
Jan 24, 2018

Conversation

Aergonus
Copy link
Collaborator

@Aergonus Aergonus commented Jan 2, 2018

Print time fixes …
standardized variables to loc
added timezone for debugging printing
specified format printout for schedule
addresses #21 and #23

Builds upon #27

Sample output

Every 2.0s: kubectl logs kube-monkey-7bccdc67d6-j6qvp --tail=50                                 Tue Jan  2 00:12:02 2018│Every 2.0s: kubectl get pods --namespace=test-system                                        Tue Jan  2 00:12:03 2018
                                                                                                                        │
I0102 00:09:13.542001       1 config.go:71] Successfully validated configs                                              │NAME                                  READY     STATUS              RESTARTS   AGE
I0102 00:09:13.542878       1 main.go:52] Starting kube-monkey with v logging level 4 and local log directory /var/log/k│counter-564db9859-lfkdj               1/1       Running             0          38s
ube-monkey                                                                                                              │counter-564db9859-nvwh2               1/1       Running             0          38s
E0102 00:09:13.542930       1 config.go:330] Expected to load root CA config from /var/run/secrets/kubernetes.io/service│counter-delete-all-6676949f5b-gtwvg   0/1       InvalidImageName    0          49s
account/ca.crt, but got err: open /var/run/secrets/kubernetes.io/serviceaccount/ca.crt: no such file or directory       │counter-delete-all-6676949f5b-krjxf   0/1       InvalidImageName    0          49s
I0102 00:09:13.547127       1 kubemonkey.go:19] Debug mode detected!                                                    │fail-deploy-6bc9b487f9-rpsxq          0/1       ImagePullBackOff    0          5h
I0102 00:09:13.547137       1 kubemonkey.go:20] Status Update: Generating next schedule in 120 sec                      │fail-deploy-6bc9b487f9-wtzvc          0/1       ImagePullBackOff    0          5h
I0102 00:11:13.547279       1 schedule.go:55] Status Update: Generating schedule for terminations                       │opt-out-counter-6cc59f984f-9dwjw      1/1       Running             0          5h
E0102 00:11:13.547751       1 config.go:330] Expected to load root CA config from /var/run/secrets/kubernetes.io/service│secret-pod-7d869dbf4b-f88gz           0/1       ContainerCreating   0          5h
account/ca.crt, but got err: open /var/run/secrets/kubernetes.io/serviceaccount/ca.crt: no such file or directory       │secret-pod-7d869dbf4b-jmwc4           0/1       ContainerCreating   0          5h
I0102 00:11:13.595900       1 schedule.go:47] Status Update: 4 terminations scheduled today                             │singlecounter-84d7b87fd8-fmxxl        1/1       Running             0          3m
I0102 00:11:13.595920       1 schedule.go:49] Deployment counter scheduled for termination at 01/01/2018 19:11:25 -0500 │
EST                                                                                                                     │
I0102 00:11:13.595925       1 schedule.go:49] Deployment counter-delete-all scheduled for termination at 01/01/2018 19:1│
1:14 -0500 EST                                                                                                          ├────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
I0102 00:11:13.595928       1 schedule.go:49] Deployment fail-deploy scheduled for termination at 01/01/2018 19:11:39 -0│# Please edit the object below. Lines beginning with a '#' will be ignored,
500 EST                                                                                                                 │# and an empty file will abort the edit. If an error occurs while saving this file will be
I0102 00:11:13.595932       1 schedule.go:49] Deployment secret-pod scheduled for termination at 01/01/2018 19:11:45 -05│# reopened with the relevant failures.
00 EST                                                                                                                  │#
I0102 00:11:13.595980       1 kubemonkey.go:63] Status Update: Waiting to run scheduled terminations.                   │apiVersion: v1
        ********** Today's schedule **********                                                                          │data:
        k8 Api Kind     Kind Name               Termination Time                                                        │  config.toml: |
        -----------     ---------               ----------------                                                        │    [kubemonkey]
        Deployment      counter         01/01/2018 19:11:25 -0500 EST                                                   │    dry_run = false
        Deployment      counter-delete-all              01/01/2018 19:11:14 -0500 EST                                   │    time_zone = "America/New_York"
        Deployment      fail-deploy             01/01/2018 19:11:39 -0500 EST                                           │    run_hour = 8
        Deployment      secret-pod              01/01/2018 19:11:45 -0500 EST                                           │    start_hour = 10
        ********** End of schedule **********                                                                           │    end_hour = 16
E0102 00:11:14.595984       1 config.go:330] Expected to load root CA config from /var/run/secrets/kubernetes.io/service│    graceperiod_sec = 10
account/ca.crt, but got err: open /var/run/secrets/kubernetes.io/serviceaccount/ca.crt: no such file or directory       │    blacklisted_namespaces = ["default","kube-system"]
I0102 00:11:14.599551       1 chaos.go:131] Terminating ALL pods for Deployment counter-delete-all                      │    [debug]
I0102 00:11:14.614742       1 kubemonkey.go:71] Termination successfully executed for Deployment counter-delete-all     │    enabled = true
I0102 00:11:14.614756       1 kubemonkey.go:74] Status Update: 3 scheduled terminations left.                           │    schedule_delay = 120
E0102 00:11:25.595916       1 config.go:330] Expected to load root CA config from /var/run/secrets/kubernetes.io/service│    force_should_kill = false
account/ca.crt, but got err: open /var/run/secrets/kubernetes.io/serviceaccount/ca.crt: no such file or directory       │    schedule_immediate_kill = true
I0102 00:11:25.599752       1 chaos.go:131] Terminating ALL pods for Deployment counter                                 │    [kubernetes]
I0102 00:11:25.608324       1 kubemonkey.go:71] Termination successfully executed for Deployment counter                │    host = "<redacted>"
I0102 00:11:25.608335       1 kubemonkey.go:74] Status Update: 2 scheduled terminations left.                           │kind: ConfigMap
E0102 00:11:39.596045       1 config.go:330] Expected to load root CA config from /var/run/secrets/kubernetes.io/service│~
account/ca.crt, but got err: open /var/run/secrets/kubernetes.io/serviceaccount/ca.crt: no such file or directory       │
E0102 00:11:39.602143       1 kubemonkey.go:69] Failed to execute termination for Deployment fail-deploy. Error: Deploym├────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
ent fail-deploy has no running pods at the moment                                                                       │---
I0102 00:11:39.602155       1 kubemonkey.go:74] Status Update: 1 scheduled terminations left.                           │  apiVersion: extensions/v1beta1
E0102 00:11:45.596051       1 config.go:330] Expected to load root CA config from /var/run/secrets/kubernetes.io/service│  kind: Deployment
account/ca.crt, but got err: open /var/run/secrets/kubernetes.io/serviceaccount/ca.crt: no such file or directory       │  metadata:
E0102 00:11:45.602093       1 kubemonkey.go:69] Failed to execute termination for Deployment secret-pod. Error: Deployme│    name: kube-monkey
nt secret-pod has no running pods at the moment                                                                         │    namespace: kube-system
I0102 00:11:45.602105       1 kubemonkey.go:74] Status Update: 0 scheduled terminations left.                           │  spec:
I0102 00:11:45.602110       1 kubemonkey.go:77] Status Update: All terminations done.                                   │    replicas: 1
I0102 00:11:45.602157       1 kubemonkey.go:19] Debug mode detected!                                                    │    template:
I0102 00:11:45.602163       1 kubemonkey.go:20] Status Update: Generating next schedule in 120 sec                      │      metadata:
                                                                                                                        │        labels:
                                                                                                                        │          app: kube-monkey
                                                                                                                        │      spec:
                                                                                                                        │        containers:
                                                                                                                        │          -  name: kube-monkey
                                                                                                                        │             command: ["/kube-monkey"]
                                                                                                                        │             args: ["-v=4", "-log_dir=/var/log/kube-monkey"]
                                                                                                                        │             image: kube-monkey:latest

Converted some logs to their appropriate type (i.e. error to info or vice versa)
Reworked schedule print
Standardized glog levels `grep -r V\([0-9]\) *`
L0: None
L1: Highest Level current status info and Errors with Terminations
L2: Successful terminations
L3: More detailed schedule status info
L4: Debugging verbose schedule and config info
L5: Auto-resolved inconsequential issues
Allows for fmt.Println(schedule)
Since we should glog the single line info
and make the table version isolated and readable
Had to run `glide cache-clear` and then `glide update --strip-vendor`
CreateClient involves creating a clientset that accesses the k8 api
It does not belong in chaos but rather kubernetes
New sublocation for all kinds
run with make gofmt
standardized variables to loc
added timezone for debugging printing
specified format printout for schedule
addresses asobti#21 and asobti#23
@asobti asobti merged commit 37b4e42 into asobti:master Jan 24, 2018
@Aergonus Aergonus deleted the timezone-print branch January 24, 2018 14:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants