Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refact(upgrade): add logs to provide status information during upgrade #1556

Merged
merged 5 commits into from
Dec 26, 2019

Conversation

shubham14bajpai
Copy link
Contributor

@shubham14bajpai shubham14bajpai commented Dec 13, 2019

Signed-off-by: shubham [email protected]

What this PR does / why we need it:
This PR improves upgrade logs by providing pool and volume status information during the upgrade and also helps in estimating the time taken for deployment pods to come up.
This PR also bumps the version file and valid current version to support 1.6 upgrades.

Sample Logs:

mayadata:kubernetes$ kubectl -n openebs logs -f cstor-spc-130140-sparse-claim-auto-q8kcv 
I1213 12:54:03.369345       1 cstor_spc.go:82] Upgrading to 1.5.0-RC1
I1213 12:54:03.697438       1 helper.go:68] Patching deployment sparse-claim-auto-e1ez
I1213 12:54:43.767379       1 helper.go:95] Deployment sparse-claim-auto-e1ez rolled out successfully
I1213 12:54:43.767414       1 csp_upgrade.go:175] patched csp deployment sparse-claim-auto-e1ez
I1213 12:54:43.773886       1 csp_upgrade.go:140] patched csp sparse-claim-auto-e1ez
I1213 12:54:43.774070       1 csp_upgrade.go:261] Verifying the reconciliation of version for sparse-claim-auto-e1ez { phase:Healthy }
I1213 12:54:53.800659       1 csp_upgrade.go:261] Verifying the reconciliation of version for sparse-claim-auto-e1ez { phase:Healthy }
I1213 12:55:03.829749       1 csp_upgrade.go:261] Verifying the reconciliation of version for sparse-claim-auto-e1ez { phase:Healthy }
I1213 12:55:13.839915       1 csp_upgrade.go:364] Upgrade Successful for csp sparse-claim-auto-e1ez
I1213 12:55:14.033861       1 request.go:530] Throttling request took 140.855641ms, request: PUT:https://10.0.16.1:443/apis/openebs.io/v1alpha1/namespaces/openebs/upgradetasks/upgrade-cstor-pool-sparse-claim-auto-qe0s
I1213 12:55:14.039840       1 helper.go:68] Patching deployment sparse-claim-auto-qe0s
I1213 12:56:04.097414       1 helper.go:95] Deployment sparse-claim-auto-qe0s rolled out successfully
I1213 12:56:04.097486       1 csp_upgrade.go:175] patched csp deployment sparse-claim-auto-qe0s
I1213 12:56:04.111865       1 csp_upgrade.go:140] patched csp sparse-claim-auto-qe0s
I1213 12:56:04.112357       1 csp_upgrade.go:261] Verifying the reconciliation of version for sparse-claim-auto-qe0s { phase:Healthy }
I1213 12:56:14.132667       1 csp_upgrade.go:261] Verifying the reconciliation of version for sparse-claim-auto-qe0s { phase:Healthy }
I1213 12:56:24.141160       1 csp_upgrade.go:261] Verifying the reconciliation of version for sparse-claim-auto-qe0s { phase:Healthy }
I1213 12:56:34.153105       1 csp_upgrade.go:364] Upgrade Successful for csp sparse-claim-auto-qe0s
I1213 12:56:34.345834       1 request.go:530] Throttling request took 140.457161ms, request: PUT:https://10.0.16.1:443/apis/openebs.io/v1alpha1/namespaces/openebs/upgradetasks/upgrade-cstor-pool-sparse-claim-auto-tdg1
I1213 12:56:34.352065       1 helper.go:68] Patching deployment sparse-claim-auto-tdg1
I1213 12:57:24.405573       1 helper.go:95] Deployment sparse-claim-auto-tdg1 rolled out successfully
I1213 12:57:24.405612       1 csp_upgrade.go:175] patched csp deployment sparse-claim-auto-tdg1
I1213 12:57:24.415037       1 csp_upgrade.go:140] patched csp sparse-claim-auto-tdg1
I1213 12:57:24.415066       1 csp_upgrade.go:261] Verifying the reconciliation of version for sparse-claim-auto-tdg1 { phase:Healthy }
I1213 12:57:34.419287       1 csp_upgrade.go:261] Verifying the reconciliation of version for sparse-claim-auto-tdg1 { phase:Healthy }
I1213 12:57:44.422949       1 csp_upgrade.go:261] Verifying the reconciliation of version for sparse-claim-auto-tdg1 { phase:Healthy }
I1213 12:57:54.432617       1 csp_upgrade.go:364] Upgrade Successful for csp sparse-claim-auto-tdg1
I1213 12:57:54.458021       1 spc_upgrade.go:148] Verifying the reconciliation of version for sparse-claim-auto
I1213 12:58:04.470232       1 spc_upgrade.go:102] Upgrade Successful for spc sparse-claim-auto


mayadata:kubernetes$ kubectl -n openebs logs -f cstor-vol-140150-pvc-8f290262-1da6-11ea-92a9-42010aac0038-8bp5b 
I1213 12:58:30.235216       1 cstor_volume.go:82] Upgrading to 1.5.0-RC1
I1213 12:58:30.386907       1 cstor_volume_upgrade.go:203] cstorvolumereplica pvc-8f290262-1da6-11ea-92a9-42010aac0038-sparse-claim-auto-e1ez patched
I1213 12:58:30.391038       1 cstor_volume_upgrade.go:492] Verifying the reconciliation of pvc-8f290262-1da6-11ea-92a9-42010aac0038-sparse-claim-auto-e1ez {id:A803B8003388FD1579A03806E151915A phase:Healthy}
I1213 12:58:40.394403       1 cstor_volume_upgrade.go:492] Verifying the reconciliation of pvc-8f290262-1da6-11ea-92a9-42010aac0038-sparse-claim-auto-e1ez {id:A803B8003388FD1579A03806E151915A phase:Healthy}
I1213 12:58:50.414229       1 cstor_volume_upgrade.go:203] cstorvolumereplica pvc-8f290262-1da6-11ea-92a9-42010aac0038-sparse-claim-auto-qe0s patched
I1213 12:58:50.418849       1 cstor_volume_upgrade.go:492] Verifying the reconciliation of pvc-8f290262-1da6-11ea-92a9-42010aac0038-sparse-claim-auto-qe0s {id:644B71B6075FC865A19E54427C42370F phase:Healthy}
I1213 12:59:00.421821       1 cstor_volume_upgrade.go:492] Verifying the reconciliation of pvc-8f290262-1da6-11ea-92a9-42010aac0038-sparse-claim-auto-qe0s {id:644B71B6075FC865A19E54427C42370F phase:Healthy}
I1213 12:59:10.437540       1 cstor_volume_upgrade.go:203] cstorvolumereplica pvc-8f290262-1da6-11ea-92a9-42010aac0038-sparse-claim-auto-tdg1 patched
I1213 12:59:10.441682       1 cstor_volume_upgrade.go:492] Verifying the reconciliation of pvc-8f290262-1da6-11ea-92a9-42010aac0038-sparse-claim-auto-tdg1 {id:63D73EFC9F0B37F8F9361692A19B05D2 phase:Healthy}
I1213 12:59:20.452892       1 cstor_volume_upgrade.go:492] Verifying the reconciliation of pvc-8f290262-1da6-11ea-92a9-42010aac0038-sparse-claim-auto-tdg1 {id:63D73EFC9F0B37F8F9361692A19B05D2 phase:Healthy}
I1213 12:59:30.468221       1 cstor_volume_upgrade.go:571] CStor volume { phase:Healthy } replicaStatuses: 
		{ id:4761571119009075862 mode:Healthy chckIOSeq:0 r/w/s:0/0/0 uptime:205 quorum:1 }
		{ id:14526460632679930052 mode:Healthy chckIOSeq:575 r/w/s:0/0/0 uptime:124 quorum:1 }
		{ id:3351440958845210662 mode:Healthy chckIOSeq:470 r/w/s:0/0/0 uptime:44 quorum:1 }
I1213 12:59:30.468538       1 helper.go:68] Patching deployment pvc-8f290262-1da6-11ea-92a9-42010aac0038-target
I1213 13:00:10.514807       1 helper.go:95] Deployment pvc-8f290262-1da6-11ea-92a9-42010aac0038-target rolled out successfully
I1213 13:00:10.514841       1 cstor_volume_upgrade.go:131] target deployment pvc-8f290262-1da6-11ea-92a9-42010aac0038-target patched
I1213 13:00:10.534662       1 helper.go:180] targetservice pvc-8f290262-1da6-11ea-92a9-42010aac0038 patched
I1213 13:00:10.549885       1 cstor_volume_upgrade.go:161] cstorvolume pvc-8f290262-1da6-11ea-92a9-42010aac0038 patched
I1213 13:00:10.549920       1 cstor_volume_upgrade.go:335] Verifying the reconciliation of version for pvc-8f290262-1da6-11ea-92a9-42010aac0038
I1213 13:00:20.554846       1 cstor_volume_upgrade.go:571] CStor volume { phase:Init } replicaStatuses: 
		{ id:4761571119009075862 mode:Healthy chckIOSeq:0 r/w/s:0/0/0 uptime:295 quorum:1 }
		{ id:14526460632679930052 mode:Healthy chckIOSeq:575 r/w/s:0/0/0 uptime:214 quorum:1 }
		{ id:3351440958845210662 mode:Healthy chckIOSeq:470 r/w/s:0/0/0 uptime:134 quorum:1 }
I1213 13:00:20.561450       1 cstor_volume_upgrade.go:599] Upgrade Successful for cstor volume pvc-8f290262-1da6-11ea-92a9-42010aac0038

Which issue this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close that issue when PR gets merged): fixes #

Special notes for your reviewer:

Checklist:

  • Fixes #
  • Labelled this PR & related issue with documentation tag
  • PR messages has document related information
  • Labelled this PR & related issue with breaking-changes tag
  • PR messages has breaking changes related information
  • Labelled this PR & related issue with requires-upgrade tag
  • PR messages has upgrade related information
  • Commit has unit tests
  • Commit has integration tests

Copy link

@mittachaitu mittachaitu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

changes are good

Copy link

@ghost ghost left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Found some fixes!

P.S. share your ideas, feedbacks or issues with us at https://github.com/fixmie/feedback (this message will be removed after the beta stage).

pkg/upgrade/upgrader/cstor_volume_upgrade.go Outdated Show resolved Hide resolved
Copy link
Contributor

@prateekpandey14 prateekpandey14 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@vishnuitta vishnuitta merged commit 8b088bf into openebs-archive:master Dec 26, 2019
shubham14bajpai added a commit to shubham14bajpai/maya that referenced this pull request Dec 27, 2019
@kmova kmova added this to the 1.6.0 milestone Jan 10, 2020
@shubham14bajpai shubham14bajpai deleted the logs branch March 17, 2020 06:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants