Skip to content
This repository has been archived by the owner on Oct 24, 2023. It is now read-only.

Upgrade VMSS node may fail with drain error #1919

Closed
weinong opened this issue Sep 6, 2019 · 1 comment · Fixed by #2013
Closed

Upgrade VMSS node may fail with drain error #1919

weinong opened this issue Sep 6, 2019 · 1 comment · Fixed by #2013
Labels
bug Something isn't working

Comments

@weinong
Copy link
Contributor

weinong commented Sep 6, 2019

Describe the bug
Draining node should be best effort. We should not fail upgrade upon failing to drain node.
https://github.com/Azure/aks-engine/blame/f415d6ffd772a2037c2eea7a5b086cf5607add10/pkg/operations/kubernetesupgrade/upgrader.go#L553

Steps To Reproduce

Expected behavior

AKS Engine version

Kubernetes version

Additional context
@weinong weinong added the bug Something isn't working label Sep 6, 2019
@jsgervais
Copy link

jsgervais commented Sep 16, 2019 •
edited
Loading

Having similar issue upgrading from 1.13.5 to 1.14.6, getting errors in the log :
msrest.exceptions : Drain did not complete within 20m0s

Deployment failed. Correlation ID: 5ed58bf1-0000-0000-0000-0000000000. Drain did not complete within 20m0s

our nodes uses deamon sets, is the drain node command ignoring them (with --ignore-daemonsets flag) ?
https://success.docker.com/article/how-to-pause-or-drain-a-node-on-kubernetes

@jackfrancis jackfrancis mentioned this issue Sep 18, 2019
Closed
@mboersma mboersma mentioned this issue Sep 23, 2019
Merged
4 tasks
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

fix: continue upgrading after node drain error mboersma/aks-engine
2 participants
@weinong @jsgervais