Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Regression on Upgrade #1224

Closed
fabriziopandini opened this issue Nov 9, 2018 · 5 comments
Closed

Regression on Upgrade #1224

fabriziopandini opened this issue Nov 9, 2018 · 5 comments
Assignees
Labels
area/HA area/test kind/bug Categorizes issue or PR as related to a bug. lifecycle/active Indicates that an issue or PR is actively being worked on by a contributor. priority/critical-urgent Highest priority. Must be actively worked on as someone's top priority right now.
Milestone

Comments

@fabriziopandini
Copy link
Member

Kubedam v1.13 raise an error on the upgrade of clusters created with kubeadm v1.12.

The error happens when connecting to etcd, and this regression was introduced by stacked etcd change.

kubeadm v1.12 cluster are deployed with etcd listening on localhost only, while kubeadm v1.13 assumes etcd is listening both on localhost and on the advertising address.

I have a fix in flight, but we should also investigate why CI test on upgrades didn't provide a signal before

@fabriziopandini fabriziopandini added kind/bug Categorizes issue or PR as related to a bug. priority/critical-urgent Highest priority. Must be actively worked on as someone's top priority right now. area/HA area/test lifecycle/active Indicates that an issue or PR is actively being worked on by a contributor. labels Nov 9, 2018
@fabriziopandini fabriziopandini added this to the v1.13 milestone Nov 9, 2018
@fabriziopandini fabriziopandini self-assigned this Nov 9, 2018
@neolit123
Copy link
Member

seems that our upgrade tests master-upgrade have been failing for other reasons:
https://storage.googleapis.com/kubernetes-jenkins/logs/ci-kubernetes-e2e-kubeadm-gce-upgrade-stable-master/2819/build-log.txt

I1109 08:15:48.277] [upgrade/config] FATAL: the ConfigMap "kubeadm-config" in the kube-system namespace used for getting configuration information was not found
I1109 08:15:48.277] [upgrade/config] In order to upgrade, a ConfigMap called "kubeadm-config" in the kube-system namespace must exist.
I1109 08:15:48.277] [upgrade/config] Without this information, 'kubeadm upgrade' won't know how to configure your upgraded cluster.
I1109 08:15:48.277] 
I1109 08:15:48.277] [upgrade/config] Next steps:
I1109 08:15:48.278]     - OPTION 1: Run 'kubeadm config upload from-flags' and specify the same CLI arguments you passed to 'kubeadm init' when you created your master.
I1109 08:15:48.278]     - OPTION 2: Run 'kubeadm config upload from-file' and specify the same config file you passed to 'kubeadm init' when you created your master.
I1109 08:15:48.278]     - OPTION 3: Pass a config file to 'kubeadm upgrade' using the --config flag.

i tried reproducing with k-a locally but failed, because k-a works for me only "sometimes".

also, i think i need to apply this patch to the older branches we support:
https://github.com/kubernetes/kubernetes/pull/68757/files

https://github.com/kubernetes/kubernetes/blob/release-1.12/test/e2e/framework/test_context.go#L236
https://github.com/kubernetes/kubernetes/blob/release-1.11/test/e2e/framework/test_context.go#L234

@AishSundar
Copy link

@nikopen @thanasisk as FYI

We should track it for 1.13 k8s cycle.

@nikopen
Copy link

nikopen commented Nov 14, 2018

as kubernetes/kubernetes#70893 got merged, I assume this is now fixed

@nikopen
Copy link

nikopen commented Nov 14, 2018

/close

@k8s-ci-robot
Copy link
Contributor

@nikopen: Closing this issue.

In response to this:

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/HA area/test kind/bug Categorizes issue or PR as related to a bug. lifecycle/active Indicates that an issue or PR is actively being worked on by a contributor. priority/critical-urgent Highest priority. Must be actively worked on as someone's top priority right now.
Projects
None yet
Development

No branches or pull requests

5 participants