No way to set cloudProvider: external in new config files

[NeilW]

BUG REPORT

Versions

kubeadm version (use kubeadm version):
kubeadm version: &version.Info{Major:"1", Minor:"11", GitVersion:"v1.11.1", GitCommit:"b1b29978270dc22fecc592ac55d903350454310a", GitTreeState:"clean", BuildDate:"2018-07-17T18:50:16Z", GoVersion:"go1.10.3", Compiler:"gc", Platform:"linux/amd64"}

Environment:

• Kubernetes version (use kubectl version):
  Client Version: version.Info{Major:"1", Minor:"11", GitVersion:"v1.11.1", GitCommit:"b1b29978270dc22fecc592ac55d903350454310a", GitTreeState:"clean", BuildDate:"2018-07-17T18:53:20Z", GoVersion:"go1.10.3", Compiler:"gc", Platform:"linux/amd64"}
  Server Version: version.Info{Major:"1", Minor:"11", GitVersion:"v1.11.0", GitCommit:"91e7b4fd31fcd3d5f436da26c980becec37ceefe", GitTreeState:"clean", BuildDate:"2018-06-27T20:08:34Z", GoVersion:"go1.10.2", Compiler:"gc", Platform:"linux/amd64"}
• Cloud provider or hardware configuration:
  Brightbox
• OS (e.g. from /etc/os-release):
  Ubuntu 18.04 LTS
• Kernel (e.g. uname -a):
  Linux srv-k1jym 4.15.0-29-generic The product_uuid and the hostname should be unique across nodes #31-Ubuntu SMP Tue Jul 17 15:39:52 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
• Others:

What happened?

When looking at upgrading the config version from kubeadm.k8s.io/v1alpha1 there doesn't appear to be a way of specifying 'cloudProvider: external' in the config

What you expected to happen?

That there is a documented way of using an external cloud provider with kubeadm, including a documented way to boot the cloud-provider as an additional add-on.

How to reproduce it (as minimally and precisely as possible)?

My v1alpaha1 config is

apiVersion: kubeadm.k8s.io/v1alpha1
kind: MasterConfiguration
clusterName: ${cluster_name}
api:
  advertiseAddress: "::"
networking:
  dnsDomain: ${cluster_domainname}
  serviceSubnet: ${service_cidr}
kubernetesVersion: ${k8s_release}
cloudProvider: external

(The variables are swapped out during terraform rendering).

The alpha2 example in the documentation here: https://kubernetes.io/docs/reference/setup-tools/kubeadm/kubeadm-init/ has nothing about cloudProvider in there.

Anything else we need to know?

The documented external cloud provider boot processes seems to expect an apiserver without any TLS connections. I've had to work out from scratch how to boot an external cloud-provider so it will work with a kubeadm created cluster - largely by pulling apart kube-proxy.

Given the move to separated cloud-providers, we could do with the process standardising.

[neolit123]

@NeilW
moving to v1alpha2 is required in 1.11.x. you can use kubeadm config migrate for that.
in v1alpha2 cloudProvider should be passed to the components of interest - kubelet, contoller-manager and apiserver.

feeding your config into kubeadm config migrate produces this for me (+some noise from defaults):

...
apiServerExtraArgs:
  cloud-provider: external
...
controllerManagerExtraArgs:
  cloud-provider: external
...
apiVersion: kubeadm.k8s.io/v1alpha2
kind: MasterConfiguration
...
nodeRegistration:
  kubeletExtraArgs:
    cloud-provider: external
...

When looking at upgrading the config version from kubeadm.k8s.io/v1alpha1 there doesn't appear to be a way of specifying 'cloudProvider: external' in the config

i will add a note about migrate in kubeadm init, but for 1.12 we want to remove the authored content from that page and also add a separate page about kubeadm + cloud providers and delegate parts of the maintenance for that to SIG cloud provider.

ref: #526

[neolit123]

@NeilW
given you are reporting the new issue (#1021) i assume that the master setup for the CP extra flags is working at least - i.e. close this issue?

[NeilW]

If we think the improved documentation of the process is captured by another ticket. The requirement to use alpha2 and the way the external flag is handled in the config just isn't in the documentation at present.

[neolit123]

yes, the alpha1-> 2 is captured in another issue:
#849

this is the ticket that tracks our missing CP docs:
#526

this here just adds a couple of notes to the kubeadm init page but is not really a full solution for the missing docs:
kubernetes/website#9638

thanks
/close