Define strategy on mapping OS images across cloud providers to the scripts that support that OS flavor + version

[rsdcastro]

• Scripts that install the required K8s on the machine should be made agnostic of cloud provider for reuse (Move Ubuntu installation script out of Cluster API GCP #596)
• Users can specify an image, or if not specified, a default OS image is picked
• OS Images are composed of the flavor (Ubuntu, Debian, CentOS) and version
• Scripts for K8s installation can support one or more versions (and potentially similar flavors) and should be easily mapped to the images that it supports in order to simplify provider actuators

[mrIncompetent]

Maybe cloud-init is an option?

[rsdcastro]

We should certainly look into cloud-init! Do you have any experience with it? How has it been?

[mrIncompetent]

We're using it to bootstrap Ubuntu machines - So far its great.
Especially compared against using ssh to provision a node, which we used in https://github.com/kube-node/kube-machine.

For Container Linux though we use Ignition.

[mrIncompetent]

Though one very important detail:
cloud-Init is more meant for a create/replace cycle, not in-place update

[rsdcastro]

I asked around, and I'd like to quote @pipejakob's notes at the time.

• cloud-init's primary (maybe only) goal is around uniformity across cloud providers / images.
• Performance isn't a non-goal, but certainly isn't a priority.
  • The tool is written in Python, executes completely serially, and has no interest in becoming a systemd-like DAG executor that has awareness of steps that could run in parallel, or in constantly driving down execution time.

Main concern with cloud-init is that performance can be a big issue when bringing up new nodes (that would go into an opposite direction of making the startup as quickly as possible).

[mrIncompetent]

Don't we have the same problems with a simple shell script for provisioning?

I'm just not a fan of having to manage the whole ssh-provisioning code and would prefer a more declarative approach.

[pipejakob]

I've been putting out feelers for tbd for a while. It's still just a proposal for a tool that hasn't been written yet, but if I understand this issue, I believe it would make all of your init scripts OS/cloud independent (because it would absorb that functionality).

If this tool existed, 1) would you use it, and 2) would it single-handedly solve this issue, or is there still a lot more OS-specific startup logic beyond installing kubeadm and its dependencies?

[roberthbailey]

@krousey

[krousey]

I filed #637 yesterday for at least GCE's side of things.

[rsdcastro]

Closing this and keeping only GCE issue for now.