[Feature] Registry for v4 & Restructure Cluster Creation

[iwilltry42]

Overview

First Level

• new command k3d registry
• new command k3d registry create
  • --cluster (multiple): clusters to attach to upon creation
    • ℹ️ only connects to the cluster network, because containerd doesn't allow a config reload yet
• new command k3d registry delete
• new command k3d registry start
• new command k3d registry stop
• 🚫 new command k3d registry connect
  • reload containerd config to include new registry
    • requires [Feature] Reload Plugin Config containerd/containerd#4478

Second Level

• use an existing registry: k3d cluster create --registry-use REGISTRY_NAME
• create a new registry for use with a new cluster: k3d cluster create --registry-create
  • all default settings: for detailed settings use k3d registry create REGISTRY_NAME and k3d cluster create --registry-use REGISTRY_NAME

Important Notes

• we need to keep track of the clusters using a registry, so that we can update the respective registries.yaml files when creating/starting/stopping/deleting a registry

Status

• draft of first level k3d registry command

Old PR: #283 (old syntax, closed in favor of this new PR)

FYI @inercia

Dependencies

Depends on #351 (rebased 03.12.2020)

❗ Additional Changes ❗

While working on this PR, some changes had to be made, which affect the overall architecture/structure/workflow:

• BREAKING: Split ClusterCreate into decoupled steps, orchestrated by ClusterRun:
  • ClusterPrep: build supporting infrastructure (like the network and volumes)
  • ClusterCreate: create the node containers (but not start them yet)
  • ClusterConfigure: do some offline configuration (like writing the registries.yaml)
  • ClusterStart: actually start the node containers (and auxiliary containers)
• BREAKING: rename pkg/cluster to pkg/client to improve consistent naming and reduce stuttering (like cluster.ClusterGet)
• Golang 1.15
• BREAKING: flag syntax noun-action
  • k3d cluster create --kubeconfig-switch-context --kubeconfig-update-default

[dweomer]

@iwilltry42 k3s registry connect seems more like wishful thinking given our conversation with @erikwilson and the lack of containerd support (unless we are okay with a cluster reboot which is probably fine, actually).

I may have missed it but if you haven't already I suggest adding a string slice of registries to connect to by name when invoking k3d cluster create (kind of like associating volumes to containers a la docker create|run)

[iwilltry42]

Thanks for the feedback @dweomer !

@iwilltry42 k3s registry connect seems more like wishful thinking given our conversation with @erikwilson and the lack of containerd support (unless we are okay with a cluster reboot which is probably fine, actually).

Fast and painless cluster reboots are on our list anyway to allow updating settings after creating the cluster. So I'll just postpone registry connect for now 👍

I may have missed it but if you haven't already I suggest adding a string slice of registries to connect to by name when invoking k3d cluster create (kind of like associating volumes to containers a la docker create|run)

Yep, I will definitely add this again. That's also how k3d 1.x handled registries (creation and connection).

[sgandon]

Will there be a way to create the registry automatically upon cluster creation, or will this have to be a second set of commands ?