Incorporate VCP vsphere.conf zones & regions into envvars.txt and manifest generation

[MnrGreg]

/kind feature

Enable infrastructure topology awareness within kubernetes clusters by supporting the configuration of vSphere Cloud Provider Label:region and Label:zones at manifest generation time.

vSphere admins apply vSphere Tags to vSphere Datacenter, Cluster and Host objects to represent Availability Regions and Availability Zones. Kubernetes clusters become aware of these topologies when bootstrapped and take them into account during scheduling.

Possible enablement through:

1. Define environment variables for Region and Tag. Make known through getting_started.md

cat <<EOF >envvars.txt
export VSPHERE_REGION_TAG="k8s-region" 
export VSPHERE_ZONE_TAG="k8s-zone"    
...
EOF

Modify /examples/generate.sh to either append to the vsphere.conf ini section:

...
[Labels]
region = ${VSPHERE_REGION_TAG}
zone = ${VSPHERE_ZONE_TAG}

or to customize cluster.yaml:

apiVersion: infrastructure.cluster.x-k8s.io/v1alpha2
kind: VSphereCluster
spec:
  cloudProviderConfiguration:
...
    labels:
      region: k8s-region
      zone: k8s-zone

2. Modify controlplane.yaml and machinedeployment.yaml kubelet configuration templates with path to vsphere.conf (might be required for pre-67u3 only?)

apiVersion: bootstrap.cluster.x-k8s.io/v1alpha2
kind: KubeadmConfig
spec:
  initConfiguration:
    nodeRegistration:
      kubeletExtraArgs:
        cloud-provider: vsphere
        cloud-config: /etc/kubernetes/vsphere.conf

Environment:

• Cluster-api-provider-vsphere version: 0.5.3
• Kubernetes version: (use kubectl version): 1.16.2
• OS (e.g. from /etc/os-release): CentOS7

[akutz]

Hi @MnrGreg,

I'm not sure if we want to continue to support the in-tree provider. I would think all new work should focus on the external cloud provider. Thoughts?

[MnrGreg]

Yes, agreed. Our use-cases still involve using in-tree but it's good motivation to get to 67u3.

Will the cluster-api-provider-vsphere/Manifests generator remain the default templating tool for the foreseeable future?

Trying to gage if is it worth ones time updating the generate.sh it to include the region & zones for the 67u3 vSphere cloud-config ConfigMap.

[akutz]

I agree this is a good ask, but it just isn't something I've had time to do. Can you possibly file a PR? Thanks!

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[MnrGreg]

This is now simple enough to do with clusterctl cluster-template.yaml overrides.

Example:

apiVersion: infrastructure.cluster.x-k8s.io/v1alpha3
kind: VSphereCluster
metadata:
  name: '${ CLUSTER_NAME }'
  namespace: '${ NAMESPACE }'
spec:
  cloudProviderConfiguration:
    labels:
      region: '${ VSPHERE_REGION_TAG }'
      zone: '${ VSPHERE_ZONE_TAG }'

I also noticed that if the vSphere Tag Categories do not exist then the vSphere Cloud Provider fails to start correctly. Probably safer to require that it be explicitly defined.

/close

[k8s-ci-robot]

@MnrGreg: Closing this issue.

In response to this:

This is now simple enough to do with clusterctl cluster-template.yaml overrides.

Example:

apiVersion: infrastructure.cluster.x-k8s.io/v1alpha3
kind: VSphereCluster
metadata:
 name: '${ CLUSTER_NAME }'
 namespace: '${ NAMESPACE }'
spec:
 cloudProviderConfiguration:
   labels:
     region: '${ VSPHERE_REGION_TAG }'
     zone: '${ VSPHERE_ZONE_TAG }'

I also noticed that if the vSphere Tag Categories do not exist then the vSphere Cloud Provider fails to start correctly. Probably safer to require that it be explicitly defined.

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.