Operating system and Container cache should live on different VMDKs

[frapposelli]

/kind feature

Describe the solution you'd like
I would like to see CAPV separate the operating system from the local container cache, this would be achieved by cloning the OS from a template, and then creating a new disk of the size specified with VSPHERE_DISK_GIB that would host the local image cache, enabling faster provisioning time and the ability to specify a disk size when using linked clones.

Anything else you would like to add:
This could also help during upgrade, the new node could simply mount the existing local image cache and that would speed up significantly the restart of workloads after an upgrade.

/cc @randomvariable @yastij @rosskukulinski

[frapposelli]

Also I think having etcd as a separate VMDK could be a good idea, since using linked clones creates a write penalty on the disk due to copy-on-write, and etcd is very sensitive to it.

[dougm]

The Etcd on data disk CAEP has been accepted and implementation merged.

The CAPA PR Etcd EBS Volumes is in progress based on the new CABPK disk support.

Is anyone looking at doing similar for CAPV yet?

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[fejta-bot]

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle rotten

[rosskukulinski]

@timmycarr can you evaluate relevance now?

[omniproc]

One downside of this approach would be that it would lower the number of PV's that can be attached since there's a hard limit of FCD's that can be added (max no. of SCSI controllers / SCSI slots)

[srm09]

/help
@timmycarr Is this something for us to tackle?

[k8s-ci-robot]

@srm09:
This request has been marked as needing help from a contributor.

Guidelines

Please ensure that the issue body includes answers to the following questions:

• Why are we solving this issue?
• To address this issue, are there any code changes? If there are code changes, what needs to be done in the code and what places can the assignee treat as reference points?
• Does this issue have zero to low barrier of entry?
• How can the assignee reach out to you for help?

For more details on the requirements of such an issue, please see here and ensure that they are met.

If this request no longer meets these requirements, the label can be removed
by commenting with the /remove-help command.

In response to this:

/help
@timmycarr Is this something for us to tackle?

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[randomvariable]

This is supported in VM Operator mode. Is this needed in non-supervisor mode?

[omniproc]

@randomvariable not sure this is ment to be related to Tanzu. This is more about additional features for vanilla CAPV, additional variables in the infrastructure provider of CAPV to be more specific.