[clusterctl] proposal: support a more expressive templating engine beyond simple variable substitution

[CecileRobertMichon]

Problem statement:

As infrastructure providers add more cluster configuration options and features, it is becoming harder to define templates for each feature. In CAPZ, we have taken the approach of using kustomize to generate template "flavors". The main downside of this approach is that 1 feature -> 1 flavor (ie. there is one flavor for managed identity, one flavor for machine pool, one flavor for out of tree provider, etc.). A user will likely want to use a combination of these features. It is not feasible to define a flavor for each possible permutation of features we support. Most of these features require more complex templating than just variable substitution (conditionals, loops, etc.).

In addition, the "quickstart" experience can be intimidating to new users which is a barrier to adoption. Quite often, the infrastructure templates require the user to specify a large number of variables, some of which might not be relevant to them because they'd just like to use the default, and doesn't provide a lot of flexibility for enabling/disabling specific features. #3189 is a first step towards improving that, but it's the bare minimum. This proposal is to propose that we go beyond that, as was originally proposed in #2339.

User Stories:

• As a provider, I would like to be able to template the cluster templates available for clusterctl consumption.

• As a user, I would like to be able to configure my cluster with a combination of features by passing in flags or environment variables (or a config file) to clusterctl without having to write my own my template from scratch.

One alternative is for each provider to use clusterctl as a library and write their own CLI wrapper around it to handle templating. The main motivations for proposing to include templating in clusterctl itself are:

1. the need for templating is generic enough for every infrastructure provider to benefit from it, and
2. having a separate CLI for each infra provider would degrade the consistency of the cluster configuration user experience

Proposed implementation:

One templating engine that would fit the needs described above is ytt (was originally brought up in the original clusterctl extensible template processing proposal).

NOTE: #3115 adds a templating interface for clusterctl such that the default "simple yaml processor" can be overridden. However, it does not support a templating engine within clusterctl itself.

Related to #3247

/kind feature
/area clusterctl

[CecileRobertMichon]

/cc @nader-ziada @devigned @wfernandes @fabriziopandini @ncdc @timothysc @vincepri

[vincepri]

@CecileRobertMichon Thoughts about milestone?

[CecileRobertMichon]

@vincepri for now just wanted to get the discussion started since I know we considered doing this in the past (#2339) and didn't end up implementing it.

[vincepri]

/milestone Next

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[fabriziopandini]

/lifecycle frozen

[vincepri]

/close

ClusterClass covers this use case with the support of inline patches

[k8s-ci-robot]

@vincepri: Closing this issue.

In response to this:

/close

ClusterClass covers this use case with the support of inline patches

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.