feat: enable easier flavor creation

[devigned]

What this PR does / why we need it:
Flavors are the way clusterctl allows providers to specify different configurations of clusters. This PR makes it easier to create new flavors and to extend existing ones by setting up an opinionated directory structure and a make goal (make generate-flavors) to generate flavors using kustomize.

Below is the proposed structure.

templates
├── addons
│   ├── calico.yaml
│   ├── cloud-controller-manager.yaml
│   └── cloud-node-manager.yaml
├── cluster-template-external-cloud-provider.yaml
├── cluster-template.yaml
├── flavors
│   ├── README.md
│   ├── base
│   │   ├── cluster-template.yaml
│   │   └── kustomization.yaml
│   ├── default
│   │   ├── kustomization.yaml
│   │   └── machine-deployment.yaml
│   └── external-cloud-provider
│       ├── kustomization.yaml
│       └── patches
│           └── external-cloud-provider.yaml
└── test
    ├── cluster-template-conformance-ci-version.yaml
    ├── cluster-template-conformance.yaml
    ├── conformance
    │   ├── kustomization.yaml
    │   └── patches
    │       └── conformance-tags.yaml
    └── conformance-ci-version
        ├── kustomization.yaml
        └── patches
            └── ci-version.yaml

• templates root: contains the generated flavors starting with cluster-template{-flavor-name}.yaml and the following folders.
  • addons: contains add-on manifests; resources that may or may not be added to a cluster
  • flavors: a list of directories which each contain a kustomization that is generated and added to the ./templates directory as a flavor.
  • test: contains all of the test / conformance kustomizations

Special notes for your reviewer:

Please confirm that if this PR changes any image versions, then that's the sole change this PR makes.

Release note:

Enable easier flavor creation. See [flavors documentation](https://github.com/kubernetes-sigs/cluster-api-provider-azure/tree/master/templates/flavors) for more details.

[devigned]

/hold

[CecileRobertMichon]

@fabriziopandini @detiber @vincepri curious what your thoughts are on this

[devigned]

/hold cancel

[CecileRobertMichon]

/hold

for discussion and reviews

[CecileRobertMichon]

@devigned maybe I should wait for this to cut a 0.4.2 release so we can try out the "release flavor templates as artifacts" story in real life, wdyt?

[devigned]

@CecileRobertMichon great idea!

[CecileRobertMichon]

thank you for making this generic so that we don't have to remember to add new templates here every time 💯

[CecileRobertMichon]

/approve

/assign @justaugustus @nader-ziada @alexeldeib

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: CecileRobertMichon, devigned

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [CecileRobertMichon]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[devigned]

nits picked :)

[CecileRobertMichon]

@devigned please add a release note

/hold cancel

[nader-ziada]

@devigned great PR 👍,
but just a question, I'm a little confused with the difference between base and default, I can see default adds a machine-deployment but when I run make generate-flavors I only get cluster-template.yaml and cluster-template-external-cloud-provider.yaml which are not corresponding to the directories names, so I was wondering if we can combine base and default or clarify it in the readme a little bit more

[devigned]

You are too kind. Thank you.

base is ignored currently as a place to hold shared structures. I hope that it goes away in a future PR where we would take the ./templates/base/cluster-template.yaml and break it up into the individual resources to be used piece by piece to build new flavors. Just thought it would be best to break that change into a couple of PRs.

What do you think?

[nader-ziada]

/lgtm