proposal for workload identity integration

[sonasingh46]

Signed-off-by: Ashutosh Kumar [email protected]

What this PR does / why we need it:

Proposal for Azure Workload Identity integration to Azure.

Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):
Design proposal for #2205

Special notes for your reviewer:

Please confirm that if this PR changes any image versions, then that's the sole change this PR makes.

TODOs:

• squashed commits
• includes documentation
• adds unit tests

Release note:

proposal for workload identity integration

[CecileRobertMichon]

Great research work @sonasingh46 and thank you @aramase for all the support. I'm excited to get this in!

[dtzar]

@sonasingh46 - thank you for this proposal and agreeing to help here. This is a very important enhancement needed by many for CAPZ. When do you think you'll have time to address feedback mentioned here and/or start on the coding work?

[sonasingh46]

@dtzar -- Thanks for chiming in. I have addressed the feedback in the proposal. I will raise a PR in parallel so that we can review and get early and quick feedback.
On a high level the only challenge right now I see is the Key Distribution to the control plane nodes. I will cc you to the relevant section on the proposal which mentions this.

[jackfrancis]

Let's prioritize starting a lazy consensus timer roughly a week before the planned 1.7.0 release date (10 Jan 2023)

[yastij]

a few comments

[aramase]

IIUC, this key pair is different from the signing keys used in the kind cluster. Why is this note part of the design doc?

[sonasingh46]

The signing keys that are used in kind cluster can be distributed to the workload cluster by creating a secret. Now, that same key pairs will be used by the apiserver and controller manager.

[CecileRobertMichon]

what's the need/use case to do that? is it optional or mandatory?

[sonasingh46]

This is optional.

Use case is the following:

• I set up a kind management cluster and have key pairs that can be used with workload identity.
• I can easily just distribute these key pairs to the workload cluster that I created using this as I will be using workload identity on the workload cluster.

[sonasingh46]

Or I will have to setup workload identity with keys that got generated for the workload cluster which resides in /etc/kubernetes/pki directory.

Or I will have to rotate the keys with the keys with which the workload identity credential is configured. (As kubeadm generate key pairs by default if not specified by the secret)

[aramase]

I can easily just distribute these key pairs to the workload cluster that I created using this as I will be using workload identity on the workload cluster.

Although this makes it easier for the user to setup WI in their workload cluster with less work in CAPZ, I wouldn't recommend doing this. As long as there is a 1:1 mapping between management cluster and workload cluster it's fine but if a single management cluster is used to create multiple workload clusters, allowing this would enable reusing same signing keys + issuer in different clusters which is insecure.

[sonasingh46]

Although this makes it easier for the user to setup WI in their workload cluster with less work in CAPZ, I wouldn't recommend doing this. As long as there is a 1:1 mapping between management cluster and workload cluster it's fine but if a single management cluster is used to create multiple workload clusters, allowing this would enable reusing same signing keys + issuer in different clusters which is insecure.

Agreed. I think this does not come as recommendation. Though if this gives in impression of a recommendation like this, I will remove this section.

If you see the user story Management Cluster Via Pivoting -- this is where this becomes useful.
https://github.com/kubernetes-sigs/cluster-api-provider-azure/pull/2814/files/d62c690a3f2cb8a3fcb2e8de44f25e42b678eaed#diff-8f168651958e0a88d9402ac98e3b8764c9a4b54d29de8284def5e2c9070dd11bR158

So what happens in this case is --

1. The cloud admin creates a kind cluster
2. The cloud admin create a workload cluster on azure ( 3 CP + 3 worker nodes )
3. The cloud admin wants to now convert this workload cluster to a mgmt cluster and use workload identity.
4. The cloud admins nukes the kind cluster.

( This is the pivoting process followed by a lot of user to create a full fledged production management cluster )

So at step 2 if keys are not explicitly specified via the secret then kubeadm generates default ones and now there are some extra steps rotating those. So just to keep the life simple this is the approach.

[sonasingh46]

Also, this will be helpful in CI.
We can pre create the secret using the same keys in the kind cluster in the CI. And now the cloud provider azure pods in workload cluster that got created will be capable of using workload identity.

[CecileRobertMichon]

/lgtm
/assign @aramase

[k8s-ci-robot]

LGTM label has been added.

Git tree hash: 8ab538a7a2faed2a2474f3dbad231168d20decd4

[aramase]

/lgtm

[k8s-ci-robot]

LGTM label has been added.

Git tree hash: ac4f2b654bc105bbabe3972eac0dae12cc3358f1

[CecileRobertMichon]

/lgtm
/approve
/hold until EOD 6/15

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: CecileRobertMichon

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [CecileRobertMichon]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[CecileRobertMichon]

/hold cancel

Congrats @sonasingh46 🎉