Create a Service Principal for Azure clusters

[sgibson91]

This PR updates our Azure Terraform config to create a service principal and secret and output these so we can use the info to login in a similar way to how GCP uses Service Accounts, and hence replacing the raw kubeconfig auth method.

This is the first piece for both #840 and #841

[yuvipanda]

\o/ This is super awesome.

Our UToronto access doesn't let us create service principals. While we should eventually get that fixed, I don't want the migration to block on that. So we should try make these temporarily conditional.

[sgibson91]

Good to know, cheers!

[sgibson91]

Ok @yuvipanda I have now made it optional to create service principals

[yuvipanda]

LGTM!

[sgibson91]

Hmmmm, seems like I still don't have enough privileges - @jhamman?

Error: Could not list existing service principals

with azuread_service_principal.service_principal[0],
on service-principal.tf line 1, in resource "azuread_service_principal" "service_principal":
1: resource "azuread_service_principal" "service_principal" {

ServicePrincipalsClient.BaseClient.Get(): unexpected status 403 with OData error: Authorization_RequestDenied: Insufficient privileges to complete the
operation.

[sgibson91]

Ok, I'm going to rescope this PR to not apply the change to carbonplan and then merge it. We can figure out the permissions to create a service principal for carbonplan later.

[jhamman]

Thanks @sgibson91 for working on this. Happy to adjust permissions as needed on our side. Confirming that right now, you do have the Owner role in the subscription being used for our deployment.

[sgibson91]

Thanks @jhamman. Knowing Azure, I am 100% confident that it's something far more complicated and annoying :) I will update the main issue tomorrow