Authenticating with Azure DevOps using SPN

[Benly-walter]

Introduce identity based azure-pipelines trigger authentication for scaledObjects / scaledJobs.

Use-Case

With the recent introduction of Azure identity based authentication for Azure DevOps - link , it would be nice if we can leverage identities in the azure-pipelines trigger. As of documentation in 2.11, it seems that only a Personal Access Token (PAT) is supported.

I am using attached startup script for my agent where AZP_CLIENTID, AZP_CLIENTSECRET and AZP_TENANT values are used instead of the PAT token to authenticate with Azure DevOps. The agents are setup as scaled Jobs. The token generated by the identity in this approach is valid for 1 hour only. For this reason, the token is regenerated again in the cleanup() block to accommodate long running jobs. This has been working seamlessly in our setup for the last 2 months. If the same values are available to be passed within the azure-pipelines trigger, it would help remove the dependency on the PAT token altogether.

FYI - Below change has additionally been done in the Dockerfile. Ref - link
ENTRYPOINT [ "./start.sh", "--once" ]

startup.txt

Specification

• Enable identity support in azure-pipelines trigger authentication for Azure Pipelines.

[Benly-walter]

any updates please?

[JorTurFer]

This is a nice addition, are you willing to contribute adding support for it?

[Benly-walter]

@JorTurFer I am happy to provide support.
I've never done a contribution so I would need some guidance around it! Thanks for the reply.

[sixeyed]

@Benly-walter this is something we're also looking at. The build agent from version 3.227 supports authentication with Service Principal credentials, so you don't need to fetch a token or worry about expiry & rotation. In the config command call you can replace --auth PAT with:

  --auth SP `
  --tenantId $env:SP_TENANT_ID `
  --clientId $env:SP_CLIENT_ID `
  --clientSecret $env:SP_CLIENT_SECRET

That works for us as we deploy from Azure Pipelines and can get the creds with addSpnToEnvironment: true in the AzureCLI@2 task (which can then filter down into a Kubernetes Secret or KeyVault).

That route could also be used by the trigger. I'm not a native Go person but I think it could be added in getAzurePipelineRequest as an option - maybe a flag to say use SP from env, and then get a credential with the client SDK.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 7 days if no further activity occurs. Thank you for your contributions.

[stale]

This issue has been automatically closed due to inactivity.

[Benly-walter]

@sixeyed That's a good approach and could work. I am not a native GO person myself too :D

[Harsha480]

is this still available? I am new here and would like to contribute.

[JorTurFer]

This is still available, but I'm not sure if this other could be related covering this one #4694
@tomkerkhove ?

[lsuarez5280]

This is still available, but I'm not sure if this other could be related covering this one #4694 @tomkerkhove ?

I'm also very interested in this issue as I've been trying to eliminate PATs and my desired hosting model for Azure Pipelines Agents in my private Azure environment is Azure Container Apps which currently do not support providing managed identity to KEDA scalers via Workload Identity, leaving me with only the PAT option in this scaler.

Just for quick context, I'm an IAM SME especially regarding OIDC and the Microsoft Entra ID implementation of that protocol. I will say that a generic OAuth provider could certainly obtain a token with the correct information to support SPN authorization. I just spent a couple weeks tussling with implementing this in PowerShell, so I'm pretty sure I can provide documented guidance for using the approach in #4694, but it might be nice to provide native support in the scaler TriggerAuthentication itself because generic OAuth will require users to provide additional configuration (i.e. tenant token endpoint, correct scope arguments) that we could automatically infer in a first-party implementation to reduce unintended errors.

I haven't built in Go before, but I'm willing to support an implementation as well as I'm eager for this feature because PATs can be so fraught. The https://github.com/AzureAD/microsoft-authentication-library-for-go repo could also make pretty direct work of managing the token lifetime for the scaler, so it's mostly a matter of feeding in configuration.

If I can be of help, I'd be delighted to tackle this.

EDIT: An implementation could be simplified if someone releases the upstream dependency at microsoft/azure-devops-go-api#141

[tmeckel]

@lsuarez5280 The OAuth Service Principal Authentication in PR https://github.com/microsoft/terraform-provider-azuredevops/pull/747/files has been added into the Terraform Provider instead of adding it to AzDO GO SDK, what would be the very correct way to do so. Because everyone would benefit from it, not only the Terraform provider.

Also refer to my comment microsoft/azure-devops-go-api#141 (comment)

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 7 days if no further activity occurs. Thank you for your contributions.

[ReneRebsdorf]

#NotStale

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 7 days if no further activity occurs. Thank you for your contributions.

[Benly-walter]

#NotStale