[Bug] Azure Identity v1.13 - Managed Identity auth failing from GitHub Actions

[sam-piper-lendus]

Library version used

4.65.0

.NET version

.NET 8

Scenario

ManagedIdentityClient - managed identity

Is this a new or an existing app?

The app is in production, I haven't upgraded MSAL, but started seeing this issue

Issue description and reproduction steps

I know there is an existing issue open for this, but we are getting the same problem with a different stack trace, so wanted to make sure our issue scenario was provided as well.

Our GitHub Actions workflow for main CI build is failing after upgrading to v1.13 of Azure.Identity, which uses MSAL 4.65 I believe for token acquisition now. The failure is coming from trying to load KeyVault secrets, works fine on v1.12.x of Azure.Identity.

The stack trace we are getting below is generated when we try to run Swashbuckle CLI on our API projects to generate swagger.json files:

Unhandled exception. Azure.Identity.AuthenticationFailedException: ManagedIdentityCredential authentication failed: [Managed Identity] Authentication unavailable. Either the requested identity has not been assigned to this resource, or other errors could be present. Ensure the identity is correctly assigned and check the inner exception for more details. For more information, visit https://aka.ms/msal-managed-identity.
  Status: BadRequest
  Content:
  {"error":"invalid_request","error_description":"Identity not found"}
  
  Headers:
  Server: IMDS/150.870.65.1475
  x-ms-request-id: d4fd1e99-4114-4673-8db6-44e1d8f35224
  Date: Tue, 22 Oct 2024 01:34:51 GMT
  
[Managed Identity] Error Code : invalid_request error Description: Identity not found [D:\a\LendUs\LendUs\apps\LendUs.Platform.Services.ConsumerApi\LendUs.Platform.Services.ConsumerApi.csproj]
  See the troubleshooting guide for more information. https://aka.ms/azsdk/net/identity/managedidentitycredential/troubleshoot
   ---> MSAL.NetCore.4.65.0.0.MsalServiceException:
  	ErrorCode: managed_identity_request_failed
  Microsoft.Identity.Client.MsalServiceException: [Managed Identity] Authentication unavailable. Either the requested identity has not been assigned to this resource, or other errors could be present. Ensure the identity is correctly assigned and check the inner exception for more details. For more information, visit https://aka.ms/msal-managed-identity.
  Status: BadRequest
  Content:
  {"error":"invalid_request","error_description":"Identity not found"}
  
  Headers:
  Server: IMDS/150.870.65.1475
  x-ms-request-id: d4fd1e99-4114-4673-8db6-44e1d8f35224
  Date: Tue, 22 Oct 2024 01:34:51 GMT
  
[Managed Identity] Error Code : invalid_request error Description: Identity not found [D:\a\LendUs\LendUs\apps\LendUs.Platform.Services.ConsumerApi\LendUs.Platform.Services.ConsumerApi.csproj]
     at Microsoft.Identity.Client.ManagedIdentity.ImdsManagedIdentitySource.HandleResponseAsync(AcquireTokenForManagedIdentityParameters parameters, HttpResponse response, CancellationToken cancellationToken)
     at Microsoft.Identity.Client.ManagedIdentity.AbstractManagedIdentity.AuthenticateAsync(AcquireTokenForManagedIdentityParameters parameters, CancellationToken cancellationToken)
     at Microsoft.Identity.Client.Internal.Requests.ManagedIdentityAuthRequest.SendTokenRequestForManagedIdentityAsync(ILoggerAdapter logger, CancellationToken cancellationToken)
     at Microsoft.Identity.Client.Internal.Requests.ManagedIdentityAuthRequest.GetAccessTokenAsync(CancellationToken cancellationToken, ILoggerAdapter logger)
     at Microsoft.Identity.Client.Internal.Requests.ManagedIdentityAuthRequest.ExecuteAsync(CancellationToken cancellationToken)
     at Microsoft.Identity.Client.Internal.Requests.RequestBase.<>c__DisplayClass11_1.<<RunAsync>b__1>d.MoveNext()
  --- End of stack trace from previous location ---
     at Microsoft.Identity.Client.Utils.StopwatchService.MeasureCodeBlockAsync(Func`1 codeBlock)
     at Microsoft.Identity.Client.Internal.Requests.RequestBase.RunAsync(CancellationToken cancellationToken)
     at Microsoft.Identity.Client.ApiConfig.Executors.ManagedIdentityExecutor.ExecuteAsync(AcquireTokenCommonParameters commonParameters, AcquireTokenForManagedIdentityParameters managedIdentityParameters, CancellationToken cancellationToken)
     at Azure.Identity.MsalManagedIdentityClient.AcquireTokenForManagedIdentityAsyncCore(Boolean async, TokenRequestContext requestContext, CancellationToken cancellationToken)
     at Azure.Identity.MsalManagedIdentityClient.AcquireTokenForManagedIdentityAsync(TokenRequestContext requestContext, CancellationToken cancellationToken)
     at Azure.Identity.ImdsManagedIdentityProbeSource.AuthenticateAsync(Boolean async, TokenRequestContext context, CancellationToken cancellationToken)
     at Azure.Identity.ManagedIdentityClient.AuthenticateCoreAsync(Boolean async, TokenRequestContext context, CancellationToken cancellationToken)
     at Azure.Identity.ManagedIdentityClient.AuthenticateAsync(Boolean async, TokenRequestContext context, CancellationToken cancellationToken)
     at Azure.Identity.ManagedIdentityCredential.GetTokenImplAsync(Boolean async, TokenRequestContext requestContext, CancellationToken cancellationToken)
  	StatusCode: 0 
  	ResponseBody:  
  	Headers: 
     --- End of inner exception stack trace ---
     at Azure.Identity.CredentialDiagnosticScope.FailWrapAndThrow(Exception ex, String additionalMessage, Boolean isCredentialUnavailable)
     at Azure.Identity.ManagedIdentityCredential.GetTokenImplAsync(Boolean async, TokenRequestContext requestContext, CancellationToken cancellationToken)
     at Azure.Core.Pipeline.TaskExtensions.EnsureCompleted[T](ValueTask`1 task)
     at Azure.Identity.ManagedIdentityCredential.GetToken(TokenRequestContext requestContext, CancellationToken cancellationToken)
     at Azure.Identity.DefaultAzureCredential.GetTokenFromSourcesAsync(TokenCredential[] sources, TokenRequestContext requestContext, Boolean async, CancellationToken cancellationToken)
     at Azure.Identity.DefaultAzureCredential.GetTokenImplAsync(Boolean async, TokenRequestContext requestContext, CancellationToken cancellationToken)
     at Azure.Identity.CredentialDiagnosticScope.FailWrapAndThrow(Exception ex, String additionalMessage, Boolean isCredentialUnavailable)
     at Azure.Identity.DefaultAzureCredential.GetTokenImplAsync(Boolean async, TokenRequestContext requestContext, CancellationToken cancellationToken)
     at Azure.Core.Pipeline.TaskExtensions.EnsureCompleted[T](ValueTask`1 task)
     at Azure.Identity.DefaultAzureCredential.GetToken(TokenRequestContext requestContext, CancellationToken cancellationToken)
     at Azure.Core.Pipeline.BearerTokenAuthenticationPolicy.AccessTokenCache.SetResultOnTcsFromCredentialAsync(TokenRequestContext context, TaskCompletionSource`1 targetTcs, Boolean async, CancellationToken cancellationToken)
     at Azure.Core.Pipeline.BearerTokenAuthenticationPolicy.AccessTokenCache.GetAuthHeaderValueAsync(HttpMessage message, TokenRequestContext context, Boolean async)
     at Azure.Core.Pipeline.TaskExtensions.EnsureCompleted[T](Task`1 task)
     at Azure.Core.Pipeline.BearerTokenAuthenticationPolicy.AccessTokenCache.TokenRequestState.GetCurrentHeaderValue(Boolean async, Boolean checkForCompletion, CancellationToken cancellationToken)
     at Azure.Core.Pipeline.BearerTokenAuthenticationPolicy.AccessTokenCache.GetAuthHeaderValueAsync(HttpMessage message, TokenRequestContext context, Boolean async)
     at Azure.Core.Pipeline.TaskExtensions.EnsureCompleted[T](ValueTask`1 task)
     at Azure.Core.Pipeline.BearerTokenAuthenticationPolicy.AuthenticateAndAuthorizeRequest(HttpMessage message, TokenRequestContext context)
     at Azure.Security.KeyVault.ChallengeBasedAuthenticationPolicy.AuthorizeRequestOnChallengeAsyncInternal(HttpMessage message, Boolean async)
     at Azure.Core.Pipeline.TaskExtensions.EnsureCompleted[T](ValueTask`1 task)
     at Azure.Security.KeyVault.ChallengeBasedAuthenticationPolicy.AuthorizeRequestOnChallenge(HttpMessage message)
     at Azure.Core.Pipeline.BearerTokenAuthenticationPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory`1 pipeline, Boolean async)
     at Azure.Core.Pipeline.TaskExtensions.EnsureCompleted(ValueTask task)
     at Azure.Core.Pipeline.BearerTokenAuthenticationPolicy.Process(HttpMessage message, ReadOnlyMemory`1 pipeline)
     at Azure.Core.Pipeline.HttpPipelinePolicy.ProcessNext(HttpMessage message, ReadOnlyMemory`1 pipeline)
     at Azure.Core.Pipeline.RedirectPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory`1 pipeline, Boolean async)
     at Azure.Core.Pipeline.TaskExtensions.EnsureCompleted(ValueTask task)
     at Azure.Core.Pipeline.RedirectPolicy.Process(HttpMessage message, ReadOnlyMemory`1 pipeline)
     at Azure.Core.Pipeline.HttpPipelinePolicy.ProcessNext(HttpMessage message, ReadOnlyMemory`1 pipeline)
     at Azure.Core.Pipeline.RetryPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory`1 pipeline, Boolean async)
     at Azure.Core.Pipeline.RetryPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory`1 pipeline, Boolean async)
     at Azure.Core.Pipeline.TaskExtensions.EnsureCompleted(ValueTask task)
     at Azure.Core.Pipeline.RetryPolicy.Process(HttpMessage message, ReadOnlyMemory`1 pipeline)
     at Azure.Core.Pipeline.HttpPipelinePolicy.ProcessNext(HttpMessage message, ReadOnlyMemory`1 pipeline)
     at Azure.Core.Pipeline.HttpPipelineSynchronousPolicy.Process(HttpMessage message, ReadOnlyMemory`1 pipeline)
     at Azure.Core.Pipeline.HttpPipelinePolicy.ProcessNext(HttpMessage message, ReadOnlyMemory`1 pipeline)
     at Azure.Core.Pipeline.HttpPipelineSynchronousPolicy.Process(HttpMessage message, ReadOnlyMemory`1 pipeline)
     at Azure.Core.Pipeline.HttpPipelinePolicy.ProcessNext(HttpMessage message, ReadOnlyMemory`1 pipeline)
     at Azure.Core.Pipeline.HttpPipelineSynchronousPolicy.Process(HttpMessage message, ReadOnlyMemory`1 pipeline)
     at Azure.Core.Pipeline.HttpPipeline.Send(HttpMessage message, CancellationToken cancellationToken)
     at Azure.Core.Pipeline.HttpPipeline.SendRequest(Request request, CancellationToken cancellationToken)
     at Azure.Security.KeyVault.KeyVaultPipeline.SendRequest(Request request, CancellationToken cancellationToken)
     at Azure.Security.KeyVault.KeyVaultPipeline.GetPage[T](Uri firstPageUri, String nextLink, Func`1 itemFactory, String operationName, CancellationToken cancellationToken)
     at Azure.Security.KeyVault.Secrets.SecretClient.<>c__DisplayClass15_0.<GetPropertiesOfSecrets>b__0(String nextLink)
     at Azure.Core.PageResponseEnumerator.<>c__DisplayClass0_0`1.<CreateEnumerable>b__0(String continuationToken, Nullable`1 pageSizeHint)
     at Azure.Core.PageResponseEnumerator.FuncPageable`1.AsPages(String continuationToken, Nullable`1 pageSizeHint)+MoveNext()
     at Azure.Pageable`1.GetEnumerator()+MoveNext()
     at Azure.Extensions.AspNetCore.Configuration.Secrets.AzureKeyVaultConfigurationProvider.Load()
     at Microsoft.Extensions.Configuration.ConfigurationManager.AddSource(IConfigurationSource source)
     at Microsoft.Extensions.Configuration.ConfigurationManager.Microsoft.Extensions.Configuration.IConfigurationBuilder.Add(IConfigurationSource source)
     at Microsoft.Extensions.Configuration.AzureKeyVaultConfigurationExtensions.AddAzureKeyVault(IConfigurationBuilder configurationBuilder, SecretClient client, AzureKeyVaultConfigurationOptions options)
     at Microsoft.Extensions.Configuration.AzureKeyVaultConfigurationExtensions.AddAzureKeyVault(IConfigurationBuilder configurationBuilder, Uri vaultUri, TokenCredential credential, KeyVaultSecretManager manager)
     at Microsoft.Extensions.Configuration.AzureKeyVaultConfigurationExtensions.AddAzureKeyVault(IConfigurationBuilder configurationBuilder, Uri vaultUri, TokenCredential credential)
     at LendUs.Platform.Infrastructure.Services.ConfigurationExtensions.AddKeyVaultSecrets(ConfigurationManager configuration) in D:\a\LendUs\LendUs\apps\LendUs.Platform.Infrastructure\Services\ConfigurationExtensions.cs:line 14
     at LendUs.Platform.Services.ConsumerApi.Program.Main(String[] args) in D:\a\LendUs\LendUs\apps\LendUs.Platform.Services.ConsumerApi\Program.cs:line 36
     at LendUs.Platform.Services.ConsumerApi.Program.<Main>(String[] args)
     at System.RuntimeMethodHandle.InvokeMethod(Object target, Void** arguments, Signature sig, Boolean isConstructor)
     at System.Reflection.MethodBaseInvoker.InvokeDirectByRefWithFewArgs(Object obj, Span`1 copyOfArgs, BindingFlags invokeAttr)
  --- End of stack trace from previous location ---
     at Microsoft.Extensions.Hosting.HostFactoryResolver.HostingListener.CreateHost() in /_/src/Swashbuckle.AspNetCore.Cli/HostFactoryResolver.cs:line 276
     at Microsoft.Extensions.Hosting.HostFactoryResolver.<>c__DisplayClass8_0.<ResolveHostFactory>b__0(String[] args) in /_/src/Swashbuckle.AspNetCore.Cli/HostFactoryResolver.cs:line 75
     at Swashbuckle.AspNetCore.Cli.HostingApplication.GetServiceProvider(Assembly assembly) in /_/src/Swashbuckle.AspNetCore.Cli/HostingApplication.cs:line 87
     at Swashbuckle.AspNetCore.Cli.Program.GetServiceProvider(Assembly startupAssembly) in /_/src/Swashbuckle.AspNetCore.Cli/Program.cs:line 190
     at Swashbuckle.AspNetCore.Cli.Program.<>c.<Main>b__0_4(IDictionary`2 namedArgs) in /_/src/Swashbuckle.AspNetCore.Cli/Program.cs:line 88
     at Swashbuckle.AspNetCore.Cli.CommandRunner.Run(IEnumerable`1 args) in /_/src/Swashbuckle.AspNetCore.Cli/CommandRunner.cs:line 68
     at Swashbuckle.AspNetCore.Cli.CommandRunner.Run(IEnumerable`1 args) in /_/src/Swashbuckle.AspNetCore.Cli/CommandRunner.cs:line 59
     at Swashbuckle.AspNetCore.Cli.Program.Main(String[] args) in /_/src/Swashbuckle.AspNetCore.Cli/Program.cs:line 159
D:\a\LendUs\LendUs\apps\LendUs.Platform.Services.ConsumerApi\LendUs.Platform.Services.ConsumerApi.csproj(35,9): error MSB3073: The command "dotnet swagger tofile --output swagger.json bin\Debug\net8.0\LendUs.Platform.Services.ConsumerApi.dll v1" exited with code -5[32](https://github.com/admin-clicklend/LendUs/actions/runs/11451536394/job/31860908395#step:10:33)462766.

Relevant code snippets

No response

Expected behavior

No response

Identity provider

Microsoft Entra ID (Work and School accounts and Personal Microsoft accounts)

Regression

No response

Solution and workarounds

No response

[gladjohn]

@sam-piper-lendus please revert to use v1.12.x of Azure.Identity until we can identify the issue.

cc: @christothes how does Azure SDK determine if the source is GitHub in Managed Identity flows? or were you using the two legged CCA flow here before? to exchange GitHub assertion for a AAD token?

[christothes]

I believe this bug is resolved by the following PR which has not yet shipped in a patch - Azure/azure-sdk-for-net#46711.

[gladjohn]

Thanks @christothes

@sam-piper-lendus marking as external as this fix was made by Azure SDK - Azure/azure-sdk-for-net#46711.

[aherrick]

+1 we are also seeing this in 13. rolling back to 12 has fixed for now.

[bgavrilMS]

@aherrick - thanks for reporting. This will be fixed in Azure SDK 1.13.1 or 1.40.0 (whichever comes first).

[christothes]

https://www.nuget.org/packages/Azure.Identity/1.13.1