Subscription ID of a resource from azurerm provider is replaced by local az cli account id

[DARB-CCM-S-20]

Is there an existing issue for this?

• I have searched the existing issues

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment and review the contribution guide to help.

Terraform Version

0.15.5

AzureRM Provider Version

3.52.0

Affected Resource(s)/Data Source(s)

azurerm_resource_group

Terraform Configuration Files

terraform {
  backend "azurerm" {
    storage_account_name = "tfstate"
    container_name       = "tfstorage"
    resource_group_name  = "Terraform"
    key                  = "terraform.tfstate"
  }
}

provider "azurerm" {
  features {}
  subscription_id = "<SUBSCRIPTION_ID X"
}

resource "azurerm_resource_group" "aks" {
  name     = "${var.prefix}-cluster"
  location = var.location

  tags = merge(local.common_tags, var.tags)
}

Debug Output/Panic Output

Error: reading resource group: resources.GroupsClient#Get: Failure responding to request: StatusCode=403 -- Original Error: autorest/azure: Service returned an error. Status=403 Code="AuthorizationFailed" Message="The client '[email protected]' with object id 'XXXXXXXXXXX' does not have authorization to perform action 'Microsoft.Resources/subscriptions/resourcegroups/read' over scope '/subscriptions/<SUBSCRIPTION_ID Y>/resourcegroups/aks-cluster' or the scope is invalid. If access was recently granted, please refresh your credentials."
│
│   with module.aks.azurerm_resource_group.aks,
│   on .terraform/modules/aks/main.tf line 11, in resource "azurerm_resource_group" "aks":
│   11: resource "azurerm_resource_group" "aks" {

Expected Behaviour

The subscription id for the resource group should be the subscription id set in the azurerm provider block, SUBSCRIPTION_ID X, and the plan should execute successfully.

Actual Behaviour

The subscription id for the resource group is replaced with the local az cli account subscription id, SUBSCRIPTION_ID Y (which needs to be set to the subscription of where the tf state file is stored) meaning the scope that is trying to be fetched is invalid.

az account set via:

az account set -s <SUBSCRIPTION_ID Y>

Steps to Reproduce

terraform plan

Important Factoids

No response

References

#14516

[austyuzhanin-klika]

This is a frustrating issue, and a regression as well.
It used to work in 3.43, but broke with 3.44 release and remains an issue to this date.

Currently, azurerm does not pass subscription id to cli authorizer, nor to any of the other authorizers it supports.
Authorizer from github.com/hashicorp/go-azure-sdk does not support subscription ids for cli either, always asking az for default account credentials, which is why az account set helps.

So this has to be fixed first in github.com/hashicorp/go-azure-sdk, and then here in github.com/hashicorp/terraform-provider-azurerm.

[austyuzhanin-klika]

To add some context to this.

In my case, we have multiple user accounts per tenant, and account user have access to different subscriptions.
In the past, before 3.44, when you specified subscription_id in provider configuration, it would be used to obtain credentials for the account that has access to that subscription.
Since 3.44, however, azurerm will obtain credentials for the "current" account within the tenant, i.e. the one configured with az account set or the one that was last used when doing az login, though both are the same thing essentially.

If you work with multiple configurations, one per subscription, you would have to constantly do az account set every time you try to work with config from a different sub.
Worse yet, having multiple aliased providers with different subscription ids is no longer possible.
In 3.43 all of this works as expected - no need for az account set and aliased providers work perfectly fine, each provider instance ending up with a different access credentials.

For this to be fixed, subscription id would have to be passed from azurerm to go-azure-sdk and there it needs to be added to az account get-access-token call.

[soerennielsen]

We are closing in on a year for this quite important bug to be closed.
For now I have the following in provider.tf, which works, but puts me about 60 versions behind. Some day I'll reach some resource that is buggy/unsupported and then I'm out of luck :(

Pretty please, someone?

provider "azurerm" {
  subscription_id = "............"
  features {
  }
}
terraform {
  required_providers {
    azurerm = {
      source  = "hashicorp/azurerm"
      version = "3.43.0" # LOCK! to force subscription id to work. https://github.com/hashicorp/terraform-provider-azurerm/issues/23068
    }
  }
}

[soerennielsen]

If anyone comes here with the same issue, I found that the following config DOES WORK in current versions:

provider "azurerm" {
subscription_id = "............"
tenant_id = "....."
features {
}
}
terraform {
required_providers {
azurerm = {
source = "hashicorp/azurerm"
}
}
}

Key is the tenant id. Without that the subscription id is simply not working.

[manicminer]

Thanks all for the feedback on this issue. I recognise this is an issue with certain configurations, but I would like to clarify the multi-subscription and/or multi-tenant scenarios which have a recommend configuration and those which currently do not work as expected.

1. Working: Subscription X and subscription Y in different tenants, authenticated principal has access to both: the tenant ID must be configured either via environment variable or the provider block, the subscription ID specified in the provider block (or via environment variable) is correctly used, and if this is absent, we call az account show to find the one you have elected as the default with Azure CLI.
2. Working: Subscription X and subscription Y in different tenants, Azure CLI signed into A & B user accounts where A has access to X, and B has access to Y: the tenant ID must be configured either via environment variable or the provider block, the subscription ID specified in the provider block (or via environment variable) is correctly used, and if this is absent, we call az account show to find the one you have elected as the default with Azure CLI.
3. Working: Subscription X and subscription Y in the same tenant, authenticated principal has access to both: the subscription ID specified in the provider block (or via environment variable) is correctly used, and if this is absent, we call az account show to find the one you have elected as the default with Azure CLI.
4. Known Issue: Subscription X and subscription Y in the same tenant, Azure CLI signed into A & B user accounts where A has access to X, and B has access to Y (more below).

In the latter scenario, it's not currently possible to apply a configuration that uses both subscriptions, and when working with a single subscription, you must use az account set to choose that subscription as the default, since access tokens will be acquired for either the default user, or at random depending on the output of az account show.

I am looking into ways to potentially resolve this - I do recognise that earlier versions passed the subscription ID to az-cli to serve as an indicator of which "account" (in az-cli terms) to use when acquiring a token; we may revert to this behavior or we may try a different approach. In the next major release (v4.0) or the provider, we will most likely be making subscription_id a required provider property, this will be factored into our approach to ensure we can support all authentication backends as best we can.

[manicminer]

#26282 will be included in this week's release and restores the earlier behavior of passing the subscription ID to Azure CLI when acquiring an access token. This should resolve the reported issue and enable authentication scenarios involving multiple user accounts and subscriptions in the same tenant.

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.