azurerm_private_dns_zone_virtual_network_link fails when linking to Virtual Network in a different subscription

[captainhook]

Is there an existing issue for this?

• I have searched the existing issues

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment and review the contribution guide to help.

Terraform Version

1.7.1

AzureRM Provider Version

3.89.0

Affected Resource(s)/Data Source(s)

azurerm_private_dns_zone_virtual_network_link

Terraform Configuration Files

# Define providers and subscriptions
provider "azurerm" {
  alias                      = "Connectivity-prod"
  skip_provider_registration = false
  features {}
  subscription_id = azurerm_subscription.Connectivity-prod-001.subscription_id
}
provider "azurerm" {
  alias                      = "Identity-prod"
  skip_provider_registration = false
  features {}
  subscription_id = azurerm_subscription.Identity-prod-001.subscription_id
}
resource "azurerm_subscription" "Identity-prod-001" {
  subscription_name = "Identity-prod-001"
  billing_scope_id  = data.azurerm_billing_mca_account_scope.Company.id
}
resource "azurerm_subscription" "Connectivity-prod-001" {
  subscription_name = "Identity-prod-001"
  billing_scope_id  = data.azurerm_billing_mca_account_scope.Company.id
}


# Defined a resource group and vnet in subscription-2
resource "azurerm_resource_group" "IdentityConnectivity-prod-001" {
  provider = azurerm.Identity-prod
  name     = "rg-IdentityConnectivity-prod-001"
  location = "UK South"
}
resource "azurerm_virtual_network" "IdentityConnectivity-prod-001" {
  provider            = azurerm.Identity-prod
  name                = "Vnet1"
  location            = "North Europe"
  resource_group_name = azurerm_resource_group.IdentityConnectivity-prod-001.name
  address_space       = ["10.80.0.0/14"]
}

# Defined a bunch of private DNZ zones:
locals {
  private_endpoint_dns = {
    # Storage
    StorageAccountBlob          = "privatelink.blob.core.windows.net"
    StorageAccountTable         = "privatelink.table.core.windows.net"
    StorageAccountQueue         = "privatelink.queue.core.windows.net"
    StorageAccountFile          = "privatelink.file.core.windows.net"
    StorageAccountWeb           = "privatelink.web.core.windows.net"
    AzureDataLakeFileSystemGen2 = "privatelink.dfs.core.windows.net"
    AzureFileSync               = "privatelink.afs.azure.net"
  }
}
resource "azurerm_private_dns_zone" "private_endpoint_dns" {
  provider            = azurerm.Connectivity-prod
  for_each            = local.private_endpoint_dns
  name                = each.value
  resource_group_name = azurerm_resource_group.rg-PrivateDNSZones-prod-001.name
}

# Then finally I am trying to create the virtual network links:
resource "azurerm_private_dns_zone_virtual_network_link" "private_endpoint_dns_vnetlink" {
  name                  = azurerm_virtual_network.IdentityConnectivity-prod-001.name
  resource_group_name   = azurerm_resource_group.rg-PrivateDNSZones-prod-001.name
  for_each              = local.private_endpoint_dns
  private_dns_zone_name = each.value
  virtual_network_id    = azurerm_virtual_network.IdentityConnectivity-prod-001.id
  provider              = azurerm.Identity-prod
}

Debug Output/Panic Output

{"location":"global","properties":{"registrationEnabled":false,"virtualNetwork":{"id":"/subscriptions/972axxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/rg-IdentityConnectivity-prod-001/providers/Microsoft.Network/virtualNetworks/Vnet1"}},"tags":{}}: timestamp=2024-01-31T11:31:11.526Z
2024-01-31T11:31:11.526Z [DEBUG] provider.terraform-provider-azurerm_v3.89.0_x5.exe: PUT https://management.azure.com/subscriptions/1cffxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/rg-PrivateDNSZones-prod-001/providers/Microsoft.Network/privateDnsZones/privatelink.azurecr.io/virtualNetworkLinks/Vnet1?api-version=2020-06-01: timestamp=2024-01-31T11:31:11.526Z
2024-01-31T11:31:11.531Z [DEBUG] provider.terraform-provider-azurerm_v3.89.0_x5.exe: AzureRM Response for https://management.azure.com/subscriptions/1cffxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/rg-PrivateDNSZones-prod-001/providers/Microsoft.Network/privateDnsZones/privatelink.blob.core.windows.net/virtualNetworkLinks/Vnet1?api-version=2020-06-01: 
HTTP/2.0 403 Forbidden
Content-Length: 457
Cache-Control: no-cache
Content-Type: application/json; charset=utf-8
Date: Wed, 31 Jan 2024 11:31:10 GMT
Expires: -1
Pragma: no-cache
Strict-Transport-Security: max-age=31536000; includeSubDomains
X-Cache: CONFIG_NOCACHE
X-Content-Type-Options: nosniff
X-Ms-Correlation-Request-Id: 4b1exxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
X-Ms-Failure-Cause: gateway
X-Ms-Request-Id: 2a86xxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
X-Ms-Routing-Request-Id: UKSOUTH:20240131T113111Z:2a86xxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
X-Msedge-Ref: Ref A: 037A768A0F13465xxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx Ref B: AMS23102061xxxx Ref C: 2024-01-31T11:31:11Z

{"error":{"code":"LinkedAuthorizationFailed","message":"The client has permission to perform action 'Microsoft.Network/virtualNetworks/join/action' on scope '/subscriptions/1cffxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/rg-PrivateDNSZones-prod-001/providers/Microsoft.Network/privateDnsZones/privatelink.blob.core.windows.net/virtualNetworkLinks/Vnet1', however the linked subscription '972axxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx' was not found. "}}: timestamp=2024-01-31T11:31:11.531Z
2024-01-31T11:31:11.531Z [ERROR] provider.terraform-provider-azurerm_v3.89.0_x5.exe: Response contains error diagnostic: diagnostic_severity=ERROR diagnostic_summary="creating/updating Virtual Network Link (Subscription: "1cffxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
Resource Group Name: "rg-PrivateDNSZones-prod-001"
Private Dns Zone Name: "privatelink.blob.core.windows.net"
Virtual Network Link Name: "Vnet1"): performing CreateOrUpdate: unexpected status 403 with error: LinkedAuthorizationFailed: The client has permission to perform action 'Microsoft.Network/virtualNetworks/join/action' on scope '/subscriptions/1cffxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/rg-PrivateDNSZones-prod-001/providers/Microsoft.Network/privateDnsZones/privatelink.blob.core.windows.net/virtualNetworkLinks/Vnet1', however the linked subscription '972axxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx' was not found. " tf_resource_type=azurerm_private_dns_zone_virtual_network_link @caller=github.com/hashicorp/[email protected]/tfprotov5/internal/diag/diagnostics.go:58 tf_proto_version=5.4 tf_provider_addr=provider tf_req_id=69c2e402-f281-8f92-d42a-fb544fdbd7ff tf_rpc=ApplyResourceChange @module=sdk.proto diagnostic_detail= timestamp=2024-01-31T11:31:11.531Z
2024-01-31T11:31:11.532Z [DEBUG] State storage *remote.State declined to persist a state snapshot
2024-01-31T11:31:11.532Z [ERROR] vertex "azurerm_private_dns_zone_virtual_network_link.private_endpoint_dns_vnetlink[\"StorageAccountBlob\"]" error: creating/updating Virtual Network Link (Subscription: "1cffxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
Resource Group Name: "rg-PrivateDNSZones-prod-001"
Private Dns Zone Name: "privatelink.blob.core.windows.net"
Virtual Network Link Name: "Vnet1"): performing CreateOrUpdate: unexpected status 403 with error: LinkedAuthorizationFailed: The client has permission to perform action 'Microsoft.Network/virtualNetworks/join/action' on scope '/subscriptions/1cffxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/rg-PrivateDNSZones-prod-001/providers/Microsoft.Network/privateDnsZones/privatelink.blob.core.windows.net/virtualNetworkLinks/Vnet1', however the linked subscription '972axxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx' was not found.

Expected Behaviour

It should successfully create a virtual network link associated with the virtual network in a different subscription to the private DNS zones.

Actual Behaviour

Fails with 'LinkedAuthorizationFailed' error.

Error: creating/updating Virtual Network Link (Subscription: "1cffxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
│ Resource Group Name: "rg-PrivateDNSZones-prod-001"
│ Private Dns Zone Name: "privatelink.azurehdinsight.net"
│ Virtual Network Link Name: "Vnet1"): performing CreateOrUpdate: unexpected status 403 with error: LinkedAuthorizationFailed: The client has permission to perform action 'Microsoft.Network/virtualNetworks/join/action' on scope '/subscriptions/1cffxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/rg-PrivateDNSZones-prod-001/providers/Microsoft.Network/privateDnsZones/privatelink.azurehdinsight.net/virtualNetworkLinks/Vnet1', however the linked subscription '972axxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx' was not found.
│
│   with azurerm_private_dns_zone_virtual_network_link.private_endpoint_dns_vnetlink["AzureHDInsight"],
│   on privateDNS.tf line 130, in resource "azurerm_private_dns_zone_virtual_network_link" "private_endpoint_dns_vnetlink":
│  130: resource "azurerm_private_dns_zone_virtual_network_link" "private_endpoint_dns_vnetlink" {

Steps to Reproduce

terraform apply

Important Factoids

No response

References

#21350 - may be the same but it is not reported as a bug
#5172
#23238

[wuxu92]

Hi @captainhook ,

This seems to be an API issue rather than a provider bug. Can you cross-link different subscriptions in the Portal?

[captainhook]

Hi @wuxu92, yes, I can create the vnet link through the Azure portal. I have tested through Azure Portal and Terraform using the same user accounts, so same rights (full).

[captainhook]

So I recreated everything with new subscriptions, and it works fine:

# Azure Provider source and version being used
terraform {
  required_providers {
    azurerm = {
      source  = "hashicorp/azurerm"
      version = "=3.90.0"
    }
  }
}

# Configure the Microsoft Azure Provider
provider "azurerm" {
  tenant_id                  = "3f94xxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
  skip_provider_registration = false
  features {
    key_vault {
      purge_soft_delete_on_destroy = false
    }
    resource_group {
      prevent_deletion_if_contains_resources = true
    }
    subscription {
      prevent_cancellation_on_destroy = true
    }
  }
}

# Create test hub and spoke subscriptions
data "azurerm_billing_mca_account_scope" "billing" {
  billing_account_name = "a614xxxx-xxxxxxxxxxxxxxxxxxxxxx"
  billing_profile_name = "T3xx-xxxx-xxx-xxx"
  invoice_section_name = "AHxx-xxxx-xxx-xxx"
}

resource "azurerm_subscription" "hub" {
  subscription_name = "test-hub"
  billing_scope_id  = data.azurerm_billing_mca_account_scope.billing.id
}

resource "azurerm_subscription" "spoke" {
  subscription_name = "test-spoke"
  billing_scope_id  = data.azurerm_billing_mca_account_scope.billing.id
}

# Configure new providers
provider "azurerm" {
  alias = "hub"
  features {}
  subscription_id            = azurerm_subscription.hub.subscription_id
  skip_provider_registration = false
}

provider "azurerm" {
  alias = "spoke"
  features {}
  subscription_id            = azurerm_subscription.spoke.subscription_id
  skip_provider_registration = false
}

# Create virtual network in hub
resource "azurerm_resource_group" "hub" {
  provider = azurerm.hub
  name     = "rg-hub-test"
  location = "UK South"
}

resource "azurerm_virtual_network" "hub" {
  provider            = azurerm.hub
  name                = "vnet-hub-test"
  location            = azurerm_resource_group.hub.location
  resource_group_name = azurerm_resource_group.hub.name
  address_space       = ["10.0.0.0/16"]

  subnet {
    name           = "subnet1"
    address_prefix = "10.0.1.0/24"
  }

  subnet {
    name           = "subnet2"
    address_prefix = "10.0.2.0/24"
  }

  depends_on = [azurerm_resource_group.hub]
}

# Create private DNS zones in spoke
resource "azurerm_resource_group" "spoke" {
  provider = azurerm.spoke
  name     = "rg-spoke-test"
  location = "UK South"
}

locals {
  private_endpoint_dns = {
    # Storage
    StorageAccountBlob  = "privatelink.blob.core.windows.net"
    StorageAccountTable = "privatelink.table.core.windows.net"
    StorageAccountQueue = "privatelink.queue.core.windows.net"
    StorageAccountFile  = "privatelink.file.core.windows.net"
    StorageAccountWeb   = "privatelink.web.core.windows.net"
  }
}
resource "azurerm_private_dns_zone" "spoke" {
  provider            = azurerm.spoke
  for_each            = local.private_endpoint_dns
  name                = each.value
  resource_group_name = azurerm_resource_group.spoke.name
  depends_on          = [azurerm_resource_group.spoke]
}

resource "azurerm_private_dns_zone_virtual_network_link" "spoke" {
  provider              = azurerm.spoke
  name                  = "${azurerm_virtual_network.hub.name}-link"
  resource_group_name   = azurerm_resource_group.spoke.name
  for_each              = local.private_endpoint_dns
  private_dns_zone_name = each.value
  virtual_network_id    = azurerm_virtual_network.hub.id
  depends_on            = [azurerm_private_dns_zone.spoke, azurerm_virtual_network.hub]
}

I then recreated the spoke vnet and it's all suddenly working fine... there must be something weird with that old vnet or I did something wrong in handling/importing it.

No bug in Terraform - lovely day!

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.