Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

auxiliary_tenant_ids LinkedAuthorizationFailed errors after azurerm v3.43.0 #21346

Closed
1 task done
nateoconnell opened this issue Apr 8, 2023 · 7 comments · Fixed by #21583
Closed
1 task done

auxiliary_tenant_ids LinkedAuthorizationFailed errors after azurerm v3.43.0 #21346

nateoconnell opened this issue Apr 8, 2023 · 7 comments · Fixed by #21583
Assignees
Labels
authentication bug sdk/requires-upgrade This is dependent upon upgrading an SDK v/3.x
Milestone

Comments

@nateoconnell
Copy link

nateoconnell commented Apr 8, 2023

Is there an existing issue for this?

  • I have searched the existing issues

Community Note

  • Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
  • Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
  • If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform Version

1.4.4

AzureRM Provider Version

3.51.0

Affected Resource(s)/Data Source(s)

azurerm_linux_virtual_machine

Terraform Configuration Files

terraform {
  required_version = "~> 1.1"
  required_providers {
    azurerm = {
      source  = "hashicorp/azurerm"
      version = "= 3.43.0"
    }
  }
}

provider "azurerm" {
  features {}
  subscription_id = "<target_subscription>"
  tenant_id       = "<target_tenant>"
  auxiliary_tenant_ids = [
    "<source_image_tenant>"
  ]
}

resource "azurerm_resource_group" "main" {
  name     = "provider-bugcheck-rg"
  location = "West US 3"
}

resource "azurerm_virtual_network" "main" {
  name                = "provider-bugcheck-vnet"
  address_space       = ["10.0.0.0/16"]
  location            = azurerm_resource_group.main.location
  resource_group_name = azurerm_resource_group.main.name
}

resource "azurerm_subnet" "main" {
  name                 = "provider-bugcheck-subnet"
  resource_group_name  = azurerm_resource_group.main.name
  virtual_network_name = azurerm_virtual_network.main.name
  address_prefixes     = ["10.0.2.0/24"]
}

resource "azurerm_network_interface" "main" {
  name                = "provider-bugcheck-nic"
  location            = azurerm_resource_group.main.location
  resource_group_name = azurerm_resource_group.main.name

  ip_configuration {
    name                          = "internal"
    subnet_id                     = azurerm_subnet.main.id
    private_ip_address_allocation = "Dynamic"
  }
}

resource "azurerm_linux_virtual_machine" "main" {
  name                = "provider-bugcheck-vm"
  resource_group_name = azurerm_resource_group.main.name
  location            = azurerm_resource_group.main.location
  size                = "Standard_A1_v2"
  admin_username      = "adminuser"
  network_interface_ids = [
    azurerm_network_interface.main.id,
  ]

  admin_ssh_key {
    username   = "adminuser"
    public_key = file("~/.ssh/id_rsa.pub")
  }

  os_disk {
    caching              = "ReadWrite"
    storage_account_type = "Standard_LRS"
  }

  source_image_id = "/subscriptions/<source_image_subscription>/resourceGroups/<source_image_resource_group>/providers/Microsoft.Compute/galleries/<source_image_gallery>/images/<source_image>/versions/<source_image_version>"
}

Debug Output/Panic Output

https://gist.github.com/nateoconnell/7148e9e71c7b0dacc08b06ddc694803a

Expected Behaviour

Terraform is able to provision a VM in one subscription using an image reference from a subscription in another tenant as long as the user running the apply has required permissions in both tenants and the image source tenant is specified in the auxiliary_tenant_ids provider configuration list.

After upgrading from azurerm v3.12.0 to v3.48.0 I found that I was no longer able to build VMs in one subscription from OS images from a compute gallery in another subscription using the auxiliary_tenant_ids argument in the azurerm provider config. It looks like some changes to auth sdks occurred between 3.43.0 and 3.44.0. I was able to confirm that I am able to build VMs successfully with 3.43.0, but run into LinkedAuthorizationFailed errors with version 3.44.0, 3.48.0, and the current latest 3.51.0 by swapping the provider version pin in the provided configuration.

Actual Behaviour

With azurerm versions >= 3.44.0 this no longer seems to work and a LinkedAuthorizationFailed error is produced.

$ terraform apply
[...]
Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following
symbols:
  + create

Terraform will perform the following actions:

  # azurerm_linux_virtual_machine.main will be created
  + resource "azurerm_linux_virtual_machine" "main" {
      [...]

Plan: 1 to add, 0 to change, 0 to destroy.

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

azurerm_linux_virtual_machine.main: Creating...
╷
│ Error: creating Linux Virtual Machine: (Name "provider-bugcheck-vm" / Resource Group "provider-bugcheck-rg"): compute.VirtualMachinesClient#CreateOrUpdate: Failure sending request: StatusCode=403 -- Original Error: Code="LinkedAuthorizationFailed" Message="The client has permission to perform action 'Microsoft.Compute/galleries/images/versions/read' on scope '/subscriptions/<target_subscription>/resourceGroups/provider-bugcheck-rg/providers/Microsoft.Compute/virtualMachines/provider-bugcheck-vm', however the current tenant '<target_tenant>' is not authorized to access linked subscription '<source_image_subscription>'."
│
│   with azurerm_linux_virtual_machine.main,
│   on main.tf line 51, in resource "azurerm_linux_virtual_machine" "main":
│   51: resource "azurerm_linux_virtual_machine" "main" {
│
╵

Steps to Reproduce

To reproduce the issue two tenants are needed, one with an OS image in a compute gallery replicated to the region in which a target VM should be built under the second tenant.

  • Set the azurerm provider version to 3.44.0, 3.48.0, or 3.51.0 in the provided terraform configuration and set the tenant, subscription, and image related values
  • Run terraform init, validate, plan, apply and the resource group and network components will be built successfully, but the VM creation will fail with the LinkedAuthorizationFailed error.
  • Now set the azurerm provider version to 3.43.0 and run terraform init -upgrade
  • Run terraform apply and the VM will build successfully.

Important Factoids

No response

References

Possibly related to #20320

@github-actions github-actions bot removed the bug label Apr 8, 2023
@myc2h6o
Copy link
Contributor

myc2h6o commented Apr 11, 2023

Hi @nateoconnell thanks for raising the issue! I'm able to repro the issue locally. And after a few rounds of testing, I'm able to workaround it with the Service Principal + Client Secret authentication https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/guides/service_principal_client_secret#configuring-the-service-principal-in-terraform. I guess there might be a misconfiguration with the az cli local environment. Would you mind trying client_id and client_secret authentication method and see if it could work? Meanwhile, we'll take a further look and see if there was something broken with the CLI auth method.

@nateoconnell
Copy link
Author

Thank you for looking into it! Currently I'm pinning to 3.43.0 for the builds that need the multitenant access, but will look into the Service Principal + Client Secret authentication as a workaround if I'm not able to maintain that pin.

@jan-hudec
Copy link

jan-hudec commented Apr 17, 2023

@myc2h6o It works in 3.42 with the same az configuration, so that shouldn't be an issue. I got the error on upgrading to 3.52, so I reverted back to 3.42 and it started working again and I didn't touch az in between at all.

… which suggests that the problem is likely to be in the code that obtains the tokens from az rather than az itself.

@manicminer
Copy link
Contributor

Many thanks for reporting this and helping to narrow down the cause of this bug. This is due to a bug in our SDK which should get fixed soon.

@manicminer manicminer added this to the v3.54.0 milestone Apr 26, 2023
@manicminer manicminer added the sdk/requires-upgrade This is dependent upon upgrading an SDK label Apr 26, 2023
@manicminer
Copy link
Contributor

Resolved with #21583

@github-actions
Copy link

This functionality has been released in v3.54.0 of the Terraform Provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template. Thank you!

Copy link

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators May 20, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
authentication bug sdk/requires-upgrade This is dependent upon upgrading an SDK v/3.x
Projects
None yet
5 participants