Subnet service delegation to Azure Databricks continuously adds and removes actions

[eliasvakkuri]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform (and AzureRM Provider) Version

Terraform v0.12.9
provider.azurerm v1.35.0

Affected Resource(s)

• azurerm_subnet

Terraform Configuration Files

resource "azurerm_resource_group" "rg" {
  # Create Landing Zone resource group
  name     = "testresourcegroup"
  location = "westeurope"
}

resource "azurerm_virtual_network" "vnet" {
  # Create VNET
  name                = "testvnet"
  resource_group_name = azurerm_resource_group.rg.name
  location            = azurerm_resource_group.rg.location
  address_space       = ["10.0.0.0/16"]
}

resource "azurerm_subnet" "subnet_databricks_public" {
  # Create public subnet for Databricks workspace
  # Used for communication from clusters to external resources
  name                      = "databricks-public"
  resource_group_name       = azurerm_resource_group.rg.name
  virtual_network_name      = azurerm_virtual_network.vnet.name
  address_prefix            = "10.0.255.0/24"
  network_security_group_id = module.nsg_databricks.nsg_id
  service_endpoints         = ["Microsoft.Storage", "Microsoft.KeyVault"]

  delegation {
    name = "databricks-del-public"

    service_delegation {
      name = "Microsoft.Databricks/workspaces"
      actions = [
        "Microsoft.Network/virtualNetworks/subnets/action",
        "Microsoft.Network/virtualNetworks/subnets/join/action",
        "Microsoft.Network/virtualNetworks/subnets/prepareNetworkPolicies/action"
      ]
    }
  }
}

Expected Behavior

Terraform should accept these values and not create a diff on subsequent runs.

Actual Behavior

On deployment, Azure apparently automatically removes "Microsoft.Network/virtualNetworks/subnets/action" and adds "Microsoft.Network/virtualNetworks/subnets/unprepareNetworkPolicies/action". On subsequent plan, Terraform then tries to reverse this.

Below a terraform plan run directly after running apply with the values above:

~ resource "azurerm_subnet" "subnet_landing_databricks_public" {
        address_prefix            = "10.0.255.0/24"
        id                        = "/subscriptions/0e951e3a-0919-45b3-92f0-140c8beb69b5/resourceGroups/eliasv-lz-rg/providers/Microsoft.Network/virtualNetworks/eliasv-vnet/subnets/lz-databricks-public"
        ip_configurations         = []
        name                      = "lz-databricks-public"
        network_security_group_id = "/subscriptions/0e951e3a-0919-45b3-92f0-140c8beb69b5/resourceGroups/eliasv-lz-rg/providers/Microsoft.Network/networkSecurityGroups/eliasv-nsg-bricks-lz"
        resource_group_name       = "eliasv-lz-rg"
        service_endpoints         = [
            "Microsoft.Storage",
            "Microsoft.KeyVault",
        ]
        virtual_network_name      = "eliasv-vnet"

      ~ delegation {
            name = "databricks-del-public"

          ~ service_delegation {
              ~ actions = [
                  + "Microsoft.Network/virtualNetworks/subnets/action",
                    "Microsoft.Network/virtualNetworks/subnets/join/action",
                    "Microsoft.Network/virtualNetworks/subnets/prepareNetworkPolicies/action",
                  - "Microsoft.Network/virtualNetworks/subnets/unprepareNetworkPolicies/action",
                ]
                name    = "Microsoft.Databricks/workspaces"
            }
        }
    }

Steps to Reproduce

1. Create plan from snippet above
2. terraform apply
3. terraform plan

[eliasvakkuri]

I also tried adding "Microsoft.Network/virtualNetworks/subnets/unprepareNetworkPolicies/action" to the action list, but AzureRM provider does not accept this.

[plindf]

I have the same behaviour when adding delegations to Microsoft.NetApp

~ resource "azurerm_subnet" "infra-vnet1-netapp" {
        address_prefix       = "XXX/28"
        id                   = "/subscriptions/XXX/resourceGroups/infra/providers/Microsoft.Network/virtualNetworks/vnet1/subnets/netapp"
        ip_configurations    = [
            "/subscriptions/XXX/resourceGroups/infra/providers/Microsoft.Network/networkInterfaces/anf-vnet1-nic-N25PP6/ipConfigurations/ipconfig1",
        ]
        name                 = "netapp"
        resource_group_name  = "infra"
        service_endpoints    = []
        virtual_network_name = "vnet1"

      ~ delegation {
            name = "delegation"

          ~ service_delegation {
              ~ actions = [
                  - "Microsoft.Network/networkinterfaces/*",
                  + "Microsoft.Network/virtualNetworks/subnets/action",
                    "Microsoft.Network/virtualNetworks/subnets/join/action",
                  + "Microsoft.Network/virtualNetworks/subnets/prepareNetworkPolicies/action",
                ]
                name    = "Microsoft.Netapp/volumes"
            }
        }
    }

When adding "Microsoft.Network/networkinterfaces/*" to the action list, AzureRM does not accept it.

[richardzone]

I've created a PR #4690 to fix the 2 issues raised by both @eliasvakkuri and @hasterhorb

[Kampfgnom]

Out of curiosity: Where did you get the possible values from?

• https://docs.microsoft.com/en-us/azure/virtual-network/virtual-network-manage-subnet#permissions
• https://docs.microsoft.com/en-us/rest/api/virtualnetwork/subnets/createorupdate#delegation
• https://docs.microsoft.com/en-us/azure/virtual-network/manage-subnet-delegation

[eliasvakkuri]

@Kampfgnom the original list of subnet delegation actions is from Terraform azurerm provider documentation: https://www.terraform.io/docs/providers/azurerm/r/subnet.html#service_delegation . Have not found any documentation on the actions on Azure side.

[richardzone]

@Kampfgnom az network vnet subnet list-available-delegations --location [region]. This can be used to get all services eligible for delegation, along with the list of actions required with each kind of service.

[myrah]

I have the same issue because Terraform gives an error when trying to add "Microsoft.Network/virtualNetworks/subnets/unprepareNetworkPolicies/action" as one of the delegation actions, but it is one of the actions if Azure Databricks service adds subnet delegation through AzureRM API.

[ghost]

This has been released in version 1.36.0 of the provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading. As an example:

provider "azurerm" {
    version = "~> 1.36.0"
}
# ... other configuration ...

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. If you feel I made an error 🤖 🙉 , please reach out to my human friends 👉 [email protected]. Thanks!