Destroy storage account with network rules is impossible (hangs forever)

[krispetkov]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform (and AzureRM Provider) Version

• Terraform v0.11.11
• Provider: v1.41

Affected Resource(s)

• azurerm_storage_account

Terraform Configuration Files

provider "azurerm" {
  subscription_id = "${var.azure_subscription_id}"
  client_id = "${var.azure_client_id}"
  client_secret = "${var.azure_client_secret}"
  tenant_id = "${var.azure_tenant_id}"
}

resource "azurerm_storage_account" "my_storage_azure" {
  name                      = "${var.myazurestorage_name}"
  resource_group_name       = "${var.resource_group_name}"
  location                  = "${var.location}"
  account_kind              = "StorageV2"
  account_tier              = "Standard"
  access_tier               = "Hot"
  account_replication_type  = "ZRS"
  enable_https_traffic_only = true

  network_rules {
    default_action             = "Deny"
    virtual_network_subnet_ids = ["${var.platform_subnets}","${var.bosh_subnets}"]
  }
}

resource "azurerm_postgresql_server" "my_postgresql_server" {
  name                = "${var.mypostgredb_name}"
  location            = "${var.location}"
  resource_group_name = "${var.resource_group_name}"

  sku {
    name     = "GP_Gen5_2"
    capacity = 2
    tier     = "GeneralPurpose"
    family   = "Gen5"
  }

  storage_profile {
    storage_mb            = 5120
    backup_retention_days = 14
    geo_redundant_backup  = "Disabled"
  }

  administrator_login          = "vcap"
  administrator_login_password = "${var.db_adm_password}"
  version                      = "10"
  ssl_enforcement              = "Enabled"
}

resource "azurerm_postgresql_database" "my_postgre_db" {
  name                = "mypostgredb"
  resource_group_name = "${var.resource_group_name}"
  server_name         = "${azurerm_postgresql_server.my_postgresql_server.name}"
  charset             = "UTF8"
  collation           = "English_United States.1252"
}

resource "azurerm_postgresql_virtual_network_rule" "network_rule_subnets" {
  count                                = "${length(var.platform_subnets)}"
  name                                 = "postgresql-vnet-rule-subnet-${count.index}"
  resource_group_name                  = "${var.resource_group_name}"
  server_name                          = "${azurerm_postgresql_server.my_postgresql_server.name}"
  subnet_id                            = "${element(var.platform_subnets, count.index)}"
  ignore_missing_vnet_service_endpoint = true
}

output "my_storage_account" {
  value = "${azurerm_storage_account.my_storage_azure.name}"
}

output "my_storage_account_primary_key" {
  value = "${azurerm_storage_account.my_storage_azure.primary_access_key}"
  sensitive = true
}

output "my_storage_account_secondary_key"{
  value = "${azurerm_storage_account.my_storage_azure.secondary_access_key}"
  sensitive = true
}

output "my_postgresql_server_name" {
  value = "${azurerm_postgresql_server.my_postgresql_server.name}"
}

output "my_postgresql_server_host" {
  value = "${azurerm_postgresql_server.my_postgresql_server.fqdn}"
}

output "my_postgresql_server_admin_user" {
  value = "${azurerm_postgresql_server.my_postgresql_server.administrator_login}"
}
output "my_postgresql_admin_pass" {
  value = "${azurerm_postgresql_server.my_postgresql_server.administrator_login_password}"
  sensitive = true
}

output "my_postgresql_db_name" {
  value = "${azurerm_postgresql_database.my_postgre_db.name}"
}

output "my_storage_primary_file_endpoint" {
  value = "${azurerm_storage_account.my_storage_azure.primary_file_endpoint}"
}

Debug Output

Recurring:
azurerm_storage_account.my_storage_azure: Still destroying... (ID: /subscriptions/<GUID>-...Microsoft.Storage/storageAccounts/test, 1h42m51s elapsed)

Trace Output

Recurring:
2020/01/17 12:05:07 [TRACE] dag/walk: vertex "output.my_storage_primary_file_endpoint (destroy)", waiting for: "output.my_storage_account_secondary_key (destroy)" 2020/01/17 12:05:07 [TRACE] dag/walk: vertex "output.my_storage_account_primary_key (destroy)", waiting for: "output.my_storage_account (destroy)" 2020/01/17 12:05:07 [TRACE] dag/walk: vertex "root", waiting for: "meta.count-boundary (count boundary fixup)" 2020/01/17 12:05:07 [TRACE] dag/walk: vertex "provider.azurerm (close)", waiting for: "azurerm_storage_account.my_storage_azure (destroy)" 2020/01/17 12:05:07 [TRACE] dag/walk: vertex "output.my_storage_account (destroy)", waiting for: "azurerm_storage_account.my_storage_azure (destroy)" 2020/01/17 12:05:07 [TRACE] dag/walk: vertex "meta.count-boundary (count boundary fixup)", waiting for: "output.my_storage_primary_file_endpoint (destroy)" 2020/01/17 12:05:07 [TRACE] dag/walk: vertex "output.my_storage_account_secondary_key (destroy)", waiting for: "output.my_storage_account_primary_key (destroy)" 2020/01/17 12:05:12 [TRACE] dag/walk: vertex "root", waiting for: "meta.count-boundary (count boundary fixup)" 2020/01/17 12:05:12 [TRACE] dag/walk: vertex "output.my_storage_primary_file_endpoint (destroy)", waiting for: "output.my_storage_account_secondary_key (destroy)" 2020/01/17 12:05:12 [TRACE] dag/walk: vertex "meta.count-boundary (count boundary fixup)", waiting for: "output.my_storage_primary_file_endpoint (destroy)" 2020/01/17 12:05:12 [TRACE] dag/walk: vertex "output.my_storage_account_primary_key (destroy)", waiting for: "output.my_storage_account (destroy)" 2020/01/17 12:05:12 [TRACE] dag/walk: vertex "provider.azurerm (close)", waiting for: "azurerm_storage_account.my_storage_azure (destroy)" 2020/01/17 12:05:12 [TRACE] dag/walk: vertex "output.my_storage_account (destroy)", waiting for: "azurerm_storage_account.my_storage_azure (destroy)" 2020/01/17 12:05:12 [TRACE] dag/walk: vertex "output.my_storage_account_secondary_key (destroy)", waiting for: "output.my_storage_account_primary_key (destroy)" azurerm_storage_account.my_storage_azure: Still destroying... (ID: /subscriptions/<GUID>...Microsoft.Storage/storageAccounts/test, 10s elapsed)

Panic Output

N/A

Expected Behavior

Destroy the storage account

Actual Behavior

Hangs forever and does not destroys the storage account

Steps to Reproduce

1. terraform apply
2. terraform destroy

Important Factoids

• Location: West Europe

References

• #3156

[krispetkov]

I have found that this issue (unable to destroy the storage account and hangs forever) is related to the network rules. If I remove:
network_rules { default_action = "Deny" virtual_network_subnet_ids = ["${var.platform_subnets}","${var.bosh_subnets}"] }
Than everything is OK and the destroy is working as it should.
But if I want to use network rules than it is impossible to destroy the storage account...

The other strange thing is that if I create the storage account with network rules and I delete the network rules manually from the Azure Portal Console and remove them from the state than it is still impossible to delete the storage account...

@tombuildsstuff can you please give some help here?

[mhaarbrink]

I've encountered a similar problem and issue #3156 provided help. In my case I deployed a storage account via TF with 2 subnets from 1 vnet and 1 subnet from another vnet. Issue 3156 suggested the issue was with multiple subnets in the same vnet. I manually deleted one of the subnets in the first vnet (leaving 1 subnet in each vnet in the rules) and re-ran the destroy and it succeeded.

I believe this happens because of the lock placed on the vnet so you can have more than one subnet in your rules as long as they are from different vnets because it will lock each vnet independently.

[krispetkov]

Hi @mhaarbrink

That's right. This is the exact problem. I have made a fix for this bug and I made a PR which is still waiting for approval. And I hope to be approved but for now no one reviewed it.
This is the PR: #5565
You can see a full description of the problem there.

[katbyte]

closed by #5565

[ghost]

This has been released in version 1.44.0 of the provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading. As an example:

provider "azurerm" {
    version = "~> 1.44.0"
}
# ... other configuration ...

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. If you feel I made an error 🤖 🙉 , please reach out to my human friends 👉 [email protected]. Thanks!