Resource group incorrect case sensitivity on terraform apply on azurerm_app_configuration_key forces replacement

[masterphenix]

Is there an existing issue for this?

• I have searched the existing issues

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment and review the contribution guide to help.

Terraform Version

1.7.4

AzureRM Provider Version

3.93.0

Affected Resource(s)/Data Source(s)

azurerm_app_configuration_key

Terraform Configuration Files

terraform {
  required_providers {
    azurerm = {
      source  = "hashicorp/azurerm"
      version = "= 3.93.0"
    }
  }
}

provider "azurerm" {
  subscription_id = "xxxx-xxxxx-xxxx"
  tenant_id       = "xxxx-xxxxx-xxxx"
  features {}
}

resource "azurerm_resource_group" "testtf-euw-dev-rg" {
  name      = "testtf-EUW-dev-RG"
  location  = "westeurope"
}

resource "azurerm_app_configuration" "ac" {
  name                = "testtf-dev-appconfig"
  location            = azurerm_resource_group.testtf-euw-dev-rg.location
  resource_group_name = azurerm_resource_group.testtf-euw-dev-rg.name
  sku                 = "standard"
  local_auth_enabled  = true
}


resource "azurerm_app_configuration_key" "main" {
  key                    = "TestKey"
  value                  = "Testvalue"
  label                  = "Default"
  configuration_store_id = azurerm_app_configuration.ac.id
}

Debug Output/Panic Output

azurerm_resource_group.testtf-euw-dev-rg: Refreshing state... [id=/subscriptions/xxxxxxx-xxxx-xxxxxxxx-xxxxxx/resourceGroups/testtf-EUW-dev-RG]
azurerm_app_configuration.ac: Refreshing state... [id=/subscriptions/xxxxxxx-xxxx-xxxxxxxx-xxxxxx/resourceGroups/testtf-EUW-dev-RG/providers/Microsoft.AppConfiguration/configurationStores/testtf-dev-appconfig]
azurerm_app_configuration_key.main: Refreshing state... [id=https://testtf-dev-appconfig.azconfig.io/kv/TestKey?label=Default]

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
-/+ destroy and then create replacement

Terraform will perform the following actions:

  # azurerm_app_configuration_key.main must be replaced
-/+ resource "azurerm_app_configuration_key" "main" {
      ~ configuration_store_id = "/subscriptions/xxxxxxx-xxxx-xxxxxxxx-xxxxxx/resourceGroups/testtf-euw-dev-rg/providers/Microsoft.AppConfiguration/configurationStores/testtf-dev-appconfig" -> "/subscriptions/xxxxxxx-xxxx-xxxxxxxx-xxxxxx/resourceGroups/testtf-EUW-dev-RG/providers/Microsoft.AppConfiguration/configurationStores/testtf-dev-appconfig" # forces replacement
      + content_type           = (known after apply)
      ~ etag                   = "prEPxH2R6Pff3Vt48poAj2uI-ClRnXhyWrOFTdN3tMs" -> (known after apply)
      ~ id                     = "https://testtf-dev-appconfig.azconfig.io/kv/TestKey?label=Default" -> (known after apply)
      - tags                   = {} -> null
        # (5 unchanged attributes hidden)
    }

Plan: 1 to add, 0 to change, 1 to destroy.

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: no

Apply cancelled.

Expected Behaviour

No changes should be detected, the resource should not be destroyed and recreated

Actual Behaviour

Terraform apply wants to recreate the azurerm_app_configuration_key

  # azurerm_app_configuration_key.main must be replaced
-/+ resource "azurerm_app_configuration_key" "main" {
      ~ configuration_store_id = "/subscriptions/9abad234-7001-4b73-a9b5-c7cf5b10d3e8/resourceGroups/testtf-euw-dev-rg/providers/Microsoft.AppConfiguration/configurationStores/testtf-dev-appconfig" -> "/subscriptions/9abad234-7001-4b73-a9b5-c7cf5b10d3e8/resourceGroups/testtf-EUW-dev-RG/providers/Microsoft.AppConfiguration/configurationStores/testtf-dev-appconfig" # forces replacement
      + content_type           = (known after apply)
      ~ etag                   = "prEPxH2R6Pff3Vt48poAj2uI-ClRnXhyWrOFTdN3tMs" -> (known after apply)
      ~ id                     = "https://testtf-dev-appconfig.azconfig.io/kv/TestKey?label=Default" -> (known after apply)
      - tags                   = {} -> null
        # (5 unchanged attributes hidden)
    }

Plan: 1 to add, 0 to change, 1 to destroy.

Steps to Reproduce

terraform init
terraform apply # first apply creates resources successfully
terraform apply # all subsequent apply will try to replace the existing azurerm_app_configuration_key

Important Factoids

N/A

References

No response

[masterphenix]

Hello,
Some more information regarding this issue:

• Using provider azurerm 3.70, the issue is not present
• Using provider azurerm 3.89, the issue is present

So, it appeared somewhere in between these 2 versions.

[wuxu92]

@masterphenix Thanks for filing this issue which seems introduced from #24023, but this is API issue in the cause, see Azure/azure-rest-api-specs#24337. I'm working on fixing this issue, you need to have an ignore_changes block to workaround it for now.

resource "azurerm_app_configuration_key" "main" {
  key                    = "TestKey"
  value                  = "Testvalue"
  label                  = "Default"
  configuration_store_id = azurerm_app_configuration.ac.id
  lifecycle {
	ignore_changes = [  configuration_store_id]
  }
}

[masterphenix]

Hello @wuxu92 , thank you for taking the time to look into this, and for the provided workaround

[cptully]

I think this this issue is wider than just resourcegroup vs resourceGroup. I have also seen it with dnszone vs dnsZone. Fundamentally, if I copy an ID string from portal.azure.com and use it in my terraform, it should work AS COPIED. I should not have to edit any part of it to comply with azurerm's weird case sensitivity rules. If the problem is that portal.azure.com give different values in different places, then azurerm should not be using case sensitive matching!

This issue has ranged from just irritating (with dnszone vs dnsZone, terraform at least gave me an intelligible error telling exactly what the problem was) to blocking when the error message told me that I did not have the rights necessary to destroy and recreate a security rule. It turned out that the problem was not my permissions, but azurerm seeing resourcegroup as different from resourceGroup and so deciding that it needed to destroy and recreate, when in reality the change I was trying to apply should have been a no-op!

Either turn off case sensitivity or validate ALL case sensitivity rues against portal.azure.com!

[cptully]

Also, I have to disagree with the tag of upstream/microsoft/blocking-api-issue Unless you can show the Microsoft is rejecting strings that were copied out of the portal website, then I believe that it falls on azurerm to fix the issue. If you have proof that it is a microsoft issue, please post a link to the issue you have opened with them so that we can all pile on and push for a fix!

[tombuildsstuff]

@cptully

Whilst typically we try and align the two - Terraform Resources aren't guaranteed to use the same Resource ID as an Azure Resource ID - the canonical source for the Resource ID for a Terraform Resource is in the import section of the documentation, not the Azure Portal.

Unfortunately the Azure Portal changes casing all the time - and despite Microsoft's messaging to the contrary, there's plenty of Azure APIs which are case sensitive too - which means making things case-insensitive would actually make the problem worse in some cases (and would make those issues extremely hard to diagnose for users).

The guidance from the ARM Team around casing has changed multiple times over the years, and as such has been implemented inconsistently across different endpoints across different API versions of different Services - as such unfortunately whilst the Azure APIs are supposed to be case-insensitive during Requests and case-sensitive in Responses (e.g. submit dnszones and return dnsZones) - there's plenty of cases where that doesn't happen. In addition there's plenty of APIs which require the casing to be specified in a particular casing (namely camelCase) - and as such whilst Resources may provision, they're ultimately not usable unless that casing is provided (e.g. you can provision the cluster but can't login to it).

In terms of an upstream issue to track that issue within the Azure API, it's linked a few comments up.

---

In order to workaround this, we now define a Type for each Resource ID within hashicorp/go-azure-sdk - using camelCase for the Key components as per the ARM Team's recommendation - and recasing these values as required coming back from the Azure API.

Whilst we have more plans around casing, unfortunately until the entire Provider is using hashicorp/go-azure-sdk we're in a bit of a holding pattern - but since we're validating user input during terraform import and validating the values for fields, we're ensuring that the data being provided matches what we're expecting going forwards. This does mean that users will need to ensure the correct casing is specified as per the import section of the documentation - but unfortunately (as mentioned above) the Resource ID for a Terraform Resource isn't necessarily the one used from the Azure Portal, and shouldn't be assumed to be as such, and therefore you'd need to check the import section of the documentation to find the Resource ID format we're expecting there.

As such whilst we've got more plans around casing once the Provider is fully onto hashicorp/go-azure-sdk - at this point in time the label applied to this issue (upstream/microsoft/blocking-api-issue) is correct, since the Azure API should be returning the correct casing here.

[cptully]

@tombuildsstuff Thank you for the detailed response! That is helpful. I have added my $0.02 to the issue linked above as well.

After reading through your discussion above, I now agree that the issue is upstream/microsoft/blocking-api-issue

And I am sure it is just as frustrating for you as it is for me!