Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

azurerm_virtual_network see constant drift if address_space array order is updated #23328

Closed
1 task done
industrialzombie opened this issue Sep 19, 2023 · 4 comments · Fixed by #23793 or #26149
Closed
1 task done
Labels
service/network upstream/microsoft/blocking-api-issue This label is applicable when there is a bug on the Azure API that is blocking. v/3.x
Milestone

Comments

@industrialzombie
Copy link

Is there an existing issue for this?

  • I have searched the existing issues

Community Note

  • Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
  • Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
  • If you are interested in working on this issue or have submitted a pull request, please leave a comment and review the contribution guide to help.

Terraform Version

0.12.31

AzureRM Provider Version

3.71.0

Affected Resource(s)/Data Source(s)

azurerm_virtual_network

Terraform Configuration Files

provider "azurerm" {
  version = "3.71.0"

  tenant_id       = "[redacted]"
  subscription_id = "[redacted]"
  client_id       = "[redacted]"
  client_secret   = "[redacted]"

  features {
  }
}

resource "azurerm_resource_group" "this" {
  name     = "test-resource-group"
  location = "centralus"
}

resource "azurerm_virtual_network" "this" {
  name                = "test-virtual-network"
  resource_group_name = azurerm_resource_group.this.name
  location            = "centralus"
  address_space       = ["10.0.1.0/24", "10.0.2.0/24", "10.0.3.0/24"]
  //address_space = ["10.0.3.0/24", "10.0.1.0/24", "10.0.2.0/24"]
}

Debug Output/Panic Output

The first call to terraform apply creates the network.
Subsequent calls correctly identify no changes.

Then, once the CIDRs in the address space are re-ordered, the plan correctly identifies the need to update the resource.
However, after the apply succeeds, the resource is not updated and subsequent plans will forever show that an update is needed.

terraform apply
azurerm_resource_group.this: Refreshing state... [id=/subscriptions/[redacted]/resourceGroups/test-resource-group]
azurerm_virtual_network.this: Refreshing state... [id=/subscriptions/[redacted]/resourceGroups/test-resource-group/providers/Microsoft.Network/virtualNetworks/test-virtual-network]

An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
  ~ update in-place

Terraform will perform the following actions:

  # azurerm_virtual_network.this will be updated in-place
  ~ resource "azurerm_virtual_network" "this" {
      ~ address_space           = [
          + "10.0.3.0/24",
            "10.0.1.0/24",
            "10.0.2.0/24",
          - "10.0.3.0/24",
        ]
        dns_servers             = []
        flow_timeout_in_minutes = 0
        guid                    = "[redacted]"
        id                      = "/subscriptions/[redacted]/resourceGroups/test-resource-group/providers/Microsoft.Network/virtualNetworks/test-virtual-network"
        location                = "centralus"
        name                    = "test-virtual-network"
        resource_group_name     = "test-resource-group"
        subnet                  = []
        tags                    = {}
    }

Plan: 0 to add, 1 to change, 0 to destroy.

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

azurerm_virtual_network.this: Modifying... [id=/subscriptions/[redacted]/resourceGroups/test-resource-group/providers/Microsoft.Network/virtualNetworks/test-virtual-network]
azurerm_virtual_network.this: Modifications complete after 7s [id=/subscriptions/[redacted]/resourceGroups/test-resource-group/providers/Microsoft.Network/virtualNetworks/test-virtual-network]

Apply complete! Resources: 0 added, 1 changed, 0 destroyed.

Expected Behaviour

When the elements of the address_space property are re-ordered, the virtual network should be updated to reflect this.
Subsequent plans should then show no changes.

Actual Behaviour

The actual behavior is that the virtual network is not being updated.

I can see from using a MITM proxy that the PUT request is correctly includes the re-ordered CIDRs. However, the response contains the original ordering.

REQUEST

{
    "location": "centralus",
    "properties": {
        "addressSpace": {
            "addressPrefixes": [
                "10.0.3.0/24",
                "10.0.1.0/24",
                "10.0.2.0/24"
            ]
        },
        "dhcpOptions": {
            "dnsServers": []
        },
        "subnets": []
    },
    "tags": {}
}

RESPONSE

{
    "etag": "W/\"[redacted]\"",
    "id": "/subscriptions/[redacted]/resourceGroups/test-resource-group/providers/Microsoft.Network/virtualNetworks/test-virtual-network",
    "location": "centralus",
    "name": "test-virtual-network",
    "properties": {
        "addressSpace": {
            "addressPrefixes": [
                "10.0.1.0/24",
                "10.0.2.0/24",
                "10.0.3.0/24"
            ]
        },
        "dhcpOptions": {
            "dnsServers": []
        },
        "enableDdosProtection": false,
        "provisioningState": "Succeeded",
        "resourceGuid": "[redacted]",
        "subnets": [],
        "virtualNetworkPeerings": []
    },
    "tags": {},
    "type": "Microsoft.Network/virtualNetworks"
}

Steps to Reproduce

  1. terraform apply
  2. update the configuration by swapping out the commented line for the address_space property
  3. terraform apply
  4. terraform apply <-- see that the plan indicates an update is required

Important Factoids

The order is important because it's coming from a map object, so the order is determined by the lexicographical order names of entries in the map which is out of my control.

References

No response

@wuxu92
Copy link
Contributor

wuxu92 commented Sep 21, 2023

hi @industrialzombie thanks for reporting this issue. I have reproduced this issue and thanks a lot for your investigation too. I believe this is a Azure API upstream issue, and I have created an internal ticket to the service team about this issue. I'll append to this thread once get response from the team.

@rcskosir rcskosir added the upstream/microsoft Indicates that there's an upstream issue blocking this issue/PR label Oct 27, 2023
@industrialzombie
Copy link
Author

Hi @wuxu92,

Please can you add the ticket number here?

I agree that this is mostly likely an upstream issue so I'd be happy to close this out if I can get get a reference to another issue on the Azure API that I can track instead.

Thanks,
Graham

@tiwood
Copy link
Contributor

tiwood commented Nov 6, 2023

@rcskosir, I'm wondering if this is really a upstream issue. The Azure API ignores ordering inside arrays in many different places (if the ordering is actually having no impact like in this case).

Shouldn't we suppress the diff on provider side?

What is the best way of doing this, making it a TypeSet or suppressing the diff?

@rcskosir rcskosir added upstream/microsoft/blocking-api-issue This label is applicable when there is a bug on the Azure API that is blocking. and removed upstream/microsoft Indicates that there's an upstream issue blocking this issue/PR labels Jan 30, 2024
@github-actions github-actions bot added this to the v3.101.0 milestone Apr 23, 2024
Copy link

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators May 24, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
service/network upstream/microsoft/blocking-api-issue This label is applicable when there is a bug on the Azure API that is blocking. v/3.x
Projects
None yet
4 participants