an issue with azurerm_cosmosdb_mongo_collection indexes detection/handling

[hokugawa]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform (and AzureRM Provider) Version

Terraform v0.12.29

azurerm 2.33.0

Affected Resource(s)

• azurerm_cosmosdb_mongo_collection

Terraform Configuration Files

resource "azurerm_cosmosdb_mongo_collection" "plugins_col" {
  name                = "plugins"
  resource_group_name = azurerm_cosmosdb_account.db_account.resource_group_name
  account_name        = azurerm_cosmosdb_account.db_account.name
  database_name       = azurerm_cosmosdb_mongo_database.mongodb.name

  default_ttl_seconds = "-1" # no expiration
  shard_key           = "namespace"
  throughput          = var.mongo_collection_plugins_throughput

//  index {
//    keys = ["_id"]
//    unique = true
//  }
  index {
    keys = ["namespace"]
    unique = true
  }
  index {
    keys = ["namespace", "uuid"]
    unique = true
  }

  timeouts {}
}

Debug Output

Panic Output

Expected/Actual Behavior

With azurerm v2.33.0, if azurerm_cosmosdb_mongodb_collection index block doesn't contain index setting for _id, terraform apply complains the following an error (terraform plan passed without an error).

Error: Error waiting on create/update future for Cosmos Mongo Collection "plugins" (Account: "...", Database: "mongodb-ho"): Code="BadRequest" Message="Index definition does not contains '_id' index specification.\r\nActivityId: ..., Microsoft.Azure.Documents.Common/2.11.0, Microsoft.Azure.Documents.Common/2.11.0, Microsoft.Azure.Documents.Common/2.11.0, Microsoft.Azure.Documents.Common/2.11.0"

If I added the index block for the _id like:

  index {
    keys = ["_id"]
    unique = true
  }

The error would be gone.
However, after that, every time I run the terraform plan, it tries to add(change) index for _id:

# module.base.azurerm_cosmosdb_mongo_collection.plugins_col will be updated in-place
  ~ resource "azurerm_cosmosdb_mongo_collection" "plugins_col" {
        account_name        = "..."
        database_name       = "mongodb-ho"
      ~ default_ttl_seconds = 0 -> -1
        id                  = "..."
        name                = "plugins"
        resource_group_name = "..."
        shard_key           = "namespace"
        system_indexes      = [
            {
                keys   = [
                    "_id",
                ]
                unique = true
            },
        ]
        throughput          = 400

      + index {
          + keys   = [
              + "_id",
            ]
          + unique = true
        }
        index {
            keys   = [
                "namespace",
                "uuid",
            ]
            unique = true
        }
        index {
            keys   = [
                "namespace",
            ]
            unique = true
        }

        timeouts {}
    }

I'm not sure it will actually trigger re-indexing. If it does, it can be a big issue, because re-indexing can cause a database query unavailable for a while.

for reference. This is output from getIndexes() command for the collection.

> db.plugins.getIndexes()
[
	{
		"v" : 1,
		"key" : {
			"_id" : 1
		},
		"name" : "_id_",
		"ns" : "mongodb-ho.plugins"
	},
	{
		"v" : 1,
		"unique" : true,
		"key" : {
			"namespace" : 1
		},
		"name" : "namespace_1",
		"ns" : "mongodb-ho.plugins"
	},
	{
		"v" : 1,
		"unique" : true,
		"key" : {
			"namespace" : 1,
			"uuid" : 1
		},
		"name" : "namespace_1_uuid_1",
		"ns" : "mongodb-ho.plugins"
	}
]

Two requests:

• If the Index block always required for ["_id"], please mention it in the document.
• Make existing indexes detection accurately

Steps to Reproduce

1. terraform apply

Important Factoids

References

• #0000

[tombuildsstuff]

hi @hokugawa

Thanks for opening this issue :)

Taking a look through this appears to be a duplicate of #8144 - rather than having multiple issues open tracking the same thing I'm going to close this issue in favour of that one; would you mind subscribing to #8144 for updates?

Thanks!

[hokugawa]

Thanks @tombuildsstuff I am subscribing the #8144. I think that detecting an index for _id correctly part is missing in the #8144.
Anyway, indexing is a kind of sensitive operation in MongoDB, hope you guys can solve the problem soon.
Thanks!

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. If you feel I made an error 🤖 🙉 , please reach out to my human friends 👉 [email protected]. Thanks!