azuread: GroupsClient.BaseClient.Post(): unexpected status 400 with OData error: Request_BadRequest: Request contains a property with duplicate values.

[kyrios]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritise this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritise the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform (and AzureAD Provider) Version

Terraform 1.0.2
Azuread 2.6.0

Affected Resource(s)

• azuread_group

Terraform Configuration Files

terraform{
  required_providers {
    azurerm = {
      source  = "hashicorp/azurerm"
      version = "~>2.80.0"
    }
    azuread = {
      source = "hashicorp/azuread"
      version = "~> 2.6.0"
    }
  }
}

data "azuread_client_config" "current" {}

resource "azuread_group" "example" {
  display_name     = "example"
  owners           = [data.azuread_client_config.current.object_id]
  security_enabled = true
}

Debug Output

https://gist.github.com/kyrios/142a0ba5350047f3ec9611ea8fe2a05d

Expected Behavior

No error

Actual Behavior

GroupsClient.BaseClient.Post(): unexpected status 400 with OData error: Request_BadRequest: Request contains a property with duplicate │ values.

Steps to Reproduce

1. terraform init
2. terraform apply

Important Factoids

We have been upgrading to 2.6.0 because of #588

References

#588

• #0000

[kyrios]

I updated the example code (most basic variant) and attached a gist.

[manicminer]

Hi @kyrios, thanks for raising this. We have seen this behavior intermittently before (see #478, #567) but we are still tracking this and raising with the service team as a breaking API change. It's possible this might be related to #588, but this error is not caused by it. At this time there's no viable workaround for us, as Terraform is, on balance, doing the right thing here. The API behavior seems to differ between some tenants and changing our payload for group creation to work around this would break more users in the process.

Sorry that we can't resolve this right now, but as mentioned we are having conversations about this and we'll keep this issue updated as we learn more. Thanks!

[kyrios]

Hi @manicminer, let me verify that I understand you correctly: There is no workaroung using terraform (the Azure API) and no bugfix planned since the bug isn't understood yet by Microsoft. It is related to our tennant and there is nothing we can do about it. That indeed is very bad news.

[manicminer]

Unfortunately that is correct. We're hoping to get some traction on this very soon but right now Terraform cannot resolve this error. I'm afraid since this doesn't affect any of our testing tenants that our ability to experiment with this is limited, however you might try adding additional arbitrary owners to the group to see if this helps bypass this error.

The error appears to stem from specifying the calling principal as the sole owner of the group. In many cases the API assigns that same principal as the default owner in the case that none are specified, but this doesn't appear to be universal, and the 'duplicate' error only seems to occur for a subset of users.

[kyrios]

Maybe as some additional information: It worked a week ago with version 2.4.0

[manicminer]

As much as this may be related to the recent API changes, despite the coincidental timing this isn't a result of any change to Terraform :(

[kyrios]

I understand. Just wanted to add every last bit of information I have.

[mlcooper]

I hit this with #567. My workaround is to assign someone other than myself as an owner of the Azure AD group. Terraform then successfully creates the AAD group and automatically makes my ID an owner as well as the other individual I specified. It got me past this "duplicate values" error.

[mlcooper]

@manicminer I actually just find a problem with my workaround. The next time I ran a terraform apply, it removed my ID as an owner of the group and left the other individual I specified as an owner. So it almost sounds like I would need to have a lifecycle {} block with this azuread_group resource so that my ID isn't removed on subsequent Terraform runs. Clearly we seem to mostly be at the mercy of this Azure API bug at the moment.

[kyrios]

That workaround seems to be working. Thanks a lot for sharing it @mlcooper!

Here is the code we are using:

owners = ["foo-bar-foo-1"]
  lifecycle {
    ignore_changes = [
      # Ignore changes to tags, e.g. because a management agent
      # updates these based on some ruleset managed elsewhere.
      owners,
    ]
  }

[gscher]

Is there any update on this issue?

[manicminer]

This issue is due to, and blocked by, an API bug which is reported at microsoftgraph/msgraph-metadata#92

[rufer7]

I faced the same error and was able to "fix" it by assigning the authenticated user principal to the directory role Groups Administrator as stated in the documentation

[ju-la-berger]

In the enterprise I am working at, it is impossible for ordinary mortals to acquire such elevated rights.

My workaround for daily use in Terraform is to create the group via Azure CLI and import it into the state for further management via Terraform, e.g. like this:

AD_APP_DISPLAY_NAME='foo.bar'
AD_APP_TF_STATE_ADDR='module.foo.azuread_group.bar'

az ad group create --display-name "${AD_APP_DISPLAY_NAME}" --mail-nickname "$(cat /proc/sys/kernel/random/uuid)"

terraform import "${AD_APP_TF_STATE_ADDR}" "$(az ad group show --group "${AD_APP_DISPLAY_NAME}" --query 'objectId' --output 'tsv')"

(It is only the creation of the group that fails. Further modifications are working.)

[robertbrandso]

I experienced this issue when I was authenticating using a service principal.

After I gave the Group.ReadWrite.All API permission to the SPN it worked.

This is also stated in the documentation:

When authenticated with a service principal, this resource requires one of the following application roles: Group.ReadWrite.All or Directory.ReadWrite.All

If using the assignable_to_role property, this resource additionally requires one of the following application roles: RoleManagement.ReadWrite.Directory or Directory.ReadWrite.All

When authenticated with a user principal, this resource requires one of the following directory roles: Groups Administrator, User Administrator or Global Administrator

I must admit it isn't the ideal solution to give such broad permissions.

[carsv]

Verified Authz issue in my organization.
"unexpected status 400" is misleading -> should be 401 Unauthorized.

[64J0]

I'm having a similar problem, but with the newer version mentioned in this issue: #789.

GroupsClient.BaseClient.Get(): unexpected status 403 with OData error: Authorization_RequestDenied: Insufficient privileges to complete the operation.

---

My problem was solved after refreshing my terminal session with new Azure login.

[TeamDman]

Activating my Azure PIM roles fixed this for me.

[SebSa]

#624 (comment)

This fixed it for me, thank you.

[jvelasquez]

If anyone comes across this issue, this did it for me (little dirty but can live with that code until this is fixed) -

locals {
  azuread_admin_group_mail_nickname = uuidv5("dns", "ca.dev.xyz")
  azuread_admin_group_owners        = concat([data.azuread_service_principal.terraform_client.id], data.azuread_user.admin_user.*.id)
  azuread_admin_group_id           = shell_script.azuread_admin_group.output.id
}

provider "shell" {}

resource "shell_script" "azuread_admin_group" {
  lifecycle_commands {
    create = "az ad group create --display-name ${var.azuread_admin_group_name} --mail-nickname ${local.azuread_admin_group_mail_nickname}"
    read   = "az ad group show --group ${var.azuread_admin_group_name}"
    update = "az ad group delete --group ${var.azuread_admin_group_name} && az ad group create --display-name ${var.azuread_admin_group_name} --mail-nickname ${local.azuread_admin_group_mail_nickname}"
    delete = "az ad group delete --group ${var.azuread_admin_group_name}"
  }
}

[SebSa]

Any news on this?

[ju-la-berger]

Not really. To my understanding, the root cause microsoftgraph/msgraph-metadata#92 is still open.

However, a colleague of mine and me did some more testing and we found out that you can actually create an AD group of type "Unified" (i.e. an O365 group).

If you we use a service principal to authenticate (our terraform apply execution), we can create groups. We will not get the Group.ReadWrite.All or Directory.ReadWrite.All in our enterprise IT environment but we found that the combination of Directory.Read.All, Group.Create, Group.Read.All, and User.Read.All (meaning all those four permissions) are sufficient.

The following code snippet illustrates our finding.

data "azuread_client_config" "current" {}

# fails when applied as user principal: "GroupsClient.BaseClient.Post(): unexpected status 400 with OData error: Request_BadRequest: Request contains a property with duplicate values."
# succeeds when applied as service principal with API permissions "Directory.Read.All", "Group.Create", "Group.Read.All", "User.Read.All"
resource "azuread_group" "test_foo" {
  display_name     = "test-foo"
  security_enabled = true
}

# succeeds when applied as user principal
# succeeds when applied as service principal with API permissions "Directory.Read.All", "Group.Create", "Group.Read.All", "User.Read.All"
resource "azuread_group" "test_bar" {
  display_name     = "test-bar"
  security_enabled = true

  mail_enabled  = true
  mail_nickname = "test-bar"
  types         = ["Unified"]
  visibility    = "Private"
}

# succeeds when applied as service principal with API permissions "Directory.Read.All", "Group.Create", "Group.Read.All", "User.Read.All"
resource "azuread_group" "test_foo_with_owners_and_members" {
  display_name     = "test-foo-with-owners-and-members"
  security_enabled = true

  members = local.owner_object_ids
  owners  = concat([data.azuread_client_config.current.object_id], local.owner_object_ids) # Do NOT forget to add the service principal's object ID here!
}

[manicminer]

Hi all, since it seems there is no traction as yet with fixing this in the API, I am proposing a potential fix for this error by retrying the group creation when this error is detected and the calling principal has been specified as an owner. The workaround relies on the other side of this API bug, that the calling principal is auto-added behind the scenes and the group is subsequently returned having the calling principal as an auto-appended owner.

If anyone is able to test out by compiling the provider locally from the bugfix/group-400-owner branch (see #1037), any such feedback will be greatly appreciated as it's not feasible to field-test this workaround in all circumstances. Thanks!

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.