Policy documents and other lists causing drift

[lijok]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform CLI and Terraform AWS Provider Version

Affected Resource(s)

• aws_ecr_repository_policy
• aws_iam_policy
• aws_iam_role
• aws_secretsmanager_secret_policy
  etc

Terraform Configuration Files

Please include all Terraform configurations required to reproduce the bug. Bug reports without a functional reproduction may be closed without investigation.

# Copy-paste your Terraform configurations here - for large Terraform configs,
# please use a service like Dropbox and share a link to the ZIP file. For
# security, you can also encrypt the files using our GPG public key: https://keybase.io/hashicorp

module "this" {
  source = "../../../submodules/ecr_repository_private"

  name        = var.name
  description = local.description
  policy      = data.aws_iam_policy_document.this
  tags        = local.tags
}

data "aws_iam_policy_document" "this" {
  version = "2012-10-17"

  statement {
    effect = "Allow"
    actions = [
      "ecr:Get*",
      "ecr:List*",
      "ecr:Describe*",
      "ecr:BatchGetImage",
      "ecr:BatchCheckLayerAvailability"
    ]
    principals {
      type = "AWS"
      identifiers = [
        for account in data.aws_organizations_organization.this.non_master_accounts :
        "arn:aws:iam::${account.id}:root"
      ]
    }
  }
}

Every 2 hours or so, our ci system runs terraform plan, and if it outputs any planned changes, marks that as drift.
Every time we get random false-positive drift like the below, where the only thing being changed is the list of AWS principals being re-arranged

# module.ecr["axo412"].module.this.aws_ecr_repository_policy.this["this"] has been changed
  ~ resource "aws_ecr_repository_policy" "this" {
        id          = "axo412"
      ~ policy      = jsonencode(
          ~ {
              ~ Statement = [
                  ~ {
                      ~ Principal = {
                          ~ AWS = [
                              - "arn:aws:iam::098...:root",
                              - "arn:aws:iam::452...:root",
                              - "arn:aws:iam::311...:root",
                              - "arn:aws:iam::859...:root",
                              - "arn:aws:iam::531...:root",
                                "arn:aws:iam::065...:root",
                              + "arn:aws:iam::509...:root",
                              + "arn:aws:iam::887...:root",
                                "arn:aws:iam::382...:root",
                              + "arn:aws:iam::311...:root",
                                "arn:aws:iam::127...:root",
                              - "arn:aws:iam::729...:root",
                              + "arn:aws:iam::859...:root",
                              + "arn:aws:iam::531...:root",
                                "arn:aws:iam::272...:root",
                              - "arn:aws:iam::403...:root",
                              - "arn:aws:iam::251...:root",
                              - "arn:aws:iam::509...:root",
                              - "arn:aws:iam::887...:root",
                                "arn:aws:iam::285...:root",
                              + "arn:aws:iam::098...:root",
                              + "arn:aws:iam::251r...:oot",
                              + "arn:aws:iam::452...:root",
                              + "arn:aws:iam::403...:root",
                              + "arn:aws:iam::729...:root",
                            ]
                        }
                        # (3 unchanged elements hidden)
                    },
                ]
                # (1 unchanged element hidden)
            }
        )
        # (2 unchanged attributes hidden)
    }

I did some digging, and finds out aws api returns those lists unsorted, so if you do the same api request twice, that list will be in a random order each time;

import json
import boto3

client = boto3.client("ecr", region_name="us-east-1")
response = client.get_repository_policy(repositoryName="axo412")
policy = json.loads(response["policyText"])
policy_principals = policy["Statement"][0]["Principal"]["AWS"]
print(json.dumps(policy_principals, indent=4, default=str))

[ewbankkit]

Potentially related: jen20/awspolicyequivalence#10.

[kylet21]

Also experiencing this issue since upgrading to Terraform v1

[github-actions]

This functionality has been released in v3.70.0 of the Terraform AWS Provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template. Thank you!

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.