Destroying a terraform created role attached to an AWS standard policy, removes all roles attached to that policy

[ghost]

This issue was originally opened by @Richard-Payne as hashicorp/terraform#23046. It was migrated here as a result of the provider split. The original body of the issue is below.

---

Terraform v0.12.8
+ provider.aws v2.31.0
+ provider.null v2.1.2

Debug Output

https://gist.github.com/Richard-Payne/80154244c7abcee7d5cb6e1cbf8a7296

Configuration

provider "aws" {
  region  = "eu-west-2"
  profile = "shared"
}

resource "aws_iam_policy_attachment" "aws_policy_attachments" {
  depends_on = [ aws_iam_role.roles ]
  name       = "tf_test_attachments"
  roles      = [ "tf_test_auto" ]
  policy_arn = "arn:aws:iam::0123456789012:policy/TF_Test"
}

resource "aws_iam_role" "roles" {
  name               = "tf_test_auto"
  assume_role_policy = local.assume_role_policy
  tags = {
    "created_by" = "terraform"
  }
}

locals {
  assume_role_policy = <<EOF
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Action": "sts:AssumeRole",
      "Principal": {
        "Service": "ec2.amazonaws.com"
      },
      "Effect": "Allow",
      "Sid": ""
    }
  ]
}
EOF
}

Expected Behavior

We have a role, created into terraform and use the aws_iam_policy_attachment resource to attach a non-terraformed policy to the role. In our case it is an AWS role, but it applies to any policy. We also have another role, created manually that has that policy attached too.

We expected that when we run terraform destroy that is will detach our terraform created role from the policy and leave the manually created role untouched.

Actual Behavior

terraform destroy unlinks the policy from all roles is attached to, not just the ones under terraform's control.

eg:

Terraform will perform the following actions:

  # aws_iam_policy_attachment.aws_policy_attachments will be destroyed
  - resource "aws_iam_policy_attachment" "aws_policy_attachments" {
      - groups     = [] -> null
      - id         = "tf_test_attachments" -> null
      - name       = "tf_test_attachments" -> null
      - policy_arn = "arn:aws:iam::0123456789012:policy/TF_Test" -> null
      - roles      = [
          - "TF_Test_Manual",
          - "tf_test_auto",
        ] -> null
      - users      = [] -> null
    }

Notice that it is stripping all of attached roles, not just the ones under terraform's control.

Steps to Reproduce

1. Create 'TF_Test' policy (any settings)
2. Create 'tf_test_manual' role
3. Attach 'TF_Test' policy to 'tf_test_manual' role
4. terraform init
5. terraform apply
6. terraform destroy

[Richard-Payne]

I see that there is now a large warning on the aws_iam_policy_attachment resource saying that it takes exclusive control of the policy.

Can I assume that switching to aws_iam_role_policy_attachment will fix my issue?

[GunArm]

I just spent 3 hours tracking down an issue where my lambdas kept losing their AWSLambdaBasicExecutionRole and thus would stop producing logs. Wrote it off as a fluke once and then it happened again. After a ton of reading cloudtrail logs, making guesses, it was NOT easy to track down, I barely managed to track it back to this: Turns out a new stack I made with a new lambda, attached that policy with aws_iam_policy_attachment, and all the other stacks were using aws_iam_role_policy_attachment. I'm not sure if ever understood the difference prior, but the behavioral result was seemingly randomly losing attachments. It is actually as described in that big red warning mentioned on the aws_iam_policy_attachment page. Changing the new stack to aws_iam_role_policy_attachment made everything work sanely again. So that is the fix.

I guess the rule is if ever attaching a policy that is aws inbuilt, and thus is not limited to your stack and might also get attached elsewhere, use aws_iam_role_policy_attachment. Technically, if the policy itself is custom and in the same stack, using aws_iam_policy_attachment should be fine, since nothing else will ever try to attach to that policy.
Really though, I marvel at this trap laid by AWS, and almost wonder if aws_iam_policy_attachment should just be discontinued as a terraform resource so people don't fall into this. I know there are probably tons of modules and code out there that uses it, so maybe making it a legacy feature you have to opt in to would be sane. It's drastic, but I wonder what benefit the existence of aws_iam_policy_attachment provides, which offsets it's potential to easily, silently, create very confusing problems.

[YakDriver]

Although it comes at this from another angle, #5904 may be of some use.

[justinretzolk]

Hey y'all 👋 Thank you for taking the time to file this issue! Given that there's been a number of AWS provider releases since you initially filed it, can you confirm whether you're still experiencing this behavior?

[MushiTheMoshi]

still happening in v0.14.7