Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

aws_iam _role upgrade to framework and inline policy change #35634

Open
wants to merge 128 commits into
base: main
Choose a base branch
from

Conversation

teddylear
Copy link
Contributor

@teddylear teddylear commented Feb 5, 2024

Description

Upgrades aws_iam_role to terraform-plugin-framework and resolve issue with inline_policy being a set in sdkv2 to a map in the terraform-plugin-framework to resolve linked issue below with inline policy plans and fully deleting then recrating policies on changes instead of just updating in place and then removing policies that are no longer present. Using terraform from that issue with base terraform update here in new version of provider:

resource "aws_iam_role" "main" {
  assume_role_policy = data.aws_iam_policy_document.trust.json
  name               = "terraform-issue-reproduction"

  inline_policies = {
    "InlinePolicy" = data.aws_iam_policy_document.inline.json
  }
}

# Inline policy document that we'll be changing
data "aws_iam_policy_document" "inline" {
  statement {
    sid     = "S3"
    actions = ["s3:ListBucket"]
    resources = [
      "arn:aws:s3:::some-bucket-a",
      # After the first apply, uncomment this next line
      # "arn:aws:s3:::some-bucket-b",
    ]
  }
}

# Minimal, arbitrary trust statement to make the role valid
data "aws_iam_policy_document" "trust" {
  statement {
    actions = ["sts:AssumeRole"]
    principals {
      type        = "Service"
      identifiers = ["apigateway.amazonaws.com"]
    }
  }
}

And the plan to add second bucket in inline policy produces this plan which is close to original one in ticket:

  # aws_iam_role.main will be updated in-place
  ~ resource "aws_iam_role" "main" {
        id                    = "terraform-issue-reproduction"
      ~ inline_policies       = {
          ~ "InlinePolicy" = jsonencode(
              ~ {
                  ~ Statement = [
                      ~ {
                          ~ Resource = "arn:aws:s3:::some-bucket-a" -> [
                              + "arn:aws:s3:::some-bucket-b",
                              + "arn:aws:s3:::some-bucket-a",
                            ]
                            # (3 unchanged attributes hidden)
                        },
                    ]
                    # (1 unchanged attribute hidden)
                }
            )
        }
        name                  = "terraform-issue-reproduction"
        # (8 unchanged attributes hidden)
    }

Plan: 0 to add, 1 to change, 0 to destroy.

Adding a third bucket produces this plan which is even more user friendly:

  # aws_iam_role.main will be updated in-place
  ~ resource "aws_iam_role" "main" {
        id                    = "terraform-issue-reproduction"
      ~ inline_policies       = {
          ~ "InlinePolicy" = jsonencode(
              ~ {
                  ~ Statement = [
                      ~ {
                          ~ Resource = [
                              + "arn:aws:s3:::some-bucket-c",
                                "arn:aws:s3:::some-bucket-b",
                                # (1 unchanged element hidden)
                            ]
                            # (3 unchanged attributes hidden)
                        },
                    ]
                    # (1 unchanged attribute hidden)
                }
            )
        }
        name                  = "terraform-issue-reproduction"
        # (8 unchanged attributes hidden)
    }

Plan: 0 to add, 1 to change, 0 to destroy.

Please let me know if there's any other test or anything else I should be adding here. I tried to document as many behavior changes as possible in the existing tests. There are some import changes which are best noted in the tests and code itself. Also please let me know if I should squash commits.

Relations

Closes #22336

Output from Acceptance Testing

Added some new tests to confirm some validation functionality as well as upgrade from previous version of prodiver. Some existing tests had to be changed as result of migration to terraform-plugin-framework. I tried to document them as best as possible in role_test.go file.

 > make testacc TESTS=TestAccIAMRole_ PKG=iam

==> Checking that code complies with gofmt requirements...
TF_ACC=1 go test ./internal/service/iam/... -v -count 1 -parallel 20 -run='TestAccIAMRole_'  -timeout 360m
=== RUN   TestAccIAMRole_basic
=== PAUSE TestAccIAMRole_basic
=== RUN   TestAccIAMRole_MigrateFromPluginSDK_basic
=== PAUSE TestAccIAMRole_MigrateFromPluginSDK_basic
=== RUN   TestAccIAMRole_description
=== PAUSE TestAccIAMRole_description
=== RUN   TestAccIAMRole_nameGenerated
=== PAUSE TestAccIAMRole_nameGenerated
=== RUN   TestAccIAMRole_namePrefix
=== PAUSE TestAccIAMRole_namePrefix
=== RUN   TestAccIAMRole_testNameChange
=== PAUSE TestAccIAMRole_testNameChange
=== RUN   TestAccIAMRole_diffsNoCondition
=== PAUSE TestAccIAMRole_diffsNoCondition
=== RUN   TestAccIAMRole_diffsCondition
=== PAUSE TestAccIAMRole_diffsCondition
=== RUN   TestAccIAMRole_badJSON
=== PAUSE TestAccIAMRole_badJSON
=== RUN   TestAccIAMRole_disappears
=== PAUSE TestAccIAMRole_disappears
=== RUN   TestAccIAMRole_policiesForceDetach
=== PAUSE TestAccIAMRole_policiesForceDetach
=== RUN   TestAccIAMRole_maxSessionDuration
=== PAUSE TestAccIAMRole_maxSessionDuration
=== RUN   TestAccIAMRole_permissionsBoundary
=== PAUSE TestAccIAMRole_permissionsBoundary
=== RUN   TestAccIAMRole_tags
=== PAUSE TestAccIAMRole_tags
=== RUN   TestAccIAMRole_InlinePolicy_basic
=== PAUSE TestAccIAMRole_InlinePolicy_basic
=== RUN   TestAccIAMRole_MigrateFromPluginSDK_InlinePolicy
=== PAUSE TestAccIAMRole_MigrateFromPluginSDK_InlinePolicy
=== RUN   TestAccIAMRole_InlinePolicy_badJSON
=== PAUSE TestAccIAMRole_InlinePolicy_badJSON
=== RUN   TestAccIAMRole_InlinePolicy_ignoreOrder
=== PAUSE TestAccIAMRole_InlinePolicy_ignoreOrder
=== RUN   TestAccIAMRole_InlinePolicy_empty
=== PAUSE TestAccIAMRole_InlinePolicy_empty
=== RUN   TestAccIAMRole_ManagedPolicy_basic
=== PAUSE TestAccIAMRole_ManagedPolicy_basic
=== RUN   TestAccIAMRole_ManagedPolicy_badARN
=== PAUSE TestAccIAMRole_ManagedPolicy_badARN
=== RUN   TestAccIAMRole_ManagedPolicy_outOfBandRemovalAddedBack
=== PAUSE TestAccIAMRole_ManagedPolicy_outOfBandRemovalAddedBack
=== RUN   TestAccIAMRole_InlinePolicy_outOfBandRemovalAddedBack
=== PAUSE TestAccIAMRole_InlinePolicy_outOfBandRemovalAddedBack
=== RUN   TestAccIAMRole_ManagedPolicy_outOfBandAdditionRemoved
=== PAUSE TestAccIAMRole_ManagedPolicy_outOfBandAdditionRemoved
=== RUN   TestAccIAMRole_InlinePolicy_outOfBandAdditionRemoved
=== PAUSE TestAccIAMRole_InlinePolicy_outOfBandAdditionRemoved
=== RUN   TestAccIAMRole_InlinePolicy_outOfBandAdditionIgnored
=== PAUSE TestAccIAMRole_InlinePolicy_outOfBandAdditionIgnored
=== RUN   TestAccIAMRole_ManagedPolicy_outOfBandAdditionIgnored
=== PAUSE TestAccIAMRole_ManagedPolicy_outOfBandAdditionIgnored
=== RUN   TestAccIAMRole_InlinePolicy_outOfBandAdditionRemovedEmpty
=== PAUSE TestAccIAMRole_InlinePolicy_outOfBandAdditionRemovedEmpty
=== RUN   TestAccIAMRole_ManagedPolicy_outOfBandAdditionRemovedEmpty
=== PAUSE TestAccIAMRole_ManagedPolicy_outOfBandAdditionRemovedEmpty
=== CONT  TestAccIAMRole_basic
=== CONT  TestAccIAMRole_MigrateFromPluginSDK_InlinePolicy
=== CONT  TestAccIAMRole_ManagedPolicy_outOfBandAdditionRemovedEmpty
=== CONT  TestAccIAMRole_InlinePolicy_outOfBandAdditionIgnored
=== CONT  TestAccIAMRole_InlinePolicy_outOfBandAdditionRemoved
=== CONT  TestAccIAMRole_badJSON
=== CONT  TestAccIAMRole_InlinePolicy_outOfBandAdditionRemovedEmpty
=== CONT  TestAccIAMRole_InlinePolicy_outOfBandRemovalAddedBack
=== CONT  TestAccIAMRole_ManagedPolicy_outOfBandRemovalAddedBack
=== CONT  TestAccIAMRole_ManagedPolicy_badARN
=== CONT  TestAccIAMRole_ManagedPolicy_basic
=== CONT  TestAccIAMRole_InlinePolicy_empty
=== CONT  TestAccIAMRole_InlinePolicy_ignoreOrder
=== CONT  TestAccIAMRole_InlinePolicy_basic
=== CONT  TestAccIAMRole_tags
=== CONT  TestAccIAMRole_permissionsBoundary
=== CONT  TestAccIAMRole_maxSessionDuration
=== CONT  TestAccIAMRole_policiesForceDetach
=== CONT  TestAccIAMRole_InlinePolicy_badJSON
=== CONT  TestAccIAMRole_ManagedPolicy_outOfBandAdditionRemoved
--- PASS: TestAccIAMRole_InlinePolicy_badJSON (63.55s)
=== CONT  TestAccIAMRole_disappears
--- PASS: TestAccIAMRole_ManagedPolicy_badARN (64.46s)
=== CONT  TestAccIAMRole_ManagedPolicy_outOfBandAdditionIgnored
--- PASS: TestAccIAMRole_badJSON (82.63s)
=== CONT  TestAccIAMRole_namePrefix
--- PASS: TestAccIAMRole_InlinePolicy_empty (375.18s)
=== CONT  TestAccIAMRole_diffsCondition
--- PASS: TestAccIAMRole_policiesForceDetach (451.16s)
=== CONT  TestAccIAMRole_diffsNoCondition
--- PASS: TestAccIAMRole_basic (451.64s)
=== CONT  TestAccIAMRole_testNameChange
--- PASS: TestAccIAMRole_namePrefix (419.93s)
=== CONT  TestAccIAMRole_description
--- PASS: TestAccIAMRole_disappears (491.02s)
=== CONT  TestAccIAMRole_nameGenerated
--- PASS: TestAccIAMRole_ManagedPolicy_outOfBandAdditionIgnored (555.98s)
=== CONT  TestAccIAMRole_MigrateFromPluginSDK_basic
--- PASS: TestAccIAMRole_InlinePolicy_outOfBandAdditionRemovedEmpty (620.75s)
--- PASS: TestAccIAMRole_InlinePolicy_outOfBandRemovalAddedBack (620.81s)
--- PASS: TestAccIAMRole_ManagedPolicy_outOfBandAdditionRemovedEmpty (620.87s)
--- PASS: TestAccIAMRole_InlinePolicy_outOfBandAdditionRemoved (621.05s)
--- PASS: TestAccIAMRole_ManagedPolicy_outOfBandAdditionRemoved (621.08s)
--- PASS: TestAccIAMRole_ManagedPolicy_outOfBandRemovalAddedBack (621.09s)
--- PASS: TestAccIAMRole_tags (673.45s)
--- PASS: TestAccIAMRole_MigrateFromPluginSDK_InlinePolicy (687.23s)
--- PASS: TestAccIAMRole_InlinePolicy_ignoreOrder (721.36s)
--- PASS: TestAccIAMRole_InlinePolicy_outOfBandAdditionIgnored (727.06s)
--- PASS: TestAccIAMRole_maxSessionDuration (747.98s)
--- PASS: TestAccIAMRole_ManagedPolicy_basic (794.56s)
--- PASS: TestAccIAMRole_InlinePolicy_basic (799.97s)
--- PASS: TestAccIAMRole_nameGenerated (248.10s)
--- PASS: TestAccIAMRole_testNameChange (380.26s)
--- PASS: TestAccIAMRole_description (405.59s)
--- PASS: TestAccIAMRole_permissionsBoundary (914.94s)
--- PASS: TestAccIAMRole_MigrateFromPluginSDK_basic (305.59s)
--- PASS: TestAccIAMRole_diffsCondition (552.75s)
--- PASS: TestAccIAMRole_diffsNoCondition (767.89s)
PASS
ok      github.com/hashicorp/terraform-provider-aws/internal/service/iam        1223.809s

@terraform-aws-provider terraform-aws-provider bot added the needs-triage Waiting for first response or review from a maintainer. label Feb 5, 2024
@teddylear teddylear changed the title [WIP] aws_iam _ole upgrade to framework and inline policy change [WIP] aws_iam _role upgrade to framework and inline policy change Feb 5, 2024
@teddylear teddylear marked this pull request as ready for review February 5, 2024 22:54
@github-actions github-actions bot added the documentation Introduces or discusses updates to documentation. label Feb 5, 2024
@teddylear teddylear changed the title [WIP] aws_iam _role upgrade to framework and inline policy change aws_iam _role upgrade to framework and inline policy change Feb 5, 2024
@justinretzolk justinretzolk added bug Addresses a defect in current functionality. and removed needs-triage Waiting for first response or review from a maintainer. labels Feb 12, 2024
@justinretzolk
Copy link
Member

Hey @teddylear 👋 Thanks for taking the time to raise this, and apologies for missing your pings on the original issue. I brought this up internally as well, to try to get a bit more traction 🙂

@teddylear
Copy link
Contributor Author

@justinretzolk Hey! No worries and thanks for the reply. If there's anything I need to add to change please let me know or if one of the maintainers needs to edit the branch directly that works as well. This is my first time doing a migration from sdkv2 to the plugin-framework so I may have missed something 😅. Also happy to help with more of these too as this was a good learning experience and could do for the aws_iam_role data source as well along with doing for other terraform provider resources.

Copy link
Contributor

@gdavison gdavison left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for submitting this, @teddylear. I have a number of comments on the PR. The biggest is that the change from inline_policy to inline_policies should be in a separate PR, and also still needs to support inline_policy because of backwards compatibility guarantees.

Comment on lines +165 to +172
"arn": schema.StringAttribute{
CustomType: fwtypes.ARNType,
Computed: true,
Optional: true,
PlanModifiers: []planmodifier.String{
stringplanmodifier.UseStateForUnknown(),
},
},
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The arn parameter should be Computed-only, without Optional. This can also be achieved using framework.ARNAttributeComputedOnly

Suggested change
"arn": schema.StringAttribute{
CustomType: fwtypes.ARNType,
Computed: true,
Optional: true,
PlanModifiers: []planmodifier.String{
stringplanmodifier.UseStateForUnknown(),
},
},
names.AttrARN: framework.ARNAttributeComputedOnly(),

TagsAll types.Map `tfsdk:"tags_all"`
}

func oldSDKRoleSchema() schema.Schema {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should use the schema version instead of "old", e.g.

Suggested change
func oldSDKRoleSchema() schema.Schema {
func roleSchemaV0() schema.Schema {

},
"inline_policy": {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because of the backwards compatibility requirements for the provider, the inline_policy parameter cannot be removed until the next major version. The map structure that you've used for inline_policies is more user-friendly, I think, so it's definitely worth adding, and deprecating inline_policy. However, it would be better to make a separate PR for that change.

Comment on lines -211 to -218
// Some partitions (e.g. ISO) may not support tag-on-create.
partition := meta.(*conns.AWSClient).Partition
if input.Tags != nil && errs.IsUnsupportedOperationInPartitionError(partition, err) {
input.Tags = nil

output, err = retryCreateRole(ctx, conn, input)
}

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should still be supported. Not all AWS partitions support tag-on-create

Comment on lines -246 to +584
if v, ok := d.GetOk(names.AttrTags); (!ok || len(v.(map[string]interface{})) == 0) && errs.IsUnsupportedOperationInPartitionError(partition, err) {
return append(diags, resourceRoleRead(ctx, d, meta)...)
}
// if v, ok := d.GetOk(names.AttrTags); (!ok || len(v.(map[string]interface{})) == 0) && errs.IsUnsupportedOperationInPartitionError(conn.PartitionID, err) {
// return append(diags, resourceRoleRead(ctx, d, meta)...)
// }

if err != nil {
return sdkdiag.AppendErrorf(diags, "setting IAM Role (%s) tags: %s", d.Id(), err)
resp.Diagnostics.AddError(
create.ProblemStandardMessage(names.IAM, create.ErrActionCreating, fmt.Sprintf("%s tags", ResNameRole), name, nil),
err.Error(),
)
return
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You'll need to invert the logic here, so that if len(plan.Tags.Elements()) == 0 && errs.IsUnsupportedOperationInPartitionError(conn.PartitionID, err) is true, execution falls through to the next part of the function where plan is populated.

}

func deleteRole(ctx context.Context, conn *iam.Client, roleName string, forceDetach, hasInline, hasManaged bool) error {
func DeleteRole(ctx context.Context, conn *iam.Client, roleName string, forceDetach, hasInline, hasManaged bool) error {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This doesn't seem to be exported

Suggested change
func DeleteRole(ctx context.Context, conn *iam.Client, roleName string, forceDetach, hasInline, hasManaged bool) error {
func deleteRole(ctx context.Context, conn *iam.Client, roleName string, forceDetach, hasInline, hasManaged bool) error {

@@ -42,6 +43,8 @@ func TestAccIAMRole_basic(t *testing.T) {
testAccCheckRoleExists(ctx, resourceName, &conf),
resource.TestCheckResourceAttr(resourceName, "path", "/"),
resource.TestCheckResourceAttrSet(resourceName, "create_date"),
resource.TestCheckResourceAttrSet(resourceName, "unique_id"),
resource.TestCheckResourceAttrSet(resourceName, "arn"),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like that you're adding a test for arn. However, you can check the specific value, e.g.

Suggested change
resource.TestCheckResourceAttrSet(resourceName, "arn"),
acctest.CheckResourceAttrGlobalARN(resourceName, "arn", "iam", fmt.Sprintf("role/%s", rName)),
,

Comment on lines +92 to +99
// NOTE: Have to update docs to reflect this because giant issue
// Can't not use PlanOnly anymore with terraform-plugin-testing
// https://github.com/hashicorp/terraform-plugin-testing/issues/256
ConfigPlanChecks: resource.ConfigPlanChecks{
PreApply: []plancheck.PlanCheck{
plancheck.ExpectEmptyPlan(),
},
},
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What was the problem you encountered with PlanOnly? We haven't seen a problem using it

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The Linked issue above describes what is happening, it was a Failed to marshal state to json: error when upgrading no matter what I did. The upstream terraform-plugin-testing plugin has updated documentation to use this instead.

Comment on lines +241 to +243
// // https://github.com/hashicorp/terraform-provider-aws/issues/23288
// // https://github.com/hashicorp/terraform-provider-aws/issues/28833
func TestAccIAMRole_diffsNoCondition(t *testing.T) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
// // https://github.com/hashicorp/terraform-provider-aws/issues/23288
// // https://github.com/hashicorp/terraform-provider-aws/issues/28833
func TestAccIAMRole_diffsNoCondition(t *testing.T) {
// https://github.com/hashicorp/terraform-provider-aws/issues/23288
// https://github.com/hashicorp/terraform-provider-aws/issues/28833
func TestAccIAMRole_diffsNoCondition(t *testing.T) {

Comment on lines +595 to +599
// Test empty value
{
Config: testAccRoleConfig_permissionsBoundary(rName, ""),
ExpectError: regexache.MustCompile(`Value "" cannot be parsed as an ARN.`),
},
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can't change the acceptable values here until the next major version. permissions_boundary will have to accept an empty string and treat it as no value. Validation should only check for a valid ARN if it's not an empty string

@ewbankkit ewbankkit added the breaking-change Introduces a breaking change in current functionality; usually deferred to the next major release. label Jul 24, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
breaking-change Introduces a breaking change in current functionality; usually deferred to the next major release. bug Addresses a defect in current functionality. documentation Introduces or discusses updates to documentation. generators Relates to code generators. service/iam Issues and PRs that pertain to the iam service. service/rds Issues and PRs that pertain to the rds service. size/XL Managed by automation to categorize the size of a PR. tests PRs: expanded test coverage. Issues: expanded coverage, enhancements to test infrastructure.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Changes to inline_policy blocks on aws_iam_role always recreates policy
5 participants