IAM Role ARN not being parsed correctly when passed to aws_dms_endpoint

[conleyst]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform Version

0.12.21
aws-2.49

Affected Resource(s)

aws_dms_endpoint

Terraform Configuration Files

data "aws_iam_policy_document" "dms_assume_role_policy_document" {
  statement {
    actions = ["sts:AssumeRole"]

    principals {
      identifiers = ["dms.amazonaws.com"]
      type        = "Service"
    }
  }
}

resource "aws_iam_role" "dms_role" {
  assume_role_policy = data.aws_iam_policy_document.dms_assume_role_policy_document.json
  name = "dms_role"
}

data "aws_iam_policy_document" "dms_s3_access_document" {
  statement {
    actions = [
      "s3:PutObject",
      "s3:DeleteObject",
      "s3:PutObjectTagging"
    ]

    resources = [
      "arn:aws:s3:::my-dms-target/*",
    ]
  }

  statement {
    actions = [
      "s3:ListBucket"
    ]

    resources = [
      "arn:aws:s3:::my-dms-target"
    ]
  }
}

resource "aws_iam_policy" "dms_s3_access" {
  name = "dms_s3_access"
  policy = data.aws_iam_policy_document.dms_s3_access_document.json
}

resource "aws_iam_role_policy_attachment" "dms_s3_access_attachment" {
  role = aws_iam_role.dms_role.id
  policy_arn = aws_iam_policy.dms_s3_access.arn
}

resource "aws_dms_endpoint" "my_dms_endpoint" {
  endpoint_id                 = "dms-s3-target"
  endpoint_type               = "target"
  engine_name                 = "s3"
  extra_connection_attributes = "dataFormat=parquet;bucketName=${aws_s3_bucket.dms_target.bucket};bucketFolder=my_data"
  service_access_role = aws_iam_role.dms_role.arn
}

Output

The arn that is being specified resolves to arn:aws:iam::012345678910:role/dms_role in the planning step.

Error: InvalidParameterValueException: Invalid role arn,  contains ARN without the required six components
	status code: 400, request id: <...>

dms.tf line 53, in resource "aws_dms_endpoint" "my_dms_endpoint":
  53: resource "aws_dms_endpoint" "my_dms_endpoint"

Expected Behavior

DMS endpoint should be created using the specified role and bucket.

Actual Behavior

Endpoint is not created with the above exception being thrown.

Steps to Reproduce

1. terraform apply

Important Factoids

Running this using terraform cloud.

I would otherwise specify the role in s3_settings, but due to the issue referenced below, I'm unable to specify the file output format in extra_connection_attributes and specify the role in s3_settings.

References

• S3 settings on aws_dms_endpoint conflict with "extra_connection_attributes" #8009
• DMS endpoint s3 target to support extra_connection_attributes #8000

[c0nfleis]

May be a dumb question, but just for testing's sake, have you tried passing in the hardcoded arn to see if it still behaves the same way?

[conleyst]

That still doesn't seem to work, unfortunately. I get the same error.

[drexler]

@conleyst taking a look at the documentation and the underlying code, there are two properties at play: service_access_role is used when the engine is dynamodb; for S3, the property is: service_access_role_arn. Let me know if this works for you:

resource "aws_dms_endpoint" "my_dms_endpoint" {
  endpoint_id                 = "dms-s3-target"
  endpoint_type               = "target"
  engine_name                 = "s3"
  extra_connection_attributes = "dataFormat=parquet;bucketName=${aws_s3_bucket.dms_target.bucket};bucketFolder=my_data"
  service_access_role_arn = aws_iam_role.dms_role.arn
}

[awahnteh]

When using service_access_role_arn, I get the error: An argument named "service_access_role_arn" is not expected here.

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thanks!