Still cannot use aws_glue_security_configuration with SSE-S3 encryption because of empty KmsKeyArn

[mckacz]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform Version

Terraform v0.12.28
AWS provider v2.69.0

Affected Resource(s)

• aws_glue_security_configuration

Terraform Configuration Files

resource "aws_glue_security_configuration" "xyz" {
  name = "xyz"

  encryption_configuration {
    s3_encryption {
      s3_encryption_mode = "SSE-S3"
    }

    cloudwatch_encryption {
      cloudwatch_encryption_mode = "DISABLED"
    }

    job_bookmarks_encryption {
      job_bookmarks_encryption_mode = "DISABLED"
    }
  }
}

Expected Behavior

1. AWS Glue job that uses this configuration ends without an error.
2. Get security configuration output does not contain empty KmsKeyArn keys

$ aws glue get-security-configuration --name xyz                                                                                                                                                                                         
{
    "SecurityConfiguration": {
        "Name": "xyz",
        "CreatedTimeStamp": "2020-07-08T12:53:22.525000+02:00",
        "EncryptionConfiguration": {
            "S3Encryption": [
                {
                    "S3EncryptionMode": "SSE-S3"
                }
            ],
            "CloudWatchEncryption": {
                "CloudWatchEncryptionMode": "DISABLED"
            },
            "JobBookmarksEncryption": {
                "JobBookmarksEncryptionMode": "DISABLED"
            }
        }
    }
}

Actual Behavior

1. AWS Glue job that uses this configuration ends with an error:

py4j.protocol.Py4JJavaError: An error occurred while calling z:com.amazonaws.services.glue.util.Job.commit.
: com.amazonaws.services.kms.model.AWSKMSException: 1 validation error detected: Value '' at 'keyId' failed to satisfy constraint: Member must have length greater than or equal to 1 (Service: AWSKMS; Status Code: 400; Error Code: ValidationException; Request ID: 9726d049-05ef-4d0c-9440-897c56b851b6)

2. Get security configuration output contains empty KmsKeyArn keys

$ aws glue get-security-configuration --name xyz
{
    "SecurityConfiguration": {
        "Name": "xyz",
        "CreatedTimeStamp": "2020-07-08T12:53:22.525000+02:00",
        "EncryptionConfiguration": {
            "S3Encryption": [
                {
                    "S3EncryptionMode": "SSE-S3"
                }
            ],
            "CloudWatchEncryption": {
                "CloudWatchEncryptionMode": "DISABLED",
                "KmsKeyArn": ""
            },
            "JobBookmarksEncryption": {
                "JobBookmarksEncryptionMode": "DISABLED",
                "KmsKeyArn": ""
            }
        }
    }
}

Steps to Reproduce

1. terraform apply
2. aws glue get-security-configuration --name xyz

References

Related issue: #12879

The resolution of the above issue removed empty KmsKeyArn only from S3Encryption list elements, but this key (if empty) should be removed also from other sections.

Workaround

1. Create security configuration via AWS CLI:
   aws glue create-security-configuration --name xyz --encryption-configuration '{"S3Encryption": [{"S3EncryptionMode": "SSE-S3"}], "CloudWatchEncryption": {"CloudWatchEncryptionMode": "DISABLED"}, "JobBookmarksEncryption": {"JobBookmarksEncryptionMode": "DISABLED"}}'
2. Import the resource to Terraform state

[DrFaust92]

duplicate of #13620

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thanks!