Cannot use aws_glue_security_configuration with SSE-S3 encryption because of spurious empty KmsKeyArn

[maparent]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform Version

Terraform v0.12.24
AWS provider v2.57

Affected Resource(s)

• aws_glue_security_configuration

Terraform Configuration Files

resource "aws_glue_security_configuration" "sec_conf" {
  name = "sec_conf"

  encryption_configuration {
    cloudwatch_encryption {
      cloudwatch_encryption_mode = "DISABLED"
    }

    job_bookmarks_encryption {
      job_bookmarks_encryption_mode = "DISABLED"
    }

    s3_encryption {
      s3_encryption_mode = "SSE-S3"
    }
  }
}

Expected Behavior

When I read the resource back with boto3 client, I expect to see the following:

import boto3
client = boto3.client("glue")
client.get_security_configuration(Name="sec_conf")

{
  "Name": "sec_conf",
  "EncryptionConfiguration": {
    "S3Encryption": [
      {
        "S3EncryptionMode": "SSE-S3"
      }
    ],
    "CloudWatchEncryption": {
      "CloudWatchEncryptionMode": "DISABLED"
    },
    "JobBookmarksEncryption": {
      "JobBookmarksEncryptionMode": "DISABLED"
    }
  }
}

and to be able to use the resource.

Actual Behavior

It is created with empty strings for "KmsKeyArn"

{
  "Name": "sec_conf",
  "EncryptionConfiguration": {
    "S3Encryption": [
      {
        "S3EncryptionMode": "SSE-S3",
        "KmsKeyArn": ""
      }
    ],
    "CloudWatchEncryption": {
      "CloudWatchEncryptionMode": "DISABLED",
      "KmsKeyArn": ""
    },
    "JobBookmarksEncryption": {
      "JobBookmarksEncryptionMode": "DISABLED",
      "KmsKeyArn": ""
    }
  }
}

Trying to use the configuration in a glue job, I get the following error message:

An error occurred while calling z:com.amazonaws.services.glue.util.Job.commit. 1 validation error detected: Value '' at 'keyId' failed to satisfy constraint: Member must have length greater than or equal to 1 (Service: AWSKMS; Status Code: 400; Error Code: ValidationException; Request ID: ...)

Workaround attempts

1. I tried setting the kms_key_arn = null in the terraform, but the empty string is still there.

2. I also tried to specify the key arn, as follows:

data "aws_kms_alias" "s3" {
  name = "alias/aws/s3"
}

resource "aws_glue_security_configuration" "sec_conf" {
  name = "sec_conf"

  encryption_configuration {
    cloudwatch_encryption {
      cloudwatch_encryption_mode = "DISABLED"
    }

    job_bookmarks_encryption {
      job_bookmarks_encryption_mode = "DISABLED"
    }

    s3_encryption {
      s3_encryption_mode = "SSE-S3"
      kms_key_arn = data.aws_kms_alias.s3.target_key_arn
    }
  }
}

In that case, terraform apply gives me the following error message:

Error: error creating Glue Security Configuration (glue_security): InvalidInputException: kmsKeyArn can only be empty for s3EncryptionMode that is not SSE_KMS

  on main.tf line 6, in resource "aws_glue_security_configuration" "sec_conf":
   6: resource "aws_glue_security_configuration" "sec_conf" {

Steps to Reproduce

1. terraform apply
2. Use the configuration in a job

References

[maparent]

Should be of interest to @bflad

[bflad]

The fix for this has been merged and will release with version 2.59.0 of the Terraform AWS Provider, later this week. Thanks to @maparent for the implementation. 👍

[maparent]

And thank you for the quick review!

[ghost]

This has been released in version 2.59.0 of the Terraform AWS provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template for triage. Thanks!

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thanks!