feat: Creating SNS/SQS policies should be optional

[bmurphey]

Description

Make the creation of SNS/SQS policies conditional based on two new variables, create_sns_policy and create_sqs_policy.

Motivation and Context

We have a use case (as I'm sure others do) where we need to send S3 bucket notifications to an external account that we don't control, and so we can't create the related SNS/SQS policies in that account (and they aren't necessary). The rest of the notification setup is valid when using the external resource ARNs, however.

Breaking Changes

None; the variables default to true so behavior will be the same.

How Has This Been Tested?

$ terraform -v
Terraform v0.12.29
+ provider.aws v3.10.0
+ provider.null v2.1.2
+ provider.random v2.3.0

Using the module as published failed to apply the notification due to the inability create the policy, so I added it manually and imported it. This snippet...:

module "redacted-bucket" {
  source  = "terraform-aws-modules/s3-bucket/aws"
  version = "v1.15.0"

  bucket        = "redacted-bucket"
  acl           = "private"
  force_destroy = false

  tags = {
    creator     = "terraform"
  }

  versioning = {
    enabled = false
  }

  # S3 bucket-level Public Access Block configuration
  block_public_acls       = true
  block_public_policy     = true
  ignore_public_acls      = true
  restrict_public_buckets = true
}

module "redacted-bucket-notifications" {
  source  = "terraform-aws-modules/s3-bucket/aws//modules/notification"

  bucket            = module.redacted-bucket.this_s3_bucket_id

  # Common error - Error putting S3 notification configuration: InvalidArgument: Configuration is ambiguously defined. Cannot have overlapping suffixes in two rules if the prefixes are overlapping for the same event type.

  sqs_notifications = {
    sqs1 = {
      queue_arn     = "redacted-queue-arn"
      events        = ["s3:ObjectCreated:Put"]
      filter_suffix = ".gz"
    }
  }
}

... results in this plan:

Terraform will perform the following actions:

  # module.redacted-bucket-notifications.aws_sqs_queue_policy.allow["sqs1"] will be created
  + resource "aws_sqs_queue_policy" "allow" {
      + id        = (known after apply)
      + policy    = jsonencode(
            {
              + Statement = [
                  + {
                      + Action    = "sqs:SendMessage"
                      + Condition = {
                          + ArnEquals = {
                              + aws:SourceArn = "arn:aws:s3:::redacted-bucket"
                            }
                        }
                      + Effect    = "Allow"
                      + Principal = {
                          + Service = "s3.amazonaws.com"
                        }
                      + Resource  = "redacted-queue-arn"
                      + Sid       = "AllowSQSS3BucketNotification"
                    },
                ]
              + Version   = "2012-10-17"
            }
        )
      + queue_url = "redacted-queue-url"
    }

Plan: 1 to add, 0 to change, 0 to destroy.

Changing the module to use this branch like so...:

module "redacted-bucket-notifications" {
  source  = "github.com/bmurphey/terraform-aws-s3-bucket//modules/notification?ref=optional-create-policy"

  bucket                   = module.redacted-bucket.this_s3_bucket_id
  create_sqs_policy = false

  # Common error - Error putting S3 notification configuration: InvalidArgument: Configuration is ambiguously defined. Cannot have overlapping suffixes in two rules if the prefixes are overlapping for the same event type.

  sqs_notifications = {
    sqs1 = {
      queue_arn     = "redacted-queue-arn"
      events        = ["s3:ObjectCreated:Put"]
      filter_suffix = ".gz"
    }
  }
}

... results in No changes. Infrastructure is up-to-date.. It appears to me that the SNS policy is created in the same manner, so I added a variable to make that policy creation conditional as well, but that hasn't been tested.

[antonbabenko]

Thanks for this addition!

v1.16.0 has been just released.

[bmurphey]

No problem, thank you!

[github-actions]

I'm going to lock this pull request because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues. If you have found a problem that seems related to this change, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.