Enhancement/cloudwatch event target adding validation #15669

philnichol · 2020-10-15T20:03:13Z

Community Note

Please vote on this pull request by adding a 👍 reaction to the original pull request comment to help the community and maintainers prioritize this request
Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for pull request followers and do not help prioritize the request

Release note for CHANGELOG:

NONE

Output from acceptance testing:

$  TF_ACC=1 go test ./aws -v -count 1 -parallel 4 -run=TestAccAWSCloudWatchEventTarget_input_transformer -timeout 60m
=== RUN   TestAccAWSCloudWatchEventTarget_input_transformer
=== PAUSE TestAccAWSCloudWatchEventTarget_input_transformer
=== CONT  TestAccAWSCloudWatchEventTarget_input_transformer
--- PASS: TestAccAWSCloudWatchEventTarget_input_transformer (39.24s)
PASS
ok  	github.com/terraform-providers/terraform-provider-aws/aws	41.304s

@gdavison hope you don't mind I went ahead and submitted this!

Added validation for max number of input_paths (10) and validation that they don't start with "AWS".
Updated Acc test to include 10 input_paths.

gdavison

Thanks for the PR, @philnichol, it's looking great! I've got one additional test I'd like to see and a couple nitpicks.
The new validators would be great to also propose for the Terraform Plugin SDK.

aws/resource_aws_cloudwatch_event_target.go

gdavison · 2020-10-15T22:19:22Z

aws/validators.go

+func MapLenBetween(min, max int) schema.SchemaValidateFunc {
+	return func(i interface{}, k string) (warnings []string, errors []error) {
+		m, ok := i.(map[string]interface{})
+		if !ok {
+			errors = append(errors, fmt.Errorf("expected type of %s to be map", k))
+			return warnings, errors
+		}
+
+		if len(m) < min || len(m) > max {
+			errors = append(errors, fmt.Errorf("expected length of %s to be in the range (%d - %d), got %d", k, min, max, len(m)))
+		}
+
+		return warnings, errors
+	}
+}


In this case, I would just check the maximum size, since there isn't really a lower limit. You could call it MapMaxLen() or match it with the MaxItems field for TypeList and TypeSet and call it MapMaxItems().

This would be a great addition to the Terraform Plugin SDK

gdavison · 2020-10-15T22:24:37Z

aws/validators.go

+func MapKeyNotMatch(r *regexp.Regexp, message string) schema.SchemaValidateFunc {
+	return func(i interface{}, k string) (warnings []string, errors []error) {
+		m, ok := i.(map[string]interface{})
+		if !ok {
+			errors = append(errors, fmt.Errorf("expected type of %s to be map", k))
+			return warnings, errors
+		}
+
+		for key := range m {
+			if ok := r.MatchString(key); ok {
+				errors = append(errors, fmt.Errorf("%s: %s: %s", k, message, key))
+			}
+		}
+
+		return warnings, errors
+	}
+}


A picky naming thing: many of the validation functions read a bit more smoothly, like StringDoesNotMatch(), StringDoesNotContainAny(), or StringIsNotWhiteSpace(). Maybe a name like MapKeyDoesNotMatch().

This would also be a great addition to the Terraform Plugin SDK

makes sense, have renamed to MapKeysDoNotMatch.
Will look into raising a proposal in the SDK plugin

aws/resource_aws_cloudwatch_event_target_test.go

philnichol · 2020-10-17T16:17:24Z

reran full ACC tests, the batch step appears to be failing before and after my changes so I assume it's unrelated? Happy to raise an issue if this is news.

$ TF_ACC=1 go test ./aws -v -count 1 -parallel 4 -run=TestAccAWSCloudWatchEventTarget_ -timeout 60m
=== RUN   TestAccAWSCloudWatchEventTarget_basic
=== PAUSE TestAccAWSCloudWatchEventTarget_basic
=== RUN   TestAccAWSCloudWatchEventTarget_missingTargetId
=== PAUSE TestAccAWSCloudWatchEventTarget_missingTargetId
=== RUN   TestAccAWSCloudWatchEventTarget_full
=== PAUSE TestAccAWSCloudWatchEventTarget_full
=== RUN   TestAccAWSCloudWatchEventTarget_ssmDocument
=== PAUSE TestAccAWSCloudWatchEventTarget_ssmDocument
=== RUN   TestAccAWSCloudWatchEventTarget_ecs
=== PAUSE TestAccAWSCloudWatchEventTarget_ecs
=== RUN   TestAccAWSCloudWatchEventTarget_ecsWithBlankTaskCount
=== PAUSE TestAccAWSCloudWatchEventTarget_ecsWithBlankTaskCount
=== RUN   TestAccAWSCloudWatchEventTarget_batch
=== PAUSE TestAccAWSCloudWatchEventTarget_batch
=== RUN   TestAccAWSCloudWatchEventTarget_kinesis
=== PAUSE TestAccAWSCloudWatchEventTarget_kinesis
=== RUN   TestAccAWSCloudWatchEventTarget_sqs
=== PAUSE TestAccAWSCloudWatchEventTarget_sqs
=== RUN   TestAccAWSCloudWatchEventTarget_input_transformer
=== PAUSE TestAccAWSCloudWatchEventTarget_input_transformer
=== CONT  TestAccAWSCloudWatchEventTarget_basic
=== CONT  TestAccAWSCloudWatchEventTarget_batch
=== CONT  TestAccAWSCloudWatchEventTarget_input_transformer
=== CONT  TestAccAWSCloudWatchEventTarget_sqs
--- PASS: TestAccAWSCloudWatchEventTarget_sqs (43.91s)
=== CONT  TestAccAWSCloudWatchEventTarget_kinesis
--- PASS: TestAccAWSCloudWatchEventTarget_basic (57.80s)
=== CONT  TestAccAWSCloudWatchEventTarget_ssmDocument
=== CONT  TestAccAWSCloudWatchEventTarget_batch
    resource_aws_cloudwatch_event_target_test.go:283: After applying this test step, the plan was not empty.
        stdout:


        An execution plan has been generated and is shown below.
        Resource actions are indicated with the following symbols:
          ~ update in-place
        -/+ destroy and then create replacement

        Terraform will perform the following actions:

          # aws_batch_job_definition.batch_job_definition must be replaced
        -/+ resource "aws_batch_job_definition" "batch_job_definition" {
              ~ arn                  = "arn:aws:batch:eu-west-1:887702914428:job-definition/tf_batch_target-5829695721356721392:1" -> (known after apply)
              ~ container_properties = jsonencode(
                  ~ {
                        command              = [
                            "ls",
                            "-la",
                        ]
                        environment          = []
                        image                = "busybox"
                        memory               = 512
                        mountPoints          = []
                      - resourceRequirements = [] -> null
                      - secrets              = [] -> null
                        ulimits              = []
                        vcpus                = 1
                        volumes              = []
                    }
                )
              ~ id                   = "arn:aws:batch:eu-west-1:887702914428:job-definition/tf_batch_target-5829695721356721392:1" -> (known after apply)
                name                 = "tf_batch_target-5829695721356721392"
              ~ revision             = 1 -> (known after apply)
                type                 = "container"
            }

          # aws_cloudwatch_event_target.test will be updated in-place
          ~ resource "aws_cloudwatch_event_target" "test" {
                arn       = "arn:aws:batch:eu-west-1:887702914428:job-queue/tf_batch_target-5829695721356721392"
                id        = "tf_batch_target-5829695721356721392-terraform-20201017154859657400000005"
                role_arn  = "arn:aws:iam::887702914428:role/event_tf_batch_target-5829695721356721392"
                rule      = "tf_batch_target-5829695721356721392"
                target_id = "terraform-20201017154859657400000005"

              ~ batch_target {
                    array_size     = 0
                    job_attempts   = 0
                  ~ job_definition = "arn:aws:batch:eu-west-1:887702914428:job-definition/tf_batch_target-5829695721356721392:1" -> (known after apply)
                    job_name       = "tf_batch_target-5829695721356721392"
                }
            }

        Plan: 1 to add, 1 to change, 1 to destroy.
--- PASS: TestAccAWSCloudWatchEventTarget_input_transformer (67.75s)
=== CONT  TestAccAWSCloudWatchEventTarget_ecsWithBlankTaskCount
--- PASS: TestAccAWSCloudWatchEventTarget_ssmDocument (23.40s)
=== CONT  TestAccAWSCloudWatchEventTarget_ecs
--- PASS: TestAccAWSCloudWatchEventTarget_ecsWithBlankTaskCount (36.55s)
=== CONT  TestAccAWSCloudWatchEventTarget_full
--- PASS: TestAccAWSCloudWatchEventTarget_kinesis (64.48s)
=== CONT  TestAccAWSCloudWatchEventTarget_missingTargetId
--- PASS: TestAccAWSCloudWatchEventTarget_ecs (32.22s)
--- PASS: TestAccAWSCloudWatchEventTarget_missingTargetId (19.91s)
--- PASS: TestAccAWSCloudWatchEventTarget_full (57.76s)
--- FAIL: TestAccAWSCloudWatchEventTarget_batch (167.33s)
FAIL
FAIL	github.com/terraform-providers/terraform-provider-aws/aws	170.132s
FAIL

gdavison · 2020-10-19T19:33:45Z

I'm not seeing the error in TestAccAWSCloudWatchEventTarget_batch on my end.

gdavison

LGTM 🚀

--- PASS: TestAccAWSCloudWatchEventTarget_sqs (26.59s)
--- PASS: TestAccAWSCloudWatchEventTarget_missingTargetId (26.69s)
--- PASS: TestAccAWSCloudWatchEventTarget_ssmDocument (29.44s)
--- PASS: TestAccAWSCloudWatchEventTarget_ecs (36.17s)
--- PASS: TestAccAWSCloudWatchEventTarget_ecsWithBlankTaskCount (37.09s)
--- PASS: TestAccAWSCloudWatchEventTarget_basic (38.35s)
--- PASS: TestAccAWSCloudWatchEventTarget_input_transformer (47.98s)
--- PASS: TestAccAWSCloudWatchEventTarget_kinesis (64.50s)
--- PASS: TestAccAWSCloudWatchEventTarget_full (65.33s)
--- PASS: TestAccAWSCloudWatchEventTarget_batch (94.51s)

ghost · 2020-10-22T21:35:18Z

This has been released in version 3.12.0 of the Terraform AWS provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template for triage. Thanks!

ghost · 2020-11-20T17:10:38Z

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thanks!

added validation to input_paths

8678951

philnichol requested a review from a team October 15, 2020 20:03

ghost added size/M Managed by automation to categorize the size of a PR. documentation Introduces or discusses updates to documentation. service/cloudwatchevents tests PRs: expanded test coverage. Issues: expanded coverage, enhancements to test infrastructure. labels Oct 15, 2020

github-actions bot added the needs-triage Waiting for first response or review from a maintainer. label Oct 15, 2020

gdavison removed the needs-triage Waiting for first response or review from a maintainer. label Oct 15, 2020

gdavison self-assigned this Oct 15, 2020

gdavison reviewed Oct 15, 2020

View reviewed changes

philnichol added 3 commits October 17, 2020 13:34

renamed validator functions

b2b78dc

added expecterror to validation test

31c1197

fixed formatting

21fa7c2

philnichol requested a review from gdavison October 17, 2020 16:17

gdavison approved these changes Oct 19, 2020

View reviewed changes

gdavison added this to the v3.12.0 milestone Oct 19, 2020

gdavison merged commit 6a66a25 into hashicorp:master Oct 20, 2020

philnichol deleted the e-adding-validation-cw-event-target branch October 20, 2020 21:08

gdavison added a commit that referenced this pull request Oct 20, 2020

Update CHANGELOG for #15669

37b8e24

ghost locked as resolved and limited conversation to collaborators Nov 20, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Enhancement/cloudwatch event target adding validation #15669

Enhancement/cloudwatch event target adding validation #15669

philnichol commented Oct 15, 2020

gdavison left a comment

gdavison Oct 15, 2020

gdavison Oct 15, 2020

philnichol Oct 17, 2020

philnichol commented Oct 17, 2020

gdavison commented Oct 19, 2020

gdavison left a comment

ghost commented Oct 22, 2020

ghost commented Nov 20, 2020

Enhancement/cloudwatch event target adding validation #15669

Enhancement/cloudwatch event target adding validation #15669

Conversation

philnichol commented Oct 15, 2020

Community Note

gdavison left a comment

Choose a reason for hiding this comment

gdavison Oct 15, 2020

Choose a reason for hiding this comment

gdavison Oct 15, 2020

Choose a reason for hiding this comment

philnichol Oct 17, 2020

Choose a reason for hiding this comment

philnichol commented Oct 17, 2020

gdavison commented Oct 19, 2020

gdavison left a comment

Choose a reason for hiding this comment

ghost commented Oct 22, 2020

ghost commented Nov 20, 2020