Add support for AWS Batch job definition timeout #4386

hectcastro · 2018-04-28T17:53:54Z

Update the existing aws_batch_job_definition resource so that it supports the newly added timeout feature. The timeout configuration is supplied through a timeout block inside aws_batch_job_definition with a single attempt_duration_seconds attribute.

This was tested via make test and by building a terraform-provider-aws binary. The binary was used with an existing Terraform project after timeout {} blocks were added to the existing aws_batch_job_definition resources.

Fixes #4186

See:

Update the existing `aws_batch_job_definition` resource so that it supports the newly added timeout feature. The timeout configuration is supplied through a `timeout` block inside `aws_batch_job_definition` with a single `attempt_duration_seconds` attribute.

bflad

Hey @hectcastro -- this is looking pretty good, can you see the below and let us know if you have any questions?

bflad · 2018-04-30T13:14:25Z

aws/resource_aws_batch_job_definition.go

@@ -123,6 +141,7 @@ func resourceAwsBatchJobDefinitionRead(d *schema.ResourceData, meta interface{})
 	d.Set("container_properties", job.ContainerProperties)
 	d.Set("parameters", aws.StringValueMap(job.Parameters))
 	d.Set("retry_strategy", flattenRetryStrategy(job.RetryStrategy))
+	d.Set("timeout", flattenTimeout(job.Timeout))


When setting non-scalar values into the Terraform state, we should perform error checking:

if err := d.Set("timeout", flattenTimeout(job.Timeout)); err != nil { return fmt.Errorf("error setting timeout: %s", err) }

I'm pretty sure in this case you would have discovered an error about trying to set an *int64 into a TypeInt, which was being silently ignored without the error check.

Got it. retry_strategy is non-scalar too, so I added error checking to that as well.

bflad · 2018-04-30T13:15:33Z

aws/resource_aws_batch_job_definition.go

+	}
+}
+
+func flattenTimeout(item *batch.JobTimeout) []map[string]interface{} {


Minor nitpick: Since we use a very flat Go package structure in this repository currently, I'd recommend renaming this to flattenBatchJobTimeout or similar.

Renamed to flattenBatchJobTimeout. Also, renamed flattenRetryStrategy to flattenBatchRetryStrategy to match.

bflad · 2018-04-30T13:16:50Z

aws/resource_aws_batch_job_definition.go

+	data := []map[string]interface{}{}
+	if item != nil {
+		data = append(data, map[string]interface{}{
+			"attempt_duration_seconds": item.AttemptDurationSeconds,


I believe this needs to be "attempt_duration_seconds": int(aws.Int64Value(item.AttemptDurationSeconds)), -- the d.Set() error checking should confirm this.

Updated attempt_duration_seconds and attempts to match.

bflad · 2018-04-30T13:20:17Z

aws/resource_aws_batch_job_definition.go

+						"attempt_duration_seconds": {
+							Type:     schema.TypeInt,
+							Required: true,
+						},


If the minimum accepted value is 60 we can add plan time validation via: ValidateFunc: validation.IntAtLeast(60), 👍

Nice. Thanks for pointing that out.

gheine · 2018-04-30T14:39:27Z

aws/resource_aws_batch_job_definition.go

+					Schema: map[string]*schema.Schema{
+						"attempt_duration_seconds": {
+							Type:     schema.TypeInt,
+							Required: true,


According to the AWS documentation, this is an optional field, see https://docs.aws.amazon.com/batch/latest/APIReference/API_JobTimeout.html

In this case, attempt_duration_seconds is the lone attribute of the parent timeout block. timeout is optional, but attempt_duration_seconds is not.

Correct, both timeout and attempt_duration_seconds are optional. I guess the idea is that at some point in the future AWS might add more ways to specify a timeout (eg. attempt_duration_minutes), in which case the individual fields can not be required.

hectcastro

Thanks for the review, @bflad. I attempted to address all of your comments.

hectcastro · 2018-04-30T23:42:03Z

aws/resource_aws_batch_job_definition.go

+						"attempt_duration_seconds": {
+							Type:     schema.TypeInt,
+							Required: true,
+						},


Nice. Thanks for pointing that out.

hectcastro · 2018-04-30T23:50:13Z

aws/resource_aws_batch_job_definition.go

+					Schema: map[string]*schema.Schema{
+						"attempt_duration_seconds": {
+							Type:     schema.TypeInt,
+							Required: true,


In this case, attempt_duration_seconds is the lone attribute of the parent timeout block. timeout is optional, but attempt_duration_seconds is not.

hectcastro · 2018-04-30T23:51:18Z

aws/resource_aws_batch_job_definition.go

@@ -123,6 +141,7 @@ func resourceAwsBatchJobDefinitionRead(d *schema.ResourceData, meta interface{})
 	d.Set("container_properties", job.ContainerProperties)
 	d.Set("parameters", aws.StringValueMap(job.Parameters))
 	d.Set("retry_strategy", flattenRetryStrategy(job.RetryStrategy))
+	d.Set("timeout", flattenTimeout(job.Timeout))


Got it. retry_strategy is non-scalar too, so I added error checking to that as well.

hectcastro · 2018-04-30T23:51:52Z

aws/resource_aws_batch_job_definition.go

+	}
+}
+
+func flattenTimeout(item *batch.JobTimeout) []map[string]interface{} {


Renamed to flattenBatchJobTimeout. Also, renamed flattenRetryStrategy to flattenBatchRetryStrategy to match.

hectcastro · 2018-04-30T23:53:11Z

aws/resource_aws_batch_job_definition.go

+	data := []map[string]interface{}{}
+	if item != nil {
+		data = append(data, map[string]interface{}{
+			"attempt_duration_seconds": item.AttemptDurationSeconds,


Updated attempt_duration_seconds and attempts to match.

Also, add validation to `attempts`.

For timeout and retry strategy.

gheine · 2018-05-01T11:37:22Z

website/docs/r/batch_job_definition.html.markdown

+
+`timeout` supports the following:
+
+* `attempt_duration_seconds` - (Required) The time duration in seconds after which AWS Batch terminates your jobs if they have not finished. The minimum value for the timeout is `60` seconds.


This still says Required (and likewise for attempts above). Both are now Optional

gheine

I'll let @bflad approve, but looks great! Thanks!

bflad

This looks great! 🚀

2 tests passed (all tests)
=== RUN   TestAccAWSBatchJobDefinition_basic
--- PASS: TestAccAWSBatchJobDefinition_basic (6.53s)
=== RUN   TestAccAWSBatchJobDefinition_updateForcesNewResource
--- PASS: TestAccAWSBatchJobDefinition_updateForcesNewResource (12.04s)

bflad · 2018-05-02T21:51:05Z

This has been released in version 1.17.0 of the AWS provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

ghost · 2020-04-06T17:25:40Z

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thanks!

ghost added the size/M Managed by automation to categorize the size of a PR. label Apr 28, 2018

bflad reviewed Apr 30, 2018

View reviewed changes

bflad added enhancement Requests to existing resources that expand the functionality or scope. service/batch Issues and PRs that pertain to the batch service. labels Apr 30, 2018

bflad mentioned this pull request Apr 30, 2018

Support timeout in aws_batch_job_definition #4393

Closed

gheine reviewed Apr 30, 2018

View reviewed changes

hectcastro added 3 commits April 30, 2018 20:00

Add validation for minimum attempt_duration_seconds

b7a064e

Add error checking to retry_strategy and timeout

e239107

Wrap attempts and attempt_duration_seconds in aws.Int64Value

61300ba

ghost added the size/M Managed by automation to categorize the size of a PR. label May 1, 2018

hectcastro commented May 1, 2018

View reviewed changes

hectcastro added 2 commits May 1, 2018 07:20

Make attempts and attempt_duration_seconds optional

773f023

Also, add validation to `attempts`.

Add additional error checking to flatten and expand

23841c8

For timeout and retry strategy.

gheine reviewed May 1, 2018

View reviewed changes

Update docs to reflect optional status

9fe5e66

gheine reviewed May 1, 2018

View reviewed changes

bflad added this to the v1.17.0 milestone May 1, 2018

bflad approved these changes May 1, 2018

View reviewed changes

bflad merged commit bcc3303 into hashicorp:master May 1, 2018

bflad mentioned this pull request May 1, 2018

Add support for aws_batch_job_definition timeout #4186

Closed

bflad added a commit that referenced this pull request May 1, 2018

Update CHANGELOG for #4386

ef01a79

hectcastro deleted the hmc/aws-batch-job-timeout branch May 1, 2018 23:48

ghost locked and limited conversation to collaborators Apr 6, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add support for AWS Batch job definition timeout #4386

Add support for AWS Batch job definition timeout #4386

hectcastro commented Apr 28, 2018

bflad left a comment

bflad Apr 30, 2018

hectcastro Apr 30, 2018

bflad Apr 30, 2018

hectcastro Apr 30, 2018

bflad Apr 30, 2018

hectcastro Apr 30, 2018

bflad Apr 30, 2018

hectcastro Apr 30, 2018

gheine Apr 30, 2018

hectcastro Apr 30, 2018

gheine May 1, 2018

hectcastro left a comment

hectcastro Apr 30, 2018

hectcastro Apr 30, 2018

hectcastro Apr 30, 2018

hectcastro Apr 30, 2018

hectcastro Apr 30, 2018

gheine May 1, 2018

gheine left a comment

bflad left a comment

bflad commented May 2, 2018

ghost commented Apr 6, 2020


		`timeout` supports the following:

		* `attempt_duration_seconds` - (Required) The time duration in seconds after which AWS Batch terminates your jobs if they have not finished. The minimum value for the timeout is `60` seconds.

Add support for AWS Batch job definition timeout #4386

Add support for AWS Batch job definition timeout #4386

Conversation

hectcastro commented Apr 28, 2018

bflad left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

hectcastro left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

gheine left a comment

Choose a reason for hiding this comment

bflad left a comment

Choose a reason for hiding this comment

bflad commented May 2, 2018

ghost commented Apr 6, 2020