Error reading JobQueue: "%!s(<nil>)"

[euclideansphere]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform Version

$ terraform -v
Terraform v0.11.3

Affected Resource(s)

• aws_batch_job_queue

Terraform Configuration Files

resource "aws_batch_job_queue" "job_queue" {
  name     = "${var.name_prefix}-job-queue"
  state    = "ENABLED"
  priority = 1

  compute_environments = [
    "${aws_batch_compute_environment.compute_environment.arn}",
  ]

  depends_on = ["aws_batch_compute_environment.compute_environment"]
}

Debug Output

I can't share the full log due to NDA/secrecy rules at the company I work for. Here are the relevant sections of logs I could find.

2018-10-02T23:01:13.640Z [DEBUG] plugin.terraform-provider-aws_v1.38.0_x4: 2018/10/02 23:01:13 [DEBUG] Job Queue "[redacted]" is already gone
2018/10/02 23:01:13 [ERROR] root.[redacted].batch_compute_environment: eval: *terraform.EvalRefresh, err: aws_batch_job_queue.job_queue: Error reading JobQueue: "%!s(<nil>)"
2018/10/02 23:01:13 [ERROR] root.[redacted].batch_compute_environment: eval: *terraform.EvalSequence, err: aws_batch_job_queue.job_queue: Error reading JobQueue: "%!s(<nil>)"

Expected Behavior

Terraform state should already know this has been removed in a previous execution of terraform destroy, or it should not have been removed in a previous execution. subsequent destroys on following failed destroys should not risk an out of date state.

Actual Behavior

The job queue was destroyed without terraform state being properly updated. In addition to this, the error message should be far more intuitive than "error reading job queue %!s(<nil)".

Steps to Reproduce

1. terraform destroy
2. terraform fails to destroy because there is a running job in the queue therefore the job queue cannot be destroyed.
3. manually terminate the running job via aws console
4. run terraform destroy again
5. terraform throws above error.

[euclideansphere]

I'm really not a go expert but I'd be happy to work on the issue. Let me know if there are contribution guidelines or anything else I should know before submitting a pull request.

[bflad]

Hi @euclideansphere 👋 Thanks for the report and sorry you're running into trouble here.

You can find our contribution documentation here: https://github.com/terraform-providers/terraform-provider-aws/blob/master/.github/CONTRIBUTING.md

Very briefly looking at this code:

https://github.com/terraform-providers/terraform-provider-aws/blob/9259be80f3d72fdc635221aff4beb09b88cc9c5b/aws/resource_aws_batch_job_queue.go#L89-L95

It looks like the batch service doesn't return a proper error (like other AWS services) when the queue does not exist, so we should remove the return fmt.Errorf(...) there and replace it with:

if jq == nil {
  log.Printf("[WARN] Batch Job Queue (%s) not found, removing from state", d.Get("name").(string))
  d.SetId("")
  return nil
}

This will prompt Terraform to show the resource as needing recreation, instead of returning the error. It can be acceptance tested by adding something similar to this pull request: #5967

Hope this helps!

[euclideansphere]

Thanks for the help @bflad, hopefully I'll have some time this weekend to dig into that this weekend. I'll ping if I have any more questions.
Thanks!

[bflad]

The fix for this has been merged into master and will release with version 1.40.0 of the AWS provider, likely middle of this week. 👍

[bflad]

This has been released in version 1.40.0 of the AWS provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thanks!