Whitespace changes in heredoc force new resources

[hashibot]

This issue was originally opened by @yissachar as hashicorp/terraform#4435. It was migrated here as part of the provider split. The original body of the issue is below.

---

I have the following ECS task:

resource "aws_ecs_task_definition" "web" {
  family = "web"
  container_definitions = <<EOF
[{
  "name": "web",
  "image": "imagelocation",
  "memory": 500,
  "essential": true,
  "portMappings": [
    {
      "containerPort": 80,
      "hostPort": 80
    }
  ]
}]
EOF
}

If I then add an extra space after one of the commas, this will force a new version of the task as Terraform thinks that it has changed.

For my particular situation this causes problems since we have developers on Mac and Windows. Since we have Git set to convert line-endings, this means the local file on Windows uses \r\n for line-endings, and Mac uses \n. Terraform sees that the container_definitions has some changes and forces an update of all the resources involved.

I understand that it might be out of scope for Terraform to handle something like this, but it would be nice if there were some way of dealing with the issue. For now we are forced to restrict Terraform plan and apply usage to one OS.

[hashibot]

This comment was originally opened by @jen20 as hashicorp/terraform#4435 (comment). It was migrated here as part of the provider split. The original comment is below.

---

Hi @yissachar - thanks for opening this issue - I can certainly see the concern here.

One option here is to normalise the line endings to the correct \n format prior to calculating the hash which determines whether there is a change, regardless of operating system. The risk there is that if a HEREDOC is used for a template_file for example, and then used in a provisioner, we might get the line endings incorrect for the target operating system, though this is likely also the case now.

@phinze or @catsby do you have a preferred approach here?

[hashibot]

This comment was originally opened by @phinze as hashicorp/terraform#4435 (comment). It was migrated here as part of the provider split. The original comment is below.

---

It seems reasonable to me that for the purposes of hashing and diff calculation that we'd normalize line endings, but for content we'd respect the line endings we see.

It would be sort of a big change though, since updating the logic would have the potential to trigger unwanted diffs across all resources for existing users. We'd probably need to write state migration code to prevent this from happening.

I wonder if this is something we could address more directly at the resource level by doing state normalization on this field. @yissachar is aws_ecs_task_definition the only place you're seeing this problem?

[hashibot]

This comment was originally opened by @yissachar as hashicorp/terraform#4435 (comment). It was migrated here as part of the provider split. The original comment is below.

---

I'm also seeing it on aws_iam_role_policy on the policy field.

[hashibot]

This comment was originally opened by @oillio as hashicorp/terraform#4435 (comment). It was migrated here as part of the provider split. The original comment is below.

---

I am seeing a similar issue in aws_sqs_queue on the policy field.

I don't know if it is the same issue or not. I have a HEREDOC setup and every time I apply, it wants to update this field. It looks like the state is being stored with all whitespace removed, but the diff is being done against the original version with whitespace.

[hashibot]

This comment was originally opened by @Morriz as hashicorp/terraform#4435 (comment). It was migrated here as part of the provider split. The original comment is below.

---

+1
Hope this be resolved

[hashibot]

This comment was originally opened by @rsalmond as hashicorp/terraform#4435 (comment). It was migrated here as part of the provider split. The original comment is below.

---

@oillio in case you haven't seen it yet the sqs policy json is now normalized (#4780), works for me as of 0.6.15.

[hashibot]

This comment was originally opened by @mitchellh as hashicorp/terraform#4435 (comment). It was migrated here as part of the provider split. The original comment is below.

---

Moving this to an AWS issue, we now have the technology to fix this:

• We need to change the StateFunc to normalize JSON (normalizeJsonString)
• We need a state migration function so that we can do this w/o breaking old states.

[repl-raymond-bolander]

We are currently seeing this issue. Not sure if its related to: issue 310

[catsby]

Hey there! Sorry for the late reply, I believe this issue to be fixed in #1195 where we normalize and ignore json to avoid diffs for json like this.

The fix went out in the AWS Provider version 1.3.0, on November 16, 2017. Are you using that version or newer?

I'm going to close for now though. Let us know!

[ghost]

@catsby I am using latest versions of aws provider and core and seeing this happen in a heredoc used for the resource aws_dynamodb_table_item. It looks like the json is being sorted by key then all whitespace removed, and terraform plan shows it as a change to the original heredoc jsonencoded, i.e. whitespace intact, quotes escaped, not sorted.

Is this resource not subject to the same comparison code? Are there any workarounds, like running the heredoc through something that sorts and removes white space? Or maybe ignore is an option.

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thanks!