terraform fmt changes Windows line endings to Unix

[bucs-fan813]

Terraform Version

Terraform v1.4.0
on linux_amd64

Terraform Configuration Files

N/A

Debug Output

N/A

Expected Behavior

terraform fmt should produce files that are compliant with the configured provider(s)

Actual Behavior

trailing whitespace.git-diff-check(error) and/or unexpected EOF error.

Steps to Reproduce

1. Unknown combination(s) of developer ide, git, os and resource provider
2. terraform init
3. terraform fmt
4. terraform apply

Additional Context

It would be nice if terraform fmt could allow for formatting files in accordance with other popular linters (besides tflint) like trunk for VSCode. The obvious workaround # trunk-ignore-all(git-diff-check) only kicks the can down the road. Developers may spend a considerable amount of time troubleshooting line terminations in herdoc blocks since it won't be obvious when/where certain tf resources (like aws lambda and codebuild) have unexpected EOF errors. For windows vscode users, Remote extensions:connect to wsl will also help but on larger teams this also only kicks the can. I'm new to terraform but it seems this issue has been mentioned previously. I tried making a fix for it myself but I'm not familiar enough with go or terraform to be able to test a PR. This may be better suited as a provider parameter since this use case may not apply to azure or other providers?

References

• terraform fmt changes Windows line endings to Unix #10381
• Add EOL conversion option in provisioner file #31003
• testing dos2unix arg #32909

[apparentlymart]

Hi @bucs-fan813,

terraform fmt considers it to be a style decision to always use Unix-style line endings on all platforms, but you're right that sometimes it can potentially change the meaning if you are using heredoc templates and the remote system is sensitive to which line endings are in use.

terraform fmt is working as designed here, but I understand that you have found a situation where its design causes a problem and so I'm going to relabel this as an enhancement so that we can use this issue to determine how to change its design to accommodate the situation you've described. (The bug report process is only for situations where Terraform's behavior does not match its design and so therefore it's "obvious" that it would be resolved by changing the implementation to match the design, but here we need to figure out what the new design ought to be first.)

---

In an ideal world the underlying grammar HCL would've already been normalizing the line endings of heredocs so that they are always the same regardless of how the source file is written, so that LF and CRLF would be entirely equivalent in all cases and terraform fmt's decision here really would just be a matter of style. However, we cannot change HCL's interpretation of heredocs now due to compatibility concerns.

Another option then would be to change the HCL formatter that terraform fmt relies on so that it leaves heredoc-bounded line endings exactly as written when formatting, to avoid changing their meaning. However, that will cause files written on Windows to contain a mixture of CRLF and LF lines, which is not supported in most text editors.

Another option would be for terraform fmt to not change line endings at all, but that's problematic for any situation where it might insert a new linebreak because it would need to somehow infer which kind of line ending to use to avoid making the file inconsistent.

I'm not sure what the best answer is. Any change to the existing line endings in a heredoc could potentially cause a problem, but not changing line endings elsewhere in the document could potentially also be problematic if they weren't already consistent. Perhaps the sweet spot is for the formatter to first check whether the existing file is 100% using Windows-style line endings, and only if that is true then rewrite the output to also use CRLF instead of LF exclusively. In all other cases it would behave as it does today, changing all of the line endings to LF so that the result is at least consistent. The edge case here is if the input is already using inconsistent line endings (a mixture of CRLF and LF), which we'd arbitrarily decide is normalized to all of them being LF because that matches how it already behaves in that case and thus it limits the potential churn caused by the change.

[anderson-zulu]

+1