--from-file does not accept \r\n line separators on Windows

[boekhold]

Describe the bug

PS > Get-Content -Raw .\helmfile.yaml | yq --from-file patch.yq
Error: 7:2: invalid input text "\r\n|\nselect(.kind..."

where patch.yq has Windows Style CrLf line endings.

Also note that yq seems to be very sensitive of the Encoding of the file specified via --from-file. It does not accept UTF-16 files (which my PowerShell produces by default), and it does not accept a UTF-8 file with BOM.

Version of yq: 4.35.2
Operating system: Windows 10 22H2
Installed via: binary release

Command
The command you ran:

Get-Content -Raw .\helmfile.yaml | yq --from-file patch.yq

[mikefarah]

Thanks for raising. The line ending issue is pretty straight forward to fix, and will be included in the next release.

I'm having trouble with the UTF encoding - theres quite a few different UTF16 encoding standards, and I haven't found a decent go library that handles them properly.

[boekhold]

@mikefarah I'm not sure if I should be reporting this here, as it seems to be related to the CrLf handling, or if I should open a new issue for this.... Let me know if I need to move this to a new issue.

Platform: Windows
Input file:

---
# document 1 start comment
test:
  sub:
   key: value
---
# document 2 start comment
foo:
  bar:
   key: value

Line Ending: CrLf

yq '.' test-in.yaml:

---
# document 1 start comment
test:
  sub:
    key: value
# document 2 start comment
---
foo:
  bar:
    key: value

Note how the "document 2 start comment" has moved from the top of the second document to the bottom of the first document!

This is causing havoc with the Kubernetes manifest that I'm manipulating with yq...

If I explicitly change the line endings of test-in.yaml to UNIX-style "Lf", the issue does not appear.

[mikefarah]

Hey, thanks for raising, this is a separate issue. I've created another card :)

[mikefarah]

CRLF issue has been fixed in 4.40.2

[mikefarah]

Closing as the main issue has been fixed. Feel free to raise a separate issue for UTF coding if desired.