Append with file redirection not working running linux tools from Windows command line

[mimaen]

On Windows (version 10.0.18362.267) I'm trying to accumulate in a file results of linux tools, but only last result is stored. Simplifying, this is the problem:

>wsl echo line1 >> file.txt
>wsl echo line2 >> file.txt
>wsl echo line3 >> file.txt
>wsl xxd file.txt
00000000: 6c69 6e65 330a                           line3.

File is always replaced. Append is not working.

[craigloewen-msft]

Looking at your input it seems like you're running wsl through another shell (I'm going to guess CMD). That shell is interpreting and capturing the >> symbol, which is why you're running into this error.

If you want to use the command in CMD escpae out the > character by using:
wsl echo line1 ^>^> file.txt

Or you can use the command in powershell where it will work as expected.

I'll close out this issue since I believe it's resolved, if that's not the case please reopen! Thanks for filing.

[mimaen]

Thanks for your response.

I still think actually there is an error in WSL interoperability:

1. Official documentation states:

https://docs.microsoft.com/en-us/windows/wsl/interop
Run Linux binaries from the Windows Command Prompt (CMD or PowerShell)
Things such as sudo, piping, and file redirection work.

2. You are right, I'm working with CMD (I know there are multiple aternatives to avoid the error, but I'm pointig to what I consider a problem in WSL). And CMD has not problems with file redirections.

I've compiled an EXE with same output as linux echo (to keep end lines):

CMD> wsl echo line1 | wsl xxd
00000000: 6c69 6e65 310a                           line1.
CMD> echoLinux.exe line1 | wsl xxd
00000000: 6c69 6e65 310a                           line1.

CMD, as usual, works correctly with append operations via file redirection of Windows executables:

CMD> (echoLinux.exe line1) > file1.txt
CMD> (echoLinux.exe line2) >> file1.txt
CMD> (echoLinux.exe line3) >> file1.txt
CMD> wsl xxd file1.txt
00000000: 6c69 6e65 310a 6c69 6e65 320a 6c69 6e65  line1.line2.line
00000010: 330a                                     3.

File redirection (appending) does not work when the executable is a linux tool launched with WSL:

CMD> (wsl echo line1) > file2.txt
CMD> (wsl echo line2) >> file2.txt
CMD> (wsl echo line3) >> file2.txt
CMD> wsl xxd file2.txt
00000000: 6c69 6e65 330a                           line3.

Having to escape redirection to avoid normal CMD work breaks interoperatibility between both worlds.

[craigloewen-msft]

Thank you for the description, and also thank you to @DHowett-MSFT for taking a look at this issue as well.

This does seem to be a WSL problem. I'll reopen the issue.

[PrasadTamgale]

@craigloewen-msft The WAR you mentioned before closing the issue for the first time does not seem to be working.
#4400 (comment)
It is again replacing the file instead of appending to it.
P.S Requirement is to run this command in CMD.

Command:
cmd /c wsl echo hi ^>^>temp.txt
cmd /c wsl echo hi ^>^>temp.txt

Output:
$ cat temp.txt
hi

[mdruiter]

To me it looks like writing to the file starts at the start of the file, even though I use >> to append. I would expect this to be cmd.exe's responsibility. But it only happens with wsl...

[cloph]

indeed it doesn't seem to truncate the file and replace the content, but indeed starts "appending" from start, the following is in git-bash:

$ cat output
cat: output: No such file or directory
$ wsl.exe echo foo >> output
$ cat output
foo
$ wsl.exe echo barbazlonger >> output
$ cat output
barbazlonger
$ wsl.exe echo foo >> output
$ cat output
foo
azlonger
$ uname
MINGW64_NT-10.0-22631

So redirects certainly won't work as expected, and at least a big fat warning should be added to the documentation.
And to throw another workaround into the mix: "wsl.exe echo foo | cat - >> output" probably the first useless use of cat that's not completely useless....

[microsoft-github-policy-service]

This issue has been automatically closed since it has not had any activity for the past year. If you're still experiencing this issue please re-file this as a new issue or feature request.

Thank you!