Problems using native editors via $env:EDITOR with Git for Windows.

[rkitover]

Setup

• Which version of Git for Windows are you using? Is it 32-bit or 64-bit?

git version 2.34.1.windows.1
cpu: x86_64
built from commit: 2ca94ab318509b3c271e82889938816bad76dfea
sizeof-long: 4
sizeof-size_t: 8
shell-path: /bin/sh
feature: fsmonitor--daemon

• Which version of Windows are you running? Vista, 7, 8, 10? Is it 32-bit or 64-bit?

Microsoft Windows [Version 10.0.22526.1000]

• What options did you set as part of the installation? Or did you choose the
  defaults?

Editor Option: VIM
Custom Editor Path:
Default Branch Option:
Path Option: Cmd
SSH Option: OpenSSH
Tortoise Option: false
CURL Option: OpenSSL
CRLF Option: CRLFAlways
Bash Terminal Option: MinTTY
Git Pull Behavior Option: Merge
Use Credential Manager: Enabled
Performance Tweaks FSCache: Enabled
Enable Symlinks: Enabled
Enable Pseudo Console Support: Disabled
Enable FSMonitor: Disabled

Details

• Which terminal/shell are you running Git from? e.g Bash/CMD/PowerShell/other

I am using the new Windows Terminal with PowerShell Core.

• What commands did you run to trigger this issue? If you can provide a
  Minimal, Complete, and Verifiable example
  this will help us understand the issue.

$env:EDITOR = 'C:/tools/neovim/Neovim/bin/nvim.exe'
git commit -a --verbose --signoff -S

• What did you expect to occur after running these commands?

I expected Neovim to start with my commit message in the same mode as it starts when run from the PowerShell prompt.

• What actually happened instead?

Neovim started with terminal support for Cygwin, with much worse colors etc..

Running the following inside Neovim shows:

:echo $TERM
" cygwin
:echo &term
" builtin_cygwin

, while starting it from PowerShell shows:

:echo $TERM
" vtpcon
:echo &term
" builtin_vtpcon

The issue is that $env:TERM is set to cygwin when Neovim launches from Git, causing it to use its builtin Cygwin terminal support which is much worse than the native vtpcon support for the new Windows console.

I opened an issue related to this in the neovim repository:

neovim/neovim#17021

There are two workarounds that I found for this issue, the first is creating a .bat file and setting it as $env:EDITOR.

This my ~/.local/bin/nvim.bat:

@echo off
set TERM=
/tools/neovim/neovim/bin/nvim %*

. Now, the following would work correctly:

$env:EDITOR = 'C:/Users/rkitover/.local/bin/nvim.bat'
git commit -a --verbose --signoff -S

. The other workaround is adding the following to ~/.gitconfig:

[core]
    editor = sh -c "'"TERM= c:/tools/neovim/Neovim/bin/nvim.exe \"$@\""'" --

.

Other Problems with $env:EDITOR on Windows

• A normal Windows path with backslashes in $env:EDITOR does not work at all:

$env:EDITOR = 'C:\tools\neovim\Neovim\bin\nvim.exe'
git commit -a --verbose --signoff -S

hint: Waiting for your editor to close the file... C:\tools\neovim\Neovim\bin\nvim.exe: C:toolsneovimNeovimbinnvim.exe: command not found
error: There was a problem with the editor 'C:\tools\neovim\Neovim\bin\nvim.exe'.
Please supply the message using either -m or -F option.

[dscho]

The issue is that $env:TERM is set to cygwin when Neovim launches from Git, causing it to use its builtin Cygwin terminal support which is much worse than the native vtpcon support for the new Windows console.

Isn't the real problem, then, that Neovim's defaults for the cygwin terminal are suboptimal?

[rkitover]

Isn't the real problem, then, that Neovim's defaults for the cygwin terminal are suboptimal?

This too, the new cygwin terminal support has almost full support for the new Windows console. However, they may want to support older systems. Further complicating this is that this is the native version of Neovim and not the Cygwin or MSYS2 version (MSYS2 is just Cygwin with a small patch for parameter-destroying.)

I should mention that the regular version of native vim does not have this problem, it ignores $env:TERM and uses its Windows console support.

I will bring this up in the Neovim issue I linked.

Regardless, I consider Git modifying $env:TERM a bug, if mine is empty I want it to stay empty when my editor or any external program launches.

Native Windows apps generally don't use $env:TERM, and there is no standard for this at present.

[dscho]

If native Windows programs don't use TERM, why do we even bother with this discussion?

[rkitover]

I tried setting:

$env:TERM = 'vtpcon'

and that works correctly with Neovim, and Git does not modify it in that case.

[dscho]

You could use one of the two approaches outlined in https://stackoverflow.com/questions/59733731/how-to-detect-if-running-in-the-new-windows-terminal or microsoft/terminal#6269 to detect in your PowerShell profile that it is running inside a Windows Terminal, and then set `$env:TERM = 'vtpcon':

1. use the presence of $env:WT_SESSION (although this might be a false positive if you spawn, say, a MinTTY window from your session)
2. walk the ParentProcessId chain to figure out whether the current session is a (transitive) child process of the Windows Terminal (it does run the same risk of a false positive, of course).

Of course, you could also change your Windows Terminal PowerShell Profile to execute with a startup command that sets this (I think -NoExit -Command '$env:TERM="vtpcon"' might work, but I haven't verified this).

[rkitover]

I took a different approach, I am setting:

$env:TERM = 'xterm-256color'
$env:COLORTERM = 'truecolor'

, in my $profile. This is currently the most widely used method on Linux as well. The $env:COLORTERM is not necessary in my case, but I set it just in case.

Both the Windows Terminal and versions of conhost made in the last many years support this just fine. All features I care about including 24 bit color work fine in Neovim. Regular vim also works fine, to the extent that it does on Windows, nano works fine.

So I have a solution for my problem and for the document I am maintaining.

But that does not solve the issue here, which is $env:TERM being set to cygwin when it is unset for native, non-Cygwin (including MSYS2 and related) apps.

I consider this a bug, if you don't, that's for you to decide.

Thank you for your work on this project.

[rkitover]

One of the terminal developers confirmed that these are good settings to use, and this works fine with the vim you ship with Git, other than the mouse not working, but that is likely a terminal issue. The mouse does not work in the native regular vim either.

[dscho]

But that does not solve the issue here, which is $env:TERM being set to cygwin when it is unset for native, non-Cygwin (including MSYS2 and related) apps.

How could the MSYS2/Cygwin runtime detect that it is connected to a Windows Terminal?

FWIW here is what the MSYS2 runtime does:

  /* If console has 24 bit color capability, TERM=xterm-256color,
     otherwise, TERM=cygwin */
  if (!sawTERM)
    envp[i++] = strdup (wincap.has_con_24bit_colors () ? xterm : cygterm);

That has_con_24bit_colors() method returns the value for the respective Windows version, as defined in wincap.cc. From what I can see, it should safely return true in your setup...

[rkitover]

I did some experiments and it seems that your version of Cygwin is setting $TERM correctly, the problem seems to be with how Git is launching $env:EDITOR.

I could take a look at this if you like, this is in editor.c.

[dscho]

I could take a look at this if you like, this is in editor.c.

If you mean to say that https://github.com/git-for-windows/git/blob/v2.35.0.windows.1/editor.c sets TERM=cygwin, I would like to learn how it does that.

[phil-blain]

Other Problems with $env:EDITOR on Windows

• A normal Windows path with backslashes in $env:EDITOR does not work at all:

$env:EDITOR = 'C:\tools\neovim\Neovim\bin\nvim.exe'
git commit -a --verbose --signoff -S

hint: Waiting for your editor to close the file... C:\tools\neovim\Neovim\bin\nvim.exe: C:toolsneovimNeovimbinnvim.exe: command not found
error: There was a problem with the editor 'C:\tools\neovim\Neovim\bin\nvim.exe'.
Please supply the message using either -m or -F option.

I also noticed that, and despite looking through the wiki and searching for "windows" in git-config(1), I could not find where this is documented.. @dscho is it documented somewhere ?

[rkitover]

I also noticed that, and despite looking through the wiki and searching for "windows" in git-config(1), I could not find where this is documented.. @dscho is it documented somewhere ?

My guess is that EDITOR is invoked with bash without proper quoting which treats backslashes as escapes. It seems that MSYS2/Cygwin bash is capable if launching Windows paths if they are properly quoted, for example, this will work:

'C:\Windows\notepad.exe'

.

[phil-blain]

On the original topic,

But that does not solve the issue here, which is $env:TERM being set to cygwin when it is unset for native, non-Cygwin (including MSYS2 and related) apps.

How could the MSYS2/Cygwin runtime detect that it is connected to a Windows Terminal?

FWIW here is what the MSYS2 runtime does:

  /* If console has 24 bit color capability, TERM=xterm-256color,
     otherwise, TERM=cygwin */
  if (!sawTERM)
    envp[i++] = strdup (wincap.has_con_24bit_colors () ? xterm : cygterm);

That has_con_24bit_colors() method returns the value for the respective Windows version, as defined in wincap.cc. From what I can see, it should safely return true in your setup...

I agree that it should set TERM=xterm-256color according to what I understand of that code. I'm on 1909, so 10.0.18363.2037, and so I should get the wincap defined for 1903. However, I'm also seeing "TERM=cygwin" on my side, though with native Vim (not Neovim). How would I go about getting GDB set up to step inside wincap.cc ? I did look through Debugging Git but it does not mention how to debug the runtime...

[rkitover]

I agree that it should set TERM=xterm-256color according to what I understand of that code. I'm on 1909, so 10.0.18363.2037, and so I should get the wincap defined for 1903. However, I'm also seeing "TERM=cygwin" on my side, though with native Vim (not Neovim). How would I go about getting GDB set up to step inside wincap.cc ? I did look through Debugging Git but it does not mention how to debug the runtime...

Did you see my example screenshot? The runtime works correctly, the problem is in Git.

I was going to look in editor.c before I got side-tracked by looking at a bunch of cmake stuff, I'd suggest you look there.

[rkitover]

Here are the PowerShell commands you can test in case someone deleted my screenshot:

ri env:TERM,env:COLORTERM -ea ignore

&'/program files/Git/bin/bash' -c 'echo $TERM'

&'/program files/Git/bin/bash' -c 'c:/tools/neovim/neovim/bin/nvim -c ''call writefile([$TERM], fnamemodify("~",":p") . "git-bash-nvim-term.txt")|q'''; gc ~/git-bash-nvim-term.txt

[phil-blain]

@rkitover I did, but the second command did not work on my side (syntax error from Vim) :) Thanks for the copy-pasteable commands!

[rkitover]

I mistyped that, sorry, the second command is:

&'/program files/Git/bin/bash' -c 'c:/tools/neovim/neovim/bin/nvim -c ''call writefile([$TERM], fnamemodify("~",":p") . "git-bash-nvim-term.txt")|q'''; gc ~/git-bash-nvim-term.txt

, I will write an equivalent for regular vim in a few minutes.

[rkitover]

So yeah, that also works for regular vim:

&'/program files/Git/bin/bash' -c 'c:/tools/vim/vim82/vim.exe -c ''call writefile([$TERM], fnamemodify("~",":p") . "git-bash-nvim-term.txt")|q'''; gc ~/git-bash-nvim-term.txt

.

[rkitover]

On the subject of Windows paths in $env:EDITOR, I just realized that this works:

$env:EDITOR = "'C:/tools/neovim/Neovim/bin/nvim.exe'"

, note the extra single quotes.

We just need a good solution to do this automatically somehow.

[phil-blain]

The second command still does not work for me (sorry, I don't know much about Vim nor Powershell!). If it matters, this is under Windows Powershell (i.e. 5.1). The following command "works":

 &"$(where.exe git)\..\..\bin\bash.exe" -c "'$(where.exe vim)' -c 'call writefile([$TERM], `"`"term.txt`"`")|q'"; cat term.txt

but the resulting file term.txt is empty...
(I have both Git and Vim installed in my account so I generalized your example to not hardcode installed locations).

[phil-blain]

OK, I was missing a backtick before TERM:

 &"$(where.exe git)\..\..\bin\bash.exe" -c "'$(where.exe vim)' -c 'call writefile([`$TERM], `"`"term.txt`"`")|q'"; cat term.txt

This does show xterm-256color for me as well.

But, is the environment setup the same when running /path/to/Git/bin/bash.exe directly vs running via /path/to/Git/cmd/git.exe, I don't know...

[rkitover]

Here is an example I came up with (backup your clone before you do this just in case.) You can do this in any clone.

git checkout -b try-branch
git filter-branch --setup 'echo $TERM' --msg-filter cat

, you will see that it will output xterm-256color.

Another test case you can try, make a trivial change in a clone, and run:

$env:EDITOR = 'echo $TERM;exit'
git commit -a --verbose

this also outputs xterm-256color, however, I still suspect the issue is in editor.c in some way. It does some special handling when it detects vim etc..

[rkitover]

Never mind, I did this wrong.

I forgot to run:

ri env:TERM,env:COLORTERM -ea ignore

, having done this, the $TERM is set to cygwin in both cases.

So you are right about the problem being with Git in general.

[rkitover]

It also appears to be a problem with the git.exe executable specifically, trying with the included MSYS2 awk:

&'/Program Files/Git/usr/bin/awk' 'BEGIN { print(ENVIRON["TERM"]); exit }'

, I also get xterm-256color.

[rkitover]

So I found the problem:

https://github.com/git-for-windows/git/blob/main/compat/mingw.c#L3365

[phil-blain]

Indeed! It dates back to baea068 and this logic was not touched since then as far as I could see.

[phil-blain]

Of course, you could also change your Windows Terminal PowerShell Profile to execute with a startup command that sets this (I think -NoExit -Command '$env:TERM="vtpcon"' might work, but I haven't verified this).

A correct syntax for this seems to be:

# From Powershell itself
powershell.exe -NoLogo -NoExit -Command '$env:TERM=\"xterm-256color\"'
# or 
powershell.exe -NoLogo -NoExit -Command '$env:TERM=''xterm-256color'''
# or
powershell.exe -NoLogo -NoExit -Command "`$env:TERM='xterm-256color'"

and in Terminal's settings.json:

"commandline": "powershell.exe -NoLogo -NoExit -Command \"$env:TERM='xterm-256color'\"",