git-bash giving unexpected results when called from Python's subprocess module

[ruaridhw]

• I was not able to find an open or closed issue matching what I'm seeing

Setup

• Which version of Git for Windows are you using? Is it 32-bit or 64-bit?

Git-2.10.1-64-bit

which was compared to the installed version:

$ git --version --build-options
git version 2.9.2.windows.1
sizeof-long: 4

• Which version of Windows are you running? Vista, 7, 8, 10? Is it 32-bit or 64-bit?

$ cmd.exe /c ver
Microsoft Windows [Version 6.1.7601]

• What options did you set as part of the installation? Or did you choose the
  defaults?

# One of the following:
> type "C:\Program Files\Git\etc\install-options.txt"
> type "C:\Program Files (x86)\Git\etc\install-options.txt"
> type "%USERPROFILE%\AppData\Local\Programs\Git\etc\install-options.txt"
$ cat /etc/install-options.txt
Path Option: Cmd
SSH Option: OpenSSH
CRLF Option: CRLFAlways
Bash Terminal Option: MinTTY
Performance Tweaks FSCache: Enabled

• Any other interesting things about your environment that might be related
  to the issue you're seeing?

Calling git-bash.exe from Python 2.7.12

Details

• Which terminal/shell are you running Git from? e.g Bash/CMD/PowerShell/other

git-bash.exe

• What commands did you run to trigger this issue? If you can provide a
  Minimal, Complete, and Verifiable example
  this will help us understand the issue.

Call from python

import subprocess
cmdargs = ["C:\Program Files\Git\git-bash.exe", "download_table.sh", "<TABLE>"]
for table in ["TABLE1","TABLE2","TABLE3"]:
	cmdargs[2] = table
	process = subprocess.call(cmdargs)

download_table.sh

TABLE_NAME=$1
sqlplus -s usr/pwd << SQL
    SPOOL $TABLE_NAME".tsv"
    SELECT * FROM $TABLE_NAME
    SPOOL OFF
SQL

• What did you expect to occur after running these commands?

For each call to subprocess to run sequentially downloading TABLE1, TABLE2, TABLE3 waiting for the current process to conclude before continuing.

• What actually happened instead?

All three processes were kicked off without waiting for the previous git-bash.exe process to terminate.

The above code was tested with Git-2.10.1-64-bit AND Git-2.9.2-64-bit. The latter yielded expected results whereas the newer version did not.

Calls to subprocess using \bin\sh.exe and \bin\bash.exe DID cause download_table.sh to be executed sequentially however these commands cause the script to be run without opening an external window therefore not allowing stdout to be read in real time (this is particularly an issue when cmdargs is then called from the multiprocessing library).

• If the problem was occurring with a specific repository, can you provide the
  URL to that repository to help us with testing?

[dscho]

Please understand that git-bash.exe is the executable that is intended to open the interactive Git Bash.

What you most likely wanted to do was to call C:\Program Files\Git\bin\bash.exe instead (which is still not the real Bash, but a redirector that sets up appropriate environment variables first).

[ruaridhw]

My question is really about why there are two completely different results for the same code between Git for Windows versions Git-2.10.1-64-bit vs Git-2.9.2-64-bit.

I did intend to call the interactive Git Bash (as opposed to \bin\bash.exe) so that the bash script runs in an "interactive" window externally to python.exe.

If I call git-bash.exe, then python.exe pauses, a new bash window is launched, the bash window closes upon completion and python.exe continues.

In comparison, \bin\bash.exe runs inside python.exe which then also continues upon bash completion. As I say, this is particularly undesired when employing multiprocessing subprocess calls because having multiple \bin\bash.exe calls running in the same window makes them impossible to follow as opposed to having a new git-bash.exe window for each call.

[dscho]

My question is really about why there are two completely different results for the same code between Git for Windows versions Git-2.10.1-64-bit vs Git-2.9.2-64-bit.

The source code has actually not changed between those releases.

So my next guess was that maybe something in the MINGW CRT package changed that lets GUI processes quit early. I even went so far was to clone https://git.code.sf.net/p/mingw-w64/mingw-w64 mingw-w64-mingw-w64 before realizing that I should check whether the mingw-w64-crt-git package actually had changed between the versions you reported. And actually, that is when I realized that there were multiple versions between Git-2.10.1-64-bit vs Git-2.9.2-64-bit. I narrowed it down (by unpacking the PortableGits and testing them) to the jump between 2.9.3(2) and 2.10.0. The version of the mingw-w64-crt-git package had not changed (as you can easily verify by comparing the included /etc/package-versions.txt files.

And when I was comparing the package versions of 2.9.3(2) vs 2.10.0, it struck me that preciously little had changed. But there was one notable jump there: the MinTTY version had jumped from 2.0.3 to 2.4.2.

And a "bug fix" in MinTTY v2.1.3 quietly changed the startup behavior to "daemonize" by default, i.e. let the main process quit while the worker process continues to run.

This comment reflects 5 hours of research.