VCS invocation merges stdout and stderr, resulting in output parsing error

[tomprince]

Environment

• pip version: 19.3.1
• Python version: 2.7
• OS: fedora

Due to a bug in mercurial, if both the split and evolve extension are enabled, mercurial will output a warning (on stderr) whenever it runs. When pip calls hg root to get the path of an editable install, pip captures both stdout and stderr, and so includes the warning as part of the path to the root of the repository and fails.

This can be reproduced with the following .hgrc:

[extensions]
evolve =
split =

and installing the evolve extensions pip install hg-evolve.

[sbidoul]

It's correct that pip captures both stdout and stderr:

pip/src/pip/_internal/utils/subprocess.py

Lines 186 to 191 in 08f61a9

	proc = subprocess.Popen(
	# Convert HiddenText objects to the underlying str.
	reveal_command_args(cmd),
	stderr=subprocess.STDOUT, stdin=subprocess.PIPE,
	stdout=subprocess.PIPE, cwd=cwd, env=env,
	)

Could you check if running hg --quiet root would silence the warning?

[tomprince]

It does not appear to silence the warning.

[uranusjr]

I’ll probably work on this after #7593. Some refactoring would improve the code touched by that PR, and would fix this bug en passant.

[uranusjr]

I’ve edited the title to be more generic. This also better reflects the problem (which is general, not limited to hg).

[godlygeek]

Copying some extra info from #8191:

1. In the git case this is pretty easy to reproduce by doing an export GIT_TRACE=1
2. The fact that stderr is redirected to stdout appears to be a historical accident. In 0cb2991 stderr was redirected to stdout to add the ability to filter tracebacks from stderr, and when that traceback filtering functionality was removed in f9429b0 stderr wasn't split back apart from stdout.

[deveshks]

I think this is similar to what I observed in svn in #7968 .

I have also created a PR for this (#7969) which removes the behavior of vcs commands of merging stderr and stdout, and makes VCS commands use their own call_subprocess. Could you try it out and see if this fixes the hg issue?

[deveshks]

Hi @tomprince

The fix for this #7969 is now merged to master.

Could you please install pip from master branch (pip install git+https://github.com/pypa/pip.git#egg=master) and verify if the fix works for you?