Improve executable path detection

[rpkilby]

Improve unix python3 detection. Right now, it's assumed that there is a python3 binary in the bin directory. This isn't necessarily the case, such as when pyenv is misconfigured.

[obestwalter]

Hi @rpkilby, I am wondering would that interpreter detection not better be a job for tox directly or another plugin? Maybe you mean something different, but I would not like us to have to do duplicate work.

[rpkilby]

Hi @obestwalter - this relates back to the issue of creating python3 venvs. There are more details in tox-dev/tox#630 (comment), but the gist is that it's not possible to create a python3 venv while working in an active virtualenv (which travis uses by default). To get around this limitation, you need to find the real python executable that was used to create the virtualenv, then use that binary to create the python3 venv. Below is how tox-venv currently determines the real executable:

tox-venv/src/tox_venv/hooks.py

Lines 29 to 41 in 5840166

	args = [str(python), '-c', 'import sys; print(sys.real_prefix)']
	# get python prefix
	process = subprocess.Popen(args, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
	output, _ = process.communicate()
	prefix = output.decode('UTF-8').strip()
	# determine absolute binary path
	if platform.system() == 'Windows':
	executable = 'python.exe'
	else:
	executable = 'bin/python3'
	path = os.path.join(prefix, executable)

The problem is that sys.real_prefix only gives us the main directory, and tox-venv has to find the binary from there. Ideally, the install has all of the pythonX/pythonX.Y/pythonX.Y.Z variants, and the above code assumes the existence of pythonX. However, there are some cases where pyenv has been misconfigured, and pythonX does not exist, instead only having pythonX.Y.Z. e.g., tox-dev/tox-travis#102.

Instead of just assuming pythonX, tox-venv should check for all three variants. The fix is fairly straightforward, but I don't have an easy way to test this, as I can't control how python is installed. The tests would have to rely on a buggy misuse of pyenv, but that's not exactly desirable.

[obestwalter]

ah o.k. that is specific to using venv then. Thanks for the detailed explanation.

[rpkilby]

Another possible case is when a linux distro provides multiple python 3 packages. eg, CentOS epel-release provides both python34 and python36 packages. The former is considered the "current" version of python, and provides both the python3 and python3.4 binaries, while the latter only provides python3.6.

Versions should be detected with decreasing specificity (3.4.1 => 3.4 => 3).