script.pip() does not work with use_module=True

[gutsytechster]

Environment

• pip version: 20.1.dev0
• Python version: 3.6.9
• OS: Linux Mint Debian Edition

Description

While I was working on this #7810, I found that when I use script.pip("freeze") with use_module as True which is the default behaviour, it does not show any package list, while when I explicitly specify use_module as False, it does.

I found that the difference is the way the command is invoked. With use_module=True, it is invoked as python -m pip freeze and with use_module=False, it is invoked as pip freeze.

Expected behavior

Both ways, it should show up the package list.

How to Reproduce

A test can be written to verify this behaviour.

def test_pip_script(script):
    assert script.pip("freeze").stdout == script.pip("freeze", use_module=False).stdout

Output

___________________________________________________ test_pip_script ___________________________________________________
[gw4] linux -- Python 3.6.9 /home/gutsytechster/Documents/Projects/Open-Source-Projects/pip/.tox/py36/bin/python

script = <tests.lib.PipTestEnvironment object at 0x7f696adfc8d0>

    def test_pip_script(script):
>       assert script.pip("freeze").stdout == script.pip("freeze", use_module=False).stdout
E       AssertionError: assert '' == 'apipkg==1.5\natomicwrites==...zeug==0.16.0\nzipp==3.0.0\n'
E         + apipkg==1.5
E         + atomicwrites==1.3.0
E         + attrs==19.3.0
E         + cffi==1.14.0
E         + coverage==5.0.3
E         + cryptography==2.8
E         + csv23==0.1.6
E         + execnet==1.7.1
E         + freezegun==0.3.15
E         + importlib-metadata==1.5.0
E         + mock==4.0.1
E         + more-itertools==8.2.0
E         + pluggy==0.13.1
E         + pretend==1.0.9
E         + py==1.8.1
E         + pycparser==2.19
E         + pytest==3.8.2
E         + pytest-cov==2.8.1
E         + pytest-forked==1.1.3
E         + pytest-rerunfailures==6.0
E         + pytest-timeout==1.3.4
E         + pytest-xdist==1.27.0
E         + python-dateutil==2.8.1
E         + PyYAML==5.3
E         + scripttest==1.3
E         + six==1.14.0
E         + virtualenv==16.7.10
E         + Werkzeug==0.16.0
E         + zipp==3.0.0

[pradyunsg]

Thanks for filing a new issue for this!

My initial response is here: #7810 (comment). Please do follow up on that here. :)

[gutsytechster]

As you suggested, here #7810 (comment), I have written the test. However, it fails with following stdout

====================================================== FAILURES =======================================================
___________________________________________________ test_pip_script ___________________________________________________
[gw3] linux -- Python 3.6.9 /home/gutsytechster/Documents/Projects/Open-Source-Projects/pip/.tox/py36/bin/python

script = <tests.lib.PipTestEnvironment object at 0x7fc79f3b5208>

    def test_pip_script(script):
        r1 = script.pip("--version")
        r2 = script.pip("--version", use_module=False)
>       assert r1.stdout == r2.stdout
E       AssertionError: assert 'pip 20.1.dev...python 3.6)\n' == 'pip 20.1.dev0...python 3.6)\n'
E         - pip 20.1.dev0 from /tmp/pytest-of-gutsytechster/pytest-0/popen-gw3/pip0/pip/src/pip (python 3.6)
E         + pip 20.1.dev0 from /home/gutsytechster/Documents/Projects/Open-Source-Projects/pip/.tox/py36/lib/python3.6/site-packages/pip (python 3.6)

[pradyunsg]

Ahhhh. That makes sense -- whoever implemented use_module did not take into account the nuances here. :)

pip in the PipTestEnvironment PATH is not the correct to-be-tested item. I'm not sure what the fix is TBH; maybe getting the full absolute path to an executable, when using use_module=False?

[gutsytechster]

Is it this line here https://github.com/pypa/pip/blob/master/tests/lib/__init__.py#L475, where the PATH for the testing environment is being set?

[pradyunsg]

Should be, yea. It might also be in the isolate fixture, so that would be worth taking a look at.

[cjc7373]

I found out this is because we don't have pip executable file in the virtual environment's Scripts folder.
So we can execute python [...]/src/pip at

pip/tests/lib/__init__.py

Line 619 in c7bde5b

exe = 'pip'

[...]/src/pip means the absolote path defined in

pip/tests/conftest.py

Line 285 in c7bde5b

pip_editable = Path(str(tmpdir_factory.mktemp('pip'))) / 'pip'

I'm not sure this is correct. But this can fix the bug.
And in

pip/tests/conftest.py

Line 294 in c7bde5b

# Drop (non-relocatable) launchers.

it removes all files in venv.bin (which is the Scripts folder) except those start with "python" or "libpy", what does non-relocatable mean?