pip freeze does not show version for in-place installs

[danielpanteleit]

Environment

• pip version: 20.1
• Python version: 3.7
• OS: Ubuntu 19.10

Description

pip freeze does not list the version for in-place installs via "pip install ." Instead it shows a @ file:///... line which is usually used in editable installs.

Expected behavior

Following the example from below, the output should be

...
+ virtualenv/bin/pip freeze
myfreeze==1.0
+ virtualenv/bin/pip list --format freeze
myfreeze==1.0
pip==20.1
setuptools==46.1.3
wheel==0.34.2

How to Reproduce

Here is a minimal environment: https://github.com/danielpanteleit/pip_freeze_test

1. Clone Repo https://github.com/danielpanteleit/pip_freeze_test
2. Run "run.sh"

Output

...
+ virtualenv/bin/pip freeze
myfreeze @ file:///home/daniel/tmp/freeze
+ virtualenv/bin/pip list --format freeze
myfreeze==1.0
pip==20.1
setuptools==46.1.3
wheel==0.34.2

[deveshks]

IIUC, the freeze format of @ .... was introduced in #7612 for pip install providing direct URL references, but I am not sure if it' applicable in your case.

Edit: I ran your example after cloning the git repo and commenting out the pip install wheel --upgrade line in your script, and I was able to get myfreeze==1.0 .

$ cat run.sh 
#!/usr/bin/env bash

set -ex

rm -rf virtualenv
python -m venv virtualenv
. virtualenv/bin/activate
pip install pip --upgrade
pip install setuptools --upgrade

pip install .

virtualenv/bin/pip freeze
virtualenv/bin/pip list --format freeze

$ ./run.sh 
+ rm -rf virtualenv
+ python -m venv virtualenv
+ . virtualenv/bin/activate
++ deactivate nondestructive
++ '[' -n '' ']'
++ '[' -n '' ']'
++ '[' -n /bin/bash -o -n '' ']'
++ hash -r
++ '[' -n '' ']'
++ unset VIRTUAL_ENV
++ '[' '!' nondestructive = nondestructive ']'
++ VIRTUAL_ENV=/Users/devesh/Desktop/pip_freeze_test/virtualenv
++ export VIRTUAL_ENV
++ _OLD_VIRTUAL_PATH=/Users/devesh/.pyenv/shims:/usr/local/opt/[email protected]/bin:/Library/Frameworks/Python.framework/Versions/3.7/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin
++ PATH=/Users/devesh/Desktop/pip_freeze_test/virtualenv/bin:/Users/devesh/.pyenv/shims:/usr/local/opt/[email protected]/bin:/Library/Frameworks/Python.framework/Versions/3.7/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin
++ export PATH
++ '[' -n '' ']'
++ '[' -z '' ']'
++ _OLD_VIRTUAL_PS1=
++ '[' 'x(virtualenv) ' '!=' x ']'
++ PS1='(virtualenv) '
++ export PS1
++ '[' -n /bin/bash -o -n '' ']'
++ hash -r
+ pip install pip --upgrade
Collecting pip
  Using cached https://files.pythonhosted.org/packages/54/2e/df11ea7e23e7e761d484ed3740285a34e38548cf2bad2bed3dd5768ec8b9/pip-20.1-py2.py3-none-any.whl
Installing collected packages: pip
  Found existing installation: pip 19.2.3
    Uninstalling pip-19.2.3:
      Successfully uninstalled pip-19.2.3
Successfully installed pip-20.1
+ pip install setuptools --upgrade
Collecting setuptools
  Using cached setuptools-46.1.3-py3-none-any.whl (582 kB)
Installing collected packages: setuptools
  Attempting uninstall: setuptools
    Found existing installation: setuptools 41.2.0
    Uninstalling setuptools-41.2.0:
      Successfully uninstalled setuptools-41.2.0
Successfully installed setuptools-46.1.3
+ pip install .
Processing /Users/devesh/Desktop/pip_freeze_test
Could not build wheels for myfreeze, since package 'wheel' is not installed.
Installing collected packages: myfreeze
    Running setup.py install for myfreeze ... done
Successfully installed myfreeze-1.0
+ virtualenv/bin/pip freeze
myfreeze==1.0
+ virtualenv/bin/pip list --format freeze
myfreeze==1.0
pip==20.1
setuptools==46.1.3

Then when I unninstalled the myfreeze package, installed wheel and installed myfreeze again, I started seeing the @ file:// format

$ pip uninstall myfreeze
Found existing installation: myfreeze 1.0
Uninstalling myfreeze-1.0:
  Would remove:
    /Users/devesh/Desktop/pip_freeze_test/virtualenv/lib/python3.8/site-packages/myfreeze-1.0-py3.8.egg-info
Proceed (y/n)? y
  Successfully uninstalled myfreeze-1.0

$ pip install --upgrade wheel
Collecting wheel
  Using cached wheel-0.34.2-py2.py3-none-any.whl (26 kB)
Installing collected packages: wheel
Successfully installed wheel-0.34.2

$ pip install .
Processing /Users/devesh/Desktop/pip_freeze_test
Building wheels for collected packages: myfreeze
  Building wheel for myfreeze (setup.py) ... done
  Created wheel for myfreeze: filename=myfreeze-1.0-py3-none-any.whl size=982 sha256=a771df8230cdcaf304164e6e5dde8daafec6b9fda04b3b90fbcc11de20897f2f
  Stored in directory: /private/var/folders/xg/blp845_s0xn093dyrtgy936h0000gp/T/pip-ephem-wheel-cache-_6j3lha_/wheels/e3/4c/70/7cc401d9a46da9128ab80c5f7f003a4d9826c6b5ad5af322b5
Successfully built myfreeze
Installing collected packages: myfreeze
Successfully installed myfreeze-1.0

$ pip freeze
myfreeze @ file:///Users/devesh/Desktop/pip_freeze_test

[sbidoul]

@danielpanteleit this change was introduced in #7612 as a solution for #609 .

The goal is to have pip freeze output requirements that have better fidelity for subsequent usage in pip install -r. When outputing name==version, information is lost when the requirement was provided as URL or, in your case, as a local directory.

If you need to obtain the installed version, I'd recommend using pip list, possibly with --format=json.

That said I had forgotten myself about pip list --format=freeze. It indeed produces a different result than pip freeze and I'm unsure if we should now consider that as a bug or as a feature that we need to document and test.

[sbidoul]

@deveshks the behaviour you observe when wheel is not installed is normal, although I understand it can be surprising. When wheel is absent, pip falls back to the legacy setup.py install behavior, and setuptools does not produce .dist-info metadata nor does it support PEP 610. In the log you show above, there is a message mentioning that: "Could not build wheels for myfreeze, since package 'wheel' is not installed.". In #8102 we are considering making that a louder warning and possibly deprecating installing without wheel.

[deveshks]

When wheel is absent, pip falls back to the legacy setup.py install behavior, and setuptools does not produce .dist-info metadata nor does it support PEP 610.

So then what happens in the presence of wheel? From the logs, it seems like pip builds the wheel in that case, which includes the .dist-info directory along with direct_url.json, and installs the package from the wheel, and hence we are able to read and display the URL in pip freeze output, but in the absense of wheel, there is no .dist-info directory and direct_url.json
to read from, hence we see the version (but we have the .egg-info directory)

[deveshks]

That said I had forgotten myself about pip list --format=freeze. It indeed produces a different result than pip freeze and I'm unsure if we should now consider that as a bug or as a feature that we need to document and test.

So IIUC, pip list --format=freeze follows the output format style of pip freeze, and so if the output of pip freeze changes in the case of installations with direct URLs, pip list --format=freeze should follow suit and do the same.

[danielpanteleit]

The goal is to have pip freeze output requirements that have better fidelity for subsequent usage in pip install -r. When outputing name==version, information is lost when the requirement was provided as URL or, in your case, as a local directory.

If you need to obtain the installed version, I'd recommend using pip list, possibly with --format=json.

We use the following workflow in our CI pipeline: We install a package in-place inside a fresh virtualenv and call pip freeze. Then we run all tests and if everything works, we store the frozen requirements.txt separately. This way, we know that each stored requirements.txt worked and we can easily roll back to an older version if problems arise by using an older requirements.txt

The alternative would be to first build the source package and install this package via pip, right? Though I would prefer to just use pip install . or any other way to install the package in-place.

[sbidoul]

The alternative would be to first build the source package and install this package via pip, right?

Would this work for you?

$ pip wheel . --wheel-dir=builddir
$ pip install packagename --no-index --find-links=builddir

[danielpanteleit]

@sbidoul
Yes, this works for me.

If this is the intended way pip install . works, I think my issue is resolved. The pip list --format=freeze vs pip freeze format is not really relevant for me and should be put in a separate issue, I think.

Thanks!

[deveshks]

The pip list --format=freeze vs pip freeze format is not really relevant for me and should be put in a separate issue, I think.

I have filed #8176 for this.

[sbidoul]

@danielpanteleit it's good if this solution works for you. I'll let you close the issue if you think all is resolved.