pip uninstall not working with editable dependencies #5193

sbidoul · 2018-04-08T16:41:02Z

Pip version: 10.0.0b2 as well as 10.0.0b2+ (8e07440)
Python version: 2.7.12
Operating system: Ubuntu 16.04

Description:

After installing an editable dependency from vcs, it cannot be uninstalled.
With pip 9 it did uninstall correctly, removing the .egg-link file.

What I've run:

$ virtualenv --version
15.0.1
$ virtualenv venv
$ venv/bin/pip --version  
10.0.0b2  # wondering why I'm getting a beta version of pip without asking but that's not the point here
$ venv/bin/pip install -e git+https://github.com/pallets/click.git#egg=click
Obtaining click from git+https://github.com/pallets/click.git#egg=click
  Updating ./venv/src/click clone
Installing collected packages: click
  Running setup.py develop for click
Successfully installed click
$ venv/bin/pip uninstall click
Can't uninstall 'click'. No files were found to uninstall.
$ venv/bin/pip list
Package       Version  Location                          
------------- -------- ----------------------------------
click         7.0.dev0 /home/sbi-local/tmp/venv/src/click
pip           10.0.0b2 
pkg-resources 0.0.0    
setuptools    39.0.1   
wheel         0.31.0

I expected click to be uninstalled at this point.

If I downgrade to pip 9.0.3, it works:

$ venv/bin/pip install pip==9.0.3
Collecting pip==9.0.3
  Using cached pip-9.0.3-py2.py3-none-any.whl
Installing collected packages: pip
  Found existing installation: pip 10.0.0b2
    Uninstalling pip-10.0.0b2:
      Successfully uninstalled pip-10.0.0b2
Successfully installed pip-9.0.3
$ venv/bin/pip uninstall click
Uninstalling click-7.0.dev0:
  /home/sbi-local/tmp/venv/lib/python2.7/site-packages/click.egg-link
Proceed (y/n)? y
  Successfully uninstalled click-7.0.dev0
$ venv/bin/pip list
DEPRECATION: The default format will switch to columns in the future. You can use --format=(legacy|columns) (or define a format=(legacy|columns) in your pip.conf under the [list] section) to disable this warning.
pip (9.0.3)
pkg-resources (0.0.0)
setuptools (39.0.1)
wheel (0.31.0)

The text was updated successfully, but these errors were encountered:

di · 2018-04-13T13:52:54Z

I can't reproduce this with virtualenv==15.0.1, virtualenv==15.2.0, or pyenv:

$ pip install virtualenv==15.0.1
Collecting virtualenv==15.0.1
  Using cached virtualenv-15.0.1-py2.py3-none-any.whl
Installing collected packages: virtualenv
Successfully installed virtualenv-15.0.1
$ python -m virtualenv env
Using base prefix '/Users/di/.pyenv/versions/3.6.4'
New python executable in /private/tmp/env/bin/python
Installing setuptools, pip, wheel...done.
$ source env/bin/activate
(env) $ pip install -U pip --pre
Requirement already up-to-date: pip in ./env/lib/python3.6/site-packages (10.0.0b2)
(env) $ pip install -e git+https://github.com/pallets/click.git#egg=click
Obtaining click from git+https://github.com/pallets/click.git#egg=click
  Cloning https://github.com/pallets/click.git to ./env/src/click
Installing collected packages: click
  Running setup.py develop for click
Successfully installed click
(env) $ pip uninstall click
Uninstalling click-7.0.dev0:
  Would remove:
    /private/tmp/env/lib/python3.6/site-packages/click.egg-link
Proceed (y/n)? y
  Successfully uninstalled click-7.0.dev0
(env) $ pip freeze

$ pip install virtualenv
Collecting virtualenv
  Using cached virtualenv-15.2.0-py2.py3-none-any.whl
Installing collected packages: virtualenv
Successfully installed virtualenv-15.2.0
$ python -m virtualenv env
Using base prefix '/Users/di/.pyenv/versions/3.6.4'
New python executable in /private/tmp/env/bin/python
Installing setuptools, pip, wheel...done.
$ source env/bin/activate
(env) $ pip install -U pip --pre
Collecting pip
  Using cached pip-10.0.0b2-py2.py3-none-any.whl
Installing collected packages: pip
  Found existing installation: pip 9.0.3
    Uninstalling pip-9.0.3:
      Successfully uninstalled pip-9.0.3
Successfully installed pip-10.0.0b2
(env) $ pip install -e git+https://github.com/pallets/click.git#egg=click
Obtaining click from git+https://github.com/pallets/click.git#egg=click
  Cloning https://github.com/pallets/click.git to ./env/src/click
Installing collected packages: click
  Running setup.py develop for click
Successfully installed click
(env) $ pip uninstall click
Uninstalling click-7.0.dev0:
  Would remove:
    /private/tmp/env/lib/python3.6/site-packages/click.egg-link
Proceed (y/n)? y
  Successfully uninstalled click-7.0.dev0
(env) $ pip freeze

$ python -m venv env
$ source env/bin/activate                                                                                                                                                                   (env) $ pip install -U pip --pre
Collecting pip
  Using cached pip-10.0.0b2-py2.py3-none-any.whl
Installing collected packages: pip
  Found existing installation: pip 9.0.3
    Uninstalling pip-9.0.3:
      Successfully uninstalled pip-9.0.3
Successfully installed pip-10.0.0b2
(env) $ pip install -e git+https://github.com/pallets/click.git#egg=click
Obtaining click from git+https://github.com/pallets/click.git#egg=click
  Cloning https://github.com/pallets/click.git to ./env/src/click
Installing collected packages: click
  Running setup.py develop for click
Successfully installed click
(env) $ pip uninstall click
Uninstalling click-7.0.dev0:
  Would remove:
    /private/tmp/env/lib/python3.6/site-packages/click.egg-link
Proceed (y/n)? y
  Successfully uninstalled click-7.0.dev0
(env) $ pip freeze

sbidoul · 2018-04-13T14:04:19Z

It looks like this issue is python 2 specific.

di · 2018-04-13T14:25:28Z

Still can't reproduce:

$ python2 -m virtualenv env
New python executable in /private/tmp/env/bin/python2
Also creating executable in /private/tmp/env/bin/python
Installing setuptools, pip, wheel...done.
$ source env/bin/activate
(env) /tmp $ virtualenv --version
15.0.1
(env) /tmp $ python --version
Python 2.7.12
(env) $ pip --version
pip 10.0.0b2 from /private/tmp/env/lib/python2.7/site-packages/pip (python 2.7)
(env) $ pip install -e git+https://github.com/pallets/click.git#egg=click
Obtaining click from git+https://github.com/pallets/click.git#egg=click
  Cloning https://github.com/pallets/click.git to ./env/src/click
Installing collected packages: click
  Running setup.py develop for click
Successfully installed click
(env) $ pip uninstall click
Uninstalling click-7.0.dev0:
  Would remove:
    /private/tmp/env/lib/python2.7/site-packages/click.egg-link
Proceed (y/n)? y
  Successfully uninstalled click-7.0.dev0
(env) $ pip freeze

$ python2 -m virtualenv env
New python executable in /private/tmp/env/bin/python2
Also creating executable in /private/tmp/env/bin/python
Installing setuptools, pip, wheel...done.
$ source env/bin/activate
(env) $ virtualenv --version
15.2.0
(env) $ python --version
Python 2.7.12
(env) $ pip --version
pip 10.0.0b2 from /private/tmp/env/lib/python2.7/site-packages/pip (python 2.7)
(env) $ pip install -e git+https://github.com/pallets/click.git#egg=click
Obtaining click from git+https://github.com/pallets/click.git#egg=click
  Cloning https://github.com/pallets/click.git to ./env/src/click
Installing collected packages: click
  Running setup.py develop for click
Successfully installed click
(env) $ pip uninstall click
Uninstalling click-7.0.dev0:
  Would remove:
    /private/tmp/env/lib/python2.7/site-packages/click.egg-link
Proceed (y/n)? y
  Successfully uninstalled click-7.0.dev0
(env) $ pip freeze

sbidoul · 2018-04-13T14:28:16Z

Also, still with pyhon 2.7, when I pip install with --src, I get the following error:

$ pip install -e git+https://github.com/pallets/click.git#egg=click --src src
$ pip uninstall click                                                                                
Not uninstalling click at /home/sbi-local/tmp/src/click, outside environment /home/sbi-local/tmp/venv2
Can't uninstall 'click'. No files were found to uninstall.

No such issue with python 3.

sbidoul · 2018-04-13T14:43:07Z

@di that is weird, I can reproduce consistently with python 2.

I just reproduced my instructions above in a fresh ubuntu 16.04 lxc container:

$ lxc launch ubuntu:16.04 ub
$ lxc exec ub bash
# apt update && apt install python virtualenv
# virtualenv venv
# source venv/bin/activate
# pip install -e git+https://github.com/pallets/click.git#egg=click
# pip uninstall click
Can't uninstall 'click'. No files were found to uninstall. 
# virtualenv --version
15.0.1
# python --version
Python 2.7.12
# pip --version
pip 10.0.0b2 from /root/venv/local/lib/python2.7/site-packages/pip (python 2.7)

mgedmin · 2018-04-13T15:51:07Z

I've also encountered the problem, however in my case the error message is slightly different:

$ pip uninstall pytest-catchlog
Not uninstalling pytest-catchlog at /home/mg/src/pytest-catchlog, outside environment /home/mg/.venv
Can't uninstall 'pytest-catchlog'. No files were found to uninstall.

Steps to reproduce:

$ virtualenv --system-site-packages ~/.venv
$ PATH=~/.venv/bin:$PATH
$ pip install -U --pre pip
$ pip --version
pip 10.0.0b2 from /home/mg/.venv/local/lib/python2.7/site-packages/pip (python 2.7)

$ pip install -e ~/src/pytest-catchlog
Obtaining file:///home/mg/src/pytest-catchlog
...
Installing collected packages: pytest-catchlog
  Running setup.py develop for pytest-catchlog
Successfully installed pytest-catchlog

$ pip uninstall pytest-catchlog
Not uninstalling pytest-catchlog at /home/mg/src/pytest-catchlog, outside environment /home/mg/.venv
Can't uninstall 'pytest-catchlog'. No files were found to uninstall.

$ pip list|grep pytest-catchlog
pytest-catchlog               1.2.2        /home/mg/src/pytest-catchlog

If I downgrade pip to version 9.0.3, I can pip uninstall editable packages in the virtualenv

$ pip install 'pip<10'
...
Successfully installed pip-9.0.3

You are using pip version 9.0.3, however version 10.0.0b2 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.

$ $ pip uninstall pytest-catchlog
Uninstalling pytest-catchlog-1.2.2:
  /home/mg/.venv/lib/python2.7/site-packages/pytest-catchlog.egg-link
Proceed (y/n)? y
  Successfully uninstalled pytest-catchlog-1.2.2

(~/src/pytest-catchlog is a checkout of https://github.com/eisensheng/pytest-catchlog)

di · 2018-04-13T15:53:17Z

@mgedmin What platform is this? What version of virtualenv did you use?

mgedmin · 2018-04-13T15:56:07Z

I'm on Ubuntu 17.10. I cannot say for sure which virtualenv version I used: ~/.venv/ was created a long time ago (Sep 10 2016, if I can trust the timestamp of the 'python2.7' symlink in ~/.venv/bin).

mgedmin · 2018-04-13T15:58:34Z

I can reproduce using the latest virtualenv:

mg@platonas: ~ $ pip install -U virtualenv
Requirement already up-to-date: virtualenv in ./.venv/lib/python2.7/site-packages (15.2.0)

mg@platonas: ~ $ rm -rf /tmp/sandbox/; virtualenv -p /usr/bin/python2.7 /tmp/sandbox
Running virtualenv with interpreter /usr/bin/python2.7
New python executable in /tmp/sandbox/bin/python2.7
Also creating executable in /tmp/sandbox/bin/python
Installing setuptools, pip, wheel...done.

mg@platonas: ~ $ /tmp/sandbox/bin/pip install -U pip
Requirement already up-to-date: pip in /tmp/sandbox/lib/python2.7/site-packages

mg@platonas: ~ $ /tmp/sandbox/bin/pip --version
pip 9.0.3 from /tmp/sandbox/local/lib/python2.7/site-packages (python 2.7)

mg@platonas: ~ $ /tmp/sandbox/bin/pip install -U pip --pre
Collecting pip
  Using cached pip-10.0.0b2-py2.py3-none-any.whl
Installing collected packages: pip
  Found existing installation: pip 9.0.3
    Uninstalling pip-9.0.3:
      Successfully uninstalled pip-9.0.3
Successfully installed pip-10.0.0b2

mg@platonas: ~ $ /tmp/sandbox/bin/pip install -e ~/src/pytest-catchlog/
Obtaining file:///home/mg/src/pytest-catchlog
Collecting py>=1.1.1 (from pytest-catchlog==1.2.2)
  Using cached py-1.5.3-py2.py3-none-any.whl
Collecting pytest>=2.6 (from pytest-catchlog==1.2.2)
  Using cached pytest-3.5.0-py2.py3-none-any.whl
Requirement already satisfied: setuptools in /tmp/sandbox/lib/python2.7/site-packages (from pytest>=2.6->pytest-catchlog==1.2.2) (39.0.1)
Collecting more-itertools>=4.0.0 (from pytest>=2.6->pytest-catchlog==1.2.2)
  Using cached more_itertools-4.1.0-py2-none-any.whl
Collecting funcsigs; python_version < "3.0" (from pytest>=2.6->pytest-catchlog==1.2.2)
  Using cached funcsigs-1.0.2-py2.py3-none-any.whl
Collecting six>=1.10.0 (from pytest>=2.6->pytest-catchlog==1.2.2)
  Using cached six-1.11.0-py2.py3-none-any.whl
Collecting attrs>=17.4.0 (from pytest>=2.6->pytest-catchlog==1.2.2)
  Using cached attrs-17.4.0-py2.py3-none-any.whl
Collecting pluggy<0.7,>=0.5 (from pytest>=2.6->pytest-catchlog==1.2.2)
Installing collected packages: py, six, more-itertools, funcsigs, attrs, pluggy, pytest, pytest-catchlog
  The scripts py.test and pytest are installed in '/tmp/sandbox/bin' which is not on PATH.
  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
  Running setup.py develop for pytest-catchlog
Successfully installed attrs-17.4.0 funcsigs-1.0.2 more-itertools-4.1.0 pluggy-0.6.0 py-1.5.3 pytest-3.5.0 pytest-catchlog six-1.11.0

mg@platonas: ~ $ /tmp/sandbox/bin/pip uninstall pytest-catchlog
Not uninstalling pytest-catchlog at /home/mg/src/pytest-catchlog, outside environment /tmp/sandbox
Can't uninstall 'pytest-catchlog'. No files were found to uninstall.

mgedmin · 2018-04-13T16:20:38Z

I could reproduce this in a fresh clean Docker container: https://gist.github.com/mgedmin/0d5f8cadac30270a997c3a7b3bd69bf6

mgedmin · 2018-04-13T16:24:30Z

It looks like this issue is python 2 specific.

I too was unable to reproduce this using Python 3.

ryan-rs · 2018-04-19T20:08:26Z

I can reproduce this issue under Ubuntu 16.04 with Python 2.7.12 and virtualenv 15.2.0. After downgrading to pip 9.0.3 I can once again uninstall editable packages.

tysonclugg · 2018-05-02T07:12:56Z

I've created a PR for pip-tools to show this issue on Python 2.7 / Ubuntu 16. Run tox -e py27 from that PR on Ubuntu to reproduce.

bucknerns · 2018-05-03T21:43:26Z

I am unable to get the first bug to show up however the one mentioned by @sbidoul consistently breaks on python2.
@di you are failing to reproduce because when you do the pip install -e git+https://... it clones the repo in the virtualenv folder under src, try using the --src flag.

asottile · 2018-05-13T19:56:25Z

Well, consider myself nerdsniped 😆

I attempted to bisect this, hope this is helpful :)

#!/usr/bin/env bash
set -euxo pipefail

export PIP_DISABLE_PIP_VERSION_CHECK=1 VIRTUALENV_NO_DOWNLOAD=1

finish() {
    git checkout -- .
    git clean -fxfd
}
trap finish ERR EXIT

# hotfix for `AttributeError: 'module' object has no attribute 'get_python_lib'
# cherry-picked from 310bcfc78fea8f1f2bc4f680b376e18bf5aeed07
if \
    grep -q '^import sysconfig$' pip/locations.py && \
    grep -q 'site_packages = sysconfig.get_python_lib()$' pip/locations.py; then
    sed -i 's/sysconfig.get_python_lib()/sysconfig.get_path("purelib")/g' pip/locations.py
fi

mkdir tdir
(
    cd tdir

    echo 'from setuptools import setup; setup(name="pkg")' > setup.py
    virtualenv venv -ppython2 > /dev/null
    venv/bin/pip install ../ > /dev/null || exit 125
    venv/bin/pip install -e . > /dev/null || exit 125
    ! (venv/bin/pip uninstall -y pkg | grep 'Not uninstalling pkg')
)

I had to apply the patch listed in 310bcfc early so that the bisect was able to differentiate between a bunch of "broken" commits which ended with:

+ venv/bin/pip install -e .
Traceback (most recent call last):
  File "venv/bin/pip", line 7, in <module>
    from pip import main
  File "/tmp/test/pip/tdir/venv/local/lib/python2.7/site-packages/pip/__init__.py", line 26, in <module>
    from pip.utils import get_installed_distributions, get_prog
  File "/tmp/test/pip/tdir/venv/local/lib/python2.7/site-packages/pip/utils/__init__.py", line 23, in <module>
    from pip.locations import (
  File "/tmp/test/pip/tdir/venv/local/lib/python2.7/site-packages/pip/locations.py", line 83, in <module>
    site_packages = sysconfig.get_python_lib()
AttributeError: 'module' object has no attribute 'get_python_lib'

git bisect start
git bisect good 9.0.3
git bisect bad HEAD
git bisect run ../bisect.sh

After the bisect run, it points at this:

904fcf1f177770a4eb05e0850d79265e4bc0907f is the first bad commit
commit 904fcf1f177770a4eb05e0850d79265e4bc0907f
Author: Donald Stufft <[email protected]>
Date:   Sat Mar 18 13:32:10 2017 -0400

    Use sysconfig instead of distutils.sysconfig

:040000 040000 dfd417efac444008f5ca7e07f85de83168e0e6de 710942f93e7ce4df8553c8534b515a60aa2feb39 M	pip
bisect run success

CC @dstufft 904fcf1

A further triage, the failure appears to come from a differing return value in dist_is_local here:

pip/src/pip/_internal/req/req_uninstall.py

Line 276 in d67d98d

if not dist_is_local(dist):

The difference appears to be the return values here:

$ venv/bin/python -c 'import sysconfig; print(sysconfig.get_path("purelib"))'
/tmp/test/venv/local/lib/python2.7/dist-packages

$ venv3/bin/python -c 'import sysconfig; print(sysconfig.get_path("purelib"))'
/tmp/test/venv3/lib/python3.5/site-packages

marc1n · 2018-08-09T15:05:26Z

Is there any solution for this problem?

(test_env) $ cd [HOME]/test_package
(test_env) $ pip install -e .
Installing collected packages: test_package
  Running setup.py develop for test_package
Successfully installed test_package

(test_env) $ pip uninstall -y test_package
Not uninstalling test_package at [HOME]/test_package, outside environment [HOME]/env/test_env
Can't uninstall 'test_package'. No files were found to uninstall.

(test_dev) $ pip --version
pip 18.0 from [HOME]/env/test_dev/local/lib/python2.7/site-packages/pip (python 2.7)

cjerdonek · 2018-08-10T06:35:16Z

@asottile So if sysconfig.get_path() is the culprit, might this mean there's a bug in sysconfig.get_path()'s implementation / return value? Can you provide steps to create a virtualenv where sysconfig.get_path() differs from distutils.sysconfig's -- is that the source of the problem? The example you gave at the end of your last comment shows the return values only for sysconfig.get_path() -- and for different virtualenvs (venv and venv3) rather than the same one -- so it's not clear to me what the difference in return values in that comment is meant to show..

asottile · 2018-08-10T07:44:56Z

@cjerdonek the key difference in the two outputs is in python2 the debianinzed dist-packages is leaking through (it's not being a real directory in virtualenvs, dist-packages is only a thing at the system level).

cjerdonek · 2018-08-10T08:09:25Z

Okay, so in other words, the following line works / gives a different answer if you substitute the distutils version?

$ venv/bin/python -c 'import sysconfig; print(sysconfig.get_path("purelib"))'

asottile · 2018-08-10T08:27:23Z

Here's the eight values:

FROM debian:stretch
RUN : \
    && apt-get update \
    && DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends \
        dumb-init \
        python \
        python3 \
        virtualenv \
    && apt-get clean \
    && rm -rf /var/lib/apt/lists
RUN : \
    && virtualenv /venv2 -ppython2.7 \
    && virtualenv /venv3 -ppython3
RUN /bin/echo -e '#!/usr/bin/env bash \n\
set -euxo pipefail \n\
/usr/bin/python -c "from distutils import sysconfig; print(sysconfig.get_python_lib())" \n\
/usr/bin/python -c "import sysconfig; print(sysconfig.get_path('"'"'purelib'"'"'))" \n\
/usr/bin/python3 -c "from distutils import sysconfig; print(sysconfig.get_python_lib())" \n\
/usr/bin/python3 -c "import sysconfig; print(sysconfig.get_path('"'"'purelib'"'"'))" \n\
/venv2/bin/python -c "from distutils import sysconfig; print(sysconfig.get_python_lib())" \n\
/venv2/bin/python -c "import sysconfig; print(sysconfig.get_path('"'"'purelib'"'"'))" \n\
/venv3/bin/python -c "from distutils import sysconfig; print(sysconfig.get_python_lib())" \n\
/venv3/bin/python -c "import sysconfig; print(sysconfig.get_path('"'"'purelib'"'"'))" \n\
' > t.sh
CMD ["dumb-init", "bash", "t.sh"]

$ docker build -q -t test . && docker run --rm -ti test
sha256:b10eb0cb39a9a9e9c641a7496e1447047b850852f1573038d3fc43280a5961ef
+ /usr/bin/python -c 'from distutils import sysconfig; print(sysconfig.get_python_lib())'
/usr/lib/python2.7/dist-packages
+ /usr/bin/python -c 'import sysconfig; print(sysconfig.get_path('\''purelib'\''))'
/usr/local/lib/python2.7/dist-packages
+ /usr/bin/python3 -c 'from distutils import sysconfig; print(sysconfig.get_python_lib())'
/usr/lib/python3/dist-packages
+ /usr/bin/python3 -c 'import sysconfig; print(sysconfig.get_path('\''purelib'\''))'
/usr/lib/python3.5/site-packages
+ /venv2/bin/python -c 'from distutils import sysconfig; print(sysconfig.get_python_lib())'
/venv2/lib/python2.7/site-packages
+ /venv2/bin/python -c 'import sysconfig; print(sysconfig.get_path('\''purelib'\''))'
/venv2/local/lib/python2.7/dist-packages
+ /venv3/bin/python -c 'from distutils import sysconfig; print(sysconfig.get_python_lib())'
/venv3/lib/python3.5/site-packages
+ /venv3/bin/python -c 'import sysconfig; print(sysconfig.get_path('\''purelib'\''))'
/venv3/lib/python3.5/site-packages

cjerdonek · 2018-08-10T10:55:51Z

Thanks. It looks like not just one but three of the four pairs have different return values (and for different reasons), with only the last the same. So does this mean that one (or perhaps both) of the two functions sysconfig.get_path() and distutils.sysconfig.get_python_lib() has a bug, or are they not supposed to behave the same?

asottile · 2018-08-10T12:59:42Z

Debian system python installs a symlink to make this work there

…

On Fri, Aug 10, 2018, 3:56 AM Chris Jerdonek ***@***.***> wrote: Thanks. It looks like not just one but three of the four pairs have different return values (and for different reasons), with only the last the same. So does this mean that one (or perhaps both) of the two functions sysconfig.get_path() and distutils.sysconfig.get_python_lib() has a bug, or are they not supposed to behave the same? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#5193 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABugn7x9otsHDQrC3p8ThqiFW1wWqKKOks5uPWbHgaJpZM4TLm1R> .

cjerdonek · 2018-08-10T17:48:11Z

My question was whether the two functions are always supposed to have the same return value.

asottile · 2018-08-10T17:57:30Z

unclear to me, but one was swapped for the other in 904fcf1 so that was at least assumed by @dstufft

pfmoore · 2018-08-10T18:10:23Z

I would imagine that they are intended to give the same result (I think sysconfig was added to the stdlib to pull certain generally-useful bits out of distutils into a more generic location). But Debian's messing round with system paths always ends up causing us problems. In this case, it may be that the amount of Debian specific special-casing already in virtualenv isn't sufficient to catch this case.

sbidoul · 2019-08-25T07:12:48Z

I propose a workaround in #6918 (a similar workaround is already in place for pypy).

pradyunsg · 2020-05-23T12:27:53Z

@kitterma This seems to be an issue introduced by Debian's patch for dist-packages. It would be wonderful if you could look into introducing a fix in Debian. :)

kitterma · 2020-05-23T13:53:32Z

I took a look a this on Debian stable as well as unstable/testing. Click is now python3 only so the original problem with that repository can't occur. I tried the reproduction steps on python3 and it uninstalled fine.

I don't know how much effort it's worth fixing python2 stuff now, but if someone will provide another example where the problem still occurs I will take a look at it.

asottile · 2020-05-23T15:53:09Z

@pradyunsg this was caused by a change in pip fwiw: #5193 (comment)

uranusjr · 2020-05-23T18:15:17Z

I think the main issue is why a platform incorrectly returns dist-packages in sysconfig though. Is it patched? Why is it patched? Should the patch be fixed instead?

asottile · 2020-05-23T18:20:08Z

🤷 it used to work though is my point

kitterma · 2020-05-23T18:24:09Z

As I said earlier, if someone can give me a current example showing the problem, I'll look into it.

sbidoul · 2020-05-23T18:30:55Z

@kitterma this should reproduce on buster.

$ mkdir editabletest
$ cd editabletest
$ echo "import setuptools; setuptools.setup(name='editabletest')" > setup.py
$ virtualenv -p python2 venv
$ venv/bin/pip install -e .
$ venv/bin/pip uninstall editabletest
...
Not uninstalling editabletest at /root/editabletest, outside environment /root/editabletest/venv
...

uranusjr · 2020-05-23T18:42:06Z

I thought about this a bit more. Python has two ways to get information on where to install stuff: distutils.sysconfig (legacy), and sysconfig (modern). The two usually should match, but may not be if one of them is incorrectly patched in an environment.

pip should be enforcing the modern scheme when it installs stuff, but can likely be more lenient when uninstalling. It could be possible for the uninstallation logic to try figure out which scheme a package is installed with, and use the right one automatically. This is worthwhilte to investigate IMO since it is still quite common for packages to be installed with the legacy scheme, either by an old pip (9.x is still in active use!) or an installation method that has not been modernised (e.g. editable).

I’ll probably take a deeper look at the implementation on Sunday if nobody else figures this out before that.

kitterma · 2020-05-23T18:47:07Z

@kitterma this should reproduce on buster.

$ mkdir editabletest
$ cd editabletest
$ echo "import setuptools; setuptools.setup(name='editabletest')" > setup.py
$ virtualenv -p python2 venv
$ venv/bin/pip install -e .
$ venv/bin/pip uninstall editabletest
...
Not uninstalling editabletest at /root/editabletest, outside environment /root/editabletest/venv
...

Thanks. Same thing happens on Unstable/Testing with python2.7. With python3.8 it works fine.

kitterma · 2020-05-23T18:59:32Z

Looking into this a bit more. On Debian Unstable, inside a python2 based virtualenv I see:

>>> import distutils.sysconfig
>>> print distutils.sysconfig.get_python_lib()
/$HOMEDIR/editabletest/venv/lib/python2.7/site-packages
>>> import sysconfig
>>> print sysconfig.get_path('purelib')
/$HOMEDIR/editabletest/venv/local/lib/python2.7/dist-packages

In a python3 virtualenv it's different:

>>> import distutils.sysconfig
>>> print(distutils.sysconfig.get_python_lib())
/$HOMEDIR/editabletest/venv3/lib/python3.8/site-packages
>>> import sysconfig
>>> print(sysconfig.get_path('purelib'))
/$HOMEDIR/editabletest/venv3/lib/python3.8/site-packages

So it's pretty clear why python3 works and python2 doesn't. Even though this may have be precipitated by a pip change, I think downstream is the correct resolution since it's connected to a Debian specific Python interpreter change (which isn't going to go away).

uranusjr · 2020-05-24T00:06:00Z

Just to confirm, has anyone in this thread tried to file an issue to Debian?

kitterma · 2020-05-24T00:12:33Z

On Saturday, May 23, 2020 8:06:13 PM EDT Tzu-ping Chung wrote: Just to confirm, has anyone in this thread tried to file an issue to Debian?

Since I'm the primary maintainer for virtualenv and pip in Debian right now, it wouldn't make me any more aware of it than I am now. Also, don't assume the previous discussion is right. I tried a patch very similar to the linked pull request and it didn't solve the problem. That may be on me for either not getting the change right or not testing correctly, but I'm not 100% sure the previous discussion is on the right path.

uranusjr · 2020-05-24T00:21:20Z

I see, thanks @kitterma.

sbidoul · 2020-05-24T11:35:13Z

@kitterma thanks for your interest in this issue.
I rebased #6918 and I confirm it still fixes it:

$ mkdir editabletest
$ cd editabletest
$ echo "import setuptools; setuptools.setup(name='editabletest')" > setup.py
$ virtualenv -p python2 venv
$ venv/bin/pip install https://github.com/sbidoul/pip/archive/editable-uninstall-2.7-sbi.zip
$ venv/bin/pip install -e .
$ venv/bin/pip uninstall editabletest
Found existing installation: editabletest 0.0.0
Uninstalling editabletest-0.0.0:
  Would remove:
    /.../editabletest/venv/lib/python2.7/site-packages/editabletest.egg-link
Proceed (y/n)? y
  Successfully uninstalled editabletest-0.0.0

kitterma · 2020-05-24T16:31:19Z

I can confirm it too now. I'm not sure what I was doing wrong before. Thanks.

I will include this in the next pip upload for Debian Unstable, which means it will support Debian Bullseye. Since the Buster (Debian 10) version of virtualenv always uses the latest version of pip from pypi, I don't think there's any benefit to me backporting the patch (please let me know if that's wrong).

rousseldenis · 2020-09-04T15:55:46Z

I can confirm I have the bug on python2.7.

What's the status of this ? @kitterma

hexagonrecursion · 2021-02-10T06:41:03Z

I can confirm I have the bug on python2.7.

What's the status of this ? @kitterma

This issue is marked as "python 2 only". pip 21.0 dropped support for Python 2. Should this be closed?

sbidoul mentioned this issue Apr 13, 2018

Plan for 10.0 #4981

Closed

tysonclugg mentioned this issue May 2, 2018

WIP: Can't uninstall editable packages with Python 2.7 jazzband/pip-tools#659

Closed

3 tasks

pradyunsg added the S: needs triage Issues/PRs that need to be triaged label May 11, 2018

sbidoul mentioned this issue May 16, 2018

Install via pip - how to proceed when a module already exists? OCA/ansible-odoo#88

Open

cjerdonek added the C: editable Editable installations label Aug 10, 2018

sbidoul changed the title ~~[10.0.0beta] pip uninstall not working with editable dependencies~~ pip uninstall not working with editable dependencies Aug 25, 2019

chrahunt mentioned this issue Oct 28, 2019

Can't uninstall package installed with sudo pip install -e - "No files were found to uninstall." #5330

Closed

sbidoul added the Python 2 only Python 2 specific label Apr 13, 2020

triage-new-issues bot removed the S: needs triage Issues/PRs that need to be triaged label Apr 13, 2020

pradyunsg added project: <downstream> When the cause/effect is related to redistributors and removed OS: linux Linux specific labels May 23, 2020

atugushev mentioned this issue Jun 13, 2020

pip-sync uninstalls main project from virtualenv jazzband/pip-tools#1158

Open

sbidoul mentioned this issue Aug 5, 2020

Editable installs do not satisfy dependencies when using pip>=20.2 and Python 2.7 #8715

Closed

uranusjr mentioned this issue Aug 14, 2020

Unable to upgrade setuptools with adopted distutils #8761

Closed

alanbriolat mentioned this issue Sep 29, 2020

Existing editable installs do not satisfy dependencies when using 2020-resolver #8939

Closed

uranusjr closed this as completed Feb 10, 2021

github-actions bot locked as resolved and limited conversation to collaborators Oct 3, 2021

pip uninstall not working with editable dependencies #5193

pip uninstall not working with editable dependencies #5193

Comments

sbidoul commented Apr 8, 2018 • edited Loading

Description:

What I've run:

di commented Apr 13, 2018

sbidoul commented Apr 13, 2018

di commented Apr 13, 2018

sbidoul commented Apr 13, 2018

sbidoul commented Apr 13, 2018

mgedmin commented Apr 13, 2018 • edited Loading

di commented Apr 13, 2018

mgedmin commented Apr 13, 2018

mgedmin commented Apr 13, 2018 • edited Loading

mgedmin commented Apr 13, 2018

mgedmin commented Apr 13, 2018

ryan-rs commented Apr 19, 2018

tysonclugg commented May 2, 2018 • edited Loading

bucknerns commented May 3, 2018

asottile commented May 13, 2018

marc1n commented Aug 9, 2018 • edited Loading

cjerdonek commented Aug 10, 2018

asottile commented Aug 10, 2018

cjerdonek commented Aug 10, 2018

asottile commented Aug 10, 2018

cjerdonek commented Aug 10, 2018

asottile commented Aug 10, 2018 via email

cjerdonek commented Aug 10, 2018

asottile commented Aug 10, 2018

pfmoore commented Aug 10, 2018

sbidoul commented Aug 25, 2019

pradyunsg commented May 23, 2020

kitterma commented May 23, 2020

asottile commented May 23, 2020

uranusjr commented May 23, 2020 • edited Loading

asottile commented May 23, 2020

kitterma commented May 23, 2020 via email

sbidoul commented May 23, 2020

uranusjr commented May 23, 2020

kitterma commented May 23, 2020

kitterma commented May 23, 2020

uranusjr commented May 24, 2020

kitterma commented May 24, 2020 via email

uranusjr commented May 24, 2020

sbidoul commented May 24, 2020

kitterma commented May 24, 2020

rousseldenis commented Sep 4, 2020

hexagonrecursion commented Feb 10, 2021

sbidoul commented Apr 8, 2018 •

edited

Loading

mgedmin commented Apr 13, 2018 •

edited

Loading

mgedmin commented Apr 13, 2018 •

edited

Loading

tysonclugg commented May 2, 2018 •

edited

Loading

marc1n commented Aug 9, 2018 •

edited

Loading

uranusjr commented May 23, 2020 •

edited

Loading