[issue 1548] Make pip install -e uninstall existing versions #1552

msabramo · 2014-02-10T15:39:02Z

by adding an additional call to requirement.check_if_exists() to
RequirementSet.install.

Example output with this PR:

$ virtualenv pip_test
...

$ cd pip_test && source bin/activate

$ pip install requests==0.5.0
Downloading/unpacking requests==0.5.0
  Using download cache from /Users/marca/.pip/download-cache/http%3A%2F%2Flocalhost%3A3141%2Froot%2Fpypi%2F%2Bf%2F6dfdc1688217d774d524e056ec6605a6%2Frequests-0.5.0.tar.gz
  Running setup.py egg_info for package requests

Installing collected packages: requests
  Running setup.py install for requests

Successfully installed requests
Cleaning up...

$ python -c 'import requests; print(requests); print(requests.__version__)'
<module 'requests' from '/private/tmp/pip_test/lib/python2.7/site-packages/requests/__init__.pyc'>
0.5.0

$ pip install -e '[email protected]:msabramo/pip.git@issue_1548_make_pip_install_editable_uninstall_existing_versions#egg=pip'
Obtaining pip from [email protected]:msabramo/pip.git@issue_1548_make_pip_install_editable_uninstall_existing_versions#egg=pip
...
Successfully installed pip
Cleaning up...

$ pip install -e ~/dev/git-repos/requests/
Obtaining file:///Users/marca/dev/git-repos/requests
  Running setup.py (path:/Users/marca/dev/git-repos/requests/setup.py) egg_info for package from file:///Users/marca/dev/git-repos/requests

Installing collected packages: requests
  Found existing installation: requests 0.5.0
    Uninstalling requests:
      Successfully uninstalled requests
  Running setup.py develop for requests

    Creating /private/tmp/pip_test/lib/python2.7/site-packages/requests.egg-link (link to .)
    Adding requests 2.0.1 to easy-install.pth file

    Installed /Users/marca/dev/git-repos/requests
Successfully installed requests
Cleaning up...

$ python -c 'import requests; print(requests); print(requests.__version__)'
<module 'requests' from '/Users/marca/dev/git-repos/requests/requests/__init__.pyc'>
2.0.1

$ pip uninstall --verbose --yes requests
Uninstalling requests:
  Removing file or directory /private/tmp/pip_test/lib/python2.7/site-packages/requests.egg-link
  Removing pth entries from /private/tmp/pip_test/lib/python2.7/site-packages/easy-install.pth:
  Removing entry: /Users/marca/dev/git-repos/requests
  Successfully uninstalled requests

$ pip uninstall --verbose --yes requests
Cannot uninstall requirement requests, not installed
Exception information:
Traceback (most recent call last):
  File "/private/tmp/pip_test/src/pip/pip/basecommand.py", line 129, in main
    status = self.run(options, args)
  File "/private/tmp/pip_test/src/pip/pip/commands/uninstall.py", line 64, in run
    requirement_set.uninstall(auto_confirm=options.yes)
  File "/private/tmp/pip_test/src/pip/pip/req/req_set.py", line 147, in uninstall
    req.uninstall(auto_confirm=auto_confirm)
  File "/private/tmp/pip_test/src/pip/pip/req/req_install.py", line 531, in uninstall
    "Cannot uninstall requirement %s, not installed" % (self.name,)
UninstallationError: Cannot uninstall requirement requests, not installed

Note that when I do pip install -e on requests above, the previous version gets found and uninstalled.

$ pip install -e ~/dev/git-repos/requests/
Obtaining file:///Users/marca/dev/git-repos/requests
  Running setup.py (path:/Users/marca/dev/git-repos/requests/setup.py) egg_info for package from file:///Users/marca/dev/git-repos/requests

Installing collected packages: requests
↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓
  Found existing installation: requests 0.5.0
    Uninstalling requests:
      Successfully uninstalled requests
↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑
  Running setup.py develop for requests

Cc: @sontek, @aconrad, @whitmo

by adding an additional call to `requirement.check_if_exists()` to RequirementSet.install Fixes pypaGH-1548

msabramo · 2014-02-10T22:10:16Z

I just amended this because with my previous commit, it would not uninstall the previous version if you installed the new editable with a git URL -- e.g.: $ pip install -e 'git+https://github.com/kennethreitz/requests.git#egg=requests'. I've fixed this by adding a few more lines to check_if_exists.

msabramo · 2014-02-11T16:25:32Z

Ugh. The tests pretty much passed -- see https://travis-ci.org/pypa/pip/builds/18658167

The only problem is that the pypy build timed out with:

I'm sorry but your test run exceeded 50.0 minutes.

qwcode · 2014-02-11T17:16:41Z

just noting that I see this. may not get the time to look at it til this weekend.
I'll want to try to dig into the history and find any gotcha's for this.

msabramo · 2014-02-11T18:09:07Z

Cool, thanks @qwcode! Let me know if you end up wanting tweaks, tests, squashing and rebasing, etc...

msabramo · 2014-02-16T04:34:25Z

@qwcode: Have you had a chance to look at this?

qwcode · 2014-02-17T03:55:03Z

pip/req/req_install.py

@@ -876,6 +876,10 @@ def check_if_exists(self):
                return True
            else:
                self.satisfied_by = pkg_resources.get_distribution(self.req)
+
+            if self.editable and self.satisfied_by:
+                self.conflicts_with = self.satisfied_by


I think I'd like to set self.satisfied_by back to None after this.
and add a comment that says something like "when installing editables, nothing pre-existing should ever satisfy"

OK, see 5b62075. Thanks!

msabramo · 2014-02-17T20:50:40Z

@qwcode: Thanks for the review. I'm AFK for a few days, but I'll try to try out the ideas you have here on Friday or so.

@qwcode

Add a comment that says something like "when installing editables, nothing pre-existing should ever satisfy". Addresses @qwcode's comment at pypa#1552 (comment)

@qwcode

Addresses @qwcode's comment at pypa#1552 (comment)

msabramo · 2014-02-23T04:02:56Z

@qwcode: Tried both of your suggestions and they seem to work -- see 5b62075 and 36cffa0.

If you think that this is a sound change to make, we should probably write a test for this. Let me know -- I can try to take a stab at it.

I'd love to have this change in pip -- I get a question probably at least twice a month that ends up with me telling someone to pip uninstall the offending library twice, because they got themselves into this state.

msabramo · 2014-02-23T04:07:54Z

Cc: @rlgomes

msabramo · 2014-02-23T20:01:38Z

Cc: @sontek, @aconrad, @whitmo

aconrad · 2014-02-23T20:20:58Z

I like it.! It would be nice indeed to not uninstall packages twice but I've been doing it for so long now, it's ingrained deep into my muscle memory.

whitmo · 2014-02-24T15:39:10Z

+1

On Sun, Feb 23, 2014 at 2:21 PM, Alexandre Conrad
[email protected]:

I like it.! It would be nice indeed to not uninstall packages twice but
I've been doing it for so long now, it's ingrained deep into my muscle
memory.

—
Reply to this email directly or view it on GitHubhttps://github.com//pull/1552#issuecomment-35842043
.

<=>
david "whit" morriss

"If you don't know where you are,
you don't know anything at all"

Dr. Edgar Spencer, Ph.D., 1995

msabramo · 2014-02-24T16:54:27Z

Cc: @KaboomFox

qwcode · 2014-02-26T07:22:42Z

@msabramo can you add a test to tests/functional/test_install.py

for your editable/non-editable pair, you can use:

-e https://github.com/pypa/[email protected]#egg=pip-test-package
tests/data/packages/pip-test-package-0.1.tar.gz

qwcode · 2014-02-26T07:28:17Z

@carljm any historical memories on why editable installs don't uninstall previously existing non-editables? this PR would change that.

Ivoz · 2014-02-26T10:54:30Z

I think warning that a package of the same name is already installed and stopping would be a less intrusive but still useful change.

carljm · 2014-02-26T15:04:51Z

I think the only historical reason was that the presence (and accuracy) of the #egg= fragment on editable URLs was not considered reliable, and that's the only clue pip has as to what ought to be uninstalled. I haven't dug into this PR at all, but I would assume it can only work if the #egg= fragment is present and correct (i.e. matches the installed name of the package). This may be just fine, but it should probably be noted in docs somewhere.

carljm · 2014-02-26T15:06:17Z

(Actually I may be wrong, maybe pip already has access to the setup.py metadata from the cloned repo by this point and can use that rather than relying on #egg=. If that's the case ignore my comment - in that case the only reason this was never done is because nobody ever did it :P)

qwcode · 2014-02-26T17:29:58Z

currently, it is dependent on the correct #egg fragment. better if it wasn't

qwcode · 2014-02-27T06:46:29Z

warning that a package of the same name is already installed and stopping
would be a less intrusive but still useful change

I don't see this as intrusive, just what pip should be doing in the first place, to be consistent with other installs, although, I would prefer it not to be based on the egg fragment.

msabramo · 2014-03-02T01:45:54Z

Maybe a silly question, but why does pip require the #egg= fragment at all? Could it always get the name from the setup.py metadata. I'm guessing not - curious why.

qwcode · 2014-03-02T02:28:48Z

but why does pip require the #egg= fragment at all

I suspect it doesn't really, if we made modifications, but see #1289 for discussion.

msabramo · 2014-03-02T12:33:45Z

@qwcode:

@msabramo can you add a test to tests/functional/test_install.py

See 3e0d727, which adds a new test called test_install_editable_uninstalls_existing

$ .tox/py27/bin/py.test --pdb -v -k test_install_editable_uninstalls_existing
============================================================================= test session starts ==============================================================================
platform darwin -- Python 2.7.6 -- py-1.4.20 -- pytest-2.5.2 -- /Users/marca/dev/git-repos/pip/.tox/py27/bin/python
collected 413 items

tests/functional/test_install.py:231: test_install_editable_uninstalls_existing PASSED

==================================================== 412 tests deselected by '-ktest_install_editable_uninstalls_existing' =====================================================
================================================================== 1 passed, 412 deselected in 14.51 seconds ===================================================================

Question: Why do other tests in test_install.py that install editables specify expect_error=True? Should I do the same in my test even though I don't see why it would have an error?

msabramo · 2014-12-20T00:43:40Z

@dstufft, @pfmoore: I wonder if this is something that could be squeezed into pip 6?

It irks me whenever I have to tell people to pip uninstall twice to make sure both versions are gone (or when I have to do it myself - I did it earlier today in fact).

pfmoore · 2014-12-20T10:46:58Z

I don't really have much opinion on it myself. I'm not against it going in, but I've not looked at it closely enough that I'd be willing to merge it myself.

One thought. All other installs require -U or -I to uninstall an existing version. Why are we allowing -e to do so without those flags? Does pip install -I -e already do what this patch does, or is it broken?

aconrad · 2014-12-20T16:14:10Z

This PR makes total sense to me, so +1 👍

-U or -I shouldn't be necessary since triggering the uninstall of a package doesn't necessarily require these options to be present with the pip install command. When pinning a different version requirement, pip already uninstalls the previous package without -I or -U:

$ pip install requests==2.5.0
Downloading/unpacking requests==2.5.0
...
Installing collected packages: requests
  Found existing installation: requests 0.5.0
    Uninstalling requests:
      Successfully uninstalled requests
...

When pip install is called with -e ~/dev/git-repos/requests/, it semantically means pinning a different version requirement, like if I had done pip install requests==~/dev/git-repos/requests/.

This is why I think this PR makes absolute sense and that it should be merged.

qwcode · 2014-12-20T17:10:26Z

To say again, the hold on this from me at the time wasn't about whether the idea in general makes sense. It does make sense.

The concern was that for vcs installs, it would work based solely on the egg= fragment being correct, which is not correct in many cases, so releasing this could end up being confusing.

But, otoh, incremental improvement is also a good argument.

at least, add something to the vcs docs explaining the situation.

https://pip.pypa.io/en/latest/reference/pip_install.html#vcs-support

aconrad · 2014-12-20T18:49:30Z

A doc update? Love it! :)

@msabramo would you mind adding some clarification in the docs about the expectations with regards to the egg= fragment? We can then later figure out how to rather use the metadata from the cloned repo and drop the egg syntax all together (if that makes sense).

msabramo · 2014-12-20T19:05:28Z

Yeah I can take a stab at docs later

msabramo · 2014-12-20T19:24:41Z

I'm AFK today so if anyone feels like taking a stab at docs, be my guest.

qwcode · 2015-03-07T23:10:45Z

@msabramo reminder, I'm willing to merge with docs updates

msabramo · 2015-03-07T23:18:51Z

Thanks it indeed fell off my radar. Will try to update with docs this week.

qwcode · 2015-03-08T00:54:48Z

tests/functional/test_install.py

+    result.assert_installed('pip-test-package', with_files=['.git'])
+    assert 'Found existing installation: pip-test-package 0.1' in result.stdout
+    assert 'Uninstalling pip-test-package:' in result.stdout
+    assert 'Successfully uninstalled pip-test-package' in result.stdout


also, would be better if you could confirm the package is removed from site-packages using result.files_deleted

qwcode · 2015-03-08T01:06:58Z

as for docs, thinking we'd need a few lines added to https://pip.pypa.io/en/latest/reference/pip_install.html#editable-installs to explain what will happen, and especially that vcs will work based on the #egg= fragment.

dstufft · 2016-05-18T11:31:14Z

Accidentally closed this, reopening. Sorry!

BrownTruck · 2016-05-19T12:02:48Z

Hello!

As part of an effort to ease the contribution process and adopt a more standard workflow pip has switched to doing development on the master branch. However, this Pull Request was made against the develop branch so it will need to be resubmitted against master. Unfortunately, this pull request does not cleanly merge against the current master branch.

If you do nothing, this Pull Request will be automatically closed by @BrownTruck since it cannot be merged.

If this pull request is still valid, please rebase it against master (or merge master into it) and resubmit it against the master branch, closing and referencing the original Pull Request.

If you choose to rebase/merge and resubmit this Pull Request, here is an example message that you can copy and paste:

by adding an additional call to requirement.check_if_exists() to
RequirementSet.install.

Fixes GH-1548

Example output with this PR:

$ virtualenv pip_test
...

$ cd pip_test && source bin/activate

$ pip install requests==0.5.0
Downloading/unpacking requests==0.5.0
  Using download cache from /Users/marca/.pip/download-cache/http%3A%2F%2Flocalhost%3A3141%2Froot%2Fpypi%2F%2Bf%2F6dfdc1688217d774d524e056ec6605a6%2Frequests-0.5.0.tar.gz
  Running setup.py egg_info for package requests

Installing collected packages: requests
  Running setup.py install for requests

Successfully installed requests
Cleaning up...

$ python -c 'import requests; print(requests); print(requests.__version__)'
<module 'requests' from '/private/tmp/pip_test/lib/python2.7/site-packages/requests/__init__.pyc'>
0.5.0

$ pip install -e '[email protected]:msabramo/pip.git@issue_1548_make_pip_install_editable_uninstall_existing_versions#egg=pip'
Obtaining pip from [email protected]:msabramo/pip.git@issue_1548_make_pip_install_editable_uninstall_existing_versions#egg=pip
...
Successfully installed pip
Cleaning up...

$ pip install -e ~/dev/git-repos/requests/
Obtaining file:///Users/marca/dev/git-repos/requests
  Running setup.py (path:/Users/marca/dev/git-repos/requests/setup.py) egg_info for package from file:///Users/marca/dev/git-repos/requests

Installing collected packages: requests
  Found existing installation: requests 0.5.0
    Uninstalling requests:
      Successfully uninstalled requests
  Running setup.py develop for requests

    Creating /private/tmp/pip_test/lib/python2.7/site-packages/requests.egg-link (link to .)
    Adding requests 2.0.1 to easy-install.pth file

    Installed /Users/marca/dev/git-repos/requests
Successfully installed requests
Cleaning up...

$ python -c 'import requests; print(requests); print(requests.__version__)'
<module 'requests' from '/Users/marca/dev/git-repos/requests/requests/__init__.pyc'>
2.0.1

$ pip uninstall --verbose --yes requests
Uninstalling requests:
  Removing file or directory /private/tmp/pip_test/lib/python2.7/site-packages/requests.egg-link
  Removing pth entries from /private/tmp/pip_test/lib/python2.7/site-packages/easy-install.pth:
  Removing entry: /Users/marca/dev/git-repos/requests
  Successfully uninstalled requests

$ pip uninstall --verbose --yes requests
Cannot uninstall requirement requests, not installed
Exception information:
Traceback (most recent call last):
  File "/private/tmp/pip_test/src/pip/pip/basecommand.py", line 129, in main
    status = self.run(options, args)
  File "/private/tmp/pip_test/src/pip/pip/commands/uninstall.py", line 64, in run
    requirement_set.uninstall(auto_confirm=options.yes)
  File "/private/tmp/pip_test/src/pip/pip/req/req_set.py", line 147, in uninstall
    req.uninstall(auto_confirm=auto_confirm)
  File "/private/tmp/pip_test/src/pip/pip/req/req_install.py", line 531, in uninstall
    "Cannot uninstall requirement %s, not installed" % (self.name,)
UninstallationError: Cannot uninstall requirement requests, not installed

Note that when I do pip install -e on requests above, the previous version gets found and uninstalled.

$ pip install -e ~/dev/git-repos/requests/
Obtaining file:///Users/marca/dev/git-repos/requests
  Running setup.py (path:/Users/marca/dev/git-repos/requests/setup.py) egg_info for package from file:///Users/marca/dev/git-repos/requests

Installing collected packages: requests
↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓
  Found existing installation: requests 0.5.0
    Uninstalling requests:
      Successfully uninstalled requests
↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑
  Running setup.py develop for requests

Cc: @sontek, @aconrad, @whitmo

---

*This was migrated from pypa/pip#1552 to reparent it to the ``master`` branch. Please see original pull request for any previous discussion.*

BrownTruck · 2016-05-26T10:58:20Z

This Pull Request was closed because it cannot be automatically reparented to the master branch and it appears to have bit rotted.

Please feel free to re-open it or re-submit it if it is still valid and you have rebased it onto master or merged master into it.

TZanke · 2018-02-12T14:39:59Z

Any news on this? When deploying systems i randomly hit this situation with cronjobs. ImportError: No module named xy because it is uninstalled for some milliseconds without any code or version change.

lock · 2019-06-02T15:06:12Z

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

[issue 1548] Make pip install -e uninstall existing versions

41dbd97

by adding an additional call to `requirement.check_if_exists()` to RequirementSet.install Fixes pypaGH-1548

Check self.ignore_installed before doing requirement.check_if_exists

3870586

qwcode reviewed Feb 17, 2014
View reviewed changes

msabramo added 2 commits February 22, 2014 19:48

Set self.satisfied_by back to None;

5b62075

Add a comment that says something like "when installing editables, nothing pre-existing should ever satisfy". Addresses @qwcode's comment at pypa#1552 (comment)

Remove requirement.check_if_exists() I added in 3870586

36cffa0

Addresses @qwcode's comment at pypa#1552 (comment)

Trigger Travis retest

bfca0f6

Add test_install_editable_uninstalls_existing

3e0d727

msabramo mentioned this pull request Dec 20, 2014

Idea: Check for packages with 2 installed versions #2235

Closed

qwcode reviewed Mar 8, 2015
View reviewed changes

piotr-dobrogost mentioned this pull request May 8, 2016

Can't uninstall projects installed with "pip install -e" #1895

Closed

dstufft closed this May 18, 2016

dstufft reopened this May 18, 2016

BrownTruck added asked to reparent auto-bitrotten PRs that died because they weren't updated and removed asked to reparent labels May 19, 2016

BrownTruck closed this May 26, 2016

This was referenced Jul 26, 2016

Remove existing package before develop mode install #3875

Closed

Pip does not reinstall with latest setuptools #3874

Closed

xavfernandez mentioned this pull request Aug 5, 2016

Make pip install -e uninstall existing versions #3898

Merged

lock bot added the auto-locked Outdated issues that have been locked by automation label Jun 2, 2019

lock bot locked as resolved and limited conversation to collaborators Jun 2, 2019

[issue 1548] Make pip install -e uninstall existing versions #1552

[issue 1548] Make pip install -e uninstall existing versions #1552

Conversation

msabramo commented Feb 10, 2014 • edited by BrownTruck Loading

Example output with this PR:

msabramo commented Feb 10, 2014

msabramo commented Feb 11, 2014

qwcode commented Feb 11, 2014

msabramo commented Feb 11, 2014

msabramo commented Feb 16, 2014

qwcode Feb 17, 2014

Choose a reason for hiding this comment

msabramo Feb 23, 2014

Choose a reason for hiding this comment

msabramo commented Feb 17, 2014

msabramo commented Feb 23, 2014

msabramo commented Feb 23, 2014

msabramo commented Feb 23, 2014

aconrad commented Feb 23, 2014

whitmo commented Feb 24, 2014

msabramo commented Feb 24, 2014

qwcode commented Feb 26, 2014

qwcode commented Feb 26, 2014

Ivoz commented Feb 26, 2014

carljm commented Feb 26, 2014

carljm commented Feb 26, 2014

qwcode commented Feb 26, 2014

qwcode commented Feb 27, 2014

msabramo commented Mar 2, 2014

qwcode commented Mar 2, 2014

msabramo commented Mar 2, 2014

msabramo commented Dec 20, 2014

pfmoore commented Dec 20, 2014

aconrad commented Dec 20, 2014

qwcode commented Dec 20, 2014

aconrad commented Dec 20, 2014

msabramo commented Dec 20, 2014

msabramo commented Dec 20, 2014

qwcode commented Mar 7, 2015

msabramo commented Mar 7, 2015

qwcode Mar 8, 2015

Choose a reason for hiding this comment

qwcode commented Mar 8, 2015

dstufft commented May 18, 2016

BrownTruck commented May 19, 2016

BrownTruck commented May 26, 2016

TZanke commented Feb 12, 2018

lock bot commented Jun 2, 2019

msabramo commented Feb 10, 2014 •

edited by BrownTruck

Loading