Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

pip install -U from github or bitbucket fails on alternate attempts on Mac OS X #3011

Closed
dkoppstein opened this issue Aug 5, 2015 · 7 comments
Closed

pip install -U from github or bitbucket fails on alternate attempts on Mac OS X #3011

dkoppstein opened this issue Aug 5, 2015 · 7 comments
Labels
auto-locked Outdated issues that have been locked by automation

Comments

@dkoppstein
Copy link

I'm running a python 3.4.3 virtual environment on Mac OS X 10.10.1 under pyenv. I first noticed that invoking pip3 install on a bitbucket repo was giving me errors every other invocation, e.g.

(venv343) ➜  ~  pip3 install -U git+https://bitbucket.org/johanneskoester/snakemake.git
DEPRECATION: --download-cache has been deprecated and will be removed in the future. Pip now automatically uses and configures its cache.
Collecting git+https://bitbucket.org/johanneskoester/snakemake.git
  Cloning https://bitbucket.org/johanneskoester/snakemake.git to /var/folders/gq/_tw_x0pn3vjcr345x2zgs72w0000gn/T/pip-ek3l2sc1-build
Installing collected packages: snakemake
  Found existing installation: snakemake 3.4
    Uninstalling snakemake-3.4:
Exception:
Traceback (most recent call last):
  File "/usr/local/opt/pyenv/versions/3.4.3/lib/python3.4/shutil.py", line 523, in move
    os.rename(src, real_dst)
FileNotFoundError: [Errno 2] No such file or directory: '/usr/local/var/lib/pyenv/versions/venv343/bin/snakemake' -> '/var/folders/gq/_tw_x0pn3vjcr345x2zgs72w0000gn/T/pip-0ywz1ocm-uninstall/usr/local/var/lib/pyenv/versions/venv343/bin/snakemake'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/opt/pyenv/versions/venv343/lib/python3.4/site-packages/pip/basecommand.py", line 223, in main
    status = self.run(options, args)
  File "/usr/local/opt/pyenv/versions/venv343/lib/python3.4/site-packages/pip/commands/install.py", line 299, in run
    root=options.root_path,
  File "/usr/local/opt/pyenv/versions/venv343/lib/python3.4/site-packages/pip/req/req_set.py", line 640, in install
    requirement.uninstall(auto_confirm=True)
  File "/usr/local/opt/pyenv/versions/venv343/lib/python3.4/site-packages/pip/req/req_install.py", line 726, in uninstall
    paths_to_remove.remove(auto_confirm)
  File "/usr/local/opt/pyenv/versions/venv343/lib/python3.4/site-packages/pip/req/req_uninstall.py", line 125, in remove
    renames(path, new_path)
  File "/usr/local/opt/pyenv/versions/venv343/lib/python3.4/site-packages/pip/utils/__init__.py", line 314, in renames
    shutil.move(old, new)
  File "/usr/local/opt/pyenv/versions/3.4.3/lib/python3.4/shutil.py", line 535, in move
    copy2(src, real_dst)
  File "/usr/local/opt/pyenv/versions/3.4.3/lib/python3.4/shutil.py", line 245, in copy2
    copyfile(src, dst, follow_symlinks=follow_symlinks)
  File "/usr/local/opt/pyenv/versions/3.4.3/lib/python3.4/shutil.py", line 108, in copyfile
    with open(src, 'rb') as fsrc:
FileNotFoundError: [Errno 2] No such file or directory: '/usr/local/var/lib/pyenv/versions/venv343/bin/snakemake'

(venv343) ➜  ~  pip3 install -U git+https://bitbucket.org/johanneskoester/snakemake.git
DEPRECATION: --download-cache has been deprecated and will be removed in the future. Pip now automatically uses and configures its cache.
Collecting git+https://bitbucket.org/johanneskoester/snakemake.git
  Cloning https://bitbucket.org/johanneskoester/snakemake.git to /var/folders/gq/_tw_x0pn3vjcr345x2zgs72w0000gn/T/pip-x_rgz4ah-build
Installing collected packages: snakemake
  Found existing installation: snakemake 3.4
    Uninstalling snakemake-3.4:
      Successfully uninstalled snakemake-3.4
  Running setup.py install for snakemake
Successfully installed snakemake-3.4
(venv343) ➜  ~  pip3 install -U git+https://bitbucket.org/johanneskoester/snakemake.git
DEPRECATION: --download-cache has been deprecated and will be removed in the future. Pip now automatically uses and configures its cache.
Collecting git+https://bitbucket.org/johanneskoester/snakemake.git
  Cloning https://bitbucket.org/johanneskoester/snakemake.git to /var/folders/gq/_tw_x0pn3vjcr345x2zgs72w0000gn/T/pip-_6p79mmo-build
Installing collected packages: snakemake
  Found existing installation: snakemake 3.4
    Uninstalling snakemake-3.4:
Exception:
Traceback (most recent call last):
  File "/usr/local/opt/pyenv/versions/3.4.3/lib/python3.4/shutil.py", line 523, in move
    os.rename(src, real_dst)
FileNotFoundError: [Errno 2] No such file or directory: '/usr/local/var/lib/pyenv/versions/venv343/bin/snakemake' -> '/var/folders/gq/_tw_x0pn3vjcr345x2zgs72w0000gn/T/pip-6h2ui3oj-uninstall/usr/local/var/lib/pyenv/versions/venv343/bin/snakemake'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/opt/pyenv/versions/venv343/lib/python3.4/site-packages/pip/basecommand.py", line 223, in main
    status = self.run(options, args)
  File "/usr/local/opt/pyenv/versions/venv343/lib/python3.4/site-packages/pip/commands/install.py", line 299, in run
    root=options.root_path,
  File "/usr/local/opt/pyenv/versions/venv343/lib/python3.4/site-packages/pip/req/req_set.py", line 640, in install
    requirement.uninstall(auto_confirm=True)
  File "/usr/local/opt/pyenv/versions/venv343/lib/python3.4/site-packages/pip/req/req_install.py", line 726, in uninstall
    paths_to_remove.remove(auto_confirm)
  File "/usr/local/opt/pyenv/versions/venv343/lib/python3.4/site-packages/pip/req/req_uninstall.py", line 125, in remove
    renames(path, new_path)
  File "/usr/local/opt/pyenv/versions/venv343/lib/python3.4/site-packages/pip/utils/__init__.py", line 314, in renames
    shutil.move(old, new)
  File "/usr/local/opt/pyenv/versions/3.4.3/lib/python3.4/shutil.py", line 535, in move
    copy2(src, real_dst)
  File "/usr/local/opt/pyenv/versions/3.4.3/lib/python3.4/shutil.py", line 245, in copy2
    copyfile(src, dst, follow_symlinks=follow_symlinks)
  File "/usr/local/opt/pyenv/versions/3.4.3/lib/python3.4/shutil.py", line 108, in copyfile
    with open(src, 'rb') as fsrc:
FileNotFoundError: [Errno 2] No such file or directory: '/usr/local/var/lib/pyenv/versions/venv343/bin/snakemake'

(venv343) ➜  ~  pip3 install -U git+https://bitbucket.org/johanneskoester/snakemake.git
DEPRECATION: --download-cache has been deprecated and will be removed in the future. Pip now automatically uses and configures its cache.
Collecting git+https://bitbucket.org/johanneskoester/snakemake.git
  Cloning https://bitbucket.org/johanneskoester/snakemake.git to /var/folders/gq/_tw_x0pn3vjcr345x2zgs72w0000gn/T/pip-xbk6g8t7-build
Installing collected packages: snakemake
  Found existing installation: snakemake 3.4
    Uninstalling snakemake-3.4:
      Successfully uninstalled snakemake-3.4
  Running setup.py install for snakemake
Successfully installed snakemake-3.4

Interestingly, I get the same error when I upgrade from the pip repo itself!

(venv343) ➜  ~  pip3 install -U git+https://github.com/pypa/pip.git@0027580
DEPRECATION: --download-cache has been deprecated and will be removed in the future. Pip now automatically uses and configures its cache.
Collecting git+https://github.com/pypa/pip.git@0027580
  Cloning https://github.com/pypa/pip.git (to 0027580) to /var/folders/gq/_tw_x0pn3vjcr345x2zgs72w0000gn/T/pip-5jc5tjwf-build
  Could not find a tag or branch '0027580', assuming commit.
Installing collected packages: pip
  Found existing installation: pip 7.2.0.dev0
    Uninstalling pip-7.2.0.dev0:
Exception:
Traceback (most recent call last):
  File "/usr/local/opt/pyenv/versions/3.4.3/lib/python3.4/shutil.py", line 523, in move
    os.rename(src, real_dst)
FileNotFoundError: [Errno 2] No such file or directory: '/usr/local/var/lib/pyenv/versions/venv343/bin/pip' -> '/var/folders/gq/_tw_x0pn3vjcr345x2zgs72w0000gn/T/pip-won625up-uninstall/usr/local/var/lib/pyenv/versions/venv343/bin/pip'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/opt/pyenv/versions/venv343/lib/python3.4/site-packages/pip/basecommand.py", line 211, in main
    status = self.run(options, args)
  File "/usr/local/opt/pyenv/versions/venv343/lib/python3.4/site-packages/pip/commands/install.py", line 311, in run
    root=options.root_path,
  File "/usr/local/opt/pyenv/versions/venv343/lib/python3.4/site-packages/pip/req/req_set.py", line 640, in install
    requirement.uninstall(auto_confirm=True)
  File "/usr/local/opt/pyenv/versions/venv343/lib/python3.4/site-packages/pip/req/req_install.py", line 716, in uninstall
    paths_to_remove.remove(auto_confirm)
  File "/usr/local/opt/pyenv/versions/venv343/lib/python3.4/site-packages/pip/req/req_uninstall.py", line 125, in remove
    renames(path, new_path)
  File "/usr/local/opt/pyenv/versions/venv343/lib/python3.4/site-packages/pip/utils/__init__.py", line 315, in renames
    shutil.move(old, new)
  File "/usr/local/opt/pyenv/versions/3.4.3/lib/python3.4/shutil.py", line 535, in move
    copy2(src, real_dst)
  File "/usr/local/opt/pyenv/versions/3.4.3/lib/python3.4/shutil.py", line 245, in copy2
    copyfile(src, dst, follow_symlinks=follow_symlinks)
  File "/usr/local/opt/pyenv/versions/3.4.3/lib/python3.4/shutil.py", line 108, in copyfile
    with open(src, 'rb') as fsrc:
FileNotFoundError: [Errno 2] No such file or directory: '/usr/local/var/lib/pyenv/versions/venv343/bin/pip'

And now I have no more pip in my virtual environment...
@dkoppstein
Copy link
Author

P.S. the pip version was 7.1.0 before it deleted itself =)

@mathieulongtin
Copy link

I just tracked the issue.
To recreate:

mkdir /tmp/mytestdir
cd /tmp
ln -s mytestdir mylink
virtualenv /tmp/mylink/myenv

Now, if you install a package that uses setuptools, with console entry points, when you want to uninstall it, it attempts to remove the executable twice, like this:

(myenv)$ pip uninstall sgetools
Uninstalling sgetools-0.1.3+ff921a2:
   /tmp/mylink/env/bin/command
   /tmp/mytestdir/env/bin/command

As it turns out, it's because bin_dir contains the symlink, but the files gotten from installed-files.txt do not contain the symlink. See pip/req/req_install.py line 620.

@mathieulongtin
Copy link

Btw, this bug was introduced sometimes after pip 6.1.

@xavfernandez xavfernandez mentioned this issue Sep 28, 2015
Closed
@xavfernandez
Copy link
Member

This is likely due to #2552.

@xavfernandez
Copy link
Member

I'm not comfortable with your solution of mathieulongtin@4dd0ab2 and I'm more in favor of something like #3142

@mathieulongtin
Copy link

You likely have a much better idea of how to fix it. I will be happy that I don't have to run updates twice anymore, regardless of the fix.

@qwcode qwcode mentioned this issue Oct 3, 2015
Merged
@qwcode
Copy link
Contributor

qwcode commented Oct 3, 2015

fixed in PR #3154

@qwcode qwcode closed this as completed Oct 3, 2015
@xavfernandez xavfernandez mentioned this issue Nov 25, 2015
Closed
pomegranited added a commit to open-craft/edx-analytics-pipeline that referenced this issue Apr 5, 2016
@pomegranited
Removes -U flag from pip install requirements/defaults.txt
48c9998 
Works around an issue when installing analytics-tasks on jenkins server,
similar to pypa/pip#3011
Issue occurs because jenkins home is a symlink, and so the virtualenv used to
install analytics-tasks is also a symlink, and alternate make installs fail
to uninstall pyinstrument
@lock lock bot added the auto-locked Outdated issues that have been locked by automation label Jun 4, 2019
@lock lock bot locked as resolved and limited conversation to collaborators Jun 4, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
auto-locked Outdated issues that have been locked by automation
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@mathieulongtin @qwcode @xavfernandez @dkoppstein