Missing setuptools.egg-info crashes pip

[mal]

This has crashed deploys on at least 9 of our production servers in the last week. I've been struggling to create a reproducable use case but can't, sometimes it's fine, other times it really isn't. Reproducable test case provided here. Any insight would be hugely appreciated!

Also worth noting that we've seen the same issue when running pip install --upgrade virtualenv so probably not package related?

$ pip install  --upgrade supervisor
Exception:
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/pip/basecommand.py", line 232, in main
    status = self.run(options, args)
  File "/usr/local/lib/python2.7/dist-packages/pip/commands/install.py", line 339, in run
    requirement_set.prepare_files(finder)
  File "/usr/local/lib/python2.7/dist-packages/pip/req/req_set.py", line 229, in prepare_files
    req_to_install.check_if_exists()
  File "/usr/local/lib/python2.7/dist-packages/pip/req/req_install.py", line 931, in check_if_exists
    self.satisfied_by = pkg_resources.get_distribution(self.req)
  File "/usr/local/lib/python2.7/dist-packages/pip/_vendor/pkg_resources/__init__.py", line 461, in get_distribution
    dist = get_provider(dist)
  File "/usr/local/lib/python2.7/dist-packages/pip/_vendor/pkg_resources/__init__.py", line 341, in get_provider
    return working_set.find(moduleOrReq) or require(str(moduleOrReq))[0]
  File "/usr/local/lib/python2.7/dist-packages/pip/_vendor/pkg_resources/__init__.py", line 870, in require
    needed = self.resolve(parse_requirements(requirements))
  File "/usr/local/lib/python2.7/dist-packages/pip/_vendor/pkg_resources/__init__.py", line 740, in resolve
    env = Environment(self.entries)
  File "/usr/local/lib/python2.7/dist-packages/pip/_vendor/pkg_resources/__init__.py", line 927, in __init__
    self.scan(search_path)
  File "/usr/local/lib/python2.7/dist-packages/pip/_vendor/pkg_resources/__init__.py", line 957, in scan
    self.add(dist)
  File "/usr/local/lib/python2.7/dist-packages/pip/_vendor/pkg_resources/__init__.py", line 973, in add
    if self.can_add(dist) and dist.has_version():
  File "/usr/local/lib/python2.7/dist-packages/pip/_vendor/pkg_resources/__init__.py", line 2658, in has_version
    self.version
  File "/usr/local/lib/python2.7/dist-packages/pip/_vendor/pkg_resources/__init__.py", line 2466, in version
    for line in self._get_metadata(self.PKG_INFO):
  File "/usr/local/lib/python2.7/dist-packages/pip/_vendor/pkg_resources/__init__.py", line 2510, in _get_metadata
    for line in self.get_metadata_lines(name):
  File "/usr/local/lib/python2.7/dist-packages/pip/_vendor/pkg_resources/__init__.py", line 1927, in get_metadata_lines
    return yield_lines(self.get_metadata(name))
  File "/usr/local/lib/python2.7/dist-packages/pip/_vendor/pkg_resources/__init__.py", line 1921, in get_metadata
    with open(self.path,'rU') as f:
IOError: [Errno 2] No such file or directory: '/usr/lib/python2.7/dist-packages/setuptools.egg-info'

$ echo $?
2

[mal]

After further investigation we've managed to consistently recreate this on a clean ubuntu 12.04 box like so (run as root):

#!/bin/sh

# Installs 0.6rc11 via 0.6.24-1ubuntu1
apt-get install -y python-setuptools

# Here we have some easy_install usage before our deploy logic
# decides if we need to execute the rest of the code below.

wget -O- https://bootstrap.pypa.io/get-pip.py | python
pip install --upgrade setuptools
pip install --upgrade virtualenv

[Igorshp]

I've started seeing this issue on our servers about 2 weeks ago.

doing pip upgrade setuptools leaves behind the symlink
/usr/lib/python2.7/dist-packages/setuptools.egg-info

it cleans up the realpath destination, but not the symlink itself.

[xamox]

I can confirm having this issue as well running under docker 12.04 container as root. Removing the symlink appears to fix it for me as well.

[bakhtiya]

I can also confirm I'm seeing this issue as well on many of my production Ubuntu 12.04 servers running on AWS EC2.

[dpresling]

Also having the same issue.

[akshah123]

I am also having the same problem.

[arruda]

+1

[rodriguezsergio]

+1 removing the broken symlink was all I needed.

[akshah123]

It would be nice to have this fixed as removing the broken symlink not a great option for automated setups via chef.

[qwcode]

this was caused by #2347 (released in pip 6.0.8), although I think that change was correct.

the problem, is that #2437 exposed another bug, that is, that pip follows symlinks when uninstalling.

but that's fixed in #2552 (which is currently unreleased, set to be released in v6.1.0))

I tested the latest pip develp branch (which has #2552) on ubuntu 12.04 and don't get the bug anymore.

for those affected, an extra confirmation that develop branch works would be great before closing this.

(cc @xavfernandez @takluyver)

[willejs]

@qwcode can you cut the release v6.1.0 to fix this please?

[qwcode]

@dstufft does them. I think he was intending releasing 6.1.0 soon.

[qwcode]

closing based on #2438 (comment)

if anyone wants to double confirm the develop branch fixes this (or doesn't), please do, and post results.