Pip Issues on Windows + ReFS

[waxhell]

Also posted to the python bug tracker: https://bugs.python.org/issue24738

Storage Spaces (RAID volumes) under Windows 8.1 / Server 2012 R2 use ReFS which does not support hard links.

If the Users directory has been moved to a ReFS volume, it causes pip to hang while calling os.link as it gets suck in the OSError exception since no timeout has been called to the initial call to acquire().

As a side effect -- this also makes it so that Python can not be uninstalled if pip is installed. The script must be terminated with a ctrl-c.

Tested under 3.5.0b3 as well as 3.4.3 (x64 as well as x32).

Traceback from ctrl-c on 'pip list':

I:\>pip list
Traceback (most recent call last):
  File "C:\Program Files\Python 3.5\lib\site-packages\pip\_vendor\lockfile\linkl
ockfile.py", line 30, in acquire
    os.link(self.unique_name, self.lock_file)
OSError: [WinError 50] The request is not supported: 'I:\\Users\\Harry\\AppData\
\Local\\pip\\Cache\\omgwtfdarkgoon-1d14.781968199621979134564595' -> 'I:\\Users\
\Harry\\AppData\\Local\\pip\\Cache\\selfcheck.json.lock'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Program Files\Python 3.5\lib\runpy.py", line 170, in _run_module_as_m
ain
    "__main__", mod_spec)
  File "C:\Program Files\Python 3.5\lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "C:\Program Files\Python 3.5\Scripts\pip.exe\__main__.py", line 9, in <mo
dule>
  File "C:\Program Files\Python 3.5\lib\site-packages\pip\__init__.py", line 217
, in main
    return command.main(cmd_args)
  File "C:\Program Files\Python 3.5\lib\site-packages\pip\basecommand.py", line
220, in main
    pip_version_check(session)
  File "C:\Program Files\Python 3.5\lib\site-packages\pip\utils\outdated.py", li
ne 134, in pip_version_check
    state.save(pypi_version, current_time)
  File "C:\Program Files\Python 3.5\lib\site-packages\pip\utils\outdated.py", li
ne 71, in save
    with lockfile.LockFile(self.statefile_path):
  File "C:\Program Files\Python 3.5\lib\site-packages\pip\_vendor\lockfile\__ini
t__.py", line 238, in __enter__
    self.acquire()
  File "C:\Program Files\Python 3.5\lib\site-packages\pip\_vendor\lockfile\linkl
ockfile.py", line 49, in acquire
    time.sleep(timeout is not None and timeout/10 or 0.1)
KeyboardInterrupt

[pfmoore]

Can you raise an issue with the lockfile project for this? The bug is caused by that project not allowing for the possibility that os.link() could fail because links aren't supported, rather than because we're waiting for the lock. The issue is in linklockfile.py in acquire() at around line 49, as noted in the traceback.

As a workaround, pip could add a timeout, but (a) that could cause potential failures if the timeout is too short, and (b) it would make pip extremely slow when links aren't supported (we'd wait for the timeout, then try the proper method).

[waxhell]

Already done.

https://bugs.launchpad.net/pylockfile/+bug/1478790

Also as a fyi, the following lockfile bug is related but not due to the same issue:

https://bugs.launchpad.net/pylockfile/+bug/1472101

[chrahunt]

Let's track this with #6954, which will remove hardlinks.