pip install git+file:// doesn't work with Windows UNC paths

[mechsin]

• Pip version:8.1.2
• Python version:2.7.11
• Operating System: Windows Server 2012 R2

Git Version: 2.8.3.windows.1

Description:

I am trying to install a package from a local repository using a UNC path. However pip seems to be mangling the path. After some inspection it seems like the code in git.py in the init function lines 39-48 are removing any slashes from the beginning of the UNC path. When this is then passed to Git it isn't a functional path.

What I've run:

1. This is an example of how I would normally format the path when using Git

(venv) PS Z:\WorkingDir> pip install git+file:////ComputerHostName/Git_Repos/gedis.git
Collecting git+file:////ComputerHostName/Git_Repos/gedis.git
  Cloning file://ComputerHostName/Git_Repos/gedis.git to c:\users\MyUserFolder\appdata\local\temp\2\pip-wf0yht-build
fatal: 'C:/Users/MyUserFolder/AppData/Local/Programs/Git/Git_Repos/gedis.git' does not appear to be a git repository
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
Command "git clone -q file://ComputerHostName/Git_Repos/gedis.git c:\users\MyUserFolder\appdata\local\temp\2\pip-wf0yht-build" failed with error code 128 in None

2. I also tried these other two alternate paths which also work with Git

(venv) PS Z:\WorkingDir> pip install git+file:///\ComputerHostName/Git_Repos/gedis.git
Collecting git+file:///\ComputerHostName/Git_Repos/gedis.git
  Cloning file:///ComputerHostName/Git_Repos/gedis.git to c:\users\MyUserFolder\appdata\local\temp\2\pip-8ltopq-build
fatal: 'C:/Users/MyUserFolder/AppData/Local/Programs/Git/ComputerHostName/Git_Repos/gedis.git' does not appear to be a git repository
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
Command "git clone -q file:///ComputerHostName/Git_Repos/gedis.git c:\users\MyUserFolder\appdata\local\temp\2\pip-8ltopq-build" failed with error code 128 in None

(venv) PS Z:\WorkingDir> pip install git+file:///\ComputerHostName/Git_Repos/gedis.git
Collecting git+file:///\ComputerHostName/Git_Repos/gedis.git
  Cloning file:///ComputerHostName/Git_Repos/gedis.git to c:\users\MyUserFolder\appdata\local\temp\2\pip-honcwh-build
fatal: 'C:/Users/MyUserFolder/AppData/Local/Programs/Git/ComputerHostName/Git_Repos/gedis.git' does not appear to be a git repository
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
Command "git clone -q file:///ComputerHostName/Git_Repos/gedis.git c:\users\MyUserFolder\appdata\local\temp\2\pip-honcwh-build" failed with error code 128 in None
(venv) PS Z:\WorkingDir>

[pradyunsg]

HI @mechsin! Thanks for filing this issue and sorry for the lack of a response all this while.

While I think what you're suggesting here makes sense, I'd like some input from @pfmoore on this.

[pfmoore]

In principle, I'd expect that if git can access a URL file:.... then pip should be able to access it as git+file:.... However, file URLs are quite complex to get right cross-platform, so I'm not surprised if there are bugs there.

[flothesof]

Hi all,
I'm running into exactly the same issue.
I think I can add the following information to the discussion. When I run with the verbose flag, it looks like the repo url transformation mangles the correct UNC path.

I run:

pip install -v git+file:////local_server/repo.git

And I get this message:

pip.exceptions.InstallationError: Command "git clone -q file://local_server/repo.git C:\Users\...\AppData\Local\Temp\pip-84s41_om-build" failed with error code 128 in None

When I try adding more slashes to my pip install command, they always get removed: the git clone -q path in the error message stays the same (no matter how many slashes I add).

Is there any way to fix this behaviour?

pip 9.0.1 from C:\Anaconda3\envs\...\lib\site-packages (python 3.6)

Thanks for your help,
Florian

[flothesof]

Update: by tinkering with the file paths, I managed to get the following working:

pip install -v git+file://\//local_server/repo.git

Notice the backslash hidden in the repo path. This partly solves my original issue.

[cjerdonek]

I took a look at this issue, and it's basically because of a bug in CPython's urllib.parse.urlunsplit() that's been around for a long time.

I filed an issue about it yesterday on CPython's issue tracker here--
https://bugs.python.org/issue34276
and others chimed in confirming the issue (with links to related issues on the tracker, etc).

To summarize, git-clone accepts UNC paths of the form: file:////<host>/<share>/<path>.

However, URL's of this form (with four or more leading slashes) fail to round-trip when using urllib.parse.urlsplit() followed by urllib.parse.urlunsplit():

file:////<host>/<share>/<path> becomes
file://<host>/<share>/<path>

This is what pip does in VersionControl.get_url_rev_and_auth():

pip/src/pip/_internal/vcs/__init__.py

Lines 231 to 244 in db12733

	scheme, netloc, path, query, frag = urllib_parse.urlsplit(url)
	if '+' not in scheme:
	raise ValueError(
	"Sorry, {!r} is a malformed VCS url. "
	"The format is <vcs>+<protocol>://<url>, "
	"e.g. svn+http://myrepo/svn/MyApp#egg=MyApp".format(url)
	)
	# Remove the vcs prefix.
	scheme = scheme.split('+', 1)[1]
	netloc, user_pass = self.get_netloc_and_auth(netloc)
	rev = None
	if '@' in path:
	path, rev = path.rsplit('@', 1)
	url = urllib_parse.urlunsplit((scheme, netloc, path, query, ''))

A couple options for us are--

1. fix the issue here so it will work in pip independent of CPython, or
2. do nothing in pip and require it to be fixed in CPython.

[pradyunsg]

I am going to close this, given the lack of activity on this issue -- I reckon that this has been resolved at some point. If not, I reckon this needs to be resolved in CPython and there isn't much more that pip should try to do here.