Link comparison fails to normalise file://localhost/

[nmonterroso]

Description

It seems like there's an issue with the new dependency resolution with local packages. I have a repo demonstrating the problem: https://github.com/nmonterroso/pip-21.1.3-resolution.

I have a repo with the following structure:

lib/
  a/
    lib_a/
      a.py
    setup.py
  b/
    lib_b/
      b.py
    setup.py
app/
  x/
    Dockerfile
    Dockerfile-legacy
    app.py
    requirements.txt

This mirrors a monorepo I'm working on, where lib_b depends on lib_a, and app x depends on both lib_b and lib_a. When running pip install -r requirements.txt for the app, I get a version conflicts for lib_a from the app and lib_b. These go away if I use the legacy resolver (via --use-deprecated=legacy-resolver).

Expected behavior

I would expect that since the files are the same files on local disk that the dependency resolution recognizes that they don't conflict and the single package lib_a can be installed.

pip version

21.1.3

Python version

3.9.6

OS

macos/Docker

How to Reproduce

1. Clone the repo: https://github.com/nmonterroso/pip-21.1.3-resolution
2. From the root directory run docker build . -f app/x/Dockerfile

Output

> [7/7] RUN pip install -r requirements.txt:
#11 1.398 Processing /pip_resolver/lib/a
#11 1.399   DEPRECATION: A future pip version will change local packages to be built in-place without first copying to a temporary directory. We recommend you use --use-feature=in-tree-build to test your packages with this new behavior before it becomes the default.
#11 1.399    pip 21.3 will remove support for this functionality. You can find discussion regarding this at https://github.com/pypa/pip/issues/7555.
#11 1.717 Processing /pip_resolver/lib/b
#11 1.912 Processing /pip_resolver/lib/a
#11 2.095 INFO: pip is looking at multiple versions of lib-a to determine which version is compatible with other requirements. This could take a while.
#11 2.096 ERROR: Cannot install -r requirements.txt (line 2) and lib-a 0.0.0 (from /pip_resolver/lib/a) because these package versions have conflicting dependencies.
#11 2.096
#11 2.096 The conflict is caused by:
#11 2.096     The user requested lib-a 0.0.0 (from /pip_resolver/lib/a)
#11 2.096     lib-b 0.0.0 depends on lib-a 0.0.0 (from /pip_resolver/lib/a)
#11 2.096
#11 2.096 To fix this you could try to:
#11 2.096 1. loosen the range of package versions you've specified
#11 2.096 2. remove package versions to allow pip attempt to solve the dependency conflict
#11 2.096
#11 2.096 ERROR: ResolutionImpossible: for help visit https://pip.pypa.io/en/latest/user_guide/#fixing-conflicting-dependencies

Code of Conduct

• I agree to follow the PSF Code of Conduct.

[nmonterroso]

Any update here?

[uranusjr]

Sorry for the delayed response. pip is an entirely volunteer-run project and it is often we don't have enough issue triaging capacity to respond to reports.

It seems like the issue is with the localhost part in the file:// URL. I removed it and everything seems to work as expected.

$ git diff
-------------------------------------------------------------------------------
modified: lib/b/setup.py
-------------------------------------------------------------------------------
@ lib/b/setup.py:9 @ setup(
    author='nmonterroso',
    packages=find_packages(include=['lib_b*']),
    install_requires=[
-     f'lib_a@file://localhost/pip_resolver/lib/a'
+     f'lib_a@file:///pip_resolver/lib/a'
    ]
)

$ docker build . -f app/x/Dockerfile
[+] Building 5.3s (12/12) FINISHED
 => [internal] load build definition from Dockerfile                                            0.1s
 => => transferring dockerfile: 32B                                                             0.0s
 => [internal] load .dockerignore                                                               0.1s
 => => transferring context: 2B                                                                 0.0s
 => [internal] load metadata for docker.io/library/python:3.9.6                                 0.0s
 => [1/7] FROM docker.io/library/python:3.9.6                                                   0.0s
 => [internal] load build context                                                               0.1s
 => => transferring context: 813B                                                               0.0s
 => CACHED [2/7] RUN mkdir -p /pip_resolver/lib                                                 0.0s
 => [3/7] COPY lib /pip_resolver/lib                                                            0.1s
 => [4/7] RUN mkdir -p /pip_resolver/app                                                        0.6s
 => [5/7] COPY app/x/ /pip_resolver/app/x/                                                      0.2s
 => [6/7] WORKDIR /pip_resolver/app/x                                                           0.2s
 => [7/7] RUN pip install -r requirements.txt                                                   3.3s
 => exporting to image                                                                          0.6s
 => => exporting layers                                                                         0.4s
 => => writing image sha256:67f579173acb6be423c4708851286437e111d4e70fb5b30ec35c223e223f4a92    0.0s

Use 'docker scan' to run Snyk tests against images to find vulnerabilities and learn how to fix them
$ docker images | head -n2
REPOSITORY        TAG           IMAGE ID            CREATED         SIZE
<none>            <none>        67f579173acb        Just now        893MB

Since localhost in a file URL is equivalent to an empty string according to RFC-8089, pip should treat these as the same location and not conflict.

[nmonterroso]

Thanks for this! It's working as expected 😄