pip-sync uninstalls main project from virtualenv

[MrkGrgsn]

In my project I have a requirements.txt produced by pip-compile from setup.py. requirements.txt does not include the packages in project. Since upgrading to pip-tools to 5.2.0 (not sure what the last version before change was sorry) running pip-sync requirements.txt causes my main project to be uninstalled from the virtual environment.

Environment Versions

1. CentOS 7
2. Python version: 3.6.8
3. pip version: 20.1.1
4. pip-tools version: 5.2.0

Steps to replicate

1. git checkout project
2. activate virtualenv
3. install project $ python setup.py install
4. sync requirements $ pip-sync requirements.txt

Expected result

I expect the requirements in requirements.txt to be synced but my main project, whose requirements I am trying to install, to be left alone.

Actual result

Requirements in requirements.txt are synced but the main project is uninstalled

[atugushev]

Hello @MrkGrgsn,

Thanks for the issue. Probably, related to #1155. Could you reproduce it with pip-tools==5.1.1?

[MrkGrgsn]

Thanks for the response. I've done some digging and actually the significant change was pip not pip-tools.

With pip-tools=5.2.0 and pip==19.3.1, I get the behaviour I am expecting however it comes with a message

Can't uninstall 'my_project'. No files were found to uninstall.

With pip-tools=5.2.0 and pip==20.0.2, I get similar behaviour with a more verbose message although the point about MyProject being outside the environment doesn't seem correct as that is where it's installed (although some scripts are installed outside the ve folder).

Found existing installation: MyProject 2.0.1
Not uninstalling mancini at /opt/mancini/src/mancini/src, outside environment /opt/my_project/ve
Can't uninstall 'MyProject'. No files were found to uninstall.

With pip-tools=5.2.0 and pip==21.1.1, it uninstalls my project:

Found existing installation: MyProject 2.0.1
Uninstalling MyProject-2.0.1:
Successfully uninstalled MyProject-2.0.1

So perhaps the behaviour I was accustomed to was an error and not intended but in any case it still seems like the sensible behaviour to me (unless there's a correct way to do what I want with pip-tools).

[MrkGrgsn]

Given the error message the change may be a result of pypa/pip#7856

[atugushev]

Thanks for the analysis! I've also found a couple of related links:

• pip uninstall not working with editable dependencies pypa/pip#5193
• WIP: Can't uninstall editable packages with Python 2.7 #659

[AndydeCleyre]

Does it help to generate the requirements.txt from a requirements.in file with just -e . in it, rather than from the setup.py?

[AndydeCleyre]

@MrkGrgsn

Have you had a chance to try the above method, to explicitly define the local package as the/a top level dependency in the .in file?

[MrkGrgsn]

@AndydeCleyre Not yet, sorry. I'll do it in the next couple of days.

[AndydeCleyre]

@atugushev

I don't know which is the intended behavior when using a setup file as pip-compile's input -- including the top package itself or not.

I would personally expect the setup-file package itself to be omitted from the requirements files, to be in line with projects that use both setup and requirements files, without necessarily using pip-tools in any capacity. And if I want it included in this case, I'll make a simple .in file.

[MrkGrgsn]

I'm now running pip-tools==6.6.0 and pip==22.0.4.

Using a requirements.in with just a reference to the current project, ie:

.  

resulting in a requirements.txt like this:

mancini @ file:///path/to/src
    # via -r requirements.in

When I run pip-sync it installs my_project but then tries and fails to uninstall it:

Building wheels for collected packages: mancini
  Building wheel for mancini (setup.py) ... done
  Created wheel for mancini: filename=Mancini-2.15.1-py3-none-any.whl size=2865624 sha256=e37b42f397269515c7728da6b07dec3b87bbd0439ad81b6eee69a98d05abcd59
  Stored in directory: /tmp/pip-ephem-wheel-cache-51nxves6/wheels/cd/25/d6/c6abda4c962e04e5dba32c9eb598ff766246df458f42001901
Successfully built mancini
Installing collected packages: mancini
  Attempting uninstall: mancini
    Found existing installation: Mancini 2.15.1
    Not uninstalling mancini at /path/to/src, outside environment /path/to/virtualenv
    Can't uninstall 'Mancini'. No files were found to uninstall.
Successfully installed mancini-2.15.1

This result is what I'm after, although for it to be properly useful I need the path to the current project to be a relative one.

I agree with @AndydeCleyre's comments regarding whether pip-compile on setup.py should include the top package or not. My initial thoughts were that pip-tools is intended for installing the requirements/dependencies of the top package not the top package itself.

If the top package is not in requirements.txt, should pip-sync requirements.txt uninstall the top package? Thinking about it now, I don't know if I have a preference. The current behaviour was a surprise for me as I was thinking pip-sync would only work on the dependencies and not the top project but once I got past that it is easy to understand and manageable, using a requirements.in file if necessary.

I'm happy to close this ticket

[AndydeCleyre]

This result is what I'm after, although for it to be properly useful I need the path to the current project to be a relative one.

I'm sorry I don't have a proper solution for you there, but FWIW I am trying to keep my rejected feature branch for relative path support updated, which you can try for example using pipx like:

$ pipx run --spec 'git+https://github.com/AndydeCleyre/pip-tools@feature/relpaths-post-6.8.0' pip-compile

or similarly with my zpy function pipz:

$ pipz runpkg 'pip-tools @ git+https://github.com/AndydeCleyre/pip-tools@feature/relpaths-post-6.8.0' pip-compile