editable packages in a constraint file causes pip-compile to fail silently

[mmerickel]

Using a -c foo.txt where foo.txt contains editable packages causes pip-compile to not work at all and fail silently (returning success).

Environment Versions

1. OS Type macos 10.14.6
2. Python version: $ python -V 3.6.8
3. pip version: $ pip --version 19.3.1
4. pip-tools version: $ pip-compile --version 4.2.0

Steps to replicate

1. make constraint file foo.in with contents -e file:src/foo#egg=foo
2. pip-compile foo.in to generate foo.txt with -e file:src/foo#egg=foo and other deps
3. make bar.in with contents -c foo.txt and other source -e file:src/bar#egg=bar (imagine src/bar/setup.py has install_requires=['foo', ...])
4. pip-compile bar.in

Expected result

The file bar.txt should contain:

#
# This file is autogenerated by pip-compile
# To update, run:
#
#    pip-compile bar.in
#
-e file:src/foo#egg=foo
six==1.13.0
... other deps

Actual result

The file bar.txt contains no dependencies, just the header:

#
# This file is autogenerated by pip-compile
# To update, run:
#
#    pip-compile bar.in
#

[atugushev]

Hell @mmerickel

Thanks for the issue! The dependencies from -c foo.txt would never be included in the requirements.txt. See:

$ cat constraints.txt
six==1.10.0

$ cat requirements.in
-c constraints.txt
pytz

$ pip-compile
#
# This file is autogenerated by pip-compile
# To update, run:
#
#    pip-compile
#
pytz==2019.3

If you still want to include constraints to the requirements.txt you have two options:

Let's say you have the following constraints:

$ cat constraints.txt
six==1.10.0

1. declare explicitly a package form constraints in requirements.in:

$ cat requirements.in
-c constraints.txt
six
pytz


$ pip-compile
#
# This file is autogenerated by pip-compile
# To update, run:
#
#    pip-compile
#
pytz==2019.3
six==1.10.0

2. include constraints file with -r constraints.txt

$ cat requirements.in
-r constraints.txt
pytz

$ pip-compile
#
# This file is autogenerated by pip-compile
# To update, run:
#
#    pip-compile
#
pytz==2019.3
six==1.10.0

[atugushev]

@mmerickel does the above resolve your issue?

[mmerickel]

Thanks @atugushev, I'm aware of -c and -r options, this isn't related to that. I'm struggling to put my finger on it in a reproducible way but it's affecting me quite often, and I think it's related to -c. For example, in my current project (doesn't actually use any -e in .in files at all), I just re-ran env/bin/pip-compile requirements/dev.in which has -c main.txt in it and the "deps of deps" disappeared. For example, dev.in has pyramid-debugtoolbar and pyramid was pulled into the txt but hupper and other dependencies of pyramid were not pulled int. The main.txt generated from main.in does not have this issue, and properly has deps of deps in it. I wish I could give a more reproducible example other than to say it's happening to me often, and inconsistently.

[atugushev]

@mmerickel

I just re-ran env/bin/pip-compile requirements/dev.in which has -c main.txt in it and the "deps of deps" disappeared

It could be related to the cache. In this case --rebuild option might help. See some weird related issues: #835, #928, #810, #683.

[atugushev]

Fixed in #1037.