celery + pylint ERROR: In --require-hashes mode, all requirements must have their versions pinned with ==

[GabLeRoux]

I recently ugraded celery in one of my projects due to a recent vulnerability. I use pip-tools to only define what I use so I can easily know what requirements are dependencies of dependencies. I also have astroid as a pylint dependency.

When running the following command in my pipelines:

pip install -r requirements.txt --upgrade

I get the following error:

[...]
Collecting setuptools>=20.0
ERROR: In --require-hashes mode, all requirements must have their versions pinned with ==. These do not:
    setuptools>=20.0 from https://files.pythonhosted.org/packages/b0/3a/88b210db68e56854d0bcf4b38e165e03be377e13907746f825790f3df5bf/setuptools-59.6.0-py3-none-any.whl#sha256=4ce92f1e1f8f01233ee9952c04f6b81d1e02939d6e1b488428154974a4d0783e (from astroid==2.8.6->-r requirements.txt (line 47))

Related lines I have in requirements.txt:

astroid==2.8.6 \
    --hash=sha256:5f6f75e45f15290e73b56f9dfde95b4bf96382284cde406ef4203e928335a495 \
    --hash=sha256:cd8326b424c971e7d87678609cf6275d22028afd37d6ac59c16d47f1245882f6
    # via
    #   pylint
    #   pylint-celery

I think removing setuptools as suggested in related #1103 might solve this issue in the future 😅 .

How to reproduce

requirements-fail.in:

celery==5.2.3
pylint==2.11.2

requirements-pass.in:

celery==5.2.2
pylint==2.11.2

FROM python:3.9.7

RUN pip install --upgrade pip && pip install pip-tools

docker build . -t example_image

# example failing
docker run --rm -v "$(pwd)":/app -w /app example_image \
  pip-compile --generate-hashes -o requirements.txt requirements-fail.in
docker run --rm -v "$(pwd)":/app -w /app example_image \
  pip install -r requirements.txt

# example succeeding
docker run --rm -v "$(pwd)":/app -w /app example_image \
  pip-compile --generate-hashes -o requirements.txt requirements-pass.in
docker run --rm -v "$(pwd)":/app -w /app example_image \
  pip install -r requirements.txt

Here's a related discussion I created on Celery project with additional workarounds/solutions inside:
celery/celery#7202

I've also setup a repository to quickly give this a try:
https://github.com/GabLeRoux/pylint-celery-setuptools-issue

[Pierre-Sassoulas]

astroid can't pin dependencies, especially setuptools. That would very quickly create impossible to solve conflict in the package using astroid as a dependency. If you want a pinned build you need to pin the dependencies yourself. See pylint-dev/pylint#5485 for a more verbose explanation.