[BUG] ERROR: No matching distribution found for setuptools_scm_git_archive>=1.0 with setuptools-62.6.0

[atodorov]

setuptools version

setuptools==62.6.0

Python version

Python 3.8

OS

Linux, Ubuntu, GitHub CI

Additional environment information

Even if setuptools is not pinned to a specific version the test job fails in GitHub CI, but passes locally on my machine.

Description

Between two subsequent executions in our GitHub CI the same job, named build-for-pypi (3.8) failed with:

      ERROR: Could not find a version that satisfies the requirement setuptools_scm_git_archive>=1.0 (from versions: 1.0, 1.1, 1.2, 1.2.1, 1.3)
      ERROR: No matching distribution found for setuptools_scm_git_archive>=1.0
      [end of output]
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error
× pip subprocess to install build dependencies did not run successfully.
│ exit code: 1
╰─> See above for output.

When failing the version of setuptools is setuptools-62.6.0

A previous commit passed the same job, with setuptools-59.6.0 (hard-coded in the test script as a workaround)

Passing commit - June 13 - kiwitcms/Kiwi@971b9f8
Failing commit - June 27 - kiwitcms/Kiwi@94efca1

The change between the 2 commits is unrelated to the failing job.

The test job itself will build our own package and then try installing it into a clean venv. First tries to install only tarballs, next tries to install only wheel packages. The failure happens when trying to install tarballs. Previously we had pinned setuptools to 59.6.0 in order to workaround django-extensions/django-extensions#1694

Expected behavior

No failure

How to Reproduce

1. git clone [email protected]:kiwitcms/Kiwi.git
2. cd Kiwi
3. ./tests/check-build

Output

https://github.com/kiwitcms/Kiwi/runs/7109573424?check_suite_focus=true

[abravalheri]

Hi @atodorov, thank you very much for reporting this.

How to Reproduce

I believe that there are missing steps here (I tried to run it and I got some unrelated errors). Probably instructions on how to set the test dependencies in a virtual env?

[abravalheri]

I did the following:

virtualenv -p py3.8 .venv
.venv/bin/python -m pip install -U pip setuptools wheel
# pip        22.1.2
# setuptools 62.6.0
# wheel      0.37.1
.venv/bin/python setup.py sdist
.venv/bin/python setup.py bdist_wheel
ls dist/*
# dist/kiwitcms-11.3.tar.gz  dist/kiwitcms-11.3-py3-none-any.whl
.venv/bin/python -m pip install -U dist/kiwitcms-11.3-py3-none-any.whl
.venv/bin/python -m pip uninstall -y kiwitcms
# Successfully installed ... kiwitcms-11.3 ...
.venv/bin/python -m pip install -U dist/kiwitcms-11.3.tar.gz
# Successfully installed kiwitcms-11.3
.venv/bin/python -m pip install twine
.venv/bin/python -m twine check dist/*
# Checking dist/kiwitcms-11.3-py3-none-any.whl: PASSED with warnings
# WARNING  `long_description_content_type` missing. defaulting to `text/x-rst`.
# Checking dist/kiwitcms-11.3.tar.gz: PASSED with warnings
# WARNING  `long_description_content_type` missing. defaulting to `text/x-rst`.

Both pip end twine seem to be dealing with things well. Maybe your test environment has dependencies that are too restrictive and conflicting with each other?

Could you please provide a minimal reproduction for your error? (e.g. isolating the conflicting dependencies?)

[atodorov]

I managed to narrow it down to:

$(which python) -m venv .venv/test-setuptools-failure
source .venv/test-setuptools-failure/bin/activate
pip install --upgrade setuptools pip
pip install --no-binary :all: jira==3.2.0

Again, works for me on Python 3.8.10, Fedora 32 but fails in GitHub CI, Ubuntu 20.04, Python 3.8.13.

[atodorov]

FTR as a workaround I did pip install setuptools-scm-git-archive before trying to install the tarballs.

[abravalheri]

That is a tricky one...

I am using Ubuntu 20.04.4 LTS, Python 3.8.10 and I could execute the commands above without any problem.

Since there is no binary available for Python 3.8.10 yet in the Ubuntu repositories, I also tried the following:

> docker run --rm -it ubuntu:20.04 bash
apt install -y git wget curl gcc build-essential libssl-dev zlib1g-dev libbz2-dev libreadline-dev libsqlite3-dev libncurses5-dev libncursesw5-dev xz-utils tk-dev libffi-dev make
# There might be some questions to answer regarding the time zone
cd /tmp
wget https://www.python.org/ftp/python/3.8.13/Python-3.8.13.tgz
tar xf Python-3.8.13.tgz
cd Python-3.8.13
./configure --enable-optimizations --with-ensurepip=install
make -j 8
make altinstall
python3.8 --version
# Python 3.8.13
python3.8 -m pip install -U setuptools pip
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh  # Yeah, apparently we need rust now !?!
# Rust will also ask you some questions, I just went with the default
python3.8 -m pip install --no-binary :all: jira==3.2.0
# Successfully installed SecretStorage-3.3.2 certifi-2022.6.15 cffi-1.15.0 charset-normalizer-2.0.12 cryptography-37.0.2 defusedxml-0.7.1 idna-3.3 importlib-metadata-4.12.0 jeepney-0.8.0 jira-3.2.0 keyring-23.6.0 oauthlib-3.2.0 pycparser-2.21 requests-2.28.0 requests-oauthlib-1.3.1 requests-toolbelt-0.9.1 typing-extensions-4.2.0 urllib3-1.26.9 zipp-3.8.0

Again there was no problem with this installation.

@atodorov, unfortunately, it seems that this problem is very specific to the environment in the Github CI. We would need a standalone reproducer to be able to investigate it...

I think it this issue is unlikely to be related to setuptools... It seems to be more of a constraint problem or cyclic dependencies. But still to be sure we would need to be able to reproduce it.

Side note: I would like to recommend not using --no-binary. There are a few problems regarding bootstrapping of packages (lately it was reported some problems with setuptools-scm and importlib-metadata). You can read more about it in pypa/packaging-problems#342.