No supported version of Pip is compatible with the given targets

[netapp-vaughan]

Describe the bug
I am unable to run pants fix. I received the error No supported version of Pip is compatible with the given targets.

Pants version
2.22.0rc2

OS
MacOS Arm64

Additional info
`
❯ pants fix src/python/
19:52:28.91 [ERROR] 1 Exception encountered:

Engine traceback:
in fix goal

ProcessExecutionFailure: Process 'Find interpreter for constraints: CPython<4,>=3.7' failed with exit code 1.
stdout:

stderr:
No supported version of Pip is compatible with the given targets:
cp310-cp310-macosx_14_0_arm64 interpreter at /Users/vaughand/.pyenv/versions/3.10.12/bin/python3.10
cp310-cp310-macosx_14_0_arm64 interpreter at /Users/vaughand/.pyenv/versions/3.10.13/bin/python3.10
cp310-cp310-macosx_14_0_arm64 interpreter at /Users/vaughand/.pyenv/versions/3.10.14/bin/python3.10
cp310-cp310-macosx_14_0_arm64 interpreter at /Users/vaughand/.pyenv/versions/3.10.9/bin/python3.10
cp310-cp310-macosx_14_0_arm64 interpreter at /opt/homebrew/Cellar/[email protected]/3.10.15/Frameworks/Python.framework/Versions/3.10/bin/python3.10
cp311-cp311-macosx_14_0_arm64 interpreter at /Users/vaughand/.pyenv/versions/3.11.1/bin/python3.11
cp311-cp311-macosx_14_0_arm64 interpreter at /Users/vaughand/.pyenv/versions/3.11.3/bin/python3.11
cp311-cp311-macosx_14_0_arm64 interpreter at /Users/vaughand/.pyenv/versions/3.11.4/bin/python3.11
cp311-cp311-macosx_14_0_arm64 interpreter at /Users/vaughand/.pyenv/versions/3.11.8/bin/python3.11
cp311-cp311-macosx_14_0_arm64 interpreter at /opt/homebrew/Cellar/[email protected]/3.11.10/Frameworks/Python.framework/Versions/3.11/bin/python3.11
cp312-cp312-macosx_14_0_arm64 interpreter at /Users/vaughand/.pyenv/versions/3.12.4/bin/python3.12
cp312-cp312-macosx_14_0_arm64 interpreter at /opt/homebrew/Cellar/[email protected]/3.12.7_1/Frameworks/Python.framework/Versions/3.12/bin/python3.12
cp313-cp313-macosx_14_0_arm64 interpreter at /opt/homebrew/Cellar/[email protected]/3.13.0_1/Frameworks/Python.framework/Versions/3.13/bin/python3.13
cp37-cp37m-macosx_14_0_arm64 interpreter at /Users/vaughand/.pyenv/versions/3.7.17/bin/python3.7
cp38-cp38-macosx_14_0_arm64 interpreter at /Users/vaughand/.pyenv/versions/3.8.10/bin/python3.8
cp39-cp39-macosx_14_0_arm64 interpreter at /Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.9/bin/python3.9
cp39-cp39-macosx_14_0_arm64 interpreter at /Users/vaughand/.pyenv/versions/3.9.12/bin/python3.9
cp39-cp39-macosx_14_0_arm64 interpreter at /Users/vaughand/.pyenv/versions/3.9.13/bin/python3.9
`
I can execute pip in the above /bin folders without trouble.

[medflowdev]

Try using pants 2.23.0. It solved the same issue for me.

[benjyw]

Sorry for the trouble. We'd need a lot more information to debug this. Ideally a small github repo that reproduces it (and whose README explains how to reproduce it). There are so many moving parts in your config, BUILD files etc, that it's not possible to say with confidence what the solution is without seeing that.

That said, if pants 2.23.0 works for you, as @medflowdev suggests, then... :)

[jsirois]

You definitely do not need more information to debug this.

The listed ICs are CPython<4,>=3.7. So a Pip that works for 3.7 & 3.13. Such a Pip does not exist since Pip depended on distutils and Python ripped that out of the stdlib in 3.12. This more or less coincided with Python 3.7 dropping off long term support; so Pip upping its lower bound to Python 3.8. There was also another Python-induced break in there with importlib deprecation removals in 3.13 + Pip's use of the removed code. IOW, both Pip and Python are semi-break-happy (unlike Pex); so you really need to stay on top of what each release of those 2 decides to break through their deprecation system. Not awesome for lower level parts of the stack.

It may be that this knowledge is spread amongst Pants people, but its definitely there. Certainly @huonw knows all this at least?

[bmcandr]

FWIW, I also encountered this after recently upgrading to 2.22. Just tried updating to 2.23 and it seems to have resolved the issue.

[jsirois]

FWIW, here are some of the existing issues documenting this problem:

• Deprecate using Python 3.7 with default tool lockfiles, preparing for pip 24.1 / Python 3.13 support #21103
• Built-in lockfiles won't work with Python 3.13, due to pip version #20852

Here is at least one of the fixes: #21389
Here is the relevant blog post: https://www.pantsbuild.org/blog/2024/08/24/venerable-pythons

[netapp-vaughan]

Thanks Guys.
An update; if I comment out this backend_package, it works.

# "pants.backend.build_files.fmt.black",

Note that it also errors on 2.23.0 if I enable that plugin.

[jsirois]

@netapp-vaughan, assuming @cburroughs knew what he was talking about, that observation confirms his note here: https://www.pantsbuild.org/blog/2024/08/24/venerable-pythons#user-content-fn-1-a1cb28

This seems to be confirmed as well by:

• 2.23:

  • pants/src/python/pants/backend/python/subsystems/python_tool_base.py

    Line 65 in 6fad8c6

    default_interpreter_constraints: ClassVar[Sequence[str]] = ["CPython>=3.7,<4"]

  • pants/src/python/pants/backend/python/lint/black/black.lock

    Lines 535 to 544 in 6fad8c6

    	"pex_version": "2.2.1",
    	"pip_version": "24.0",
    	"prefer_older_binary": false,
    	"requirements": [
    	"black<24,>=22.6.0",
    	"typing-extensions>=3.10.0.0; python_version < \"3.10\""
    	],
    	"requires_python": [
    	"<4,>=3.7"
    	],

• 2.24:

  • pants/src/python/pants/backend/python/subsystems/python_tool_base.py

    Line 65 in b399758

    default_interpreter_constraints: ClassVar[Sequence[str]] = ["CPython>=3.8,<4"]

  • pants/src/python/pants/backend/python/lint/black/black.lock

    Lines 287 to 296 in b399758

    	"pex_version": "2.16.2",
    	"pip_version": "24.2",
    	"prefer_older_binary": false,
    	"requirements": [
    	"black<24,>=22.6.0",
    	"typing-extensions>=3.10.0.0; python_version < \"3.10\""
    	],
    	"requires_python": [
    	"<4,>=3.8"
    	],

@netapp-vaughan maybe you missed this bit of the blog post, snipped:

Starting in Pants version 2.23 2.24 1, the default tool lockfile bundled with Pants will no longer support Python 3.7. Pants will still build and test Python 3.7 code! But you will first need to create your own tool lockfiles.

That teaches you how to solve this problem without disabling black in older Pants versions.

[bmcandr]

Is there any other change in 2.23 that may have resolved this? I can reliably reproduce the error by downgrading to 2.22 and resolve it by upgrading to 2.23. Both 2.22 and 2.23 show the Find interpreter for constraints: CPython<4,>=3.7 message when executing goals like fix which makes sense if the default interpreter constraint hasn't changed.

I use the black backend too and have not created a lockfile for it.

[jsirois]

@bmcandr do you have a Python 3.13 installed and on the PATH?

[bmcandr]

Yep. Here's the error from 2.22:

No supported version of Pip is compatible with the given targets:
cp310-cp310-macosx_15_0_x86_64 interpreter at /Users/brendanmcandrew/.pyenv/versions/3.10.13/bin/python3.10
cp310-cp310-macosx_15_0_x86_64 interpreter at /Users/brendanmcandrew/.pyenv/versions/3.10.14/bin/python3.10
cp310-cp310-macosx_15_0_x86_64 interpreter at /Users/brendanmcandrew/.pyenv/versions/3.10.4/bin/python3.10
cp310-cp310-macosx_15_0_x86_64 interpreter at /usr/local/Cellar/[email protected]/3.10.15/Frameworks/Python.framework/Versions/3.10/bin/python3.10
cp311-cp311-macosx_15_0_x86_64 interpreter at /Users/brendanmcandrew/.pyenv/versions/3.11.6/bin/python3.11
cp311-cp311-macosx_15_0_x86_64 interpreter at /usr/local/Cellar/[email protected]/3.11.10/Frameworks/Python.framework/Versions/3.11/bin/python3.11
cp312-cp312-macosx_15_0_x86_64 interpreter at /usr/local/Cellar/[email protected]/3.12.7_1/Frameworks/Python.framework/Versions/3.12/bin/python3.12
cp313-cp313-macosx_15_0_x86_64 interpreter at /usr/local/Cellar/[email protected]/3.13.0_1/Frameworks/Python.framework/Versions/3.13/bin/python3.13
cp38-cp38-macosx_15_0_x86_64 interpreter at /Users/brendanmcandrew/.pyenv/versions/3.8.13/bin/python3.8
cp38-cp38-macosx_15_0_x86_64 interpreter at /Users/brendanmcandrew/.pyenv/versions/3.8.17/bin/python3.8
cp38-cp38-macosx_15_0_x86_64 interpreter at /Users/brendanmcandrew/.pyenv/versions/3.8.18/bin/python3.8
cp39-cp39-macosx_15_0_x86_64 interpreter at /Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.9/bin/python3.9
cp39-cp39-macosx_15_0_x86_64 interpreter at /Users/brendanmcandrew/.pyenv/versions/3.9.13/bin/python3.9
cp39-cp39-macosx_15_0_x86_64 interpreter at /Users/brendanmcandrew/.pyenv/versions/3.9.18/bin/python3.9
cp39-cp39-macosx_15_0_x86_64 interpreter at /usr/local/Cellar/[email protected]/3.9.20/Frameworks/Python.framework/Versions/3.9/bin/python3.9

[jsirois]

Aha, but you do not have 3.7 installed like @netapp-vaughan. The later Pips supported by Pants 2.23 support Python >=3.8. In Pants 2.22 the latest available Pips supported >=3.7

[jsirois]

Unless I'm mistaken, a Pants maintainer can close this is an answered question. AFAICT there are 0 mysteries left.

[jsirois]

@bmcandr perhaps to make things more clear: even though Pip claims a Requires-Python of >=3.7 for older versions and >=3.8 for newer versions, it turns out these claims can be lies on the open end of the range. Pex tests all versions of Pip (it supports) against all versions of Python it supports (>=2.7,!=3.0.*,!=3.1.*,!=3.2.*,!=3.3.*,!=3.4.*,<3.14) and knows when Pip lies; so it writes this down.

Relevant here, Pip 24.0 for Pants 2.22:

• https://github.com/pex-tool/pex/blob/50ff75aacb0ee0efb797a9956dfac2f13c23ac4f/pex/pip/version.py#L269-L276

And Pip 24.2 for Pants 2.24.0a0:

• https://github.com/pex-tool/pex/blob/50ff75aacb0ee0efb797a9956dfac2f13c23ac4f/pex/pip/version.py#L299-L304

You'll note the former does not cover your range (which is cp38-cp313, cp313 being the not covered version), where the latter does (Pex is currently running test shards against 3.14 alphas; so knows Pip works with 3.14; thus the <3.15 upper bound).

[bmcandr]

Thanks @jsirois, that was a helpful explanation.

[benjyw]

Thanks @jsirois !