Add support for --extra-pip-requirement.
#2461
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
You can now inject extra requirements into the isolated Pip PEXes Pex uses to resolve distributions. The motivating use case for this is to use the feature Pip 23.1 introduced for forcing `--keyring-provider import`. Pex already supported using a combination of the following to force non-interactive use of the keyring: 1. A `keyring` script installation that was on the `PATH` 2. A `--pip-version` 23.1 or newer. 3. Specifying `--use-pip-config` to pass `--keyring-provider subprocess` to Pip. You could not force `--keyring-provider import` though since the Pips Pex uses are themselves hermetic PEXes without access to extra installed keyring requirements elsewhere on the system. With `--extra-pip-requirement` you can do this with the primary benefit over `--keyring-provider subprocess` being that you do not need to add the username to index URLs. This is ultimately because the keyring CLI requires username whereas the API does not; but see https://pip.pypa.io/en/stable/topics/authentication/#keyring-support for more information.
jsirois
force-pushed
the
pip/keyring/full-support
branch
from
f003932
to
91b1ed6
Compare
jsirois
commented
Jul 11, 2024
| @@ -119,61 +117,11 @@ def mitmdump_venv(shared_integration_test_tmpdir): | |||
|
|
|||
|
|
|||
| @pytest.fixture | |||
| def run_proxy( | |||
There was a problem hiding this comment.
I factored this out to testing/mitmproxy.py to re-use mitmproxy in the new test in its reverse proxy mode to wrap PyPI / devpi with required auth.
jsirois
commented
Jul 11, 2024
Comment on lines
+108
to
139
| if not resolver: | ||
| raise ValueError( | ||
| "A resolver is required to install extra {requirements} for vendored Pip: " | ||
| "{extra_requirements}".format( | ||
| requirements=pluralize(extra_requirements, "requirement"), | ||
| extra_requirements=" ".join(map(str, extra_requirements)), | ||
| ) | ||
| ) | ||
|
|
||
| # This indirection works around MyPy type inference failing to see that | ||
| # `iter_distribution_locations` is only successfully defined when resolve is not None. | ||
| extra_requirement_resolver = resolver | ||
|
|
||
| def iter_distribution_locations(): | ||
| # type: () -> Iterator[str] | ||
| for location in expose_vendored(): | ||
| yield location | ||
|
|
||
| for resolved_distribution in extra_requirement_resolver.resolve_requirements( | ||
| requirements=tuple(map(str, extra_requirements)), | ||
| targets=Targets.from_target(LocalInterpreter.create(interpreter)), | ||
| pip_version=PipVersion.VENDORED, | ||
| extra_resolver_requirements=(), | ||
| ).distributions: | ||
| yield resolved_distribution.distribution.location | ||
|
|
||
| return _pip_installation( | ||
| version=PipVersion.VENDORED, | ||
| iter_distribution_locations=lambda: third_party.expose( | ||
| ("pip", "setuptools"), interpreter=interpreter | ||
| ), | ||
| iter_distribution_locations=iter_distribution_locations, | ||
| interpreter=interpreter, | ||
| fingerprint=_fingerprint(extra_requirements), | ||
| ) |
There was a problem hiding this comment.
N.B.: The PR was motivated by Pip keyring but is more generic, allowing extra pip requirements for any reason under the sun that comes up. As such, even though keyring support does not work with vendored Pip, --extra-pip-requirement should work with vendored Pip.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
You can now inject extra requirements into the isolated Pip PEXes Pex
uses to resolve distributions. The motivating use case for this is to
use the feature Pip 23.1 introduced for forcing
--keyring-provider import.Pex already supported using a combination of the following to force
non-interactive use of the keyring:
keyringscript installation that was on thePATH--pip-version23.1 or newer.--use-pip-configto pass--keyring-provider subprocessto Pip.
You could not force
--keyring-provider importthough since the PipsPex uses are themselves hermetic PEXes without access to extra installed
keyring requirements elsewhere on the system. With
--extra-pip-requirementyou can do this with the primary benefit over--keyring-provider subprocessbeing that you do not need to add theusername to index URLs. This is ultimately because the keyring CLI
requires username whereas the API does not; but see
https://pip.pypa.io/en/stable/topics/authentication/#keyring-support for
more information.