pex fails with InvalidSpecifier if any dist has a bad version specifier #2289

travisdowns · 2023-11-16T15:24:35Z

After upgrading from 2.1.100 to 2.1.151 ~~and so pulling #2264~~ pex started failing to execute locally with the following error:

Traceback (most recent call last):
  File "/usr/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "~/.pex/unzipped_pexes/003cdfcf7d5635cb324a06f8b49939c255f2add1/__main__.py", line 108, in <module>
    bootstrap_pex(__entry_point__, execute=__execute__, venv_dir=__venv_dir__)
  File "pex/pex_bootstrapper.py", line 627, in bootstrap_pex
    pex.PEX(entry_point).execute()
  File "pex/pex.py", line 542, in execute
    self.activate()
  File "pex/pex.py", line 242, in activate
    self.patch_sys()
  File "pex/pex.py", line 445, in patch_sys
    new_sys_path, new_sys_path_importer_cache, new_sys_modules = self.minimum_sys(inherit_path)
  File "pex/pex.py", line 415, in minimum_sys
    sys_path, sys_path_importer_cache = self.minimum_sys_path(isolated_sys_path, inherit_path)
  File "pex/pex.py", line 351, in minimum_sys_path
    site_distributions.update(all_distribution_paths(path_element))
  File "pex/pex.py", line 345, in all_distribution_paths
    locations = set(dist.location for dist in find_distributions([path]))
  File "pex/pex.py", line 345, in <genexpr>
    locations = set(dist.location for dist in find_distributions([path]))
  File "pex/dist_metadata.py", line 1004, in find_distributions
    location=location, metadata=DistMetadata.from_metadata_files(metadata_files)
  File "pex/dist_metadata.py", line 758, in from_metadata_files
    requires_python=requires_python(metadata_files),
  File "pex/dist_metadata.py", line 516, in requires_python
    return SpecifierSet(python_requirement)
  File "pex/vendor/_vendored/packaging_23_1/packaging/specifiers.py", line 714, in __init__
    parsed.add(Specifier(specifier))
  File "pex/vendor/_vendored/packaging_23_1/packaging/specifiers.py", line 245, in __init__
    raise InvalidSpecifier(f"Invalid specifier: '{spec}'")
pex.vendor._vendored.packaging_23_1.packaging.specifiers.InvalidSpecifier: Invalid specifier: '>=3.5.*'

We managed to trace this back to apache_libcloud-3.2.0.egg-info/PKG-INFO which has the following line:

Requires-Python: >=3.5.*, <4

Although this seems to be a bad specifier , some components presumably accept it as the package was working in that state for years, but not the packaging stuff used by pex.

Now this was a bug in libcloud [1], but I think pex should not fail entirely if some package has this in its version info. ~~This was exposed by #2264 because before that this PKG-INFO file was not parsed at all.~~

In any case libcloud has been fixed years ago apparently but of course not every transitive dependency has necessarily been updated to use a fixed version, etc.

The text was updated successfully, but these errors were encountered:

jsirois · 2023-11-16T17:10:03Z

@travisdowns I think #2264 may be a bad diagnosis. What prior version of Pex was this same scenario working with?

jsirois · 2023-11-16T17:35:03Z

Yeah, the bisect on Pex versions leads to some change in 2.1.140:

$ pex --python python3.8 pex==2.1.139 -cpex -- apache_libcloud==3.2.0 -- -c 'import libcloud; print(libcloud.__file__)'
/home/jsirois/.pex/installed_wheels/e41b8670b5b2cb9b8bf1557f099a0e75dcb0ed9e9b2d667a3a0c1fbec0fa11bb/apache_libcloud-3.2.0-py2.py3-none-any.whl/libcloud/__init__.py
$ pex --python python3.8 pex==2.1.140 -cpex -- apache_libcloud==3.2.0 -- -c 'import libcloud; print(libcloud.__file__)'
Invalid specifier: '>=3.5.*'

jsirois · 2023-11-16T17:44:51Z

Yeah, this bisects to #2188 which upgraded vendored packaging from the PyPA to 23.1 for Python>=3.7 (packaging 21.3 is used for Python 3.6 and packaging 20.9 is used for any older Python). IOW, the official way to parse any metadata (whether from a dist-info dist (wheel) or PKG-INFO), got stricter in enforcing the PyPAs own spec in PEP-440.

jsirois · 2023-11-16T17:48:16Z

So, @travisdowns Pex could contemplate ignoring bad Requires-Python metadata, except that it actually uses the metadata as part of the tests used at boot-time to decide which embedded dists to activate. This is part of providing multi-platform PEXes. So to drop the check when it fails would mean Pex yolos on your behalf which is definitely against the grain of Pex philosophy of hermetic, exact, verified.

Instead of continuing down that line of thought, perhaps this can be pre-empted by just fixing libcloud? VCS specifiers are very handy for this sort of thing.

jsirois · 2023-11-16T17:54:31Z

Closing the loop on the exact error cause. It looks to be right here (which was included in the packaging 23.1 release): pypa/packaging@068a0b5#diff-91bbf04171e694df3ef07a8fffc168bfa43e2f00cb531ee37035987210ae0371R230

IOW, PyPA adding better enforcement of their own spec.

travisdowns · 2023-11-16T17:58:24Z

@travisdowns I think #2264 may be a bad diagnosis.

You could be right: I thought before this change the .egg-info wasn't examined at all, which would avoid this, but I could be mistaken there.

What prior version of Pex was this same scenario working with?

2.1.100

travisdowns · 2023-11-16T18:02:34Z

Instead of continuing down that line of thought, perhaps this can be pre-empted by just fixing libcloud? VCS specifiers are very handy for this sort of thing.

Libcloud was fixed a couple of years ago but I can't go and fix all the transitive dependees to update their version. Among other places this version of libcloud is the default shipped with the most recent Ubuntu LTS 22.04, a very popular distro.

except that it actually uses the metadata as part of the tests used at boot-time to decide which embedded dists to activate. This is part of providing multi-platform PEXes. So to drop the check when it fails would mean Pex yolos on your behalf which is definitely against the grain of Pex philosophy of hermetic, exact, verified.

I don't know much about pex to make a very intelligent comment, but wouldn't treating a failed check in the same way as an unsatisfied python version be better? I.e., don't activate the dist.

which is definitely against the grain of Pex philosophy of hermetic, exact, verified.

Fair enough, but as it is it is totally not hermetric: installing any random package (even non-python one) on the host may bring in this file and its mere existence on the filesystem means that any existing pex just stops working. So in this sense it seems like it is not at all hermetic.

jsirois · 2023-11-16T18:05:31Z

You could be right: I thought before this change the .egg-info wasn't examined at all, which would avoid this, but I could be mistaken there.

It would absolutely not avoid this. The same metadata is present in the whl METADATA file.

I don't know much about pex to make a very intelligent comment, but wouldn't treating a failed check in the same way as an unsatisfied python version be better? I.e., don't activate the dist.

Ok. But then you'd fail in a different way when the code that needs libcloud failed to import it!

Fair enough, but as it is it is totally not hermetric: installing any random package (even non-python one) on the host may bring in this file and its mere existence on the filesystem means that any existing pex just stops working.

That is just straight up false. That is not what is happening here. See my note above about .whl METADATA.

Libcloud was fixed a couple of years ago but I can't go and fix all the transitive dependees to update their version. Among other places this version of libcloud is the default shipped with the most recent Ubuntu LTS 22.04, a very popular distro.

It's as easy as this...
So, for example just specify apache_libcloud @ git+https://github.com/jsirois/libcloud@2cde5937fdfae56113f80f18c7005a4c691a06e5 as a direct requirement or via a constraints file:

$ pex -V
2.1.151
$ pex --python python3.11 "apache_libcloud @ git+https://github.com/jsirois/libcloud@2cde5937fdfae56113f80f18c7005a4c691a06e5" -- -c 'import libc
loud; print(libcloud.__file__)'
/home/jsirois/dev/jsirois/libcloud/libcloud/__init__.py

That is my fork of the project with a patch applied on top of the v3.2.0 tagged commit.

jsirois · 2023-11-16T18:09:03Z

@travisdowns let me know if my explanation and workaround are sticking for you. If so, I'd like to close this as an answered question. If not, we can discuss in more detail why the workaround is not sufficient or other means of amelioration.

travisdowns · 2023-11-16T18:14:29Z

It would absolutely not avoid this. The same metadata is present in the whl METADATA file.

We did not find any METADATA file with this string, only the PKG-INFO. It comes from the python3-libcloud apt package in 22.04 ubuntu repos.

Ok. But then you'd fail in a different way when the code that needs libcloud failed to import it!

That seems like a good approach: after all libcloud is broken, so it would be up to the user to update it. "Breaking any pex that tries to use libcloud" seems much better than "breaking every pex on the system as soon as the libcloud package is even installed". Note that we did not need libcloud in any way, it was just incidentally installed on the system for some other reason (probably not even explicitly but just as a transitive dependency of some other package).

That is just straight up false. That is not what is happening here. See my note above about .whl METADATA.

I am reporting what I saw. pex fails on Ubuntu 22.04 if python3-libcloud is installed. After removing this package, pex immediately works. If installing an unrelated package which isn't used by any pex and is not used by pex itself causes pex bootstrap to break that is "not hermetic", I think? I didn't understand how the .whl file relates to hermeticity, sorry.

So, for example just specify apache_libcloud @ git+https://github.com/jsirois/libcloud@2cde5937fdfae56113f80f18c7005a4c691a06e5 as a direct requirement or via a constraints file:

As above, and perhaps I did not make this clear enough initially, we are not using libcloud. It was just brought in at some unknown point by the host package manager, probably as a transitive dependency.

We don't need a fix: we have already "fixed" this issue by removing the unused package, before this bug was reported. I thought I'd report it here for the benefit of pex maintainers and other users but I'm sure starting to regret my choice.

jsirois · 2023-11-16T18:39:01Z

As above, and perhaps I did not make this clear enough initially, we are not using libcloud. It was just brought in at some unknown point by the host package manager, probably as a transitive dependency.

We don't need a fix: we have already "fixed" this issue by removing the unused package, before this bug was reported. I thought I'd report it here for the benefit of pex maintainers and other users but I'm sure starting to regret my choice.

Yeah, that was totally unclear. You did alot of work figuring out what was going on it seems, which is great, but it was not detailed here. If the bug report included the Pex build time command line and runtime command line (this is a runtime error), for example, it would have made it clear libcloud was not a direct or transitive requirement. You have provided more details now though, which is great.

I am reporting what I saw. pex fails on Ubuntu 22.04 if python3-libcloud is installed. After removing this package, pex immediately works. If installing an unrelated package which isn't used by any pex and is not used by pex itself causes pex bootstrap to break that is "not hermetic", I think?

Ok, that is new information - for me. Thank you.

I repro as such:

$ cat Dockerfile
FROM ubuntu:22.04

RUN apt update && \
    apt install -y \
        python3-dev \
        python3-distutils \
        python3-venv \
        python3-libcloud

RUN python3 -m venv pex.venv && \
    pex.venv/bin/pip install pex

ENV PATH=pex.venv/bin:$PATH

$ docker build -tpex-issues-2289 .
$ docker run --rm -it pex-issues-2289
root@83e1b2ff9d09:/# pex -V
2.1.152
root@83e1b2ff9d09:/# pex cowsay -c cowsay -o cowsay.pex
root@83e1b2ff9d09:/# ./cowsay.pex
Traceback (most recent call last):
  File "/usr/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/root/.pex/unzipped_pexes/994f8ec4b043ab1eb64b76a7f53d81e99fce2604/__main__.py", line 108, in <module>
    bootstrap_pex(__entry_point__, execute=__execute__, venv_dir=__venv_dir__)
  File "/root/.pex/unzipped_pexes/994f8ec4b043ab1eb64b76a7f53d81e99fce2604/.bootstrap/pex/pex_bootstrapper.py", line 627, in bootstrap_pex
    pex.PEX(entry_point).execute()
  File "/root/.pex/unzipped_pexes/994f8ec4b043ab1eb64b76a7f53d81e99fce2604/.bootstrap/pex/pex.py", line 542, in execute
    self.activate()
  File "/root/.pex/unzipped_pexes/994f8ec4b043ab1eb64b76a7f53d81e99fce2604/.bootstrap/pex/pex.py", line 242, in activate
    self.patch_sys()
  File "/root/.pex/unzipped_pexes/994f8ec4b043ab1eb64b76a7f53d81e99fce2604/.bootstrap/pex/pex.py", line 445, in patch_sys
    new_sys_path, new_sys_path_importer_cache, new_sys_modules = self.minimum_sys(inherit_path)
  File "/root/.pex/unzipped_pexes/994f8ec4b043ab1eb64b76a7f53d81e99fce2604/.bootstrap/pex/pex.py", line 415, in minimum_sys
    sys_path, sys_path_importer_cache = self.minimum_sys_path(isolated_sys_path, inherit_path)
  File "/root/.pex/unzipped_pexes/994f8ec4b043ab1eb64b76a7f53d81e99fce2604/.bootstrap/pex/pex.py", line 351, in minimum_sys_path
    site_distributions.update(all_distribution_paths(path_element))
  File "/root/.pex/unzipped_pexes/994f8ec4b043ab1eb64b76a7f53d81e99fce2604/.bootstrap/pex/pex.py", line 345, in all_distribution_paths
    locations = set(dist.location for dist in find_distributions([path]))
  File "/root/.pex/unzipped_pexes/994f8ec4b043ab1eb64b76a7f53d81e99fce2604/.bootstrap/pex/pex.py", line 345, in <genexpr>
    locations = set(dist.location for dist in find_distributions([path]))
  File "/root/.pex/unzipped_pexes/994f8ec4b043ab1eb64b76a7f53d81e99fce2604/.bootstrap/pex/dist_metadata.py", line 1004, in find_distributions
    location=location, metadata=DistMetadata.from_metadata_files(metadata_files)
  File "/root/.pex/unzipped_pexes/994f8ec4b043ab1eb64b76a7f53d81e99fce2604/.bootstrap/pex/dist_metadata.py", line 758, in from_metadata_files
    requires_python=requires_python(metadata_files),
  File "/root/.pex/unzipped_pexes/994f8ec4b043ab1eb64b76a7f53d81e99fce2604/.bootstrap/pex/dist_metadata.py", line 516, in requires_python
    return SpecifierSet(python_requirement)
  File "/root/.pex/unzipped_pexes/994f8ec4b043ab1eb64b76a7f53d81e99fce2604/.bootstrap/pex/vendor/_vendored/packaging_23_1/packaging/specifiers.py", line 714, in __init__
    parsed.add(Specifier(specifier))
  File "/root/.pex/unzipped_pexes/994f8ec4b043ab1eb64b76a7f53d81e99fce2604/.bootstrap/pex/vendor/_vendored/packaging_23_1/packaging/specifiers.py", line 245, in __init__
    raise InvalidSpecifier(f"Invalid specifier: '{spec}'")
pex.vendor._vendored.packaging_23_1.packaging.specifiers.InvalidSpecifier: Invalid specifier: '>=3.5.*'
root@83e1b2ff9d09:/# apt remove -y python3-libcloud
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
The following packages were automatically installed and are no longer required:
  python3-certifi python3-cffi-backend python3-chardet python3-cryptography python3-idna python3-lockfile python3-pkg-resources python3-requests python3-simplejson python3-six python3-urllib3
Use 'apt autoremove' to remove them.
The following packages will be REMOVED:
  python3-libcloud
0 upgraded, 0 newly installed, 1 to remove and 12 not upgraded.
After this operation, 19.6 MB disk space will be freed.
(Reading database ... 14211 files and directories currently installed.)
Removing python3-libcloud (3.2.0-2) ...
root@83e1b2ff9d09:/# ./cowsay.pex -t 'Moo!'
  ____
| Moo! |
  ====
    \
     \
       ^__^
       (oo)\_______
       (__)\       )\/\
           ||----w |
           ||     ||
root@83e1b2ff9d09:/#

travisdowns · 2023-11-16T18:58:01Z

With this reproducer where which pex commit does the failure bisect to?

jsirois · 2023-11-16T19:02:49Z

I'm sure its the one you identified. That seems clear without bisecting. The fuller picture is what I needed here. I apologize for not being able to read the tea leaves of the initial bug report.

jsirois · 2023-11-16T19:25:02Z

Alrighty, this should clean up easily. The Pex runtime was doing alot of un-necessary work since ~this date 9 years ago it appears in its sys.path scrubbing implementation. Before this change, each scrubbed distribution in each scrubbed path element was logged, but no more. Since then even that bit of use has been absent, and collecting dists was un-needed work.

jsirois · 2023-11-16T19:28:27Z

Aha, thinking more, in 2014 there were egg zips as possible sys.path sub-elements; so find_distributions was necessary at the time to find such un-exploded eggs. Since none of the Pex machinery, 1st or (vendored) 3rdparty, handles eggs any longer, this is safe to remove.

travisdowns · 2023-11-16T19:35:27Z

Thanks for the RCA, and apology accepted.

jsirois · 2023-11-16T19:36:16Z

I am becoming pretty frustrated with OSS expectations. What part of my behavior do you consider unsatisfactory? I apologized for nothing here. I was just trying to run down a bug, and sometimes that takes 20 questions.

jsirois · 2023-11-16T19:37:49Z

Ah, I guess I did apologize - but that was a bit tongue in cheek.

jsirois · 2023-11-16T19:39:24Z

FWIW this bug will be fixed and released in under 8 hours start to finish. I would guess you don't get that good a response time in many OSS projects; especially those maintained gratis. I am not payed for this or any other software work.

This was a relic of the days of egg zips; and the `dist.location` data is now fully redundant with the `sys.path` element input to find the dist in the 1st place. Fixes pex-tool#2289

jsirois · 2023-11-16T19:47:44Z

The fix will go out in 2.1.153: #2291

travisdowns · 2023-11-16T20:26:09Z

Thanks, the quick turnaround is much appreciated.

This was a relic of the days of egg zips; and the `dist.location` data is now fully redundant with the `sys.path` element input to find the dist in the 1st place. Fixes #2289

jsirois · 2023-11-16T21:36:26Z

Ok, using the same Dockerfile from above, the repro case is now fixed:

$ docker run --rm -it pex-issues-2289
root@c60f39ca3347:/# pex -V
2.1.152
root@c60f39ca3347:/# pex cowsay -ccowsay -o cowsay.pex
root@c60f39ca3347:/# ./cowsay.pex
Traceback (most recent call last):
  File "/usr/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/root/.pex/unzipped_pexes/994f8ec4b043ab1eb64b76a7f53d81e99fce2604/__main__.py", line 108, in <module>
    bootstrap_pex(__entry_point__, execute=__execute__, venv_dir=__venv_dir__)
  File "/root/.pex/unzipped_pexes/994f8ec4b043ab1eb64b76a7f53d81e99fce2604/.bootstrap/pex/pex_bootstrapper.py", line 627, in bootstrap_pex
    pex.PEX(entry_point).execute()
  File "/root/.pex/unzipped_pexes/994f8ec4b043ab1eb64b76a7f53d81e99fce2604/.bootstrap/pex/pex.py", line 542, in execute
    self.activate()
  File "/root/.pex/unzipped_pexes/994f8ec4b043ab1eb64b76a7f53d81e99fce2604/.bootstrap/pex/pex.py", line 242, in activate
    self.patch_sys()
  File "/root/.pex/unzipped_pexes/994f8ec4b043ab1eb64b76a7f53d81e99fce2604/.bootstrap/pex/pex.py", line 445, in patch_sys
    new_sys_path, new_sys_path_importer_cache, new_sys_modules = self.minimum_sys(inherit_path)
  File "/root/.pex/unzipped_pexes/994f8ec4b043ab1eb64b76a7f53d81e99fce2604/.bootstrap/pex/pex.py", line 415, in minimum_sys
    sys_path, sys_path_importer_cache = self.minimum_sys_path(isolated_sys_path, inherit_path)
  File "/root/.pex/unzipped_pexes/994f8ec4b043ab1eb64b76a7f53d81e99fce2604/.bootstrap/pex/pex.py", line 351, in minimum_sys_path
    site_distributions.update(all_distribution_paths(path_element))
  File "/root/.pex/unzipped_pexes/994f8ec4b043ab1eb64b76a7f53d81e99fce2604/.bootstrap/pex/pex.py", line 345, in all_distribution_paths
    locations = set(dist.location for dist in find_distributions([path]))
  File "/root/.pex/unzipped_pexes/994f8ec4b043ab1eb64b76a7f53d81e99fce2604/.bootstrap/pex/pex.py", line 345, in <genexpr>
    locations = set(dist.location for dist in find_distributions([path]))
  File "/root/.pex/unzipped_pexes/994f8ec4b043ab1eb64b76a7f53d81e99fce2604/.bootstrap/pex/dist_metadata.py", line 1004, in find_distributions
    location=location, metadata=DistMetadata.from_metadata_files(metadata_files)
  File "/root/.pex/unzipped_pexes/994f8ec4b043ab1eb64b76a7f53d81e99fce2604/.bootstrap/pex/dist_metadata.py", line 758, in from_metadata_files
    requires_python=requires_python(metadata_files),
  File "/root/.pex/unzipped_pexes/994f8ec4b043ab1eb64b76a7f53d81e99fce2604/.bootstrap/pex/dist_metadata.py", line 516, in requires_python
    return SpecifierSet(python_requirement)
  File "/root/.pex/unzipped_pexes/994f8ec4b043ab1eb64b76a7f53d81e99fce2604/.bootstrap/pex/vendor/_vendored/packaging_23_1/packaging/specifiers.py", line 714, in __init__
    parsed.add(Specifier(specifier))
  File "/root/.pex/unzipped_pexes/994f8ec4b043ab1eb64b76a7f53d81e99fce2604/.bootstrap/pex/vendor/_vendored/packaging_23_1/packaging/specifiers.py", line 245, in __init__
    raise InvalidSpecifier(f"Invalid specifier: '{spec}'")
pex.vendor._vendored.packaging_23_1.packaging.specifiers.InvalidSpecifier: Invalid specifier: '>=3.5.*'
root@c60f39ca3347:/# pex.venv/bin/pip install -U pex
Requirement already satisfied: pex in /pex.venv/lib/python3.10/site-packages (2.1.152)
Collecting pex
  Downloading pex-2.1.153-py2.py3-none-any.whl (3.0 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3.0/3.0 MB 9.4 MB/s eta 0:00:00
Installing collected packages: pex
  Attempting uninstall: pex
    Found existing installation: pex 2.1.152
    Uninstalling pex-2.1.152:
      Successfully uninstalled pex-2.1.152
Successfully installed pex-2.1.153
root@c60f39ca3347:/# pex -V
2.1.153
root@c60f39ca3347:/# pex cowsay -ccowsay -o cowsay.pex
root@c60f39ca3347:/# ./cowsay.pex -t 'Moo!'
  ____
| Moo! |
  ====
    \
     \
       ^__^
       (oo)\_______
       (__)\       )\/\
           ||----w |
           ||     ||
root@c60f39ca3347:/#
exit

travisdowns · 2023-11-16T21:39:37Z

Very thorough and quick, thanks John.

oseiberts11 · 2023-11-28T14:40:25Z

For the record, we ran into a similar case which seems to be fixed with this fix:

/usr/lib/python3/dist-packages/httpie-1.0.3.egg-info/requires.txt which contains among others

[python_version == "3.0" or python_version == "3.1"]
argparse>=1.2.1

which got glued together to a bad requirement string (an attempt at nesting quotes):

Traceback (most recent call last):
  File "/root/.pex/unzipped_pexes/83f486fa8e217f5631f0c37bbc88e1e6cfbc3dfa/.bootstrap/pex/vendor/_vendored/packaging_23_1/packaging/requirements.py", line 35, in __init__
    parsed = _parse_requirement(requirement_string)
  File "/root/.pex/unzipped_pexes/83f486fa8e217f5631f0c37bbc88e1e6cfbc3dfa/.bootstrap/pex/vendor/_vendored/packaging_23_1/packaging/_parser.py", line 64, in parse_requirement
    return _parse_requirement(Tokenizer(source, rules=DEFAULT_RULES))
  File "/root/.pex/unzipped_pexes/83f486fa8e217f5631f0c37bbc88e1e6cfbc3dfa/.bootstrap/pex/vendor/_vendored/packaging_23_1/packaging/_parser.py", line 83, in _parse_requirement
    tokenizer.expect("END", expected="end of dependency specifier")
  File "/root/.pex/unzipped_pexes/83f486fa8e217f5631f0c37bbc88e1e6cfbc3dfa/.bootstrap/pex/vendor/_vendored/packaging_23_1/packaging/_tokenizer.py", line 140, in expect
    raise self.raise_syntax_error(f"Expected {expected}")
  File "/root/.pex/unzipped_pexes/83f486fa8e217f5631f0c37bbc88e1e6cfbc3dfa/.bootstrap/pex/vendor/_vendored/packaging_23_1/packaging/_tokenizer.py", line 165, in raise_syntax_error
    raise ParserSyntaxError(
pex.vendor._vendored.packaging_23_1.packaging._tokenizer.ParserSyntaxError: Expected end of dependency specifier
    argparse>=1.2.1; extra == "python_version == "3.0" or python_version == "3.1""
                                                 ^

jsirois self-assigned this Nov 16, 2023

travisdowns changed the title ~~pex fails if any dist has a bad version~~ pex fails with InvalidSpecifier if any dist has a bad version specifier Nov 16, 2023

jsirois added the bug label Nov 16, 2023

jsirois added the in progress label Nov 16, 2023

jsirois mentioned this issue Nov 16, 2023

Remove Pex runtime scrubbing dist discovery. #2290

Merged

jsirois closed this as completed in #2290 Nov 16, 2023

jsirois removed the in progress label Nov 16, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

pex fails with InvalidSpecifier if any dist has a bad version specifier #2289

pex fails with InvalidSpecifier if any dist has a bad version specifier #2289

travisdowns commented Nov 16, 2023 •

edited

Loading

jsirois commented Nov 16, 2023

jsirois commented Nov 16, 2023

jsirois commented Nov 16, 2023

jsirois commented Nov 16, 2023

jsirois commented Nov 16, 2023

travisdowns commented Nov 16, 2023

travisdowns commented Nov 16, 2023

jsirois commented Nov 16, 2023 •

edited

Loading

jsirois commented Nov 16, 2023

travisdowns commented Nov 16, 2023 •

edited

Loading

jsirois commented Nov 16, 2023

travisdowns commented Nov 16, 2023

jsirois commented Nov 16, 2023

jsirois commented Nov 16, 2023

jsirois commented Nov 16, 2023 •

edited

Loading

travisdowns commented Nov 16, 2023

jsirois commented Nov 16, 2023

jsirois commented Nov 16, 2023

jsirois commented Nov 16, 2023

jsirois commented Nov 16, 2023

travisdowns commented Nov 16, 2023

jsirois commented Nov 16, 2023

travisdowns commented Nov 16, 2023

oseiberts11 commented Nov 28, 2023 •

edited

Loading

pex fails with InvalidSpecifier if any dist has a bad version specifier #2289

pex fails with InvalidSpecifier if any dist has a bad version specifier #2289

Comments

travisdowns commented Nov 16, 2023 • edited Loading

jsirois commented Nov 16, 2023

jsirois commented Nov 16, 2023

jsirois commented Nov 16, 2023

jsirois commented Nov 16, 2023

jsirois commented Nov 16, 2023

travisdowns commented Nov 16, 2023

travisdowns commented Nov 16, 2023

jsirois commented Nov 16, 2023 • edited Loading

jsirois commented Nov 16, 2023

travisdowns commented Nov 16, 2023 • edited Loading

jsirois commented Nov 16, 2023

travisdowns commented Nov 16, 2023

jsirois commented Nov 16, 2023

jsirois commented Nov 16, 2023

jsirois commented Nov 16, 2023 • edited Loading

travisdowns commented Nov 16, 2023

jsirois commented Nov 16, 2023

jsirois commented Nov 16, 2023

jsirois commented Nov 16, 2023

jsirois commented Nov 16, 2023

travisdowns commented Nov 16, 2023

jsirois commented Nov 16, 2023

travisdowns commented Nov 16, 2023

oseiberts11 commented Nov 28, 2023 • edited Loading

travisdowns commented Nov 16, 2023 •

edited

Loading

jsirois commented Nov 16, 2023 •

edited

Loading

travisdowns commented Nov 16, 2023 •

edited

Loading

jsirois commented Nov 16, 2023 •

edited

Loading

oseiberts11 commented Nov 28, 2023 •

edited

Loading