Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Can't run PEX file when a dependency's wheel includes a build tag #964

Closed
luislew opened this issue May 9, 2020 · 0 comments
Closed

Can't run PEX file when a dependency's wheel includes a build tag #964

luislew opened this issue May 9, 2020 · 0 comments
Labels

Comments

@luislew
Copy link
Contributor

luislew commented May 9, 2020

We recently encountered a runtime issue with a successfully built PEX file:

(nwpy-ml-clustering_linux-x86-64_cp3.7) app@8bceeea547ee:/srv/nerdwallet/nwpy-ml-clustering$ ./build/nwpy-ml-clustering.pex
Traceback (most recent call last):
  File "/srv/nerdwallet/nwpy-ml-clustering/build/nwpy-ml-clustering.pex/.bootstrap/pex/pex.py", line 391, in execute
  File "/srv/nerdwallet/nwpy-ml-clustering/build/nwpy-ml-clustering.pex/.bootstrap/pex/pex.py", line 89, in _activate
  File "/srv/nerdwallet/nwpy-ml-clustering/build/nwpy-ml-clustering.pex/.bootstrap/pex/environment.py", line 232, in activate
  File "/srv/nerdwallet/nwpy-ml-clustering/build/nwpy-ml-clustering.pex/.bootstrap/pex/environment.py", line 375, in _activate
  File "/srv/nerdwallet/nwpy-ml-clustering/build/nwpy-ml-clustering.pex/.bootstrap/pex/environment.py", line 209, in _update_candidate_distributions
  File "/srv/nerdwallet/nwpy-ml-clustering/build/nwpy-ml-clustering.pex/.bootstrap/pex/environment.py", line 227, in can_add
  File "/srv/nerdwallet/nwpy-ml-clustering/build/nwpy-ml-clustering.pex/.bootstrap/pex/vendor/_vendored/packaging/packaging/tags.py", line 75, in parse_tag
ValueError: too many values to unpack (expected 3)

We tracked this down to the following installed wheel: 'llvmlite-0.29.0-1-cp37-cp37m-manylinux1_x86_64.whl'. The current wheel filename parsing code in environment.py assumes that there is no build tag: https://github.com/pantsbuild/pex/blob/916e61e04634c60d09ee25859c76ba4f27282e51/pex/environment.py#L223

As a result, it includes the build tag with wheel_tags (e.g. 1-cp37-cp37m-manylinux1_x86_64) and parse_tag fails.

Per PEP 427, wheel filenames may include an optional build tag: https://www.python.org/dev/peps/pep-0427/#file-name-convention


Proposed fix: pex already vendors wheel, which includes a regex matcher for wheel filenames. Just use the WHEEL_INFO_RE to parse the wheel filename.

Even simpler - split the whole filename, and gather up the trailing 3 tags:

wheel_tags = '-'.join(filename.split('-')[-3:])
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants