Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Pex skipping pandas activation #1017

Closed
blackstardlb opened this issue Aug 3, 2020 · 2 comments · Fixed by #1105
Closed

Pex skipping pandas activation #1017

blackstardlb opened this issue Aug 3, 2020 · 2 comments · Fixed by #1105
Assignees
@jsirois
Labels
bug resolver

Comments

@blackstardlb
Copy link

blackstardlb commented Aug 3, 2020 •
edited
Loading

Running pex pandas==1.0.5 -v using python 3.6.2 outputs.
pex: Skipping activation of `pandas==1.0.5; python_version >= "3.6.1"` due to environment marker de-selection
Thus import pandas results in :
Traceback (most recent call last): File "<console>", line 1, in <module> ModuleNotFoundError: No module named 'pandas'
@blackstardlb
Copy link
Author

blackstardlb commented Aug 3, 2020 •
edited
Loading

It works if i use pex==2.1.6 and fails for everything about that version

@jsirois
Copy link
Member

jsirois commented Aug 7, 2020

In pex 2.1.7 we started honoring python_requires metadata in #923 fixing #898 which explains your observation that this problem does not exist in pex 2.1.6.

It looks like that fix has a bug when converting Requires-Python wheel metadata into an equivalent environment marker:
https://github.com/pantsbuild/pex/blob/e5b62caddbf4070521980f54984a01ed55251bf9/pex/resolver.py#L103-L117

Whereas Requires-Python allows for full version specifiers as per PEP-440, The python_version environment marker just uses the major and minor versions (https://www.python.org/dev/peps/pep-0508/#environment-markers) so converting all Requires-Python constraints to python_version constraints is incorrect. For any constraint that includes a patch level in the version constraint the python_full_version environment marker should be used.

Thanks for pinning down the Pex version where things went wrong, that was helpful pinpointing this.

@jsirois jsirois self-assigned this Aug 7, 2020
@jsirois jsirois mentioned this issue Aug 7, 2020
Closed
3 tasks
@jsirois jsirois mentioned this issue Sep 14, 2020
Closed
4 tasks
This was referenced Oct 2, 2020
Closed
Closed
jcuquemelle added a commit to criteo/cluster-pack that referenced this issue Oct 13, 2020
@jcuquemelle
pandas pinned to < 1.0.0
89192d2 
because pex-tool/pex#1017
@Eric-Arellano Eric-Arellano mentioned this issue Oct 15, 2020
Closed
7 tasks
@jsirois jsirois mentioned this issue Nov 6, 2020
Closed
jsirois added a commit to jsirois/pex that referenced this issue Nov 7, 2020
@jsirois
Fix Requires-Python environment marker mapping.
702e54b 
Previously `Requires-Python` metadata was unconditionally mapped to
`python_version` environment marker clauses. This was incorrect for any
`Requires-Python` clauses that included version information beyond
`X.Y` and prevented activation of requirements in PEX files at runtime
in some cases. A previously failing test is added for the pandas case
that revealed this issue.

Fixes pex-tool#1017
@jsirois jsirois mentioned this issue Nov 7, 2020
Merged
jsirois added a commit that referenced this issue Nov 7, 2020
@jsirois
Fix Requires-Python environment marker mapping. (#1105)
ef99275 
Previously `Requires-Python` metadata was unconditionally mapped to
`python_version` environment marker clauses. This was incorrect for any
`Requires-Python` clauses that included version information beyond
`X.Y` and prevented activation of requirements in PEX files at runtime
in some cases. A previously failing test is added for the pandas case
that revealed this issue.

Fixes #1017
@jsirois jsirois mentioned this issue Aug 1, 2021
Closed
nateagr pushed a commit to nateagr/cluster-pack that referenced this issue Aug 2, 2021
Unipined version of Pex
8a367e3 
Torch could not be packaged with the previously pinned version due
to a bug fixed in 2.1.21
(same bug as described here pex-tool/pex#1017)
@nateagr nateagr mentioned this issue Aug 2, 2021
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@jsirois jsirois

Labels
bug resolver
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Fix Requires-Python environment marker mapping. jsirois/pex
2 participants
@jsirois @blackstardlb