Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement support for preserving and injecting Python args. #2427

Merged
merged 6 commits into from
Jun 12, 2024

Conversation

jsirois
Copy link
Member

@jsirois jsirois commented Jun 11, 2024

Support preserving and injecting Python args.

Pex now supports --inject-python-args similar to --inject-args but
for specifying arguments to pass to the Python interpreter as opposed to
arguments to pass to the application code. This is supported for both
zipapp and venv execution modes as well as standard shebang launching,
launching via a Python explicitly or via the --sh-boot mechanism.

In addition, PEX files now support detecting and passing through Python
args embedded in shebangs or passed explicitly on the command line for
all Pythons Pex supports save for PyPy<3.10 where there appears to be
no facility to retrieve the original argv PyPy was executed with.

Closes #2422

This is the code that ends up in PEX `__main__.py` and it could benefit
from formatting, linting and type checking.
This support does not work for `PyPy<3.10` but it does work for all
other Pythons Pex supports.
@jsirois jsirois requested review from zmanji, benjyw and huonw June 11, 2024 23:33
@jsirois
Copy link
Member Author

jsirois commented Jun 11, 2024

This was a good idea from @bkad and it completes the work started by @BrandonTheBuilder in #1746 at PyCon US 2022. I think Pex now supports all Python interpreter args that make sense in all modes of operation.

@zmanji
Copy link
Collaborator

zmanji commented Jun 11, 2024

The tests make sense to me and capture the original request of passing -u to the interpreter.

Copy link
Collaborator

@huonw huonw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nifty

Comment on lines +175 to +180
orig_args = orig_argv()
if orig_args:
for index, arg in enumerate(orig_args[1:], start=1):
if os.path.exists(arg) and os.path.samefile(entry_point, arg):
python_args.extend(orig_args[1:index])
break
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added some manual timing around this during development - I was a bit worried about the ctypes impl since there is no opt-out even if you don't use Python args and this is in the hot path. The timings were consistently sub-ms for both sys.orig_argv and the ctypes implementation. Almost all the ~O(100us) time for the ctypes version was spent in the imports above where the function is defined. The call here was much faster.

@jsirois jsirois merged commit 451e507 into pex-tool:main Jun 12, 2024
26 checks passed
@jsirois jsirois deleted the issues/2422 branch June 12, 2024 03:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

A way to configure sys.executable flags at PEX runtime
3 participants