Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Introduce pex3 venv inspect. #2135

Merged
merged 3 commits into from
Apr 28, 2023
Merged

Conversation

jsirois
Copy link
Member

@jsirois jsirois commented Apr 28, 2023

This command allows inspecting venvs created by Pex as well as those
created by other tools.

Work towards #1752 and #2110

This command allows inspecting venvs created by Pex as well as those
created by other tools.

Work towards pex-tool#1752 and pex-tool#2110
@jsirois jsirois requested review from zmanji and huonw April 28, 2023 00:53
@jsirois
Copy link
Member Author

jsirois commented Apr 28, 2023

N.B.: pex3 venv create will follow this, but this establishes the tool sub-command namespace and is useful in its own right to, for example, determine the proper site-packages directory for a pex venv (there can be more than one!) if you want to copy out its contents.

@jsirois
Copy link
Member Author

jsirois commented Apr 28, 2023

For example, a Pants venv created by the ./pants script:

$ python -mpex.cli venv inspect ~/.cache/pants/setup/bootstrap-Linux-x86_64/2.14.1_py39/ -i2
{
  "venv_dir": "/home/jsirois/.cache/pants/setup/bootstrap-Linux-x86_64/2.14.1_py39",
  "provenance": {
    "created_by": "virtualenv 20.4.7",
    "is_pex": false,
    "pex_version": null
  },
  "include_system_site_packages": false,
  "interpreter": {
    "binary": "/home/jsirois/.cache/pants/setup/bootstrap-Linux-x86_64/2.14.1_py39/bin/python",
    "base_binary": "/usr/bin/python3.9",
    "version": "3.9.16",
    "sys_path": [
      "/usr/lib/python39.zip",
      "/usr/lib/python3.9",
      "/usr/lib/python3.9/lib-dynload",
      "/home/jsirois/.cache/pants/setup/bootstrap-Linux-x86_64/2.14.1_py39/lib/python3.9/site-packages"
    ]
  },
  "script_dir": "/home/jsirois/.cache/pants/setup/bootstrap-Linux-x86_64/2.14.1_py39/bin",
  "scripts": [
    "normalizer",
    "pants",
    "pex",
    "pex-tools",
    "pex3",
    "pip",
    "pip3",
    "pip3.9",
    "wheel",
    "wheel-3.9",
    "wheel3",
    "wheel3.9"
  ],
  "site_packages": "/home/jsirois/.cache/pants/setup/bootstrap-Linux-x86_64/pants.2KoUMU/install/lib/python3.9/site-packages",
  "distributions": [
    "PyYAML==6.0",
    "ansicolors==1.1.8",
    "certifi==2022.12.7",
    "charset-normalizer==3.0.1",
    "fasteners==0.16.3",
    "humbug==0.2.7",
    "idna==3.4",
    "ijson==3.1.4",
    "importlib-resources==5.0.7",
    "packaging==21.3",
    "pantsbuild.pants==2.14.1",
    "pex==2.1.108",
    "pip==23.0",
    "psutil==5.9.0",
    "pyparsing==3.0.9",
    "python-lsp-jsonrpc==1.0.0",
    "requests==2.28.2",
    "setproctitle==1.2.2",
    "setuptools==63.4.3",
    "six==1.16.0",
    "toml==0.10.2",
    "types-PyYAML==6.0.3",
    "types-setuptools==62.6.1",
    "types-toml==0.10.8",
    "typing_extensions==4.3.0",
    "ujson==5.7.0",
    "urllib3==1.26.14",
    "wheel==0.36.2"
  ]
}

Copy link
Collaborator

@zmanji zmanji left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This code lgtm.

It might be a good idea to write an integration test against a more modern virtualenv or other widely used virtualenv creating tool (pipenv?).

@jsirois
Copy link
Member Author

jsirois commented Apr 28, 2023

It might be a good idea to write an integration test against a more modern virtualenv or other widely used virtualenv creating tool (pipenv?).

Good, idea - done for virtualenv. Pipenv is not even official any more if you follow PyPA discourse ... it's Hatch. I'll just stick to virtualenv if you agree since that invented all this and is used by several tools behind the scenes. I may rue the choice though, I'm sure there are PEP-405 violators out there and I'll find that out the hard way :/.

return self.__pyvenv_cfg

@property
def created_by(self):
Copy link
Collaborator

@huonw huonw Apr 28, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The code that writes a pyvenv.cfg file above explicitly sets created_by = ..., but it seems like this created_by method wouldn't reflect that value back out, instead showing virtualenv ...; is that expected?

Copy link
Member Author

@jsirois jsirois Apr 28, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is expected. I did choose a confusing name I guess reading the code, but the name was picked for the reader of the file.

The created_by, as with all fields except home and include-system-site-packages, are arbitrary and un-defined by PEP-405. I just wanted to leave a marker that this was not created by virtualenv to help avoid confusing someone poking about in the venv later. Although modern virtualenv does create a pyvenv.cfg, this ancient single-file version of virtualenv Pex vendors does not.

Clearly though I'm just causing different confusion! I did consider emitting a #-style comment line with the same content.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All good, sounds reasonable. If it really mattered (it doesn't to me), one could use a synonym in the pyvenv.cfg, e.g. creator = ... or something, unless there's some non-PEP-405 convention that's guiding the choice.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No convention. There is in fact no similar field at all in any pyvenv.cfg I've seen. This appears to be unique.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I may do something about this, at least comment, in the pex3 venv create PR CI burn cycle.

@jsirois jsirois merged commit f1646b7 into pex-tool:main Apr 28, 2023
@jsirois jsirois deleted the pex3/venv/inspect branch April 28, 2023 05:07
jsirois added a commit to jsirois/pex that referenced this pull request Apr 28, 2023
This code as well as selection of a unique target to resolve for will
be re-used by the forthcoming `pex3 venv create` command.

Along the way, clean up confusion noted in pex-tool#2135 in the introduction
of the `pex3 venv` subcommand surrounding `pyvenv.cfg` files generated
by `Virtualenv`.

Work towards pex-tool#1752 and pex-tool#2110.
jsirois added a commit that referenced this pull request Apr 28, 2023
This code as well as selection of a unique target to resolve for will be
re-used by the forthcoming `pex3 venv create` command.

Along the way, clean up confusion noted in #2135 in the introduction of
the `pex3 venv` sub-command surrounding `pyvenv.cfg` files generated by
`Virtualenv`.

Work towards #1752 and #2110.
@jsirois jsirois mentioned this pull request May 1, 2023
4 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants