feature: Perform operations in a venv without switching to it.

[frostming]

Is your feature request related to a problem? Please describe.

The original idea comes from this comment by @hynek

Describe the solution you'd like

Introduce a new option --venv VENV so that when given a name, the matching virtual environment will be used as the environment for the particular operation, or PDM will raise an error if none matches.

• The supported operations: install, sync, add, remove, update, list, info, run, show, use.
• This requires python.use_venv to be true.

Some other ideas

We will not extend the PDM scripts to automatically provision an environment when running pdm run <task>, since 1) it will require to supply with more information, such as which python to use to create that environment, which lockfile to use, and which groups to install; and 2) we don't want to compete with tox and nox, it should be easy to achieve the desired behavior with the said tools. Users need to prepare the environments as a preceding step.

[hynek]

Let’s brainstorm workflows real quick.

Currently, my initialization looks something like this:

PY=$(command -v python3.11)
pdm venv create $PY
pdm use -i .venv/bin/python
pdm sync --dev

The minimal implementation of above would be something like this:

pdm venv create -n lint $PY
pdm sync —-venv=lint -G lint

And then:

pdm run —venv=lint lint

I think that’s a pretty good low-level primitive that opens PDM up to new use-cases!

---

I would still like to point out that you shouldn’t be that dismissive of tox-like behavior. When I develop a package, I use something like tox / nox but for apps – and that’s what I use PDM for due to it’s great locking – I would be happy to drop them!

For most people, apps have only one Python version and don’t need any kind of matrices and that’s where I feel the line should be. Even if you develop a package you usually have one main Python version that you use but don’t necessarily want the dependencies from sphinx in the venv that runs your tests. And you run [tn]ox to verify it works across versions.

Don’t get me wrong: the above feature is already great and I’ll hide the invocations behind a Justfile or Makefile, but it would be really cool if you would consider a way to define venvs and map them to commands:

[tool.pdm.venvs]
lint = { groups=["lint"] }

[tool.pdm.scripts]
lint = { venv= "lint", command="flake8" }

Then you can say something like: pdm venv --init-from-config path/to/python

That would keep it declarative yet explicit.

The venv="lint" part would be nice to have in any case and would just imply pdm run --venv=lint

[frostming]

Don’t get me wrong: the above feature is already great and I’ll hide the invocations behind a Justfile or Makefile, but it would be really cool if you would consider a way to define venvs and map them to commands:

It seems not that different from a tox/hactch env, comparing:

[tool.pdm.scripts.lint]
groups = ["lint"]
command = "flake8"

If you map the env with a script, there is no need to create a venv manually for the script anymore. Running pdm run lint will create a venv named lint with the specific dependencies under the hood.

[hynek]

I’m not 100% sure how to parse your response. Are agreeing it would be nice, saying it’s bad, or even suggesting it already works and I’m bad at reading docs? 😅

[frostming]

i mean it would be a step too far into the field of tox, and i'd rather leave the possibility to a plugin on it's own, instead of building into pdm.

[hynek]

I’m open to building my own plugins (like I did with hatch-fancy-pypi-readme :)).

By your estimation: does PDM have all the necessary APIs to build a plugin that does what I’ve outlined?

[frostming]

By your estimation: does PDM have all the necessary APIs to build a plugin that does what I’ve outlined?

I think so, but there will be a refactor around the environments module, which is relevant to the plugin.