Should pip install -e foo be interpreted as a directory, or a package name?

[charliermarsh]

Description

(Apologies for using the issue tracker to ask a question, but I figured it'd be nice if we have similar behavior here across pip and uv.)

If foo is an empty local directory, then...

• pip install foo fails with Could not find a version that satisfies the requirement foo (i.e., it treats it as a PEP 508 requirement without a version specifier).
• pip install ./foo fails with Invalid requirement: './foo' (i.e., it treats it as an unnamed requirement).

However:

• pip install -e foo fails with foo does not appear to be a Python project (i.e., it treats it as an unnamed requirement).
• pip install -e ./foo fails with foo does not appear to be a Python project (i.e., it treats it as an unnamed requirement).

So there's a difference right now in interpretation between pip install foo vs. pip install -e foo, presumedly because PEP 508 editables aren't yet supported.

My question is: when PEP 508 editables are supported, should pip install -e foo be interpreted as the path foo, or as a named PEP 508 requirement (e.g. foo==*)?

Expected behavior

No response

pip version

24.0

Python version

3.12.3

OS

macOS

How to Reproduce

mkdir foo
pip install foo
pip install -e foo

Output

No response

Code of Conduct

• I agree to follow the PSF Code of Conduct.

[pfmoore]

Can you clarify what you mean by "PEP 508 editables"? PEP 508 doesn't refer to editabe installs anywhere (and the docs for -e state that it takes a path or URL, not a requirement).

[charliermarsh]

I am referring to the idea that --editable should some day support the PEP 508 syntax as described here: #1289 (comment). I know it does not support PEP 508 syntax today. My question is: if PEP 508 syntax were supported for --editable (assuming that is the intent), how would -e foo be interpreted?

[pfmoore]

Ah, OK. I'm not sure I agree with that comment, as I don't actually think that (for example) pip install -e requests>=2.0 makes any sense¹. But if we do, backward compatibility probably means that we'd have to keep the current behaviour, at least for a deprecation period.

I don't believe that equivalence between normal and editable installs is a useful principle - they are very different operations, used for very different purposes. The fact that we use install -e rather than some other command (like develop, following from the setuptools original design) is mostly a historical wart, IMO.

(This is another case where I'd prefer it if uv designed a better UI, rather than following pip's "because it's what people expect". But that's not my decision to make.)

Footnotes

1. I'm not even convinced of the usefulness of allowing a URL instead of a local path, but people do use that functionality, so I guess we're stuck with it. ↩

[uranusjr]

Personally I don’t think installing a PEP 508 requirement as an editable makes sense. Either a wheel or sdist, the fetched artifact cannot always be “edited” in a way meaningful to end users. It may be possible to come up with a quasi-reasonable behaviour for some of the most trivial cases, but it falls apart quite quickly. The only possible alternative to the current behaviour, IMO, would be to make pip install -e foo always fail, which I don’t think is an improvement.

[charliermarsh]

Completely agree that pip install -e "requests>=2.0" does not make sense and don't mean to imply that it should be supported. However, pip install -e "flask[dotenv] @ file:///path/to/requests" does make sense, right? That is also a PEP 508 requirement and is not supported today. Does pip intend to support it?

[charliermarsh]

(For what it's worth, we do not support editables for VCS or URL dependencies in uv -- we only support local paths.)

[pfmoore]

Does pip intend to support it?

As of right now, no-one has provided a PR. There's #9471, but that got closed unmerged. And that was solely for name@vcs_url - there was no intention that I could see to support non-VCS URLs, or extras. Personally, I think it would be misleading to support a subset of PEP 508, so we'd be better with a custom syntax (which is what the #egg= fragment is today, but we want to deprecate the term "egg", so maybe just rename it?)

@sbidoul is, I believe, the pip maintainer with most interest in this, so he may have further insight.

[sbidoul]

Does pip intend to support it?

I personally think we should support -e "name @ URL", to get rid of our last use of #egg fragments. URL must be restricted to a file URL pointing to a directory, or a VCS URL (side note: I personally consider the latter as legacy). The tracking issue for this is #1289

deprecate the term "egg", so maybe just rename it?

I had not thought about that. That would certainly be simpler to implement. On the other hand, it's another non standard syntax to learn for users, so I'd go for the PEP 508 syntax.

Should pip install -e foo be interpreted as a directory, or a package name?

A directory, I think, because considering it as a package name would not make much sense, and that is what pip does today.

[pfmoore]

On the other hand, it's another non standard syntax to learn for users, so I'd go for the PEP 508 syntax.

While I don't have a strong opinion here, I will point out that the PEP 508 style is also non standard, as it's similar to, but not identical to PEP 508. People will expect name[extra] @ url to work (see above), and the limitation to file and VCS URLs will catch some people out as well.

I'm happy to go with whatever the person writing the PR chooses, though.

[uranusjr]

Since I haven’t seen this mentioned, for completeness, pip install -e path/to/project[extras] should alreay work (IIRC).

[pfmoore]

Does it? That's not documented anywhere, as far as I can see. The docs say:

Install a project in editable mode (i.e. setuptools “develop mode”) from a local project path or a VCS url.

There's no elaboration on "local project path", so the assumption has to be that's just a path to a directory, and VCS URLs for editable installs are documented here, with the allowed fragments, #egg and #subdirectory documented here. There's no mention of extras being supported anywhere.

Looking at the code, parse_editable seems inconsistent, even with its own documentation comment. Extras are returned for file paths and file URLs, but not for VCS URLs, although the "requirement name" specified via #egg= can include extras, which are returned in a different place (in the requirement name, not in the "extras" return value). 🙁

I feel like we should either not allow extras with -e (my preference) or properly document what is allowed, and how it will behave.

Have I mentioned recently that I hate extras (and for that matter I hate editables)? 🙂

or a VCS URL (side note: I personally consider the latter as legacy)

@sbidoul I support that view. Maybe we should document that editable VCS installs are legacy only here?

[charliermarsh]

Yes, pip install -e path/to/project[extras] works. Though the behavior differs between the command-line and requirements.txt.

In requirements.txt, the general algorithm seems to be:

• Split at the first whitespace.
• Discard everything after. (This is why -e path/to/project ; python_version < '3.7' "works" -- pip just throws out the markers.)
• If the URL ends in a hard bracket, find open bracket, split on it, and parse the extras between them. You can have multiple comma-separated extras, but you cannot have a space between them, e.g., path/to/project[a, b] gets interpreted as the exact file path path/to/project[a,.

On the command-line, -e "path/to/project[a, b]" is allowed (space between extras). However, -e "path/to/project[a, b] ; python_version < '3.7'" is not (pip says it's not a valid editable).

[pfmoore]

Note that none of that is documented (at best, there's some examples that suggest what the expected behaviour is).

If we want any sort of consistency between pip and uv, we should start by defining something that we all agree is a reasonable set of functionality and syntax. Once we have that, we can then consider how to implement it, and which project needs to change. I certainly wouldn't commit pip to keeping the current behaviour unchanged.