Permit --no-use-pep517 with editable mode in more cases.

[cjerdonek]

Fixes #6434.

The two new messages this PR introduces are as follows:

1. "build-backend" present and use_pep517=None raises InstallationError:

Error installing 'my-package': editable mode is not supported for pyproject.toml-style projects. This project is pyproject.toml-style because it has a pyproject.toml file and a "build-backend" key for the "build_system" value, but editable mode is undefined for pyproject.toml-style projects. Since the project has a setup.py, you may pass --no-use-pep517 to opt out of pyproject.toml-style processing. However, this is an unsupported combination. See PEP 517 for details on pyproject.toml-style projects.

2. "build-backend" present and use_pep517=False logs warning:

WARNING: Installing 'my-package' in editable mode, which is not supported for pyproject.toml-style projects: this project is pyproject.toml-style because it has a pyproject.toml file and a "build-backend" key for the "build_system" value, but editable mode is undefined for pyproject.toml-style projects. See PEP 517 for details on pyproject.toml-style projects.

And for comparison, I'm including one of the messages that already exists from before and remains the same.

3. "build-backend" not present and use_pep517=None raises InstallationError:

Error installing 'my-package': editable mode is not supported for pyproject.toml-style projects. pip is processing this project as pyproject.toml-style because it has a pyproject.toml file. Since the project has a setup.py and the pyproject.toml has no "build-backend" key for the "build_system" value, you may pass --no-use-pep517 to opt out of pyproject.toml-style processing. See PEP 517 for details on pyproject.toml-style projects.

[gaborbernat]

Works for me, looks good, let's get this shipped.

[cjerdonek]

I tweaked the news entry, FYI.

[gaborbernat]

Works for me 👍 Any chance for a release today so I can apply a fix in tox today too?

[cjerdonek]

That would be up to @dstufft as he'd most likely be the one to cut the release (assuming he can do it). Also, there are at least a couple other post-release fixes that should go out IMO, too, so there's a question of whether to put those in the same release or not.

[pfmoore]

No time to do more than a quick review, but this LGTM, and I specifically like the approach you've taken.

+1 on merging. I guess it's then up to @dstufft (as the person handling the 19.1 release) to look at getting it released (although if he's unavailable, someone else could take that on).

[cjerdonek]

Thanks, @pfmoore. I need to go off-line, but someone else is welcome to click the merge button.

@gaborbernat Perhaps @jaraco can offer the $50 to @dstufft instead as an enticement. :)

[jaraco]

I appreciate the work on this. Although I understand why you settled on this approach and that it’s superior to the status quo, it still will require updating every affected project to support pip 19.1.1, which was the busy work I was hoping to avoid when offering the bounty.

Edit:
Nevertheless, I’ll have to do that busy work twice, once for the 19.1 release and again for the 19.1.1 release. So I guess the value is still to be had I’m cutting a release ASAP.

@dstufft, the bounty is yours if this can release today... let me know if you want to split it with Chris.

[gaborbernat]

all tox driven projects will not need update (as tox will auto enable the feature).

[pradyunsg]

@cjerdonek @pfmoore I've been AWOL, so sorry if it feels like I'm barging out of place - is there any specific reason we're requiring packages using a setuptools backend to have to specify --no-pep-517 for editable installs?

We already didn't support other use cases for editable anyway and the PEP doesn't explicitly disallow use of editable install mechanisms, since it's outside of the scope of the PEP.

[cjerdonek]

is there any specific reason we're requiring packages using a setuptools backend to have to specify --no-pep-517 for editable installs?

@pradyunsg This is where @pfmoore first provided the rationale when I asked him about this case: #6331 (comment)

[pfmoore]

My reasoning is that what we now have does the job, while remaining with the spirit of the PEP 517 changes - we take the use of the PEP 517 pyproject.toml keys as "opt in" to PEP 517, requiring a deliberate "opt out" via --no-use-pep517 if the user wants it. And "there's no support for editable installs" is a documented (if inconvenient) aspect of PEP 517, for better or worse¹.

We could have taken -e as another means for people to opt out, but I'm not sure of the ongoing impact of that, particularly when we come to the point where PEP 517 gets support for editable installs, and we want -e to use PEP 517 :-( And I was strongly against adding yet more weird and wonderful flags without taking the time to understand the implications, so --really-dont-use-pep517 options were never something I was happy with.

Beyond that, it was mostly about finding something that we could implement quickly, but without leaving us with maintenance issues down the line that we didn't understand the consequences of.

What I mostly want to come from this is for people to look at actually adding editable mode to PEP 517. If that doesn't happen, then I'll be very disappointed, because this issue has demonstrated that there's a significant need for that work.

¹ The only real reason PEP 517 doesn't support editable installs was because the participants at the time were basically burned out working on what we did get. No-one thought they were insignificant, or that a PEP without them was the final answer. But we had to stop somewhere, and start implementing the design, or we'd never have any real world feedback to validate what we did have.

[pradyunsg]

Basically, my main concern with this approach is that we're going to have some legacy overhead for editable installs until that void is filled within our standards, but disrupting user workflows as a part of the transition would negatively affect how folks perceive pip and PyPA broadly.

"there's no support for editable installs" is a documented (if inconvenient) aspect of PEP 517

-e being an opt-out makes sense to me, precisely because PEP 517 doesn't specify the behavior. IMO it meant we should probably continue our status-quo behavior, possibly printing a warning if pyproject.toml exists, that "Hey, PEP 517 support is disabled on this because this is an editable install and the PEP doesn't cover that yet". As for the consequences of doing this, I don't think this adds any maintenance burden in the short term, and in the long term, we'll have a similar transition as legacy -> PEP 517 builds either way.

OTOH, requiring users to specify --no-pep-517 creates a lot of busy work that has to be undone once the editable discussion is done. It pushes us to have it soon, but we don't know if and when folks will have the time and energy to.

[pradyunsg]

And I was strongly against adding yet more weird and wonderful flags without taking the time to understand the implications, so --really-dont-use-pep517 options were never something I was happy with.

FWIW, I agree that adding more options is not a good idea.

With use_pep517 == True, editable installs should definitely fail. When it's None, I don't think we should fail them precisely because of the disruption it can/has caused.

[pfmoore]

I'm happy to duck out at this point and leave the decision to others. If the consensus is to revert @cjerdonek's change in favour of an alternative approach that's fine with me. There was a lot of pressure yesterday for a quick solution, and if as a result I let myself be pushed into arguing for the wrong approach, then I apologise.

[cjerdonek]

I think it would be better to discuss this on the Discourse category instead of on this closed issue.

[xavfernandez]

I'm in phase with @pradyunsg analysis.

[cjerdonek]

Please discuss here: https://discuss.python.org/t/pip-19-1-and-installing-in-editable-mode-with-pyproject-toml/1553

[pradyunsg]

@pfmoore You don't need to apologize for anything. I hope I didn't come across wrong as I don't mean to undermine the work that you and Chris have done for this release cycle. Thank you for doing the post-release firefighting and sorry for not being there.

In fact, you've stated your reasoning and position very clearly and that has definitely been very helpful.

[lock]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.