Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Un-deprecate --process-dependency-links until an alternative is implemented #4187

Closed
multimeric opened this issue Dec 19, 2016 · 72 comments · Fixed by #5571 or #5726
Closed

Un-deprecate --process-dependency-links until an alternative is implemented #4187

multimeric opened this issue Dec 19, 2016 · 72 comments · Fixed by #5571 or #5726
Labels
auto-locked Outdated issues that have been locked by automation type: maintenance Related to Development and Maintenance Processes
Milestone

Comments

@multimeric
Copy link

  • Pip version: 8.1.1
  • Python version: 3.5.2
  • Operating System: Ubuntu 16.04

Description:

The --process-dependency-links was re-added to pip a while ago because there are a number of valid use-cases for the flag: #1955

For this reason, it should probably be un-deprecated until an implementation of PEP 508/440 is in pip. The dependency_links argument to setuptools's setup is also still documented and not deprecated: http://setuptools.readthedocs.io/en/latest/setuptools.html#dependencies-that-aren-t-in-pypi

What I've run:

$ pip install request --process-dependency-links
Collecting request
  Downloading request-0.0.12.tar.gz
  DEPRECATION: Dependency Links processing has been deprecated and will be removed in a future release.
... 
@Kxrr
Copy link

Kxrr commented May 11, 2017

Agreed. Current behaviour of pip for dependency-links really sucks.
#4295
#3610

@jleclanche
Copy link

[Wrote in #3939, moving my comment here]

So, here's the main issue: If I don't want to use a requirements.txt file (because, say, I want declarative dependencies all specified in setup.cfg), how am I supposed to specify a URL as dependency?

This doesn't work:

[options]
install_requires = 
  requests==2.18.4
  https://github.com/example/example_lib/archive/master.zip

I also think dependency links are weird and fine to drop, but canonically how is this use case served if not with those?

@jleclanche
Copy link

Another issue is that dependency_links apparently cannot be specified in setup.cfg, only as setup.py arguments. If they are to be undeprecated, that should be fixed.

@robertour
Copy link

Is there any news regarding this? Is there any alternative in progress? dependency_links_ seems to be the only option for internal/private distribution of libraries.

Instead of deprecating it, it should be fixed, a lot of people forget or get confuse because they forget to put both the name-version of the library in the the install_requires and the dependency link that contains the tarbal together with the egg. As an alternative, I would love to see that @jleclanche has suggested, a simple link to the tarball in the install_requires (specifying the egg could be optional).

I don't think we are alone: https://stackoverflow.com/questions/3472430/how-can-i-make-setuptools-install-a-package-thats-not-on-pypi

@RonnyPfannschmidt
Copy link
Contributor

@robertour tools like devpi support a more clear approach since years now

@RonnyPfannschmidt
Copy link
Contributor

@dstufft btw - i would like to argue that the existene of devpi demonstrates a more clean solution to the problem making dependency links unnecessary

IMHO index inheritance + white listing/blacklisting is more consistent and more comprehensible at a ops level

@robertour
Copy link

robertour commented Sep 20, 2017

@RonnyPfannschmidt, thanks for the suggestion, it seems I am missing something important (discussions about this deprecation have been going for too long) but generally speaking I cannot agree more with this comment from reddit:

devpi could be a solution, but bringing another service into the game that needs maintenance and having to wait for some administrator to setup, will not get me into working very soon. Also we still would have the need to build packages and then put them onto devip (or has devpi a feature to track git repositories??)

Ideally, I would like an alternative that won't take me more than adding a file in my library, or few lines in the setup.py.

I basically don't understand the underlying decision of removing it, I can see how it can lead to bad practices and insecurities, but as long as it is keep for internal use, it should not be a major issue.

@pradyunsg pradyunsg added the state: needs discussion This needs some more discussion label Oct 2, 2017
@pradyunsg
Copy link
Member

#4175 means that PEP 508 URL support will be there in pip 10. I think this can be closed.

Thoughts @pfmoore @dstufft?

@pfmoore
Copy link
Member

pfmoore commented Oct 2, 2017

I thought 86b0779 meant that the @ syntax cannot be used for dependencies, so this means #4175 isn't a solution for this issue. The comments over there were that we shouldn't implement @ support for dependencies until PyPI was able to block uploads that used it (because we don't want installing something from PyPI to be able to grab stuff from arbitrary URLs, presumably).

@rgerkin
Copy link

rgerkin commented Oct 13, 2017

It shouldn't be deprecated until there is an alternative in place, i.e. PEP 508 support. Until then, too many people are using it.

@ghost
Copy link

ghost commented Nov 21, 2017

What's the proper workflow for this?

Let 's say there's a library that I need to add a feature to. I fork it on Github. Feature isn't going to be merged any time soon, so I have a tool that I set to use my Github fork instead of PyPi.

Now all of my users have to add --process-dependency-links when installing my tool with pip, which is a deprecated and even once removed flag?

Is there some option in setup.py that I'm missing or is there really no way around this? It seems like the only viable way to do this is to push up a forked pypi package. That is just going to add to user's confusion once your pull request does get merged, anyway.

@pradyunsg pradyunsg added the type: maintenance Related to Development and Maintenance Processes label Mar 5, 2018
@pradyunsg
Copy link
Member

Let's come around to this.

--process-dependency-links does not really have an alternative today. So, I'll suggest that we go ahead and un-deprecate it.

However, the flag does mean that pip would reach out to arbitrary URLs -- it should have a warning attached to it about this.

@ziima
Copy link

ziima commented Jan 29, 2019

Hi guys, I just like to note that the deprecation pathway in this case was way too short. I know that dependency links have been marked as deprecated for quite some time, but PEP 508 URLs, which can be used to replace them, haven't been implemented until 18.1. As a result there was only a 3 month long window to switch from dependency links to URL requirements, which is very short time for large projects.

newskylabs added a commit to newskylabs/newskylabs-temple that referenced this issue Mar 6, 2019
deKlerk referenced this issue in deKlerk/ghpythonremote Mar 6, 2019
Replaced pip.main() call with subprocess.check_call()
Solved installation issue reported in pilcru#12
dojeda added a commit to quetz-al/quetzal-client that referenced this issue Mar 21, 2019
Found a suitable solution to dependency_link hell using @ at the dependency
declaration as suggested in:
pypa/pip#4187 (comment)
@KevinMars
Copy link

KevinMars commented May 9, 2019

@rgerkin Hi, I am trying to follow your instructions to no avail,

Searching for PACKAGE@ git+ssh://[email protected]:OWNER/PACKAGE.git@BRANCH
Reading https://pypi.org/simple/PACKAGE/
Couldn't find index page for 'PACKAGE' (maybe misspelled?)
Scanning index of all packages (this may take a while)
Reading https://pypi.org/simple/

PACKAGE@ git+ssh://[email protected]:OWNER/PACKAGE.git@BRANCH, this is in install_requires.

Would you have an idea why I am getting the above?

@rgerkin
Copy link

rgerkin commented May 9, 2019

@KevinMars There are a few differences between my example and what you have, including the use of git_ssh, bitbucket, a.git suffix, a named branch, and no version tag. Maybe one or more of those things is leading pip to search on PyPI instead of at your URL. What version of pip are you using?

@osjerick
Copy link

To remark something that I've found: Using setup.py to install the package with python setup.py install still requires declarations of external dependencies in dependency_links.

In your setup.py file:

import os, sys
from setuptools import setup, find_packages

def read_requirements():
    """Parse requirements from requirements.txt."""
    reqs_path = os.path.join('.', 'requirements.txt')
    with open(reqs_path, 'r') as f:
        requirements = [line.rstrip() for line in f]
    return requirements

setup(
    ..., # Other stuff here
    install_requires=read_requirements(),
    )

@jw-cpnet
Copy link

@rgerkin Thanks for sharing this solution. But what if I'm using pbr to setup my Python package? How to adapt this to fit pbr?

@justinsantoro
Copy link

@KevinMars I have the same exact issue. Did you ever figure out the fix? I am trying to require a specific branch of a private bitbucket repo over SSH.

@PhilippeFerreiraDeSousa
Copy link

PhilippeFerreiraDeSousa commented Jul 1, 2019

I just realized --process-dependency-links didn't existed anymore. I'm grateful for all the work the community does. Trying to justify the decision in never ending discussions and a maze of issue closing and redirection was the chosen solution but I still think leaving this option would have harmed nobody.

@lock lock bot added the auto-locked Outdated issues that have been locked by automation label Jul 31, 2019
@lock lock bot locked as resolved and limited conversation to collaborators Jul 31, 2019
@pradyunsg pradyunsg removed state: needs discussion This needs some more discussion state: awaiting PR Feature discussed, PR is needed labels Nov 7, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
auto-locked Outdated issues that have been locked by automation type: maintenance Related to Development and Maintenance Processes
Projects
None yet