Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Command to know what file would be downloaded for a requirement #6430

Closed
uranusjr opened this issue Apr 24, 2019 · 12 comments · Fixed by #10771
Closed

Command to know what file would be downloaded for a requirement #6430

uranusjr opened this issue Apr 24, 2019 · 12 comments · Fixed by #10771
Labels
state: needs discussion This needs some more discussion type: feature request Request for a new feature

Comments

@uranusjr
Copy link
Member

uranusjr commented Apr 24, 2019

What's the problem this feature will solve?

I want to know what version (and potentially ahich artifact) would be downloaded/installed when I pip install a package. Currently I do something like pip download --no-deps --dest %TMP% and read the output, but that can take a long time if the package is large (e.g. numpy).

Describe the solution you'd like

Something like pip find numpy to only execute the finding part, but instead of downloading (and inspecting) the package, print the link that would be downloaded instead:

$ py -3.7 -m pip find numpy
https://files.pythonhosted.org/packages/4e/9d/c129d78e6b942303b762ccfdf1f8339de80c5e6021b14ef0c99ec5bdc6aa/numpy-1.16.3-cp37-cp37m-win_amd64.whl

Alternative Solutions

My current approach is described above. Alternative design includes pip install --dry-run, but that does not feel right because it implies dependency resolution (which requires the package to be downloaded).

An alternative output is to list all applicable links, instead of only the best match.

Additional context

A command that automatically download packages (to a temporary location) and produce a requirements.txt could be useful as well, but I can write a script using existing pip commands easily enough, so pip don’t really need to provide that functionality.

@dstufft
Copy link
Member

dstufft commented Apr 24, 2019

I don't think it's entirely possible (and I think the initial snippet is wrong) once we get a real version solver in place. It's entirely possible that pip install foo will traverse the dependency graph and find that the latest version of foo is being excluded for some reason by one of it's dependencies (or we cannot install the latest version because of a conflict) and an older version will instead be installed.

However, the hypothetical pip find foo that does not do dependency resolution will just grab the latest version that matches the specifier, which would be different than what pip install foo would do in the above scenario.

@uranusjr
Copy link
Member Author

Good point, I did not think of that. How about defining this as “outputting all possible links for this specification” instead? This don’t need to guarentee what would be actually installed (or anything at all), only list the links found.

My use case is exactly to feed the output to a resolver, but I don’t want to re-implement all the configuration and edge cases in pip.

@di
Copy link
Member

di commented Jul 19, 2019

In #6740 I made a similar request:

What's the problem this feature will solve?
Currently there is no way for third-party projects to use PackageFinder's logic via the command line. Instead, projects like pip-compile and pipenv need to import from the _internal API.

Describe the solution you'd like
Ideally this would mimic PackageFinder.find_all_candidates() as much as possible, and would have a human-friendly and machine-friendly format. Something like:

$ pip find bar
bar    0.1.0    https://files.pythonhosted.org/packages/2c/85/d6397edeceb1c81b5b99b1046033c8854abac3e50971c8d2915e63ce54c0/bar-0.1.0.tar.gz#sha256=6a711b5b06a662fe3f8e897310456b02127fb15d89310161fc8af7f415e270a1 (from https://pypi.org/simple/bar/)
bar    0.2.0    https://files.pythonhosted.org/packages/ba/bf/81ef8a2c80f4f86e9e6570605b7b5a277950cbd0ff9be901633159f4110e/bar-0.2.0.tar.gz#sha256=4a97e8664a22eec73f65d522cc36b4c693b61f0557760a856bb18a8fb67a8881 (from https://pypi.org/simple/bar/)
bar    0.2.1    https://files.pythonhosted.org/packages/77/d6/760c7a0ee7385ba4b0abb1d38af97e8c2f34cb47fe301315e48204088047/bar-0.2.1.tar.gz#sha256=b0a71c865729f9e8ac0f74e666b46c5ed1f5addac3425be7e06ca4828cec90ef (from https://pypi.org/simple/bar/)

$ pip find bar --format json
[
  {
    "name": "bar",
    "version": "0.1.0",
    "link": "https://files.pythonhosted.org/packages/2c/85/d6397edeceb1c81b5b99b1046033c8854abac3e50971c8d2915e63ce54c0/bar-0.1.0.tar.gz#sha256=6a711b5b06a662fe3f8e897310456b02127fb15d89310161fc8af7f415e270a1",
    "index": "https://pypi.org/simple/bar/"
  }, {
    "name": "bar",
    "version": "0.2.0",
    "link": "https://files.pythonhosted.org/packages/ba/bf/81ef8a2c80f4f86e9e6570605b7b5a277950cbd0ff9be901633159f4110e/bar-0.2.0.tar.gz#sha256=4a97e8664a22eec73f65d522cc36b4c693b61f0557760a856bb18a8fb67a8881",
    "index": "https://pypi.org/simple/bar/"
  }, {
    "name": "bar",
    "version": "0.2.1",
    "link": "https://files.pythonhosted.org/packages/77/d6/760c7a0ee7385ba4b0abb1d38af97e8c2f34cb47fe301315e48204088047/bar-0.2.1.tar.gz#sha256=b0a71c865729f9e8ac0f74e666b46c5ed1f5addac3425be7e06ca4828cec90ef",
    "index": "https://pypi.org/simple/bar/"
  }
]

Alternative Solutions

Additional context
Recent internal API changes in 19.2 will break pip-compile: jazzband/pip-tools#853

@chrahunt chrahunt added state: needs discussion This needs some more discussion type: feature request Request for a new feature labels Jul 21, 2019
@triage-new-issues triage-new-issues bot removed S: needs triage Issues/PRs that need to be triaged labels Jul 21, 2019
@cjerdonek
Copy link
Member

FYI, I posted PR #6910, which makes progress towards this.

@cjerdonek
Copy link
Member

Also, FWIW, re: the original issue, I like the idea of a pip find command.

@cjerdonek
Copy link
Member

FYI PR #7016, which was just posted, relates to this.

@ziebam
Copy link
Contributor

ziebam commented Sep 23, 2021

Similarly to #7975, I think this issue was solved by #8978 and further improvements are currently tracked #10052.

Please excuse my nit-pickiness, just checking out the issues that CodeTriage sends me. 😅

@uranusjr
Copy link
Member Author

Neither pip index version now and #10052 covers what I originally needed. I was looking for the information “what file will be downloaded when I install this requirement”. Both features you linked only operate at the version level, but I need the information at the file level. Numpy 1.16.3, for example, contains 22 wheels and 1 sdist, I want to know which one will be selected.

@pradyunsg
Copy link
Member

Is this a subset of the broader "do a dry-run, and tell me what you would've done" ALA #53?

@uranusjr
Copy link
Member Author

Yes if we provide good enough dry-run logging (I want to parse the filename in a script).

@sbidoul
Copy link
Member

sbidoul commented Jun 24, 2022

Do we reckon that this is adequately resolved by #10771 (pip install --dry-run --report), together with #11111 (PEP 658) ?

@pradyunsg
Copy link
Member

I think so, yes.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Aug 15, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
state: needs discussion This needs some more discussion type: feature request Request for a new feature
Projects
None yet
Development

Successfully merging a pull request may close this issue.

8 participants