Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pip install --find-links not expanding "~" #6804

Closed
j0057 opened this issue Jul 29, 2019 · 5 comments · Fixed by #6827
Closed

pip install --find-links not expanding "~" #6804

j0057 opened this issue Jul 29, 2019 · 5 comments · Fixed by #6827
Labels
auto-locked Outdated issues that have been locked by automation C: finder PackageFinder and index related code type: bug A confirmed bug or unintended behavior

Comments

@j0057
Copy link
Contributor

j0057 commented Jul 29, 2019

Environment

  • pip version: >19.0.3
  • Python version: 3.7.3
  • OS: ArchLinux

At the moment I have pip 19.0.3 installed in my user-site directory. I noticed a change in behavior when using a newer pip (19.2.1) while experimenting with PYTHONUSERBASE as an alternative to virtualenvs.

Description
The --find-links option used to expand tildes (~) into the user's $HOME, but it no longer does so.

Expected behavior
The --find-links option should expand tildes like it used to.

How to Reproduce

I was able to bisect it to commit 7d08bb3 as the first commit that no longer expands tildes: (All done from the root of the pip repo)

  1. Use the current master version of pip to download a wheel such as pytz: PYTHONPATH=src python -m pip download --destination-dir='~/temp' pytz
  2. Start bisecting: git bisect start HEAD 19.0.3
  3. Repeatedly install pytz into a custom PYTHONUSERBASE: PYTHONPATH=src PYTHONUSERBASE=/tmp/x git bisect run sh -c "rm -rf /tmp/x; python -m pip --isolated --no-cache-dir --disable-pip-version-check install --no-index --find-links='~/temp' --user pytz"

(Clean everything up: rm -rf /tmp/x ~/temp)

Output

jjm@muon pip master %= $ PYTHONPATH=src python -m pip download --destination-dir='~/temp' pytz
Collecting pytz
  Downloading https://files.pythonhosted.org/packages/3d/73/fe30c2daaaa0713420d0382b16fbb761409f532c56bdcc514bf7b6262bb6/pytz-2019.1-py2.py3-none-any.whl (510kB)

  Saved /home/jjm/temp/pytz-2019.1-py2.py3-none-any.whl
Successfully downloaded pytz

jjm@muon pip master %= $ git bisect start HEAD 19.0.3
Bisecting: 251 revisions left to test after this (roughly 8 steps)
[f44344f1220bce5cd942641f832cb407ed477f06] Merge pull request #6515 from johnthagen/svn-interactive-final

jjm@muon pip (f44344f1...) %|BISECTING $ PYTHONPATH=src PYTHONUSERBASE=/tmp/x git bisect run sh -c "rm -rf /tmp/x; python -m pip --isolated --no-cache-dir --disable-pip-version-check install --no-index --find-links='~/temp' --user pytz"
running sh -c rm -rf /tmp/x; python -m pip --isolated --no-cache-dir --disable-pip-version-check install --no-index --find-links='~/temp' --user pytz
Looking in links: /home/jjm/temp
Collecting pytz
Installing collected packages: pytz
Successfully installed pytz-2019.1
Bisecting: 126 revisions left to test after this (roughly 7 steps)
[edbd0822dc50f49ef92dc7bee27d5728ed3babb7] Rename InstallRequirement.setup_py -> setup_py_path
running sh -c rm -rf /tmp/x; python -m pip --isolated --no-cache-dir --disable-pip-version-check install --no-index --find-links='~/temp' --user pytz
Looking in links: ~/temp
Collecting pytz
  WARNING: Url '~/temp' is ignored. It is either a non-existing path or lacks a specific scheme.
  ERROR: Could not find a version that satisfies the requirement pytz (from versions: none)
ERROR: No matching distribution found for pytz
Bisecting: 62 revisions left to test after this (roughly 6 steps)
[3f8df2487597d47279b167f29d93622e5632c044] Merge pull request #6600 from lkollar/consolidate-path-opt-parsing
running sh -c rm -rf /tmp/x; python -m pip --isolated --no-cache-dir --disable-pip-version-check install --no-index --find-links='~/temp' --user pytz
Looking in links: /home/jjm/temp
Collecting pytz
Installing collected packages: pytz
Successfully installed pytz-2019.1
Bisecting: 30 revisions left to test after this (roughly 5 steps)
[c3c61b5c5198b702aa506cc72c3cdf78974b88b7] Merge pull request #6624 from cjerdonek/add-search-scope
running sh -c rm -rf /tmp/x; python -m pip --isolated --no-cache-dir --disable-pip-version-check install --no-index --find-links='~/temp' --user pytz
Looking in links: /home/jjm/temp
Collecting pytz
Installing collected packages: pytz
Successfully installed pytz-2019.1
Bisecting: 15 revisions left to test after this (roughly 4 steps)
[a447be47858f980b9b022891ab4d45aebd0ac967] Update PackageFinder to support "yanked" files (PEP 592).
running sh -c rm -rf /tmp/x; python -m pip --isolated --no-cache-dir --disable-pip-version-check install --no-index --find-links='~/temp' --user pytz
Looking in links: ~/temp
Collecting pytz
  WARNING: Url '~/temp' is ignored. It is either a non-existing path or lacks a specific scheme.
  ERROR: Could not find a version that satisfies the requirement pytz (from versions: none)
ERROR: No matching distribution found for pytz
Bisecting: 7 revisions left to test after this (roughly 3 steps)
[36430e2a43b8e820af60b0eb43195fd9f323e5f2] Merge pull request #6639 from sbidoul/imp_looks_like_hash-sbi
running sh -c rm -rf /tmp/x; python -m pip --isolated --no-cache-dir --disable-pip-version-check install --no-index --find-links='~/temp' --user pytz
Looking in links: ~/temp
Collecting pytz
  WARNING: Url '~/temp' is ignored. It is either a non-existing path or lacks a specific scheme.
  ERROR: Could not find a version that satisfies the requirement pytz (from versions: none)
ERROR: No matching distribution found for pytz
Bisecting: 3 revisions left to test after this (roughly 2 steps)
[a38a0eacd4650a7d9af4e6831a2c0aed0b6a0329] Merge pull request #6631 from cjerdonek/finder-search-scope
running sh -c rm -rf /tmp/x; python -m pip --isolated --no-cache-dir --disable-pip-version-check install --no-index --find-links='~/temp' --user pytz
Looking in links: ~/temp
Collecting pytz
  WARNING: Url '~/temp' is ignored. It is either a non-existing path or lacks a specific scheme.
  ERROR: Could not find a version that satisfies the requirement pytz (from versions: none)
ERROR: No matching distribution found for pytz
Bisecting: 0 revisions left to test after this (roughly 1 step)
[cab7316e2b6f994d95527c0577c14f6ae2164126] Change PackageFinder.create() to accept a SearchScope object.
running sh -c rm -rf /tmp/x; python -m pip --isolated --no-cache-dir --disable-pip-version-check install --no-index --find-links='~/temp' --user pytz
Looking in links: ~/temp
Collecting pytz
  WARNING: Url '~/temp' is ignored. It is either a non-existing path or lacks a specific scheme.
  ERROR: Could not find a version that satisfies the requirement pytz (from versions: none)
ERROR: No matching distribution found for pytz
Bisecting: 0 revisions left to test after this (roughly 0 steps)
[761b2064f42e6904bba189d89b29100f7c03ca1f] Remove models/search_scope.py's dependency on download.py.
running sh -c rm -rf /tmp/x; python -m pip --isolated --no-cache-dir --disable-pip-version-check install --no-index --find-links='~/temp' --user pytz
Looking in links: /home/jjm/temp
Collecting pytz
Installing collected packages: pytz
Successfully installed pytz-2019.1
cab7316e2b6f994d95527c0577c14f6ae2164126 is the first bad commit
commit cab7316e2b6f994d95527c0577c14f6ae2164126
Author: Chris Jerdonek <[email protected]>
Date:   Fri Jun 21 09:10:03 2019 -0700

    Change PackageFinder.create() to accept a SearchScope object.

 src/pip/_internal/cli/base_command.py | 16 ++++------------
 src/pip/_internal/cli/cmdoptions.py   | 26 ++++++++++++++++++++++++++
 src/pip/_internal/commands/list.py    | 15 ++++++---------
 src/pip/_internal/commands/wheel.py   |  5 -----
 src/pip/_internal/index.py            | 10 ++--------
 src/pip/_internal/utils/outdated.py   |  6 ++++--
 tests/lib/__init__.py                 |  9 +++++++--
 tests/unit/test_build_env.py          |  4 +++-
 tests/unit/test_cmdoptions.py         | 31 ++++++++++++++++++++++++++++++-
 tests/unit/test_index.py              | 11 +++++++++--
 tests/unit/test_unit_outdated.py      |  4 ++--
 11 files changed, 93 insertions(+), 44 deletions(-)
bisect run success

Bisect log

# bad: [f76014efac796c0656ad61bd2bbcc23b67d66495] Add global options and no user config args to make_setuptools_shim_args (#6706)
# good: [a8de4eb49a45035aefd9a122b6d1d67915663d5f] Generate NEWS
git bisect start 'HEAD' '19.0.3'
# good: [f44344f1220bce5cd942641f832cb407ed477f06] Merge pull request #6515 from johnthagen/svn-interactive-final
git bisect good f44344f1220bce5cd942641f832cb407ed477f06
# bad: [edbd0822dc50f49ef92dc7bee27d5728ed3babb7] Rename InstallRequirement.setup_py -> setup_py_path
git bisect bad edbd0822dc50f49ef92dc7bee27d5728ed3babb7
# good: [3f8df2487597d47279b167f29d93622e5632c044] Merge pull request #6600 from lkollar/consolidate-path-opt-parsing
git bisect good 3f8df2487597d47279b167f29d93622e5632c044
# good: [c3c61b5c5198b702aa506cc72c3cdf78974b88b7] Merge pull request #6624 from cjerdonek/add-search-scope
git bisect good c3c61b5c5198b702aa506cc72c3cdf78974b88b7
# bad: [a447be47858f980b9b022891ab4d45aebd0ac967] Update PackageFinder to support "yanked" files (PEP 592).
git bisect bad a447be47858f980b9b022891ab4d45aebd0ac967
# bad: [36430e2a43b8e820af60b0eb43195fd9f323e5f2] Merge pull request #6639 from sbidoul/imp_looks_like_hash-sbi
git bisect bad 36430e2a43b8e820af60b0eb43195fd9f323e5f2
# bad: [a38a0eacd4650a7d9af4e6831a2c0aed0b6a0329] Merge pull request #6631 from cjerdonek/finder-search-scope
git bisect bad a38a0eacd4650a7d9af4e6831a2c0aed0b6a0329
# bad: [cab7316e2b6f994d95527c0577c14f6ae2164126] Change PackageFinder.create() to accept a SearchScope object.
git bisect bad cab7316e2b6f994d95527c0577c14f6ae2164126
# good: [761b2064f42e6904bba189d89b29100f7c03ca1f] Remove models/search_scope.py's dependency on download.py.
git bisect good 761b2064f42e6904bba189d89b29100f7c03ca1f
# first bad commit: [cab7316e2b6f994d95527c0577c14f6ae2164126] Change PackageFinder.create() to accept a SearchScope object.
@triage-new-issues triage-new-issues bot added the S: needs triage Issues/PRs that need to be triaged label Jul 29, 2019
@cjerdonek cjerdonek added C: finder PackageFinder and index related code type: bug A confirmed bug or unintended behavior labels Jul 29, 2019
@triage-new-issues triage-new-issues bot removed the S: needs triage Issues/PRs that need to be triaged label Jul 29, 2019
@cjerdonek
Copy link
Member

I was able to bisect it to commit 7d08bb3 as the first commit that no longer expands tildes: (All done from the root of the pip repo)

I think you mean commit cab7316?

It looks like the error is that this line should have been SearchScope.create() rather than SearchScope(): cab7316#diff-863212c0ea2f09c6463884971d77230dR371

@j0057
Copy link
Contributor Author

j0057 commented Jul 29, 2019

Yes, you're right, I did mean cab7316.

I tested the fix you propose, that indeed fixes it. I'll send a PR for it!

@cjerdonek
Copy link
Member

Thanks. In your PR you should add another unit test of that function testing that e.g. normalize_path is called on such values.

@cjerdonek
Copy link
Member

I posted PR #6827 to fix this, with tests, and added it to the 19.2 milestone.

@pradyunsg
Copy link
Member

Released pip 19.2.2 containing a fix for this issue. :)

@lock lock bot added the auto-locked Outdated issues that have been locked by automation label Sep 10, 2019
@lock lock bot locked as resolved and limited conversation to collaborators Sep 10, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
auto-locked Outdated issues that have been locked by automation C: finder PackageFinder and index related code type: bug A confirmed bug or unintended behavior
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants