Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Setuptools backtrack hack no longer required for Pip #11836

Closed
1 task done
notatallshaw opened this issue Mar 5, 2023 · 0 comments · Fixed by #11837
Closed
1 task done

Setuptools backtrack hack no longer required for Pip #11836

notatallshaw opened this issue Mar 5, 2023 · 0 comments · Fixed by #11837
Labels
type: bug A confirmed bug or unintended behavior

Comments

@notatallshaw
Copy link
Member

notatallshaw commented Mar 5, 2023

Description

When the Pip new resolver was added a hack to delay backtracking on setuptools was introduced in PR #9249, this was from the observation #9187 (comment) as part of the reason for causing long times backtracking in #9203.

However since then the ecosystem has drastically improved and it is very difficult to reproduce this use case without pypi-timemachine.

If you do use "pypi-timemachine" to reproduce this you will find that the hack is no longer required on pip main, probably thanks to recent improvements in resolvelib.

Expected behavior

Shouldn't need specific hacks for packages, if backtracking is powerful enough individual hacks are likely to cause more problems than they solve.

pip version

main

Python version

3.7+

OS

any

How to Reproduce

  1. Install pypy-timemachine
  2. Run like so: pypi-timemachine 2020-12-02 --port 9999
  3. Create constraints.txt with the contents argparse>=1
  4. Run pip like so python3.7 -m pip download apache-airflow[all]==1.10.13 -c constraints.txt -d downloads --index-url http://localhost:9999

Output

To come up with an objective assessment of if changes to the resolver had a positive or negative impact we will take the total number of packages Pip had to download as part of backtracking and subtract the number of packages that were saved to the download folder as part of the download command. This number is the "extra nodes" that Pip had to include in the backtracking graph, and is likely represents an exponential increase in time spent backtracking.

Pip Version Total Downloaded Backtracking Final Download Count Difference
Pip main (30fd549) 400 355 45
Pip main (30fd549) + setuptools hack removed 400 355 45

There was therefore no difference between having the setuptools hack or not.

I tested on Pip 23.0.1 I couldn't get resolution to finish (though the setuptools hack changed backtracking path quite significantly), I also tested using resolvelib main (e383b40) and saw no difference between that and Pip main in this test case.

Code of Conduct

@notatallshaw notatallshaw added S: needs triage Issues/PRs that need to be triaged type: bug A confirmed bug or unintended behavior labels Mar 5, 2023
@pradyunsg pradyunsg removed the S: needs triage Issues/PRs that need to be triaged label Mar 17, 2023
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Apr 17, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
type: bug A confirmed bug or unintended behavior
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants