Improve Performance of Picking Best Candidate from Indexes

[jbylund]

• wheel.supported and wheel.support_index_min both currently depend on a linear scan of what can possibly be a large number of tags. Combining these into one step and pre-computing the {tag: index_in_list} dictionary makes it a lot less expensive to evaluate.

• wheel.supported - if we happen to have the tags in a constant time lookup data structure doing the intersection might be nice, but if we're only using it once it may not make sense to do?

• I was unable to get all tests to pass on master, it's likely I'm doing something wrong and there can maybe just be an edit to the readme

• Is there a preferred method of profiling/a performance regression suite?

[ichard26]

FYI perf isn't a valid news type. Your choices are process|removal|feature|bugfix|doc|vendor|trivial. No idea which one would be ideal. Is there an issue you could point to? If so, bugfix might work. ex. PR #7962

[jbylund]

Here's a very coarse benchmark:

On main:

	Command being timed: "bash -c for i in $(seq 10); do python -m pip wheel boto3; done"
	User time (seconds): 23.97
	System time (seconds): 0.59
	Percent of CPU this job got: 83%
	Elapsed (wall clock) time (h:mm:ss or m:ss): 0:29.31

On this branch:

	Command being timed: "bash -c for i in $(seq 10); do python -m pip wheel boto3; done"
	User time (seconds): 19.61
	System time (seconds): 0.53
	Percent of CPU this job got: 87%
	Elapsed (wall clock) time (h:mm:ss or m:ss): 0:23.02

[jbylund]

Profile of 30 calls of find_best_candidate of boto3 from main (triggered by doing python -m pip wheel boto3, instrumentation on just package_finder.find_best_candidate):

The same from the branch:

[uranusjr]

The logic looks good to me, awesome work! I feel the attribute and function for this can be named a bit better though, something like _wheel_tag_preferences and find_most_preferred_tag. A comment explaining lower means more preferred would be nice as well.

[uranusjr]

Re: news type, I think feature is more appropriate since this is arguably not a bug. The text can be slightly more specific as well, something like Improve performance when picking the best file from indexes during pip install.

[jbylund]

Should I add a comment in the __init__ of CandidateEvaluator explaining why the thing exists or is it clear enough?

[uranusjr]

Should I add a comment in the __init__ of CandidateEvaluator

A comment near _wheel_tag_preferences definition telling the user to read docstring of find_most_preferred_tag would be useful.

[pradyunsg]

This is super neat! ^>^

[pradyunsg]

FYI: I've cancelled the CI run for one of the older commits.

[uranusjr]

🚢

[pradyunsg]

Thanks @jbylund! ^.^