Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Take priority into account within the pending queue #11032

Merged
merged 2 commits into from
Sep 13, 2022
Merged

Take priority into account within the pending queue #11032

merged 2 commits into from
Sep 13, 2022

Commits on Sep 2, 2022

  1. sort the pending queue according to cost/priority 

    If multiple pieces of work are waiting in the pending queue, we can sort it according to
their priorities: higher priorities should be scheduled sooner.

They are more often than not wider than pure chains, and this should create more parallelism
opportunities earlier in the pipeline: a high priority piece of work represents more future
pieces of work down the line.

This is a scheduling tradeoff that behaves differently for each project, machine configuration,
amount of available parallelism at a given point in time, etc, but seems to help more often than
hinders, at low-core counts and with enough units of work to be done, so that there is jobserver
token contention where choosing a "better" piece of work to work on next is possible.
    @lqd
    lqd committed Sep 2, 2022
    Configuration menu
    Copy the full SHA
    9ef2958 View commit details
    Browse the repository at this point in the history

  2. store jobs priorities in the pending queue 

    This cleans up the priority-sorted scheduling by removing the need
for a priority accessor that would hash the nodes, and allows inserting
in the queue at the correctly sorted position to remove the insert +
sort combination.
    @lqd
    lqd committed Sep 2, 2022
    Configuration menu
    Copy the full SHA
    4ffe98a View commit details
    Browse the repository at this point in the history