Influence the build order

[travisdowns]

We have a build of ~400 source files which takes about 6 minutes. The build is highly parallel: almost all 400 jobs are available shortly into the build (typical of C and C++ projects where compilation has few prerequisites).

Most files take 10s of seconds to build, but one file takes 2 minutes. Ninja always starts this file right at the end of the build, so it's building all by its lonesome for almost 2 minutes, wasting (for example) 31 out of 32 cores. If Ninja started this job any time in the first couple minutes of the build, we'd get a speedup factor of almost 1.5x for the entire build.

So is there any way to influence ninja's build order today?

I've looked at many of the existing issues regarding prioritization and scheduling, but I wanted to create this issue for a clear 2023 answer on this. Even relatively dirty hacks are open for consideration because of the large benefit.

[jhasse]

No, there's no way to do this today, see #232.

[jhasse]

Also see PR #2177.

[travisdowns]

Thanks for the link to #232, I hadn't seen that one. So it seems like if we get lucky and the allocator returns increasing addresses we get control by changing the rule order and empirically this seems to work out... thanks!

I've looked at #2177 too, but for us it will do exactly the wrong thing, since this large file is actually at a very shallow depth/height in the build graph, so it would inevitably be built at the end. So I'm actually much happier with current behavior. If 2177 is merged it would be nice to be able to turn it off.

[ilyapopov]

@travisdowns There was an older PR #2019 that used estimated time of the critical path, not depth. It was closed in favor of #2177.