Scheduler throttling time/count limit per frame

[nhelfman]

Current scheduler in Chromium and Firefox behaves differently and inconsistently in terms of how many async tasks get executed per animation frame.

For example, the following code:

 for (let i = 0; i < 1000; i++) {
   setTimeout(() => {
     spin(); // do some work

     // paint something...
     document.getElementById("count").innerText = count++;

     tasks.push(`task ${i}`);
   }, 0);
 }

Will cause in Chromium around 9 tasks to be executed per frame – regardless of how long each task takes. In Firefox the behavior is different and it schedules 1-3 tasks per animation frame.
This behavior makes it easy for many async tasks to accumulate in a single frame and reduce UI responsiveness.

With the new proposal I don’t see that this crucial problem is being addressed since there is no definition for browser implementers how many tasks should be scheduled per frame.
Users of the new scheduling API have to implement a wrapping layer on top of it to ensure that when a task needs to be scheduled – enough time is remaining in the current frame to prevent long blocking periods of the main thread.

When experimenting with this feature in it looks like that tasks with ‘user-visible’ and ‘user-blocking’ priorities get scheduled in correct order relative to one another. However – almost no throttling happens and like the setTimeout tasks they get accumulated (up to 30 tasks in Chromium) and executed in the same animation frame. This behavior limits the usefulness of these 2 priorities since if they are executed in the same frame but just in different order it doesn’t much help address the original need to prioritizing tasks since they blocking time will be the same.

The tasks scheduled with ‘background’ priority behave in Chromium a similar way to how setTimeout behaves in Firefox in terms of how many get executed per animation frame which makes this priority very useful.

I hope my explanation above makes sense.

My proposal to address this issue is to add a frame throttling limit (or any other name) for each priority, which will allow users define how many tasks from a given priority they are willing to accept.

So perhaps something like this:

scheduler.userBlockingThrottleCount = 10; 
scheduler.userVisibleThrottleCount = 5; 
scheduler.backgroundThrottleCount = 1; 

Another, maybe simpler (?), option is allow users to set on the scheduler how long tasks should be scheduled to be executed on the main thread in a single animation frame before they get throttled.

For example:

scheduler.maxPerFrameBudgetMs = 10;

So in this case - if a tasks execute on the main thread for over 10 ms in the same animation frame - no other tasks will be scheduled in that frame regardless of their priority.

This approach is used for example by https://github.com/justinfagnani/queue-scheduler

Alternatively there could reasonable throttling defaults which enable sufficient differentiation between the priorities besides order of execution. E.g.: ‘background’: 2, ‘user-visible’: 5, ‘user-blocking’: 30.

I think if we have some way to limit the tasks executed every frame it would eliminate the need to web developers for using additional wrapping code on top of native browser scheduler - which I believe is one of the main goals of the main-thread-scheduling proposal.

Attaching a file with experimentation with the new APIs which helped me understanding the behaviors and the main downside I see.

priority.zip

[shaseley]

Thanks for the detailed feedback (and sorry for the late reply)! I think the intersection of rendering and task scheduling is both important and complex, and something we hope to address, but it isn't a goal for v1 of this API.

The priorities give developers a way to better coordinate across an app and give the browser more information to help schedule between other async tasks (aside from rendering). This also gives us a path forward to further improve coordination, for example exposing these priorities for other async work, e.g. fetch(), IDB, etc. While unfortunately this will limit the usefulness for some use cases (sorry!), our thought has been that we want to limit the scope/complexity of the initial proposal/spec and iterate/expand from there.

Current scheduler in Chromium and Firefox behaves differently and inconsistently in terms of how many async tasks get executed per animation frame.

For example, the following code:

 for (let i = 0; i < 1000; i++) {
   setTimeout(() => {
     spin(); // do some work

     // paint something...
     document.getElementById("count").innerText = count++;

     tasks.push(`task ${i}`);
   }, 0);
 }

Will cause in Chromium around 9 tasks to be executed per frame – regardless of how long each task takes. In Firefox the behavior is different and it schedules 1-3 tasks per animation frame.
This behavior makes it easy for many async tasks to accumulate in a single frame and reduce UI responsiveness.

Browsers have a lot of flexibility as far as when to render (see step 11 in the event loop processing model), which you're teasing out here. Coincidentally I did a similar investigation a while back with postMessage (see this bug), which resulted in some changes to rendering prioritization in Chromium recently (M81) to reduce visual lag. I could imagine us doing something different in Chromium with our rendering prioritization in a postTask world, but would need to think through that a bit more.

FWIW you might also get slightly different behavior with postMessage, which a lot of userspace schedulers use instead of setTimeout. In Chromium, how rendering and other tasks interleave depends on time elapsed, whether or not input is detected, and whether or not the other tasks are delayed tasks or not (and note, in Chromium setTimeout(0) still has a minimum 1 ms delay).

With the new proposal I don’t see that this crucial problem is being addressed since there is no definition for browser implementers how many tasks should be scheduled per frame.
Users of the new scheduling API have to implement a wrapping layer on top of it to ensure that when a task needs to be scheduled – enough time is remaining in the current frame to prevent long blocking periods of the main thread.

When experimenting with this feature in it looks like that tasks with ‘user-visible’ and ‘user-blocking’ priorities get scheduled in correct order relative to one another. However – almost no throttling happens and like the setTimeout tasks they get accumulated (up to 30 tasks in Chromium) and executed in the same animation frame. This behavior limits the usefulness of these 2 priorities since if they are executed in the same frame but just in different order it doesn’t much help address the original need to prioritizing tasks since they blocking time will be the same.

One thing that's not explored in your example is how user-visible and user-blocking work with in conjunction with other (non-rendering) async work, e.g. setTimeout, fetch(), etc. The way we plan to spec this (currently WIP) is that the order of the new tasks are well-defined based on priority (as you've noted), but that UAs have flexibility to schedule between these new task types and other task types. Other browsers (Firefox specifically, on the TAG review thread) has expressed a strong desire to retain the current flexibility w.r.t. scheduling between postTask tasks and other tasks.

The tasks scheduled with ‘background’ priority behave in Chromium a similar way to how setTimeout behaves in Firefox in terms of how many get executed per animation frame which makes this priority very useful.

I hope my explanation above makes sense.

My proposal to address this issue is to add a frame throttling limit (or any other name) for each priority, which will allow users define how many tasks from a given priority they are willing to accept.

So perhaps something like this:

scheduler.userBlockingThrottleCount = 10; 
scheduler.userVisibleThrottleCount = 5; 
scheduler.backgroundThrottleCount = 1; 

Another, maybe simpler (?), option is allow users to set on the scheduler how long tasks should be scheduled to be executed on the main thread in a single animation frame before they get throttled.

For example:

scheduler.maxPerFrameBudgetMs = 10;

So in this case - if a tasks execute on the main thread for over 10 ms in the same animation frame - no other tasks will be scheduled in that frame regardless of their priority.

Do you have any use cases in mind for Excel or other apps where you might set different values? Does this change depending on what the user is doing, or is it relatively static?

FWIW I like the time version over the task-count version since tasks can be arbitrarily long and the task length can vary depending on device, etc.

One question I have is whether or not this only affects postTask tasks, i.e. for a smooth experience don't all the other async tasks matter as well?

An alternative might be to expose some direct control over rendering, rather than affecting it indirectly by controlling other tasks. For example, a frame-rate API has been proposed in the past, which would allow developers to set a target frame rate. I could imagine being able to set the importance of rendering could be useful in conjunction with this.

[nhelfman]

our thought has been that we want to limit the scope/complexity of the initial proposal/spec and iterate/expand from there.

I agree with this statement as I'm a proponent of incremental improvement!

Do you have any use cases in mind for Excel or other apps where you might set different values? Does this change depending on what the user is doing, or is it relatively static?

At least for Excel Online use case I think it would be a dynamic value based only the activity the user in engaged in. For example - if the user is performing selection activity - I would like to minimize the frames which exceeds 16ms so any other async tasks which takes longer to execute should have lower priority and postponed until that activity is complete. So I imagine logic should be added to dynamically change the frame budget based on the context.

One question I have is whether or not this only affects postTask tasks, i.e. for a smooth experience don't all the other async tasks matter as well?

I imagine that if there is useful API for prioritized task scheduling the app can be adapted to use it for almost all async tasks. This can be done creating a task dispatcher which controls the relative priority of each task based on the context. For example - if a fetch arrives - handling the result can be done based on the importance of the result to the app user experience and current user activity. This of course would be app specific although generic frameworks to abstract the complexity could be built.

An alternative might be to expose some direct control over rendering, rather than affecting it indirectly by controlling other tasks. For example, a frame-rate API has been proposed in the past, which would allow developers to set a target frame rate. I could imagine being able to set the importance of rendering could be useful in conjunction with this.

I must say that I need to think a bit more about this concept. I'm not sure I have good intuition what would provide better API usability to software engineers.

[mmocny]

I wonder if this should be closed now?

I think that the literal count of task scheduled between rendering updates should not be a fixed number. I think it never was, and the patterns Noam points out were more coincidences that spawned out of the typical induced delays between setTimeout(0) calls and typical delays before rAF() calls.

Instead, as Noam points out, the problem is actually just to ensure that needless tasks aren't scheduled during periods where frame latency is more important (say, during interactions) than general task throughput (say, during page loading).

[nhelfman]

Probably it can be closed if there is an alternative to address the high-level concern (many frequent tasks block interaction). From Scott's answer it seems like the scheduler has a way to handle it reliably in a more optimal way by limiting the frame by default and prioritizing render tasks after user interaction. I wonder if the default frame limit (is it 100ms?) is sufficient or we need something shorter for highly interactive apps.

[mmocny]

In Chromium:

• Rendering gets scheduled in FIFO order at normal priority, and so could get starved by other tasks-- until 100ms passes, where it is boosted up to highest priority.
• Immediately after Interactions, rendering is also boosted.
• The proposed scheduler.yield({ wairForRender: true }) api (previously known as scheduler.render()) might also become a signal to boost rendering priority, since its needed for the continuation which itself is already a higher priority.

Even with all of the above, there might still be a gap of time between when the (e.g. event dispatch) task ends, and the rendering task actually becomes scheduled. The scheduler today might allow arbitrary js tasks to become scheduled to take advantage of this (potentially minimal) idle time gap. That risks long tasks which could block rendering. Scott is launching some experiments to change scheduling policies to reduce that risk... at the cost of some potential task scheduling throughput.

Even more experiments are likely to follow, such as changing the way events are dispatched in the first place (i.e. not flushing all events before rendering)

---

All to say, there is existing flexibility for scheduler to prioritize rendering after interactions in ways that are quite a lot better than today, and yield({ waitForRender: true }) might offer a way for sites to opt-in to some amount of prioritized rendering, too.