[ResponseOps][Task Manager][mget Claimer] fix problem with limited concurrency starvation of unlimited concurrency tasks #184937
Labels
Feature:Task Manager
Team:ResponseOps
Label for the ResponseOps team (formerly the Cases and Alerting teams)
v8.16.0
In PR implement task claiming strategy mget #180485 we implemented an alternative task claiming strategy, but it has the following problem:
If the number of limited concurrency tasks to run next exceeds the count of the tasks we do the initial search for, no tasks with unlimited concurrency will be returned for this polling cycle, as they would be "starved" by the limited concurrency tasks.
This is because we are searching all task types in one search, then applying limited concurrency on the results. We need to change this to do multiple searches - the unlimited concurrency search should not search on any limited concurrency tasks, and vice versa. Presumably we'd be doing one search for unlimited concurrency, and an additional search for each task type with limited concurrency.
I think we can do this with an
_msearch
. I'm thinking we put all the common matching clauses we need into the top-levelfilter
of the query, in hopes that they make use of the potential filter caching. Then the query can provide the specific matching clauses for that sub-search.The text was updated successfully, but these errors were encountered: