[Task Manager] Add caching to the task partitioning logic

[mikecote]

The Kibana discovery service provides an API to get a list of discovered Kibana nodes. The Task Manager calls this API during every claiming cycle to determine which partitions to query for when using the mget task claiming strategy.

kibana/x-pack/plugins/task_manager/server/task_claimers/strategy_mget.ts

Line 290 in 47c7137

const partitions = await taskPartitioner.getPartitions();

This adds more load to Elasticsearch when the polling interval is lowered for each Kibana node, and then all the node's assigned partitions are recalculated every time.

Proposed Solution

Within the task partitioning logic, we add a mechanism that keeps the node's calculated partitions in cache for 10 seconds before calling the discovery service again and recalculating them.

Definition of Done

• The kibana assigned partitions are kept in memory for 10s
• Requests errors to the discovery service are not cached (and rather retried on the next call)
• Tests

[elasticmachine]

Pinging @elastic/response-ops (Team:ResponseOps)