[Heartbeat] Limit number of parallel monitors, by type

[andrewvc]

Today, heartbeat has the heartbeat.scheduler.limit option, which limits the total number of internal tasks heartbeat can run. For lightweight checks this corresponds to the number of open file descriptors.

This doesn't work well for synthetics, where each check is made of a single recursive run of many tasks (one per JSON line from the agent) that don't correlate to FDs at all.

We should add a new setting:

heartbeat.jobs.{icmp|http|tcp|browser}.limit which lets users define on a job-type basis how many concurrent tasks may be run.

Docs are updated to clarify how these relate and should be used

[elasticmachine]

Pinging @elastic/uptime (Team:Uptime)

[dominiqueclarke]

Working as expected for icmp, tcp, http, and browser monitors in 7.15.0.