[Alerting] Gracefully handle errors when retrieving task document on rule disable

[ymao1]

When rules are disabled, active alerts for that rule are auto-resolved to avoid zombie active alerts. I noticed while writing a functional test for another issue that if the task manager doc for the rule is missing, the disable call throws a 404

{
    "statusCode": 404,
    "error": "Not Found",
    "message": "Saved object [task/329798f0-b0b0-11ea-9510-fdf248d5f2a4] not found"
}

and does not disable the rule. We may want to handle this case better, as if the task document is missing and we can't disable the rule, we can't then re-enable it to create a new task document.

We should figure out what the expected behavior should be. Maybe only catch 404s and continue disabling the rule and throw other types of errors? Or allow rule to be disabled regardless (possibly leading to zombie active alerts?)

[elasticmachine]

Pinging @elastic/kibana-alerting-services (Team:Alerting Services)

[pmuellr]

For reference, the auto-recover PR #111671 was merged in 7.16.0

This feels pretty important, as we do see cases where rules reference task documents that don't exist, and our remediation is to disable and then re-enable the rule. So if we can't do that because of this problem, the only remediation for these cases would be to delete the rule and re-create it.

In terms of behaviour, I think we'll want to have disable do as much as it possibly can - delete the task, invalidate the API key, recover instances, etc. When we run into issues, log and continue. If we can't recover instances because we can't get the task document, log that and continue. Etc.