Add DevOps metric Change Failure Rate

[Sebastiaan127001]

User story

As a quality manager I want to measure change failure rate so that I can monitor the DevOps effectiveness of my team.

Change failure rate is defined as the percentage of deployments causing a failure in production.

This metric requires a "look back period" that the user can configure.

Because this metric is a "rate", it needs a numerator (the number of failed deployments in the look back period) and a denominator (the number of deployments in the look back period).

Acceptance criteria

• Users can add a change-failure-rate metric
• Users can configure Azure DevOps as source for the metric (both for number of deployments and number of failures). AzureDevOps needs to be added as source only once.
• Users can configure Jira as source for the metric (but only the number of failures) and either Jenkins or GitLab CI as source for the metric (for the number of deployments). #7374

Assumptions

• The number of failures can be retrieved from Jira and Azure DevOps in the same way as issues are retrieved, i.e. using a JQL or WIQL query. This allows for ignoring e.g. minor bugs or including only bugs with a certain label.
• Deployments are automated and can be retrieved from a CI-tool. In other words, we don't support deployments that are manually tracked in issue trackers.
• Bugs have a timestamp. Deployments have a timestamp. Bugs are caused by the deployment with the most recent timestamp <= timestamp of the bug.
• The sources need a "look back period". We assume users won't shoot themselves in the foot and configure different lookback periods for different sources.

Out of scope

• GitLab as source for issues, i.e. failed deployments in this case.
• Manual Number is integer only, so can only support this metric but adding it twice, once for the numerator and once for the denominator. That's too confusing so out of scope.
• Azure DevOps metrics should return datetimes #7677

Tasks

Components

Shared code

• Add metric change_failure_rate to the data model, with Jira, GitLab CI, Jenkins, and Azure DevOps as supporting sources. Add a documentation attribute (comparable to the documentation of the test cases metric, explaining to users that two sources need to be added to this metric.

Collector

• Add ChangeFailureRate metric collector in src\metric_collectors\change_failure_rate.py
• Add AzureDevOpsChangeFailureRate collector
• Add JiraChangeFailureRate collector
• Add JenkinsChangeFailureRate collector
• Add GitLabChangeFailureRate collector

Documentation

• Update changelog

[wkoot]

NB: entities could also include links to the issues relating to them