Skip to content

Commit

Permalink
scripts: add script to measure percentage of commits with failed status
Browse files Browse the repository at this point in the history
This is to start measuring the test flakyness and see the numbers improving once we improve and deflake flaky tests

Fixes etcd-io#13167
  • Loading branch information
karuppiah7890 committed Jul 3, 2021
1 parent af9b5e7 commit b447aaf
Show file tree
Hide file tree
Showing 2 changed files with 23 additions and 1 deletion.
3 changes: 2 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -22,4 +22,5 @@ hack/tls-setup/certs
*.bak
.gobincache/
/Documentation/dev-guide/api_reference_v3.md
/Documentation/dev-guide/api_concurrency_reference_v3.md
/Documentation/dev-guide/api_concurrency_reference_v3.md
commit-and-check-data.json
21 changes: 21 additions & 0 deletions scripts/measure-test-flakyness.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
#!/bin/bash

set -e
set -o pipefail

if [[ -z ${GITHUB_TOKEN} ]]
then
echo "Please set the \$GITHUB_TOKEN environment variable for the script to work"
exit 1
fi

curl --fail --show-error --silent -H "Authorization: token ${GITHUB_TOKEN}" \
-X POST \
-d '{
"query": "query { repository(owner: \"etcd-io\", name: \"etcd\") { defaultBranchRef { target { ... on Commit { history(first: 100) { edges { node { ... on Commit { commitUrl statusCheckRollup { state } } } } } } } } } }"
}' \
https://api.github.com/graphql | jq . > commit-and-check-data.json

failure_percentage=$(jq '.data.repository.defaultBranchRef.target.history.edges | reduce .[] as $item (0; if $item.node.statusCheckRollup.state == "FAILURE" then (. + 1) else . end)' commit-and-check-data.json)

echo "Commit status failure percentage is - ${failure_percentage} %"

0 comments on commit b447aaf

Please sign in to comment.