Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

✨ Update score for branch protection with levels #1287

Merged
merged 11 commits into from
Nov 20, 2021

Conversation

laurentsimon
Copy link
Contributor

@laurentsimon laurentsimon commented Nov 16, 2021

Update score to work like 'stages':

  • first stage: force push, force deletion. if admin, enforce admins. Those give 3/10
  • second stage: review >= 1. for admins, up-to-date branch. Those gives 6/10. Maybe up-to-date branch could go to first stage?
  • third stage: status checks defined. Gives 8/10.
  • fourth stage : reviews >= 2. Gives 9/10
  • fifth stage: for admins, dismissal (very hard to use). Gives 10/10

If a stage is not satisfied, the score won't go up regardless of the settings of the next stages. Warn/Info messages are shown for all stage, regardless of the score.
I scorecard is run without admin access, the admin settings are ignored.

@olivekl how shall I word this in the checks.yaml?

FYI, I'm working on the unit tests right now but that should not hamper the review.

@laurentsimon laurentsimon changed the title ✨ [DRAFT: do not review] Update score for branch protection fr basic/advanced levels ✨ [DRAFT: do not review] Update score for branch protection with levels Nov 16, 2021
@laurentsimon laurentsimon marked this pull request as draft November 17, 2021 02:10
@laurentsimon laurentsimon marked this pull request as ready for review November 17, 2021 21:47
@laurentsimon laurentsimon marked this pull request as draft November 17, 2021 21:53
@laurentsimon laurentsimon marked this pull request as ready for review November 17, 2021 22:21
@laurentsimon laurentsimon changed the title ✨ [DRAFT: do not review] Update score for branch protection with levels ✨ Update score for branch protection with levels Nov 17, 2021
@laurentsimon
Copy link
Contributor Author

friendly ping. ISE would like this check in the cron job's data in a couple weeks.

Copy link
Contributor

@asraa asraa left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

going to also comment offline as well..

checks/branch_protection.go Outdated Show resolved Hide resolved
checks/branch_protection.go Outdated Show resolved Hide resolved
@olivekl
Copy link
Contributor

olivekl commented Nov 19, 2021

Suggestions for wording this in checks.yaml (use | before to preserve formatting):

This test has tiered scoring. Each tier must be fully satisfied to achieve points at the next tier. For example, if you fulfill the Tier 3 checks but do not fulfill all the Tier 2 checks, you will not receive any points for Tier 3.

Note: If Scorecard is run without an administrative access token, the requirements that specify “For administrators” are ignored.

Tier 1 Requirements (3/10 points):

  • Force push
  • Force deletion
  • For administrators: Include administrators

Tier 2 Requirements (6/10 points):

  • Required reviewers >=1 ​
  • For administrators: Strict status checks (require branches to be up-to-date before merging)

Tier 3 Requirements (8/10 points):

  • Status checks defined

Tier 4 Requirements (9/10 points):

  • Required reviewers >= 2

Tier 5 Requirements (10/10 points):

  • For administrators: Dismiss stale reviews

checks/branch_protection.go Show resolved Hide resolved
checks/branch_protection.go Outdated Show resolved Hide resolved
checks/branch_protection.go Outdated Show resolved Hide resolved
checks/branch_protection.go Show resolved Hide resolved
checks/branch_protection.go Outdated Show resolved Hide resolved
Copy link
Contributor

@asraa asraa left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM besides the small comment nits

docs/checks.md Show resolved Hide resolved
@laurentsimon
Copy link
Contributor Author

laurentsimon commented Nov 19, 2021

Thanks @asraa and @azeemsgoogle . Did not mean to be too pushy on this one... kinda came across this way though. @asraa please feel free to start an issue/PR to separate this check, if you're interested in tackling it.
I think one ugly part is the max score which I added for sanity check, it complicates the code and makes it less readable...

@laurentsimon laurentsimon enabled auto-merge (squash) November 19, 2021 23:00
@laurentsimon
Copy link
Contributor Author

FYI Makefile:261: *** GITHUB_AUTH_TOKEN is undefined. Stop. Error: Process completed with exit code 2.

@laurentsimon
Copy link
Contributor Author

@naveensrinivasan @azeemshaikh38 please approve my integration tests. Thanks

@laurentsimon
Copy link
Contributor Author

Note: the PR has 2 LGTMs but this does not trigger the automatic approval of the integration test :/

@laurentsimon laurentsimon temporarily deployed to integration-test November 20, 2021 01:19 Inactive
@laurentsimon
Copy link
Contributor Author

Looks like I'm able to approve my own integration tests!

@github-actions
Copy link

Integration tests success for
[00a3e48]
(https://github.com/ossf/scorecard/actions/runs/1483384351)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants