Describe and write down the CI policy

[cpanato]

Describe the CI policy for jobs, like:

• having contact information
• resources defined

We have a policy for blocking and informing jobs, https://github.com/kubernetes/sig-release/blob/master/release-blocking-jobs.md .
If we compare this policy with what is proposed in kubernetes/test-infra#18599, what would we add? what would we change?
We should evaluate what changes we need to make to help ensure we are acting on useful information and check tht CI jobs are maintained and well.

/area release-eng
/area ci
/kind documentation
/priority important-soon
/milestone v1.20

[k8s-ci-robot]

@cpanato: The label(s) area/ci cannot be applied, because the repository doesn't have them

In response to this:

Describe the CI policy for jobs, like:

• having contact information
• resources defined

Parent Issue: kubernetes/sig-release#1215

/area release-eng
/area ci
/kind documentation
/priority important-soon
/milestone v1.20

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[cpanato]

will close because it is a duplicate of kubernetes/test-infra#18551

/close

[k8s-ci-robot]

@cpanato: Closing this issue.

In response to this:

will close because it is a duplicate of kubernetes/test-infra#18551

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[alejandrox1]

Reopening this to evaluate how the current release blocking and release informing policy compares with whats proposed in kubernetes/test-infra#18599 .

/reopen

[k8s-ci-robot]

@alejandrox1: Reopened this issue.

In response to this:

Reopening this to evaluate how the current release blocking and release informing policy compares with whats proposed in kubernetes/test-infra#18599 .

/reopen

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[spiffxp]

I would recommend we strive to make the criteria something that can be enforced via tests or automation. I still think the final decision should come down to humans, but it's not clear to me how often people really check against adherence to these criteria.

Taking a look at release-blocking criteria

Have the average of 75% percentile duration of all runs for a week finishing in 120 minutes or less

This used to be charted; p75_duration in http://storage.googleapis.com/k8s-metrics/job-health-latest.json is daily not weekly

Run at least every 3 hours

If every job is a prowjob, we could statically check the job configs that use interval; if not we could approximate by using runs from http://storage.googleapis.com/k8s-metrics/job-health-latest.json and alert if it's less than 8

Be able to pass 3 times in a row against the same commit

We don't measure this currently, is it possible for us to do so? Or should we use some other measure?

Be Owned by a SIG, or other team, that is responsive to addressing failures, and whose alert email is configured in the job.

Ownership is enforced via static checks against the testgrid config. "That is responsive" though, I'm not sure how we measure that?

Have passed 75% of all of its runs in a week, and have failed for no more than 10 runs in a row

This used to be charted; failure_rate in http://storage.googleapis.com/k8s-metrics/job-health-latest.json is daily not weekly

I think testgrid's summary page shows how many out of 10 recent columns passed, but I'm not sure if we measure this over time?

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[cpanato]

/remove-lifecycle stale

[cpanato]

added this to my next week backlog

[saschagrunert]

Took my freedom to rename the issue to optically match with others.

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-contributor-experience at kubernetes/community.
/lifecycle stale

[spiffxp]

/remove-lifecycle stale

[spiffxp]

I would like to move this over to kubernetes/sig-testing with the intent of tackling this in v1.22, any objections?

[saschagrunert]

I would like to move this over to kubernetes/sig-testing with the intent of tackling this in v1.22, any objections?

Sounds good! Thank you for catching up with this 🙏

[lasomethingsomething]

/sig testing

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-contributor-experience at kubernetes/community.
/lifecycle stale

[k8s-triage-robot]

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Send feedback to sig-contributor-experience at kubernetes/community.
/lifecycle rotten

[k8s-triage-robot]

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Reopen this issue or PR with /reopen
• Mark this issue or PR as fresh with /remove-lifecycle rotten
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close

[k8s-ci-robot]

@k8s-triage-robot: Closing this issue.

In response to this:

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Reopen this issue or PR with /reopen
• Mark this issue or PR as fresh with /remove-lifecycle rotten
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[spiffxp]

/reopen
/remove-lifecycle rotten
/lifecycle frozen

[k8s-ci-robot]

@spiffxp: Reopened this issue.

In response to this:

/reopen
/remove-lifecycle rotten
/lifecycle frozen

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[k8s-ci-robot]

@cpanato: The label(s) area/release-eng, area/ci cannot be applied, because the repository doesn't have them.

In response to this:

Describe the CI policy for jobs, like:

• having contact information
• resources defined

We have a policy for blocking and informing jobs, https://github.com/kubernetes/sig-release/blob/master/release-blocking-jobs.md .
If we compare this policy with what is proposed in kubernetes/test-infra#18599, what would we add? what would we change?
We should evaluate what changes we need to make to help ensure we are acting on useful information and check tht CI jobs are maintained and well.

/area release-eng
/area ci
/kind documentation
/priority important-soon
/milestone v1.20

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[spiffxp]

/sig testing

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue or PR as fresh with /remove-lifecycle stale
• Mark this issue or PR as rotten with /lifecycle rotten
• Close this issue or PR with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[k8s-triage-robot]

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue or PR as fresh with /remove-lifecycle rotten
• Close this issue or PR with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

[k8s-triage-robot]

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Reopen this issue or PR with /reopen
• Mark this issue or PR as fresh with /remove-lifecycle rotten
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close

[k8s-ci-robot]

@k8s-triage-robot: Closing this issue.

In response to this:

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Reopen this issue or PR with /reopen
• Mark this issue or PR as fresh with /remove-lifecycle rotten
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.