Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[docs/component-stability.md] Add criteria for graduating between stability levels #11864

Draft
wants to merge 4 commits into
base: main
Choose a base branch
from

Conversation

mx-psi
Copy link
Member

@mx-psi mx-psi commented Dec 12, 2024

Description

Code ownership and maintenance of components continues to be an issue, with varying levels of support across contrib. As we approach 1.0 and the ability to mark components as stable, we want to make sure that components that we deem as 'stable' have a healthy community around them. We have three datapoints that we can leverage here: how many codeowners a component has, how diverse these are in terms of employers and how actively the codeowners have been responding to issues/PRs in the recent past.

We need criteria that

  1. Are reasonable predictors of the component health over the short/medium term
  2. Are not too onerous on the code owners

This excludes the

Some notes:

  1. Some beta components do not meet the criteria listed on the PR. This will be the case even after the transition for some components. This PR makes no claim as to what should happen to these components stability (so, de facto, they will stay as is).
  2. The OTLP receiver and exporters do not meet this criteria today because they don't have listed code owners. We can solve this either by carving out an exception or by listing code owners.
  3. We need automation and templates to enforce this.

Link to tracking issue

Fixes #11850

@mx-psi mx-psi added the Skip Changelog PRs that do not require a CHANGELOG.md entry label Dec 12, 2024
Copy link

codecov bot commented Dec 12, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 91.58%. Comparing base (8bd803f) to head (859ece3).

Additional details and impacted files
@@           Coverage Diff           @@
##             main   #11864   +/-   ##
=======================================
  Coverage   91.58%   91.58%           
=======================================
  Files         448      448           
  Lines       23755    23755           
=======================================
  Hits        21757    21757           
  Misses       1623     1623           
  Partials      375      375           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Comment on lines +114 to +118
3. Within the 30 days prior to the graduation request, the code owners MUST have reviewed and
replied to at least 80% of the issues and pull requests opened against the component. This
excludes general PRs or issues that are not specific to the component itself (e.g. repo-wide API
updates). It is not necessary that the issues and PRs are closed or merged, but that they have
been reviewed and replied to appropriately.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is formalizing the existing practice and I agree with it.

## Beta to stable

To graduate any signal from beta to stable on a component:
1. The component MUST have at least three active code owners.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think there should be a commitment from codeowners that there is a SLA for first response on bug issues.
The commitment should be measured in days.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This raises some questions for me including:

  • What happens when people go on vacation/have a kid/[insert activity here that leads to a prolonged period of absence]?
  • What happens if people don't follow this SLA? Typically an SLA means that you pay if you don't meet a certain standard, how do you "pay" here?

Copy link
Member

@julianocosta89 julianocosta89 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we shouldn't make it too hard to have community components.
I think vendor components and widely used components will not have any issue to follow the guidelines.

When we think about components that add value to the overall project, but may not be interesting/priority to vendors, they may struggle to get folks involved on it, and that would disqualify them from being moved to stable.

I do understand that we need to provide a way to ensure maintainability of stable components, but maybe we could draw something over the ideas of:

  • Being active
  • Replying to issues related to its components timely
  • Fixing reported bug timely
  • ...

If we have a component that is not vendor related, but maintained by 2 folks from a single employer, that wouldn't allow them to move on.

Also, let's imagine the following scenario:

  • 3 folks are codeowners of a component, 2 from one company and another one from another company.
  • They graduate to stable.
  • A couple of months later the person that was from the other company moves to the same company of the other 2. Would the component be demoted?

I don't think it should be, if they are active and responsive in issues related to that component.
I know it is a corner case, and may never happen, but it still can.

My main point here, is actually that we shouldn't make it too hard to have community components.
I know a couple of companies that develop internal components to solve their customers' issues. It would be awesome to have a couple of those contributed back to upstream, and let the community grow together.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Skip Changelog PRs that do not require a CHANGELOG.md entry
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Establish codeowners minimum criteria for moving up through the stability ladder
4 participants