Stale icebox issues cleanup experiment

[heng-liu]

Creating this issue to track the stale icebox issues cleanup experiment we're rolling out in NuGet/Home. Please refer to the original issue in microsoft/contributor-community-experiments#2 for more background.

Hypotheses

In the past, we triaged issues with lower priority as icebox issues (compared to backlog issues). While now in NuGet/Home repo, there is a substantial number of icebox issues that have not been updated for over 2 years. In theory, we could keep all issues open no matter if what will happen in the future. But that makes it hard to understand what has realistic chances to ever make it into the repository.

This is a proposal to roll out an old low priority issues cleanup experiment. The experiment is based on a couple of hypotheses:

• Reducing issue sizes allows for more efficient planning.
• Close issues which we don't have the capacity to work on, makes it clear which issue has the realistic chances to ever make it into the repository. So that the community would not be surprised when the issue get closed 5 years later with no fix.
• Several major repos including non-Microsoft repos have an issue cleanup/issue bankruptcy policy. We have received positive feedback from partner teams that successfully applied such policies to their backlogs.

Experiment

There are a number of parameters to be considered for the experiment:

Selection Criteria

• Focus issues that have been triaged as icebox in the past.
• Focus issues that have not been updated in over 2 years.
• Exclude popular issues, e.g. issues that has more than 1 comments or more than 1 upvotes [Not currently possible due to FB scheduled search restriction, can explore this avenue in the future].
• Incorporate a manual opt-out mechanism (e.g. by applying a Status:Excluded from icebox cleanup label).

Execution

1. FabricBot applies the Icebox cleanup candidate label to selected issues and writes a comment explaining that the issue has been marked for closure, linking to this issue and encouraging users to submit their feedback on the process.
2. If no response has been provided within 14 days, the issue will be closed automatically by FabricBot. Otherwise, FabricBot will add a label Triage:NeedsTriageDiscussion and the issue will remain open and we will discuss about this issue in our triage meeting.

The policy would initially cleanup issues in bulk, but will remain in place so that any other issues going over the 2 years threshold are subsequently marked for closure.

Risks

• Initial bulk cleanup might trigger a flurry of activity, substantially increasing workload for triaging meetings.
• Automated closure of issues can be controversial with the community. We hope that a carefully worded message pointing out this is an experiment and linking back to this issue should serve to mitigate any such concerns.

[ghost]

@heng-liu Issue is missing Type label, remember to add a Type label

[loic-sharma]

I strongly dislike auto-closing issues. Opening an issue, especially from an outside customer, is often a difficult task. You need to find the right repository. You need to provide relevant debugging information. You may even need to write in a language you're uncomfortable with. Having a bot auto-close their issues feels cold and unwelcoming.

[heng-liu]

All above linked icebox issues with comments are discussed in recent triaging meetings. Some of them are re-triaged as backlog issues, some of them are closed with explanations.
The rest are closed automatically.
Close it as the experiment is completed.