Update dotnet/runtime issue management guide

[eiriktsarpalis]

This draft is an attempt to document and formalize an issue management workflow/guide/playbook intended for reference by both repo maintainers and community members. The guide aspires to

1. Provide a common set of triage guidelines for reference by area owners.
2. Make issue management decisions more transparent and accessible to the wider community.
3. Be used as a basis for further automation of common issue management policies.

This document is being developed by the responsiveness working group as part of the experiment outlined in #1. TL;DR we hypothesize that a well-documented issue management workflow backed by automation of common policies can positively impact responsiveness-related metrics of an OSS repo (e.g. "time to triage", "time to resolution", etc.) by

1. Aiding area owners adopt more efficient triage workflows.
2. Using automation to catch any stagnating issues.
3. Letting maintainers more effectively identify issues where "the ball is in their court".

This document uses triage practices and terminology drawn specifically from dotnet/runtime repo, since we intend to use this document as a basis for conducting responsiveness-related experiments in the dotnet/runtime repo. That being said, many of the concepts described here do transfer to other similar OSS repos as well.

The contents are intended as a supplement to the existing dotnet/runtime issue guide, drawing inspiration from the triage guidelines published by the VS Code and aspnetcore repos.

This is by no means intended to be a definitive or exhaustive guide, however it does aspire to cover most issues (the 20% of known workflows accounting for 80% of all issues). Finally, do note that this draft intentionally omits pull requests, as they are currently not the focus of our experimentation.

[JulieLeeMSFT]

@eiriktsarpalis I added more text yesterday in my repo. Please feel free to take texts from there.

JulieLeeMSFT/contributor-community-experiments@1a38805

[danmoseley]

this doc is specific for dotnet/runtime, right? it's in this repo just as an example.
assuming so, this should probably explain area labels, and link to the area list.
there are also os- and arch- labels that inform ownership.

[JulieLeeMSFT]

@eiriktsarpalis, I have Labels section and link to the area list in the text from my repo.
-- List of runtime areas and owners can be found in https://github.com/dotnet/runtime/blob/main/docs/area-owners.md.

this should probably explain area labels, and link to the area list.
there are also os- and arch- labels that inform ownership.

[eiriktsarpalis]

this doc is specific for dotnet/runtime, right? it's in this repo just as an example.

The idea is to author an unofficial, experimental and incomplete issue management guide specifically to drive our experiments in dotnet/runtime. Any automated actions conducted as part of the experiments would link to the guide in this repo.

[danmoseley]

should this say something about how triage decisions are made? what is considered when we decide whether keep an issue or not? (eg., we would take a PR for it)

[danmoseley]

that's actually potentially more interesting than the process details, and likely quite opaque (perhaps even within the team). what is a bug bar? do we have one (implicitly?)
also, late in the cycle, what is the bug bar?
what considerations play into our decisions (eg risk, alignment to our strategy, community input, team resources, code complexity..)

how do we use milestones?
how do we use assignment? (eg., we consciously do not assign most bugs out -- we used to, but it achieved little and sucked the air out of the repo)
what other labels might triage apply eg., up for grabs.

[JulieLeeMSFT]

how triage decisions are made

@eiriktsarpalis, I have Criteria to determine milestone and Closing an Issue sections that are related to triage decisions.

[eiriktsarpalis]

should this say something about how triage decisions are made? what is considered when we decide whether keep an issue or not? (eg., we would take a PR for it)

that's actually potentially more interesting than the process details, and likely quite opaque (perhaps even within the team). should this say something about how triage decisions are made?
what is a bug bar? do we have one (implicitly?)
how do we use milestones?
how do we use assignment?
what other labels might triage apply eg., up for grabs.

My intention with this draft is to emphasize what issue state transitions are possible rather than why or how they are being made. As you mention the latter can vary depending on area, timing, business priorities, etc. From a responsiveness perspective there is value in documenting just the workflows since it provides a rough set of guardrails to maintainers, lets us identify opportunities to introduce automated policies and makes it easier to identify issues where maintainer action is required.

The triage documents from aspnetcore and VSCode are similar in that regard, as they generally don't expand a lot on why many of these decisions are being made. Perhaps then calling it an "Issue Management Guide" is misleading, this could most likely be an "Issue Management Workflows" chapter within a larger guide.

[danmoseley]

Thanks for working on this.

I like that we're couching this as an example of (something we hope to iterate towards) an effective guide might look like, and NOT as a reference-quality triage process (I have no reason to believe that dotnet/runtime has an excellent process worth others modeling, or even that it's relevant to others, and wouldn't know how to prove it was)

Is there any reason you didn't make this a PR in dotnet/runtime? That's where the community is, so they can see it and give feedback. Once we have something we can merge, and proven (by surveys or other means) that it is worthy of becoming an example, we can put it in this repo.

I do suspect the most interesting part here would be policy and philosophy, rather than mechanics. how we make our decisions. what communication we will do. how others can provide input into the decisions. etc. That is the part that's least codified (even internally, arguably)

[eiriktsarpalis]

I added more text yesterday in my repo. Please feel free to take texts from there.

@JulieLeeMSFT I seem to have permission issues with your fork: that particular link 404's. If you've added more text I'm not able to see it. I have also enabled edits by maintainers, so feel free to push any changes directly to my branch.

[eiriktsarpalis]

Is there any reason you didn't make this a PR in dotnet/runtime? That's where the community is, so they can see it and give feedback. Once we have something we can merge, and proven (by surveys or other means) that it is worthy of becoming an example, we can put it in this repo.

I felt that getting an "Issue management guide" approved for dotnet/runtime would require a substantially wider scope and review than what might be necessary for the purposes of running a localized experiment. I'm certainly hoping that feedback from the experiment could contribute to both a repo-wide guide being incorporated in dotnet/runtime, as well as incorporating repo-agnostic recommendations here.

[JulieLeeMSFT]

permission issues with your fork

It was marked as private. I made it public just now. You would be able to see it now.

[eiriktsarpalis]

@JulieLeeMSFT I incorporated the contents of your draft, essentially verbatim. Following feedback from @danmoseley I renamed my section "Issue Workflows" and integrated it as a section in your text which is establishing a broader scope. There is a slight amount of duplication, particularly in defining our triage process but I thought I'd merge the two documents first and refine later.

[Pilchie]

I took a look at this, and I definitely like it.