Audit, reorganize, and document labels (and probably also milestones)

[handrews]

We've accumulated quite a few issue/PR labels (48) created at different times and with different usage in mind. As @lornajane discovered, our TDC meeting issue template expects certain labels to be used in ways that they are not (which we did not notice because we almost never get to those parts of the agenda). Other labels are for specific projects that are no longer of interest.

Some labels are documented in various parts of DEVELOPMENT.md, and a few have descriptions in the label list, but there is no comprehensive documentation. And the ones that are documented in DEVELOPMENT.md are kind of buried in it. Sometimes this makes sense, as with the section on PR/issue automation.

We should probably do the same with milestones, particularly as there is a weird overlap in labels and milestones.

---

I'm assigning this to myself – I'll take a look at the current usage and propose something.

[handrews]

Here's the basic audit (as of 19 Jan 2024, unless otherwise notes). I'll comment later with suggestions, once I've had more time to think on it.

All entries are labels unless otherwise specified.

Release Management

Milestones

All entries in this subsection are milestones.

• v3.x-Candidate
  • Issues: 7 open, 3 closed
  • PRs: 3 rejected
• v3.2.0
  • Issues: 2 open, 1 closed
• v3.1.1
  • Issues: 2 open

Closed Milestones:

• v3.0.0-rc1, v3.0.0-rc2, v3.0.0-final
• v3.1.0-rc1, v3.1.0-rc2, v3.1.0-final
• 3.2.0-dev

Past Releases

• 2.0
  • Issues: 5 closed
  • PRs: 3 merged, 2 rejected
• 3.0.0-RC1
  • Issues: 20 closed
  • PRs: 15 merged, 1 rejected
• 3.0.0-RC2
  • Issues: 23 closed
• 3.0.1
  • Issues: 1 closed
• Old Versions
  • Issues: 3 closed

Current Defined Releases / Ongoing Work

• 3.0 needs documentation
  • Issues: 2 open, 1 closed
• 3.0.4
  • Issues: 14 open, 5 closed, 1 not planned
  • PRs: 4 merged, 3 rejected
• 3.1.patch
  • PRs: 1 rejected

Future Releases

• OpenAPI.Next Proposal
  • Issues: 52 open, 99 closed
  • PRs: 19 merged, 4 rejected
• Post 3.0 Proposal
  • Issues: 13 open, 5 closed
  • PRs: 3 rejected
• Rejected Proposal
  • not yet used, or all uses removed

Draft process

Documented in DEVELOPMENT.md under "Draft Features", although the labels it mentions are draft-feature, draft:proposal, draft:pilot, draft:abandoned, and draft:graduated (one punctuation typo, plus two labels never added).

• draft:pilot
  • Issues: 1 closed
• draft:proposal
  • PRs: 2 open, 1 rejected
• draft:feature
  • Issues: 1 closed

Issue Workflow

Manual

• help wanted
  • Issues: 7 open
• review
  • Issues: 24 open, 15 closed
  • PRs: 13 merged, 3 rejected
• wontfix
  • not yet used, or all uses removed
• Moved to Moonwalk
  • Issues: 1 open, 11 closed [as of 25 Jan 2024]

Automated Issue Closure

Documented in DEVELOPMENT.md under "Automated closure of issues Process"

• Needs attention
  • not yet used, or all uses removed
• Needs author feedback
  • not yet used, or all uses removed
• No recent activity
  • not yet used, or all uses removed

TDC Agenda Automation

Documented in DEVELOPMENT.md under "Automated TDC agenda issues Process", but the label is also mentioned under "Tracking Process" with various other labels. Both usages seem to be present.

• Housekeeping
  • Issues: 11 open, 263 closed
  • PRs: 6 merged, 3 rejected

Infrastructure Maintenance Automation

• dependencies
  • PRs: 7 open, 2 merged, 1 rejected
• javascript
  • PRs: 7 open

Issue Types

GitHub-Defined

• bug
  • Issues: 1 open, 8 closed, 1 not planned
• duplicate
  • never used, or all uses have been removed
• enhancement
  • Issues: 3 open, 5 closed
• invalid
  • never used, or all uses have been removed
• question
  • Issues: 14 open, 23 closed
• regression
  • not yet used, or all uses removed

OAI-Defined

Note: Housekeeping probably also belongs here (see "TDC Agenda Automation" above")

• Meta Issue
  • Issues: 6 open, 2 closed
• Minor Change
  • Issues: 1 open
• OAI-scope
  • Issues: 1 open, 2 closed
• Outreach
  • Issues: 1 open
• Process
  • Issues: 2 open
• Sub Issue
  • Issues: 32 open, 68 closed
• Tooling
  • Issues: 3 closed
• TSC Minutes
  • Issues: 1 closed

Issue Topics

Specifications

Projects

This is the only project defined for the repository:

• OpenAPI Specification
  • Issues: 1 open

Labels

• discriminator
  • Issues: 9 open
  • PRs: 1 open, 1 draft, 1 rejected
• links
  • Issues: 7 open, 1 closed
• overlay-candidate
  • Issues: 2 open
• $ref
  • Issues: 18 open, 6 closed
• schema-object
  • Issues: 9 open, 25 closed
  • PRs: 2 merged, 1 rejected
• security
  • Issues: 18 open, 3 closed
  • PRs: 1 open
• xml
  • Issues: 8 open

Other Deliverables

Milestones

All entries in this subsection are milestones.

• Ancillary
  • Issues: 2 open, 3 closed, 1 not planned
  • PRs: 3 merged, 6 rejected
• Security Definition Proposals
  • Issues: 1 open

Labels

• Documentation
  • Issues: 5 open, 14 closed, 1 not planned
• format-registry
  • PRs: 7 merged
• Getting Started Guide
  • Issues: 1 open, 2 closed
• Schema
  • Issues: 4 open, 7 closed
  • PRs: 2 merged, 1 rejected

[miqui]

@handrews - nice! So, what are you proposing? Say for example, refactor labels for issue topics ? How is a new label proposed?

[handrews]

@miqui I'll be proposing a refactor here. If you or anyone else wants to propose a refactor you are welcome to do so, either before or after I publish mine. I expect to get to this within the next week or at most two, as I am working on many things right now. I'm also trying out some things and looking more deeply into label usage before I propose anything.

I would suggest holding off on proposing individual new labels until we have at least one refactor proposal available to discuss. Although if there is another issue that you think can be solved by a label, you can propose one in that issue (like I proposed "Moved to Moonwalk" in #3508).

[handrews]

While I'm not quite ready to propose a complete refactor, here are my observations so far:

Labels used in automation workflows should not be used for anything else

We should prefix all lables used in automation so that folks know that using them has specific effects. Labels like Housekeeping that are both used by automation and used by humans in an ad-hoc fashion should be split up.

Don't use milestones for things that don't have clear end conditions.

I've closed Security definition proposals (last touched 7 years ago - one issue which is a Meta-Issue, see below), Ancillary (one remaining open issue, about media types, which we aren't going to forget as there's a whole IETF I-D about it happening elsewhere), and v3.x-Candiate (most things closed as either completed or moved to Moonwalk, and the rest are sufficienlty labeled that we'll consider them anyway as we sort out the 3.x(.y) releases.

Use milestones to manage release contents, not labels

I've added a v3.0.4 milestone alongside the existing v3.1.1 and v3.2.0 milestones.

Since an issue can only be part of one milestone at a time, I've put all issues labeled 3.0.4 in the v3.0.4 milestone, and all issues labeled 3.1.patch but not 3.0.4 in the v3.1.1 milestone.

I think it's straightforward that anything that goes in a 3.0.y is expected to roll forward into the next v3.1.y and (if relevant) and further v3.x.y lines. There is some sort of script that ports PRs across branches, or so I'm told, which should impact how we are managing our PRs and ensure that we do in fact roll fixes forwards.

I'll be deleting all of the labels named after releases, including 3.0 needs documentation which was only on one issue that was also labeled with 3.0.4.

QUESTION: Do we want to make sure closed issues tagged with old releases are in the right historical milestone? This would probably mean creating some milestones for old releases like 2.0. I'll hold off on deleting until that's answered, although I will un-label the open issues still using these labels since the milestone info is sufficient.

Topic/theme labels are more flexible and intuitive than Meta-Issue/Sub-Issue

I've noticed that our issues clump. A lot. For release planning purposes, it makes sense to group them so that we can have some coherent themes to 3.x(.y) releases. It will also help us make decisions about scope.

For example, there are a ton of requests that are really JSON Schema requests. Unless we want to define more OAS-specific keywords, we should just close these as out-of-scope. If we do want to define more OAS-specific keywords, we should probably do all of them at once.

There are also themes like requests for more/better examples, or for wording clarification. These identify types of work that specific people might want to do. And also ones that might make nice themes for patch releases in particular. We could do a patch release focused on clarification, and then one focused on better examples (just to illustrate - idk if that is really the right way to organize things).

I plan to add more topic/theme labels. I also think that it's fine to delete such labels once they have served their purpose.

Meta-Issue and Sub-Issue were ways that folks organized this sort of thing in the past. I think it worked quite well for them, but it's now stale. It's also less intuitive than simply searching on labels, which is how this works in most repos.

The concept of a meta- issue can be fine as an ad-hoc workflow management (e.g. #3516), but they should be short-lived.

[handrews]

To extract some themes from the previous comment:

• Make it clear which labels impact automation, and don't use them for anything else
• Use labels to organize topics for release planning
• Use milestones to organize releases, putting issues in the milestone for the oldest relevant release line
• Use labels to group types of work that different volunteers might find appealing (e.g. improving examples, which usually does not require the same level of debate as spec changes)

[lornajane]

Good summary! For the question on we want to categorising old issues, I'd say that we don't - or at least let's start with the ones that are open so we can operate the project without so much baggage.

The labels proposals sound good although I'd distinguish between the ones that have side effects, and those that don't. We use housekeeping for all the operational stuff, and the automation also creates issues and applies that label, but applying the label doesn't make anything happen. The "needs author feedback" however does kick off some automation.

Milestones are a mystery, I think we need some more opinions on how they have previously been used. We release very infrequently and I'm not sure we're super clear about what has been added to the various version branches in the last few years, or what the policy on back-porting things to 3.0 when they're added to 3.1 is. Does the issue close when it's merged to a version branch? How do we build the changelogs? I'm open to patch releases to help checkpoint where were are up to on the 3.0 and 3.1 branches as we document or adjust the processes.

[handrews]

@lornajane all good points regarding milestones. It's on my list of things to do this week to assess whether there's anything in 3.0.4 that needs to be ported to 3.1.1, and vice versa. I think once we make sure those are in sync, we can define a process, and I'll advocate strongly for "start with the oldest release line that is relevant and immediately port forward to stay in sync."

I think this is the first time we'll be maintaining multiple release lines. 3.0.3 went out before 3.1.0, and there was mild debate over whether we should even do a 3.0.4. Which resolved fairly quickly to "yes, we should at least do that, but maybe not a 3.0.5."

I see what you mean about Housekeeping, although I've been tending to use Process for more things recently because I feel like may Housekeeping is more about mechanical day-to-day management of things. But I just made that up, so...

[handrews]

Regarding release frequency, I've made some proposals in #3528 and #3529 which I expect are too aggressive. But they do state rationales for more frequent releases (as often as we want to do them for 3.x.y, and no faster than most tooling vendors can keep up for 3.x).

[handrews]

I have created quite a few more labels to help group the (still quite large) backlog and let patterns emerge to help us plan for the 3.x(.y) releases. So for the moment, we have even more labels.

I've been quite liberal in applying them, but will continue to refine them as the patterns of user requests become more clear. We can also change the names - I decided it was better to do stuff while I had time than wait to find consensus on the label names. They are easy enough to fix later. I will also clean up any excess / overlapping / not-actually-as-useful-as-I-thought-they'd-be labels after I've finished this whole exercise - please bear with the labeling excess in the meantime!

[miqui]

@miqui I'll be proposing a refactor here. If you or anyone else wants to propose a refactor you are welcome to do so, either before or after I publish mine. I expect to get to this within the next week or at most two, as I am working on many things right now. I'm also trying out some things and looking more deeply into label usage before I propose anything.

I would suggest holding off on proposing individual new labels until we have at least one refactor proposal available to discuss. Although if there is another issue that you think can be solved by a label, you can propose one in that issue (like I proposed "Moved to Moonwalk" in #3508).

@handrews - Issue Topics: current labels you have listed (imho) provide good+ context for pro
jects,milestones. Issue Types: given the nature of the repo (content) I am wondering if the github+OAI provide a reasonable combo-context. No labels to indicate urgency. Never seen this applied to the spec. Is this by design? Draft: a good set of labels. Issue Workflow: moved to Moonwalk - great idea.

[miqui]

To extract some themes from the previous comment:

• Make it clear which labels impact automation, and don't use them for anything else
• Use labels to organize topics for release planning
• Use milestones to organize releases, putting issues in the milestone for the oldest relevant release line
• Use labels to group types of work that different volunteers might find appealing (e.g. improving examples, which usually does not require the same level of debate as spec changes)

@handrews great summary.

[miqui]

I'm open to patch releases to help checkpoint where were are up to on the 3.0 and 3.1 branches as we document or adjust the processes.

Agreed.

[handrews]

I'm still experimenting with the labels we'll want to keep, but I'd like to propose removing the following labels:

• enable triage team to transfer these issues to other repos:

  • Outreach (-> Outreach repo)
  • Tooling (-> Tooling repo)
  • Documentation (-> learn.openapis.org repo)
  • Getting Started Guide (-> learn.openapis.org repo)

• Superseded topic labels

  • $ref (superseded by new re-use:* labels)

• Outdated processes

  • OpenAPI.Next Proposal (no replacement - a very old definition of "Next")
  • Post 3.0 Proposal (no replacement, other labels sufficient)
  • Rejected Proposal (superseded by the draft:* labels/process?)
  • Meta-Issue (mostly replaced by topic labels)
  • Sub-Issue (mostly replaced by topic labels)
  • Minor Change (no replacement, unused)
  • TSC Minutes (only used once, on a typical TSC meeting issue)

• Use milestones instead (if we agree to that and how to manage issues across release lines)

  • 2.0 (no replacement)
  • 3.0.0-RC1 (could be added to closed v3.0.0-rc1 milestone)
  • 3.0.0-RC2 (all in closed v3.0.0-rc2 milestone)
  • 3.0.1 (no replacement)
  • Old Versions (no replacement)
  • 3.0 needs documentation (no repalcement, only 2 issues, both closed)(
  • 3.0.4 (already moved to v3.0.4 milestone)
  • 3.1.patch (already moved to v3.1.1 or v3.0.4 milestones)

[lornajane]

I'm +1 on all these suggestions - and we can always re-create and/or re-label as we go along if we want to evolve the strategy.

[earth2marsh]

Discussed on today's TDC call. Thanks for thinking this through, please move ahead.

[handrews]

All of the proposed deletions are done except for the ones that are blocked on the ability to transfer issues. I've filed OAI/community#13 and tagged Ron to get that admin work done (also paging @darrelmiller and @earth2marsh but I think we already determined that only Ron has all the necessary permissions).

[handrews]

The labels I created seem to either be working fine, or at least not bothering anyone enough to complain.

I think the only remaining labels that are good candidates for deletion are:

• duplicate
  • currently unused, we just tend to say "this is a duplicate of #..." when closing things
• invalid
  • currently unused, we just tend to explain why things aren't valid when closing
• regression
  • currently unused, and not all that relevant to spec work (as opposed to software)
• review (NOTE: but see my next comment for an alternative)
  • we don't really use this, it's just on a lot of issues from past ways of working
  • the triage link to unlabled issues in the current weekly meeting agenda seems to be a better way to get attention on not-yet-considered issues
  • otherwise, folks who want a review in the weekly meeting are pretty good at putting them on each week's agenda issue, which seems like a better way to focus on active concerns
• wontfix
  • currently unused; there's now a "Close as not planned" option that is better anyway

The only label I feel a bit ambiguous about is enhancement. We don't use it much, but we do use bug and question so it could be a nice high-level partition. If we keep it, we should use it more consistently, but I'd be fine with saying "if it's not labeled bug or question or maybe Process we can assume it's an enhancement."

I think resolving these will let us close out this issue. Any further changes can be handled by @OAI/triage in its likely-soon-expanded form now that we have the new TSC in place.

[handrews]

Alternatively, review could be used to direct TSC attention to issues where we need to decide whether we're going to take action or not. There are quite a few issues that could result in a spec change but probably don't make sense to do for a variety of reasons. We need a way to make decisions on these and either queue them for work or close them as not planned.

[handrews]

Discussed in the TDC call today. Decisions:

• delete all of the ones listed above except (for now) review (which I will do now)
• keep `enhancement
• For now, use review to bring issues to the TSC's attention for decision on whether to implement or not - in particular, if we aren't going to do something, we should close it
• @baywet noted that for things that are approved, a help wanted label is a common practice
• @karenetheridge mentioned some other options rather than the generic review, such as triage and... several others I forgot, and I didn't save the chat this week, so hopefully she can chime in here

[lornajane]

I think @karenetheridge 's list was review, triage, and discussion-needed. If we can nail down when to apply each of those, we can put something in the contributing guide and create those labels

[handrews]

I have filed the following issues for the remaining open tasks so I can close this mammoth issue - I kept forgetting there were specific action items here, plus this is assigned to me but the follow-ups are not things I can take on right now:

• Define and add new process labels and document general label usage in CONTRIBUTING.md #3848
• Document milestone usage in DEVELOPMENT.md #3847

Therefore I am closing this as complete, as the major audit and re-organization is done, and the labels in question are mostly self-documenting (if you read the Labels list) feature areas. The ones needing process documentation are covered in #3848.