Streamlining the Process Document

[wseltzer]

@plehegar and I did some analysis of the length of the Process document over time. Using the same stylesheet and font/layout, we found that the process has recently grown from 60 pages to 72 in the Everblue draft (historical data below). Keeping in mind that we already have had need for a "Process for Busy People," I'd ask that we look carefully for text we can cut. In particular, text that's merely descriptive or details that could be left to /Guide could be moved out of this governing document.

I hope we can agree on brevity as a goal, even if we disagree on specific proposed cuts.

year/draft | length
1999 | 54
2001 | 37
2001 | 41
2003 | 61
2004 | 63
2005 | 61
2014 | 60
2015 | 53
2017 | 55
2018 | 58
2019 | 61
20 ED | 66
20 EB | 72

[jeffjaffe]

I agree with Wendy that this is a problem.

It is very difficult to make substantial change and streamline the document at the same time. In Process 2020 we are introducing substantial change which is part of what is causing the growth in length.

I think we desperately need a clean-up/streamlining effort. My instincts are that we should strive for a quick turnaround Process 2021 - certainly adding new critical items, but mostly focused on bug-fix and streamlining.

[frivoal]

Overall, I agree we should strive to make the document shorter, and I agree with Jeff that a good time for doing that is probably P2021, and to do that exercise in combination with a general clean-up. I think the actual goal is readability / understand-ability. Brevity is one aspect of that, and there's certainly room for improvement on that aspect, but it needs to be balanced against other factors. Even with that nuance, I do agree that we should be able to shorten things.

At the same time, I'll note a few things:

• Bikeshed now generates an index of terminology defined in the document. This is useful, but increases the page count by about 2 A4 pages on by machine.
• The change section grow over time, yet that seems useful, and I don't think it is a source of concern. "Changes since Process 2019" on the master branch increases the page count by about 1 A4 page on my machine

So there's still some growth, but it may be less dramatic than a first look would suggest.

[dwsinger]

I am also concerned. Something that has 12 pages of text to document the process should give us all pause.

[chaals]

A long time ago, I hoped to reduce the size of the process significantly. It seems for a while that worked, but that we are going backwards again.

As @frivoal noted there are some things that mitigate this - the change section, index, and the images (which were not there when we started what became Process 2014).

That said, I think we try to list many examples, instead of keeping things brief. On the other hand there has been feedback that people otherwise find the Process document confusing.

There are also sections that seem very verbose, but unnecessary to the process itself. My experience in the past was that getting agreement to remove them was difficult or impossible - eventually I gave up, to focus my available effort on other things that seemed more likely to justify the time.

[nigelmegitt]

I'm going to chip in contrarily here:

I disagree that this is a major problem, at least at first glance. I dislike long documents as much as anyone else, but in this case nobody is expected to recite or memorise the process. I would prioritise navigability, clarity and the correct scope higher than brevity. Focusing on brevity is likely to distract from more important issues.

What are the reasons for being concerned about the length of the document? Is there evidence that it is causing particular problems? What are those problems?

[dwsinger]

What are the reasons for being concerned about the length of the document? Is there evidence that it is causing particular problems? What are those problems?

People can't remember what a long/complex process says. People might not even be able to understand it, if it's long and complex. It's mostly about comprehensibility and remembering.

[nigelmegitt]

People can't remember what a long/complex process says. People might not even be able to understand it, if it's long and complex. It's mostly about comprehensibility and remembering.

All good hypothetical and general points. Do we have evidence that they apply specifically in the case of the Process? And if we do, and the problem is primarily comprehensibility, then we should target fixing that over merely reducing the length of the document. Hopefully in some cases they'll coincide. In other cases, say if there is some dense precise spec terminology in the Process that is hard to understand but that is important, it might be better to add information boxes that explain it more simply, at the risk of lengthening the document.

I'm not challenging this issue purely to be contrary. Large scale editing exercises can be a huge time sink, not just in the up front effort but also in the review time to check that the new document actually means the right thing, and there's a high risk of not achieving the end goal. We should not set out on this path unless we have some really clear objectives. If the objective is comprehensibility then we should look for a better metric than document length.

[frivoal]

For process 2021, I intent to spend a good chunk of time focusing on readability, understand-ability, non ambiguity, logical simplifications, untangling spaghetti balls, and the like. I also expect that as part of that exercise, we'll uncover redundant or unnecessary parts of the process. I don't intent to pursue brevity for brevity's sake, but I do suspect that after that exercise, the document will end up being somewhat shorter.

[chaals]

@nigelmegitt wrote

nobody is expected to recite or memorise the process

People don't have to recite it because they can go and look. But people do present pieces of the process from memory "all the time". IMHO it is a feature that most of a Working Group knows the process they work under, rather than having to look it up.

The longer the document, the harder it is to understand the entirety (especially with the lack of illustration for what is substantially a set of flowcharts). In addition, the process has been changing, more or less every year, sections that some groups don't use even once a year.

I agree that there are metrics we should be using beyond length, but in my experience overall document length - and to @frivoal's points above "length of the bit you have to read carefully", and "length of the summary checklists or flowcharts you can rely on to remind you" are important metrics.

[dwsinger]

Perhaps we need to split this into two issues (a) textual changes that make the document shorter, simpler, and easier to read but that don't make significant changes and (b) simplifications such as in #346 that both make the process functionally simpler and the document shorter.

It would also help to identify rarely-encountered edge cases and reduce the text for them or make it easier to realize that they are sidelines.

[michaelchampion]

I think the hierarchy of goals for the Process is:

1. Clarity / Consistency. There should not be disagreements about what the Process "really means". So issues such as (sniff, my favorite) Clarify the voting process #60 should be fixed one way or the other so there is no discrepancy between what W3C does and what the Process says it should do. [Obligatory rant -- I suggest making the actual operation comply with the current wording and run an STV election for each open seat, but I know you're not going to and I'm not around anymore to object]

2. Simplicity. The average W3C participant should have a mental model of how the Process works that works for most ordinary situations. For example, pick a way to resolve situations where there are a significant number of dissenters to a WG's, AC, or W3M majority opinion, and use that consistently. I suspect supermajority voting is the most common and effective way to resolve dissent without a an engaged BDFL, but feel free to pick another if it is easy to understand and apply. [Obligatory rant: I think Everblue is too complex as currently drafted for most participants to have an accurate mental model of it, but you've already overruled my dissent, and time will tell].

3. Brevity. So long as you don't sacrifice clarity or simplicity, make the document itself as short and readable as possible. That might mean putting examples and discussing complex corner cases in a FAQ (as the Patent Policy does).

[jeffjaffe]

I marked this as priority for P2021. The Process Document has become much more complex. It needs streamlining badly.

[dwsinger]

Here's a proposal: we find all the material that is explanatory, and trim it to a minimum, and move the bulk out into a document or documents (like the W3C Process for Dummies). We use linking to link to background or explanatory material much more. We try to remove repetitions, and rely more on links.

Specifically, I suspect that the changes to Horizontal/Wide Review #401 could be split this way, link in the team's draft Document Review wiki.

The process would be shorter, it would be easier to keep it consistent, and the guidance material would be easier to find and maintain.

[frivoal]

Here's a proposal: we find all the material that is explanatory, and trim it to a minimum, and move the bulk out into a document or documents

I support that, and as you say #401 seems like a good candidate / example.

Not sure if that was included in the suggestion or not, but I do not think this should imply reducing the number of "Notes" to zero. Some might be removable, some seem useful to help make sense of the surrounding normative text. My goal is to improve understandability of the Process, and brevity is one tool towards that goal, but I do not support brevity when it comes at the expense of understandability.

We may disagree about the importance of various notes and that's fine, but I don't want to agree to removing all the notes as a matter of principle.

[nigelmegitt]

Adding to the sense of "be careful" I get from @frivoal : it would be easy to go too far if you strip out everything that isn't normative from the Process. The Process does define "the letter of the law" but it would be much weaker if, where there are deliberate ambiguities or openness, it does not also define "the spirit of the law". It's important that non-normative statements that fit into the latter category are retained in the Process document.

@jeffjaffe , since you have deemed this a P2021 priority, please could you respond to #356 (comment) and tell us what you think should be the primary objective of this "streamlining"? It is far from clear to me what sorts of changes you have in mind.

For example, you wrote in #356 (comment) that the document has become too complex; is it possible/desirable to separate the document's complexity from the Process's complexity? If they're really the same thing, then wouldn't it be more practicable to identify specific areas of the Process that are too complex and propose simplifications, as individual issues?

[jeffjaffe]

@nigelmegitt I deemed this as a P2021 priority because I found Wendy's statement of the issue to be compelling. It is getting too long and complex. It is not that I have identified a primary objective to the streamlining.

I do agree with your implication that both the document and the process have become too complex.

[TzviyaSiegman]

I believe both /guide and Process for Busy People were attempts to streamline the Process. My (I think unpopular) opinion is that the Process as it stands is a tome that very few people look at. Even if we claim that it is not meant to be read end to end but as a reference, it is too dense.

@fantasai summarized a version of evergreen with bullets that was highly readable. I think something like this would convey the information well. Look, for example at section 2.5.2, which explains AB and TAG elections. It is a wall of text that most people won't care to read. Can we make this a bullet list (at least para 4 that explains requirements for running in an election)?

[nigelmegitt]

@jeffjaffe I don't think we should begin an exercise like this without a defined goal. @wseltzer's analysis was on length, but you seem to be worried about complexity. As @TzviyaSiegman just pointed out, in some cases breaking a block of text into bullets makes for easier comprehension, perhaps without changing complexity, but if you analyse it in length terms, you would conclude it has made the Process document worse because you take up more pages.

Taken to an extreme, we could improve the process according to a length-based analysis by removing bullet points, white space etc but that would be unlikely to be considered an improvement!

We could argue for a long time about this but in the end I don't think it should be an open issue: it is poorly defined and not easily actionable or completable; rather it should be a guiding principle, for example taken into account when prioritising which issues to work on.

Issue template suggestion:

• Does this proposal make the Process easier to comprehend?
• Does this proposal make the Process shorter?
• Does this proposal simplify the Process?
• Does this proposal have to be implemented within the Process? (vs in some other external guidance document, say)

Make this part of the requirement for raising any new issue, and prioritise the issues based on which boxes are ticked.

[dwsinger]

yes, when I say reduce to a minimum, I meant a minimum that leaves the process comprehensible and integral, not zero

[jeffjaffe]

@nigelmegitt I believe it is length and complexity.

You want a defined goal. A defined goal is to reduce complexity. You might want some metrics for that. But I'm not aware of metrics for complexity. But to say that we shouldn't reduce complexity simply because it is too hard to measure seems wrong.

Here is an example of complexity that I would like to reduce. Chapter 6 of Process 2019 has 25 sub, sub-sub, and sub-sub-sub headings. Chapter 6 of Process 2020 has 35 such sub*-headings. My instincts are that we introduced all of that additional machinery so that we could be as clear as possible about how to execute P2020 - which made sense at the time. In some sense, it was a brute force attempt to cover every case. But my instincts also tell me that a refactoring of the process elements can probably achieve the same result with less complexity in the document.

[nigelmegitt]

I believe it is length and complexity.

Those are two goals! They might not always pull in the same direction.

In addition to a defined goal, I think it is important that there is consensus for that goal in the CG, i.e. that it is an agreed goal.

You might want some metrics for that.

That might be nice to have but I think this is in the eye of the beholder, and that's fine. I believe there are algorithms for computing the complexity of text, but I don't know if they're appropriate here.

Here is an example of complexity that I would like to reduce.

Great! At the risk of "teaching a grandmother to suck eggs" (sorry for the idiom; I'm pretty sure you're not a grandmother, @jeffjaffe !), why not raise that as a specific single issue? It's a concrete section where you think simplification can be achieved. It'd be helpful to have some proposals for how it can be achieved of course...

[dwsinger]

Speaking as chair, I am hoping that we can come with specific suggestions for

• simplification of structure and streamlining of text, to make the document simpler to read
• reducing guidance and narrative material to guidance documents, to shorten the normative process

This 'umbrella' issue isn't going to help very much. We need specific places where simplification seems possible, issues filed on those specific issues, and then, of course, pull requests.

I am hoping we can close this issue in favor of more specific ones, at some point.

[TzviyaSiegman]

The Process is too long and too complex. Ask someone for whom English is not a first language if they can parse it. @dwsinger - is the plan to do individual PRs on the full document?

[dwsinger]

Yes, I am looking for specific proposals resulting in pull requests.

[jeffjaffe]

Here is an example of complexity that I would like to reduce.

Great! At the risk of "teaching a grandmother to suck eggs" (sorry for the idiom; I'm pretty sure you're not a grandmother, @jeffjaffe !), why not raise that as a specific single issue? It's a concrete section where you think simplification can be achieved. It'd be helpful to have some proposals for how it can be achieved of course...

I apologize @nigelmegitt but your idiom is incomprehensible to me.

In terms of a concrete section, 6.2.11.5 is a great place to start. I'm unconvinced that we need three different processes called Last Call for Review of Proposed Corrections, Last Call for Review of Proposed Additions, and Last Call for Review of Proposed Corrections and Additions. To avoid that, requires some rethinking of the options that we have - we may need to drop process features to get some simplification. I had raised this with the editors prior to P2020 going to the membership but we ran out of time so I accepted an approach that I believe to be overly complex.

Candidly, though, if we fix 6.2.11.5 as a point issue, but do not have a broad commitment to improve the Process Document in general - enhancing 6.2.11.5 by itself will not fix the larger problem. For that reason, I would not support closing Wendy's issue and asking people to raise individual issues. There needs to be a broad commitment to simplification for this to work.

[dwsinger]

I think a specific issue "Can 6.2.11.5 be streamlined and simplified?" would be a great specific issue. I look for more...

[frivoal]

"Can 6.2.11.5 be streamlined and simplified?" would be a great specific issue

#358 (comment)

I look for more...

• Remove section of the Process on Workshop and Symposia #419
• Remove mentions of when various groups were created #420
• Simplify member submissions #421
• What's the point of what the Process has to say about Liasons? #422
• Concerns about Amended Recommendations #137 (comment)

I have a few more ideas, but here's a start.

[jwrosewell]

It isn’t sufficiently clear for it to be used as an acceptable process. Firstly, it is obscure, secondly it can be changed unintentionally, thirdly it is unclear who if anyone independent is taking a record that can be used in the defence of participants to demonstrate that they complied with their obligations. The process needs to be clear and intelligible and independently overseen and a credible record needs to be taken by independent W3C people.

Voluminous process that is not read or understood is the equivalent of no process at all and assures neither fairness of process nor outcome. The rules of the W3C and its processes must be amended with simplification in mind.

This will also facilitate ensuring that members do not stray from a process which would protect them and the organisation from risk of liability.

[frivoal]

It isn’t sufficiently clear for it to be used as an acceptable process.

Does "it" refer to the W3C Process in the above sentence? I'm assuming so, as I cannot see what other meaning this sentence may have.

While no text written by humans is ever perfect, and indeed this one could certainly be improved in a number of ways, claiming that the W3C Process is so unclear as to be unusable is quite a claim to make, given how many people have been relying on it for a great many years.

If you have concrete suggestions to make individual points clearer or simpler, they are most welcome. This is the very reason for this group to exist in the first place. However, vague claims the the document as a whole is useless without concrete proposals about what to do about it are not constructive, and frankly, insulting to the many people who have been involved in writing it, reviewing it, improving it, or who have voted in favor of adopting it over the years.

[jwrosewell]

"It" refers to the W3C Process in the prior comment. However some of the points I make can equally be applied to the relationship between the W3C Process and other documents, particularly the Membership Agreement and referenced documents.

As the Legal Entity (LE) is setup then there may be similar issues with the by-laws associated with the LE. Do we have visibility of the proposed by-laws or other documents that outline the purpose of the LE, expectations of statutory Directors, and the relationship to the W3C Process? Have we moved away from trying to retain the existing Membership Agreement and it's relationship with the W3C Process?

When did we last poll members to understand the role and use of the process? Is there a summary for new members that I've missed? My comments were made ~8 months after joining the W3C and having devoted some time - more than I suspect others do - to try and understand what is going on. They relate to use of the document in practice, recording of decision / meetings and, changes by virtue of inclusion of a myriad of referenced documents.

I'm not in a position to propose concrete changes to such a complex document. I don't have the time and I don't believe my views have consensus among the vocal followers of these issues.

I would rather see the results of general alignment to other documents, and simplification of documents by W3C Team factoring in the comments made here and elsewhere rather than making concrete changes myself. If there is a simpler document, that can be summarised for new members thus making the W3C Process more usable in practice, and address inconsistencies and overlap with other documents then that would be a great step forward.

Here is an example of an inconsistency. The W3C describes the TAG "is limited to technical issues about Web architecture". However TAG produce documents on ethics. How does such a document relate to the vision and values of the W3C? Is this issue inside the scope of the W3C Process or not? This member at least remains confused.

[jeffjaffe]

Several comments, @jwrosewell

1. Many of your comments are orthogonal to issue Streamlining the Process Document #356 which is not (yet) focused on the Legal Entity time frame. Some of your comments about the Legal Entity might be better placed as new Issues in https://github.com/w3c/AB-memberonly/issues.
2. While on the topic, though, I would say that many of your questions about the relationship between document in LE time frame still need to be done. Some indeed are open issues at the AB (https://github.com/w3c/AB-memberonly/issues/5)
3. Your input on simplifications of documents is well received. That is a principle that we generally align to.
4. You might choose to move your remarks about the TAG to https://github.com/w3c/AB-memberonly/issues/49

[TallTed]

Page count (especially when the grand range is 37–72 over 20 years, during which there's been a fair amount of change in the process being documented, especially in the current iteration) is a terrible metric to use for the sort of considerations I see in this thread.

It's especially important to remember that additional subheads, paragraph breaks, and various other things tend to increase the page count, but arguably make the document easier to read.

Word count is a somewhat better metric, even though there are times where more words are better to express something; they generally get balanced by using the fewest possible to be clear. I'm curious to see similar analysis as started this issue, putting word counts alongside the page counts.

[dwsinger]

Exporting into HTML and into Word gives me word counts: 28844 for 2020, 32095 for the current draft. We added Registries, W3C Statements, and a Tooling Policy, all new.

I agree, simplification can be other things than length, as we showed this year by re-structuring for flow, grouping, and readability.

[dwsinger]

More specific proposals and issues sought, please. We've done most of the easy stuff

[dwsinger]

closing, not because we disagree, but because we'd need specific requests on what/how to do it. we remain open to suggestions/requests to simplify – but they to tell us in what way