Enumerate the requirements for wide review #130
Comments
|
Read and understood. Will discuss at next CG meeting. |
|
taking up, and we should clarify if this is practice or process |
|
It is certainly a problem that chairs do not know what the minimum requirements are. In the past, people have wanted this treated as a practice rather than a process so that the Director might apply judgment in individual cases. But if chairs are totally confused, it might make sense to make this into a process issue. |
|
By the way, since I raised this, it's become clear that horizontal review is considered to be a subset of wide review, so that needs to be brought into scope. Additionally, I think that other W3C groups that are listed in the WG's Charter as dependencies also need to be notified. One more point: there's some confusion about whether wide review is needed to enter CR, to exit CR, or both. The usual practice and expectation is that it is required in order to enter CR, but the Process clearly states that it is also required in order to enter PR. The ambiguity here is whether or not the same wide review can be used for entering CR and PR; since a spec cannot get to PR except from a CR, by definition all specs must have demonstrated wide review already. So why require it again explicitly for PR? Is it supposed to be a wide review of any deltas between the intended PR and the CR? |
|
I think concrete suggestions to help clarify would be very welcome. |
|
I think Section 6.2.3.1 has most of the information, but it's just not written in a way that's easy to parse.
If we can get some agreement around the high level points, I'll take a run at some words. |
|
@LJWatson Yes, and I'd expand "what should be done with the resulting feedback" a bit to include the follow-up step with the commenter, in other words:
and add:
I don't agree that Section 6.2.3.1 has most of the information. That section currently describes the objectives of the Wide Review, which is definitely worth doing, but it does not address your summary bullet points. I think the edit needed is mainly additive rather than a change to the existing words. |
|
I agree, the distinction between the review per se, and the consequent discussion, and possible action, may need to be clearer. The discussions can be effectively interminable; but the group needs to get past the milestone. Advice on how to handle that would be good. |
|
I don't know where to draw the line between formal Process and guidance on how to implement the policy in a way that will satisfy the Director. But from watching the pain chairs go through, it seems to me that wide review needs to be less labor intensive to track and quicker to conclude. That said, "wide review" is one of the features that distinguish W3C Recommendations from lighter-way alternative ways to define specs, so finding the right balance is critical. Things I'd like to see in the process or guidance:
|
Why "Editors"? I think this is a WG responsibility.
@michaelchampion I disagree - it is too onerous on external individuals and organisations to force them around this loop, and it is courteous to conclude the conversation that they began. It is also useful information if they agreed or disagreed with the WG's disposition, since the Director can rapidly filter out the "agreed"s and focus only on the "disagreed"s which is where the attention is likely needed. |
|
The question I'm asking is whether we want to make it onerous for chairs / editors to process feedback at the cost of making the standards process slow and bureaucratic or to make it more onerous for external reviewers to track the resolution of their issues and contest the WG / editor resolution. I see a lot of work that once might have gone to W3C being done in WHATWG or open source projects that operate by "lazy consensus" http://en.osswiki.info/concepts/lazy_consensus. They give editors / committers clear ownership, and puts the burden on reviewers who don't agree to speak up and make their case to do something different. But I agree it's a difficult balance to strike -- a value of W3C standards over those from some more agile organizations is a broad base of stakeholders who have given specs wide review. High quality input from external experts who will engage to make their case is definitely valuable, but such reviewers are generally happy to engage directly in the WG's issues list in my experience. |
|
We want good results without it being onerous (indefinite amounts of effort) for anyone, or taking indefinite amounts of time. We have to find a better way to do this.
On Feb 28, 2018, at 8:53 , Michael Champion ***@***.***> wrote:
The question I'm asking is whether we want to make it onerous for chairs / editors to process feedback at the cost of making the standards process slow and bureaucratic or to make it more onerous for external reviewers to track the resolution of their issues and contest the WG / editor resolution.
David Singer
Manager, Software Standards, Apple Inc.
|
@michaelchampion Sometimes they do, but often they do not: a lot of feedback comes my way from other standards organisations representing a constituency who are giving collective feedback with the goal of adopting/referencing W3C standards within other specifications. Their preferred engagement model is much more ponderous, and consists of liaison messages. Obviously where possible we do try to engage more directly, but members of other standards organisations do not always want to come out in the open. |
|
I've taken a first pass at some words. My hunch is that this is guidance rather than process, even assuming it captures what we hope it will. In the interests of having something concrete to work with though... the purpose of a wide review is to seek feedback on a document from the entire web community, including the general public. The goal is to make the document as robust as possible before it becomes a Recommendation. A wide review can be conducted at different stages of a document's maturity. Depending on the reason for the wide review, the set of stakeholders may vary. As a minimum, a comprehensive wide review should be conducted as the document prepares to transition to Candidate Recommendation. At this stage a wide review request should be sent to the following stakeholders:
In addition, a notice should be sent to [insert email here]. A wide review can also be conducted at the following times:
To facilitate wide review, a document should have a log of changes made since its previous publication on TR. Issues raised as a consequence of a wide review should be filed on the Github repository for the document. This streamlines the management of issue discussion and resolution, and makes sure the person or organisation that files the issue is informed of its progress. |
|
It's a start, but I think the hard questions are around "what is a review? when is it complete? what level of reaction is needed? how much response is needed? how much consensus with the commenter is needed? can review to-and-fro be allowed to drag on forever? can deadlines be put on the stages? how much documentation trail is needed? are tagged GH issues good enough?" |
|
The Process shouldn't say where issues should be raised because it's a WG decision about how to work and what tools to use. However the SOTD in the document being reviewed must do so I think. If it's not already a requirement then it needs to be made one. The "other" stakeholders need to include any external liaisons registered as having an interest in the topic (see Liaisons for the list) or listed in the WG's Charter, and the general public, as notified minimally on the w3.org homepage but also by any other W3 communications channel such as twitter, RSS etc. I'm slightly troubled by the idea of differing sets of stakeholders for wide review depending on maturity level. It would be simpler to have one process that is wide review, and reuse it wherever needed. For what it's worth, my pet name for a review process that only goes to other W3 groups is "narrow review". |
|
Team is working on this and we expect a proposal |
|
@swickr can you share updates from your conversations with i18n and a11y? |
frivoal
added
the
DoC
I think that this is sufficiently similar to what others and I have proposed — that we tersely state the normative requirements in the process, and link to a Guide document that is more helpful to read — that we'll need to work out whether we can do this separation cleanly as I hope, or have normative statements outside the process. The advantage of the second document bding a Guide is that its update process can be relaxed; unlike say CEPC, which because it's normative we subject to W3M, ASB, and AC scrutiny, we could have a lighter hand. I doubt we'll find out which approach can be made to work without trying, though. |
|
I think that this is sufficiently similar to what others and I have
proposed — that we tersely state the normative requirements in the
process, and link
to a Guide document that is more helpful to read — that we'll need to
work out whether we can do this separation cleanly as I hope, or have
normative statements
outside the process."
It is the balance of separation that is important I think. If the
requirements documented in /Guide can be considered normative, and the
Process can normatively state that they must be followed, I would be ok
with that approach.
Otherwise I feel strongly that the Process itself needs to be
unequivocal about the requirements.
If one or other of these things is not possible, then I fear we will
have failed to address the feedback we heard from multiple team
contacts, chairs and editors during the AB priority task review.
|
|
I would not be okay with an approach that made novel requirements stated in /Guide (and thus outside the jurisdiction of the Process, and the AB/AC) normative. Any normative requirements must go through the same process as updates to the Process itself, or there is no escalation path. |
|
re:
I am truly hesitant to do the former; linked documents that are normative have to go through much greater scrutiny. I think we should try to do the latter instead: normative requirements tersely stated in the process, and an informative and readable guide that introduces the tooling, best practices, considerations, and so on, and is actually helpful; and because it's informative, it can be easily updated without W3M, AB, AC and Director review and formal approval. If we fail on this separation and determine it can't be done, then I'll grudgingly consider a linked document with normative requirements... |
|
I think there is Process and then there is process. The normative bits should go in the Process doc--you have to get HR within such-and-such time (e.g. the putative 60 days) and either resolve issues to the satisfaction of the HR group (which can include deferring the issue) or be prepared to document the dispute at transition time. Etc. I think these can be fairly succinct declarative statements. And then there are the minutiae of conducting HR: where to file requests, how to label issues, where to look for tooling, where to get self-review checklists, when to do review, etc. That sort of thing probably belongs in the Guide and doesn't probably require W3M/AB/AC/Director scrutiny. It's a Best Practices document that borders on normative (lots of SHOULD, reserving MUST for things found in the Process). A WG would not be well-received if they hewed to the requirements of the Process but ignored the best practices. |
|
It might be useful to think about the questions the various process-related documents address:
@LJWatson and @plehegar it's apparently true that the Process today links to normative documents that are not subject to a community-wide consensus process. I think that's a bug not a feature. Or perhaps that it's a feature of a Director-controlled community in which the Director can delegate the implementation details but is ultimately responsible for them just as they[*] are ultimately responsible for declaring consensus . Either way, it's not sustainable going forward. [* ]Both a gender-neutral singular reference to the current nominal Director and a plural reference to the W3M consensus that seems to be the de facto Director today. |
I'm beginning to think we're going around in circles. The reason for the proposed text was to address feedback that the requirements for wide/horizontal review were not clear, were not mandated, and were consequently not happening properly. I think the proposed text addresses the first two parts of that feedback, and will hopefully realise the third. But there have been requests to move requirements out of the Process and requests to keep it in; requests to make the requirements more specific and requests to make it more vague, and somewhere in the background we've touched on the wider intent to streamline the process as a whole. I'm exhausted. |
|
one note on this: /Guide on wide review was refreshed and there is now https://www.w3.org/Guide/documentreview/ |
|
Thanks @plehegar . Question to everyone on this thread... Now this guidance exists, does it impact what should/should not go into the process, and if so, how? |
|
we need specific wording, ideally a Pull Request, please, if any changes need to be made for P2021 |
|
(We could also improve the Guide material) |
|
The problems to solvce include at least
|
|
We think the new /Guide document helps with these, and it's not entirely clear what text would need to be added to the Process other than what's there to support it. If something is needed, we need a more specific suggestion. |
Yes. Several of the minimal requirements I suggested in this issue are still undocumented, and I'm not clear whether that's because they're optional, or stated some other place, but I don't see them in the Guide or the Process. Specifically the required handling of comments is not documented. Something along the lines of (quoting myself):
I raised w3c/guide#276 to capture this somewhat. My view is that it would be better for the minimal requirements to be documented in the Process, and additional good practices to be documented in the Guide. If I've understood correctly, there is no minimal requirement on comment handling in the Process, but there is a historical expectation that things will be done with an appropriate level of formality, in order to demonstrate the review has been completed. I'd like to see all WGs work to the same rules here, and that means defining something in the Process, even if that something isn't very detailed. |
|
@nigelmegitt I believe the requirement to handle comments is already covered by the combination of these two sections: |
|
@fantasai thank you, yes, and apologies for the noise caused by my not noticing those sections previously. I withdraw my comment that "handling of comments is not documented" (with some embarrassment!). If there are no other unresolved comments on this issue, then, having raised it, I'd be happy to close it, or for someone else to. |
|
Why has this been closed? |
|
closed because we thought it was covered in all the /Guide updates and no more process CG action was needed. If you disagree, can you either re-open and say why, or file new issues? |
|
@dwsinger @LJWatson -- I think it worth noting that the person who raised the issue (@nigelmegitt) believed it had been resolved, so it was ready to be closed. |
and others
I've observed that there is confusion about what constitutes a good wide review. It would be helpful to clarify in the Process the minimal requirements for processing review feedback, for example to match the good practice that was embedded in the old disposition of comments tool, namely:
This is minimal - obviously groups that enter into discussions with the commenter and can document a conversation in which the WG and the commenter arrive at a mutually acceptable conclusion would satisfy these requirements.
The text was updated successfully, but these errors were encountered: